Noos
PT EN
RAN-DBDEA-00042
Ranvier · DBDEA

Exam DBDEA Topics: transformation Difficulty: medium

Two regional extracts can overlap: the same order row may appear in both files. An engineer combines them expecting SQL UNION semantics, where duplicate rows collapse into one: ```python combined = extract_eu.union(extract_us) ``` The result still contains duplicated rows. Why, and how should the code be fixed?

Options (Reading Mode) Click to filter discussion →
  1. A. DataFrame.union() deduplicates only when both sides have identical column names — rename the columns first
  2. B. union() is UNION ALL by design — rows stack positionally; append .distinct() to deduplicate
  3. C. unionAll() is the deduplicating variant — switch to extract_eu.unionAll(extract_us)
  4. D. union() runs asynchronously, so duplicates appear until the result is materialized — call .cache() and count() first
Reveal answer

Correct answer: union() is UNION ALL by design — rows stack positionally; append .distinct() to deduplicate

Reference: https://docs.databricks.com/aws/en/pyspark/reference/functions

Train this exam on Ranvier →

02 — The Myelin Layer · Canonical Discussion

Where the exam tries to trick you — and how those who passed solved it.

0 Passed

contributed to this question

Answer Distractor Matrix Click an option to filter or open its dissection.
✦ Consolidated Synthesis (Wiki) rev. #1 (baseline) · CC BY-SA 4.0
Start synthesis for this question →

No community synthesis consolidated for this question yet. Below is the verifiable official starting point — if you passed the exam, open the option dissection.

Official Answer (B): union() is UNION ALL by design — rows stack positionally; append .distinct() to deduplicate

Ranvier · DBDEA

Contribute to this Question

Your contribution builds your Orbital Sheath

Peer contributions

No public contribution yet. Be the first who passed to explain why the distractors trick candidates — your contribution builds your Orbital Sheath.