Noos
PT EN
RAN-DBDEA-00143
Ranvier · DBDEA

Exam DBDEA Topics: transformation Difficulty: easy

Before promoting a new PySpark transformation into the nightly production job, an engineer wants to see what Spark intends to do with it — the planned join strategy, the filters, and the projection — without spending cluster resources actually running the query. Which action gives her this preview?

Options (Reading Mode) Click to filter discussion →
  1. A. Call count() after every transformation and compare the elapsed time of each stage
  2. B. Nothing — lazy evaluation makes the plan unknowable until the query has fully executed
  3. C. Call df.explain() to print the logical and physical plans without executing anything
  4. D. Run the job in production and inspect the Spark UI, the only place where plans appear
Reveal answer

Correct answer: Call df.explain() to print the logical and physical plans without executing anything

Reference: https://docs.databricks.com/aws/en/pyspark/reference

02 — The Myelin Layer · Canonical Discussion

Where the exam tries to trick you — and how those who passed solved it.

0 Passed

contributed to this question

Answer Distractor Matrix Click an option to filter or open its dissection.
✦ Consolidated Synthesis (Wiki) rev. #1 (baseline) · CC BY-SA 4.0
Start synthesis for this question →

No community synthesis consolidated for this question yet. Below is the verifiable official starting point — if you passed the exam, open the option dissection.

Official Answer (C): Call df.explain() to print the logical and physical plans without executing anything

⚓ Official Documentation Anchor: https://docs.databricks.com/aws/en/pyspark/reference
Ranvier · DBDEA

Contribute to this Question

Your contribution builds your Orbital Sheath

Peer contributions

No public contribution yet. Be the first who passed to explain why the distractors trick candidates — your contribution builds your Orbital Sheath.