Noos
PT EN
RAN-DBDEA-00014
Ranvier · DBDEA

Exam DBDEA Topics: ingestion Difficulty: easy

A team copies this snippet from an old tutorial to ingest JSON files that keep arriving in a Unity Catalog volume: ```python df = (spark.read .format("cloudFiles") .option("cloudFiles.format", "json") .load("/Volumes/main/ingest/events/")) ``` The job runs on a schedule, but each run never picks up files that arrived after the very first execution, and no schema inference is configured. Which change turns this into a working Auto Loader stream with schema inference and evolution?

Options (Reading Mode) Click to filter discussion →
  1. A. Switch the reader to spark.readStream.format("cloudFiles") and keep everything else unchanged, because schema inference needs no additional options
  2. B. Keep spark.read and add .option("cloudFiles.schemaLocation", ...) to the reader, since the batch reader supports Auto Loader options too
  3. C. Use spark.readStream.format("cloudFiles") with cloudFiles.format and a durable cloudFiles.schemaLocation for schema inference and evolution
  4. D. Replace format("cloudFiles") with format("streamingFiles") and set checkpointLocation on the reader, because schemaLocation only applies to Delta targets
Reveal answer

Correct answer: Use spark.readStream.format("cloudFiles") with cloudFiles.format and a durable cloudFiles.schemaLocation for schema inference and evolution

Reference: https://docs.databricks.com/aws/en/ingestion/cloud-object-storage/auto-loader/

Train this exam on Ranvier →

02 — The Myelin Layer · Canonical Discussion

Where the exam tries to trick you — and how those who passed solved it.

0 Passed

contributed to this question

Answer Distractor Matrix Click an option to filter or open its dissection.
✦ Consolidated Synthesis (Wiki) rev. #1 (baseline) · CC BY-SA 4.0
Start synthesis for this question →

No community synthesis consolidated for this question yet. Below is the verifiable official starting point — if you passed the exam, open the option dissection.

Official Answer (C): Use spark.readStream.format("cloudFiles") with cloudFiles.format and a durable cloudFiles.schemaLocation for schema inference and evolution

Ranvier · DBDEA

Contribute to this Question

Your contribution builds your Orbital Sheath

Peer contributions

No public contribution yet. Be the first who passed to explain why the distractors trick candidates — your contribution builds your Orbital Sheath.