Noos
PT EN
RAN-DBDEA-00026
Ranvier · DBDEA

Exam DBDEA Topics: ingestion Difficulty: medium

A logistics company must ingest shipment telemetry from a supplier's bucket: roughly 5,000 new Parquet files per day arriving around the clock, volume growing about 20% per year, and the producer occasionally adds columns. The team wants low operational overhead with schema handled automatically. The SQL-first team lead proposes a scheduled COPY INTO statement that runs every 15 minutes. What should the data engineering team recommend?

Options (Reading Mode) Click to filter discussion →
  1. A. Accept COPY INTO — it scales to any file count and absorbs column changes as long as it runs frequently enough
  2. B. Replace it with a nightly spark.read.parquet overwrite of the entire bucket, which avoids streaming checkpoints entirely
  3. C. Use Auto Loader (e.g. a streaming table): incremental file tracking, very large scale, and schema inference with evolution
  4. D. Query the supplier's operational database over JDBC instead, because file-based ingestion cannot keep up with around-the-clock arrivals
Reveal answer

Correct answer: Use Auto Loader (e.g. a streaming table): incremental file tracking, very large scale, and schema inference with evolution

Reference: https://docs.databricks.com/aws/en/ingestion/cloud-object-storage/auto-loader/

Train this exam on Ranvier →

02 — The Myelin Layer · Canonical Discussion

Where the exam tries to trick you — and how those who passed solved it.

0 Passed

contributed to this question

Answer Distractor Matrix Click an option to filter or open its dissection.
✦ Consolidated Synthesis (Wiki) rev. #1 (baseline) · CC BY-SA 4.0
Start synthesis for this question →

No community synthesis consolidated for this question yet. Below is the verifiable official starting point — if you passed the exam, open the option dissection.

Official Answer (C): Use Auto Loader (e.g. a streaming table): incremental file tracking, very large scale, and schema inference with evolution

Ranvier · DBDEA

Contribute to this Question

Your contribution builds your Orbital Sheath

Peer contributions

No public contribution yet. Be the first who passed to explain why the distractors trick candidates — your contribution builds your Orbital Sheath.