Noos
PT EN
RAN-DBDEA-00029
Ranvier · DBDEA

Exam DBDEA Topics: ingestion Difficulty: hard

A notebook task extracts a 400-million-row table from a PostgreSQL data warehouse over JDBC. The current read runs on a single connection and takes hours: ```python orders = (spark.read.format("jdbc") .option("url", jdbc_url) .option("dbtable", "sales.orders") .option("user", user) .option("password", pwd) .load()) ``` How should the team parallelize the read, and what is the effect?

Options (Reading Mode) Click to filter discussion →
  1. A. Set numPartitions=8 alone; Spark opens eight simultaneous queries against sales.orders without needing any column or bound options
  2. B. Add partitionColumn (indexed, uniform), lowerBound, upperBound, and numPartitions; Spark runs concurrent range queries
  3. C. Set fetchSize=100000 so each round trip pulls more rows through the existing single connection, which parallelizes the fetch
  4. D. Set partitionColumn to the primary key and numPartitions=500 to maximize the number of simultaneous queries against the warehouse
Reveal answer

Correct answer: Add partitionColumn (indexed, uniform), lowerBound, upperBound, and numPartitions; Spark runs concurrent range queries

Reference: https://docs.databricks.com/aws/en/archive/connectors/jdbc

Train this exam on Ranvier →

02 — The Myelin Layer · Canonical Discussion

Where the exam tries to trick you — and how those who passed solved it.

0 Passed

contributed to this question

Answer Distractor Matrix Click an option to filter or open its dissection.
✦ Consolidated Synthesis (Wiki) rev. #1 (baseline) · CC BY-SA 4.0
Start synthesis for this question →

No community synthesis consolidated for this question yet. Below is the verifiable official starting point — if you passed the exam, open the option dissection.

Official Answer (B): Add partitionColumn (indexed, uniform), lowerBound, upperBound, and numPartitions; Spark runs concurrent range queries

Ranvier · DBDEA

Contribute to this Question

Your contribution builds your Orbital Sheath

Peer contributions

No public contribution yet. Be the first who passed to explain why the distractors trick candidates — your contribution builds your Orbital Sheath.