A re-delivered bronze file duplicated customer records: the same customer_id now appears up to three times, and the copies differ in their updated_at values, so no two rows are fully identical. For the silver table, the team must end up with exactly one row per customer_id, and keeping any one of the copies is acceptable. A teammate proposes calling distinct() on the bronze DataFrame. What is the difference between the two candidate calls?

1. A. distinct() compares every column, so rows that differ in updated_at all survive; dropDuplicates(['customer_id']) keeps a single row per key ✓ A
2. B. distinct() and dropDuplicates() are interchangeable, so the teammate's call would already deduplicate on the customer_id key 💬 B
3. C. distinct() accepts a column list, as in distinct('customer_id'), and deduplicates on just the columns passed to it 💬 C
4. D. dropDuplicates(['customer_id']) cannot work by itself; it must be combined with a row_number() window to pick the surviving row 💬 D

Reveal answer

Correct answer: distinct() compares every column, so rows that differ in updated_at all survive; dropDuplicates(['customer_id']) keeps a single row per key

Reference: https://docs.databricks.com/aws/en/sql/language-manual/sql-ref-functions