For months, a nightly COPY INTO has loaded new files from a landing volume into main.ops.bookings. During a schema dispute, an engineer drops and re-creates the table with the correct columns. The next run reloads every file still present in the landing directory, duplicating months of history. Where does COPY INTO keep its record of already-loaded files?

1. A. In a workspace-level ingestion service that tracks every loaded file globally, shared by all tables that read from the same cloud storage account 💬 A
2. B. In the source path itself: each landing directory maintains its own tracking metadata, independent of whichever table the data lands in 💬 B
3. C. In the Auto Loader checkpoint directory that backs the job, which should have been backed up before the table was re-created 💬 C
4. D. In the target table's Delta transaction log: dropping and re-creating the table destroyed the loaded-file history, so everything was reloaded ✓ D

Reveal answer

Correct answer: In the target table's Delta transaction log: dropping and re-creating the table destroyed the loaded-file history, so everything was reloaded

Reference: https://docs.databricks.com/aws/en/ingestion/copy-into