A bronze streaming table in a Lakeflow pipeline ingests device events. Rows with a null device_id arrive occasionally; they must never reach the table, but the team wants the failure counts visible in pipeline metrics. The current definition is: ```sql CREATE OR REFRESH STREAMING TABLE main.bronze.device_events AS SELECT * FROM STREAM read_files('/Volumes/main/bronze/devices/'); ``` How should the pipeline handle the invalid rows?

1. A. Add an expectation with the default warn action: invalid rows still land in the table, and only failure counts appear as metrics 💬 A
2. B. Add a CONSTRAINT ... EXPECT (device_id IS NOT NULL) ON VIOLATION DROP ROW clause: invalid rows are dropped and their count is logged ✓ B
3. C. Leave the definition as is and filter WHERE device_id IS NOT NULL in every downstream dashboard and query that reads the table 💬 C
4. D. Direct all raw events into a separate quarantine table and have an on-call engineer delete the invalid rows manually after every pipeline refresh 💬 D

Reveal answer

Correct answer: Add a CONSTRAINT ... EXPECT (device_id IS NOT NULL) ON VIOLATION DROP ROW clause: invalid rows are dropped and their count is logged

Reference: https://docs.databricks.com/aws/en/ldp/concepts/streaming-tables