A healthcare analytics team is implementing a dimensional model in Delta Lake for patient care analysis. They have a date dimension table and are evaluating design options to ensure it supports a wide range of time-based analyses. Which design approach for the date dimension will support efficient time-based querying and aggregation?

• A. Pre-calculate attributes like fiscal_period, quarter, month_name, day_of_week, and holiday.
• B. Store the date as string in ISO format (YYYY-MM-DD) for readability.
• C. Create separate dimension tables for different calendar systems (fiscal, academic, etc.)
• D. Store only the date value and calculate all time attributes in queries.

A data engineer is configuring a Databricks Asset Bundle to deploy a job with granular permissions.
The requirements are:
- Grant the data-engineers group CAN_MANAGE access to the job.
- Ensure the auditors' group can view the job but not modify/run it.
- Avoid granting unintended permissions to other users/groups.
How should the data engineer deploy the job while meeting the requirements?

• A. resources:
  jobs:
  my-job:
  name: data-pipeline
  tasks: [...]
  job_clusters: [...]
  permissions:
  - group_name: data-engineers
  level: CAN_MANAGE
  - group_name: auditors
  level: CAN_VIEW
• B. resources:
  jobs:
  my-job:
  name: data-pipeline
  tasks: [...]
  job_clusters: [...]
  permissions:
  - group_name: data-engineers
  level: CAN_MANAGE
  - group_name: auditors
  level: CAN_VIEW
  - group_name: admin-team
  level: IS_OWNER
• C. permissions:
  - group_name: data-engineers
  level: CAN_MANAGE
  - group_name: auditors
  level: CAN_VIEW
  resources:
  jobs:
  my-job:
  name: data-pipeline
  tasks: [...]
  job_clusters: [...]
• D. resources:
  jobs:
  my-job:
  name: data-pipeline
  tasks: [...]
  job: [...]
  permissions:
  - group_name: data-engineers
  level: CAN_MANAGE
  permissions:
  - group_name: auditors
  level: CAN_VIEW

Given the following PySpark code snippet in a Databricks notebook:
filtered_df = spark.read.format("delta").load("/mnt/data/large_table")
\
.filter("event_date > '2024-01-01'")
filtered_df.count()
The data engineer notices from the Query Profiler that the scan operator for filtered_df is reading almost all files, despite the filter being applied.
What is the probable reason for poor data skipping?

• A. The Delta table lacks optimization that enables dynamic file pruning.
• B. The event_date column is outside the table's partitioning and Z-ordering scheme.
• C. The filter is executed only after the full data scan, preventing data skipping.
• D. The filter condition involves a data type excluded from data skipping support.

A company processes semi-structured JSON files from an external source using Auto Loader in a classic Databricks job. Occasionally, records arrive with null critical fields, invalid types, or unexpected nested schema variations. The engineer must ensure that malformed or non- conforming records are not dropped silently and are captured in a separate quarantine table. The pipeline should continue processing good records into the Bronze layer without failing the job, and the approach must support both batch and streaming ingestion.
The data engineer needs to build a robust ingestion pattern that automatically routes bad records to a quarantine Delta table, while still ingesting good records into the Bronze layer for further processing.
Which approach fulfills the quarantine mechanism in this ingestion architecture?

• A. Create a notebook job with inferSchema=True, write a streaming query with .foreachBatch() and catch exceptions using try/except to redirect failed batches to quarantine.
• B. Use Lakeflow Spark Declarative Pipelines with a SQL pipeline; configure it to drop rows with nulls using where critical_fields is not null, and rely on audit logs for malformed data.
• C. Use Auto Loader with failFast mode to set to false, and enable schema evolution; invalid records will be silently ignored during ingestion.
• D. Use Auto Loader with LDP and implement an EXPECT () constraint with a record audit logic to route bad records.

A data ingestion task requires a one-TB JSON dataset to be written out to Parquet with a target part-file size of 512 MB. Because Parquet is being used instead of Delta Lake, built-in file-sizing features such as Auto-Optimize & Auto-Compaction cannot be used.
Which strategy will yield the best performance without shuffling data?

• A. Set spark.sql.shuffle.partitions to 2,048 partitions (1TB*1024*1024/512), ingest the data, execute the narrow transformations, optimize the data by sorting it (which automatically repartitions the data), and then write to parquet.
• B. Set spark.sql.files.maxPartitionBytes to 512 MB, ingest the data, execute the narrow transformations, and then write to parquet.
• C. Set spark.sql.adaptive.advisoryPartitionSizeInBytes to 512 MB bytes, ingest the data, execute the narrow transformations, coalesce to 2,048 partitions (1TB*1024*1024/512), and then write to parquet.
• D. Ingest the data, execute the narrow transformations, repartition to 2,048 partitions (1TB*
  1024*1024/512), and then write to parquet.
• E. Set spark.sql.shuffle.partitions to 512, ingest the data, execute the narrow transformations, and then write to parquet.