Online Access Free CDP-3002 Exam Questions

How does Apache NiFi support schema inference during data flow management?

• A.By enforcing static schemas for all data flows
• B.By manually specifying schemas for each data source
• C.Utilizing external schema repositories exclusively for schema management
• D.Through processors that automatically detect data formats and schemas

In Apache Airflow, how can you ensure that a data quality check task is only executed if a previous data ingestion task succeeded and the ingested data volume exceeds a certain threshold?

• A.Utilize the ShortCircuitOperator with a custom Python function to check both conditions.
• B.Configure the trigger_rule parameter to all_success and use a PythonOperator to verify the data volume.
• C.Use a BranchPythonOperator with a condition that checks the success of the ingestion task and the data volume.
• D.Implement a PythonOperator with a pre_execute hook for the condition check.

You have a DataFrame containing sales data with columns "product_id", "customer id", and "amount". How can you efficiently calculate the total sales per customer?

• A.Utilize spark SQL's GROUP BY and SUM functions
• B.Implement a custom function to group and sum the sales
• C.Leverage Spark's machine learning library (MLIiB. for aggregation
• D.Use a loop to iterate through the DataFrame and accumulate the sales for each customer

You encounter an error during a data quality check within your Airflow DAG. How can you access detailed information about the error to aid in troubleshooting?

• A.The Airflow web UI task details page might display error messages related to the failed check.
• B.You can directly inspect the custom Python script used for the check to identify potential issues.
• C.All of the above
• D.The Airflow logs (scheduler, worker) might contain relevant error messages from the failing task.

Explain the role of Spark MLIib and its functionalities in building machine learning pipelines on Spark.

• A.Spark MLIib is a comprehensive library for machine learning tasks within Spark
• B.It provides pre-built machine learning algorithms and tools for building and evaluating models on Spark.
• C.It offers functionalities for data pre-processing, feature engineering, and model training within Spark.
• D.It allows integration with external machine learning libraries like TensorFlow or scikit-learn.