Your new customer has requested daily reports that show their net consumption of Google Cloud compute resources and who used the resources. You need to quickly and efficiently generate these daily reports. What should you do?
You need to create a SQL pipeline. The pipeline runs an aggregate SOL transformation on a BigQuery table every two hours and appends the result to another existing BigQuery table. You need to configure the pipeline to retry if errors occur. You want the pipeline to send an email notification after three consecutive failures. What should you do?
Correct Answer: D
To create a robust and resilient SQL pipeline in BigQuery that handles retries and failure notifications, consider the following: BigQuery Scheduled Queries: This feature allows you to schedule recurring queries in BigQuery. It is a straightforward way to run SQL transformations on a regular basis without requiring extensive setup. Error Handling and Retries: While BigQuery Scheduled Queries can run at specified intervals, they don't natively support complex retry logic or failure notifications directly. This is where additional Google Cloud services like Pub/Sub and Cloud Functions come into play. Pub/Sub for Notifications: By configuring a BigQuery scheduled query to publish messages to a Pub/Sub topic upon failure, you can create a decoupled and scalable notification system. Cloud Functions: Cloud Functions can subscribe to the Pub/Sub topic and implement logic to count consecutive failures. After detecting three consecutive failures, the Cloud Function can then send an email notification using a service like SendGrid or Gmail API. Implementation Steps: Set up a BigQuery Scheduled Query: Create a scheduled query in BigQuery to run your SQL transformation every two hours. Configure the scheduled query to publish a notification to a Pub/Sub topic in case of a failure. Create a Pub/Sub Topic: Create a Pub/Sub topic that will receive messages from the scheduled query. Develop a Cloud Function: Write a Cloud Function that subscribes to the Pub/Sub topic. Implement logic in the Cloud Function to track failure messages. If three consecutive failure messages are detected, the function sends an email notification. Reference: BigQuery Scheduled Queries Pub/Sub Documentation Cloud Functions Documentation SendGrid Email API Gmail API
Professional-Data-Engineer Exam Question 33
You are designing a pipeline that publishes application events to a Pub/Sub topic. You need to aggregate events across hourly intervals before loading the results to BigQuery for analysis. Your solution must be scalable so it can process and load large volumes of events to BigQuery. What should you do?
Correct Answer: B
Professional-Data-Engineer Exam Question 34
You are designing storage for two relational tables that are part of a 10-TB database on Google Cloud. You want to support transactions that scale horizontally. You also want to optimize data for range queries on nonkey columns. What should you do?
Correct Answer: C
Professional-Data-Engineer Exam Question 35
Google Cloud Bigtable indexes a single value in each row. This value is called the _______.
Correct Answer: C
Cloud Bigtable is a sparsely populated table that can scale to billions of rows and thousands of columns, allowing you to store terabytes or even petabytes of data. A single value in each row is indexed; this value is known as the row key.
