You are selecting services to write and transform JSON messages from Cloud Pub/Sub to BigQuery for a data pipeline on Google Cloud. You want to minimize service costs. You also want to monitor and accommodate input data volume that will vary in size with minimal manual intervention. What should you do?
Correct Answer: D
Professional-Data-Engineer Exam Question 17
You are administering a BigQuery on-demand environment. Your business intelligence tool is submitting hundreds of queries each day that aggregate a large (50 TB) sales history fact table at the day and month levels. These queries have a slow response time and are exceeding cost expectations. You need to decrease response time, lower query costs, and minimize maintenance. What should you do?
Correct Answer: A
To improve response times and reduce costs for frequent queries aggregating a large sales history fact table, materialized views are a highly effective solution. Here's why option A is the best choice: Materialized Views: Materialized views store the results of a query physically and update them periodically, offering faster query responses for frequently accessed data. They are designed to improve performance for repetitive and expensive aggregation queries by precomputing the results. Efficiency and Cost Reduction: By building materialized views at the day and month level, you significantly reduce the computation required for each query, leading to faster response times and lower query costs. Materialized views also reduce the need for on-demand query execution, which can be costly when dealing with large datasets. Minimized Maintenance: Materialized views in BigQuery are managed automatically, with updates handled by the system, reducing the maintenance burden on your team. Steps to Implement: Identify Aggregation Queries: Analyze the existing queries to identify common aggregation patterns at the day and month levels. Create Materialized Views: Create materialized views in BigQuery for the identified aggregation patterns. For example CREATE MATERIALIZED VIEW project.dataset.sales_daily_summary AS SELECT DATE(transaction_time) AS day, SUM(amount) AS total_sales FROM project.dataset.sales GROUP BY day; CREATE MATERIALIZED VIEW project.dataset.sales_monthly_summary AS SELECT EXTRACT(YEAR FROM transaction_time) AS year, EXTRACT(MONTH FROM transaction_time) AS month, SUM(amount) AS total_sales FROM project.dataset.sales GROUP BY year, month; Query Using Materialized Views: Update existing queries to use the materialized views instead of directly querying the base table. Reference Links: BigQuery Materialized Views Optimizing Query Performance
Professional-Data-Engineer Exam Question 18
Which of these are examples of a value in a sparse vector? (Select 2 answers.)
Correct Answer: C,D
Categorical features in linear models are typically translated into a sparse vector in which each possible value has a corresponding index or id. For example, if there are only three possible eye colors you can represent 'eye_color' as a length 3 vector: 'brown' would become [1, 0, 0], 'blue' would become [0, 1, 0] and 'green' would become [0, 0, 1]. These vectors are called "sparse" because they may be very long, with many zeros, when the set of possible values is very large (such as all English words). [0, 0, 0, 1, 0, 0, 1] is not a sparse vector because it has two 1s in it. A sparse vector contains only a single 1. [0, 5, 0, 0, 0, 0] is not a sparse vector because it has a 5 in it. Sparse vectors only contain 0s and 1s. Reference: https://www.tensorflow.org/tutorials/linear#feature_columns_and_transformations
Professional-Data-Engineer Exam Question 19
You want to migrate an on-premises Hadoop system to Cloud Dataproc. Hive is the primary tool in use, and the data format is Optimized Row Columnar (ORC). All ORC files have been successfully copied to a Cloud Storage bucket. You need to replicate some data to the cluster's local Hadoop Distributed File System (HDFS) to maximize performance. What are two ways to start using Hive in Cloud Dataproc? (Choose two.)
Correct Answer: B,C
Professional-Data-Engineer Exam Question 20
Your new customer has requested daily reports that show their net consumption of Google Cloud compute resources and who used the resources. You need to quickly and efficiently generate these daily reports. What should you do?