Online Access Free Associate-Developer-Apache-Spark-3.5 Exam Questions

A data engineer is working on the DataFrame:

(Referring to the table image: it has columns Id, Name, count, and timestamp.) Which code fragment should the engineer use to extract the unique values in the Name column into an alphabetically ordered list?

• A.df.select("Name").distinct()
• B.df.select("Name").orderBy(df["Name"].asc())
• C.df.select("Name").distinct().orderBy(df["Name"])
• D.df.select("Name").distinct().orderBy(df["Name"].desc())

30 of 55.
A data engineer is working on a num_df DataFrame and has a Python UDF defined as:
def cube_func(val):
return val * val * val
Which code fragment registers and uses this UDF as a Spark SQL function to work with the DataFrame num_df?

• A.num_df.register("cube_func").select("num").show()
• B.num_df.select(cube_func("num")).show()
• C.spark.createDataFrame(cube_func("num")).show()
• D.spark.udf.register("cube_func", cube_func)
  num_df.selectExpr("cube_func(num)").show()

Which configuration can be enabled to optimize the conversion between Pandas and PySpark DataFrames using Apache Arrow?

• A.spark.conf.set("spark.sql.execution.arrow.enabled", "true")
• B.spark.conf.set("spark.sql.arrow.pandas.enabled", "true")
• C.spark.conf.set("spark.sql.execution.arrow.pyspark.enabled", "true")
• D.spark.conf.set("spark.pandas.arrow.enabled", "true")

A data scientist is working on a large dataset in Apache Spark using PySpark. The data scientist has a DataFrame df with columns user_id, product_id, and purchase_amount and needs to perform some operations on this data efficiently.
Which sequence of operations results in transformations that require a shuffle followed by transformations that do not?

• A.df.withColumn("purchase_date", current_date()).where("total_purchase > 50")
• B.df.filter(df.purchase_amount > 100).groupBy("user_id").sum("purchase_amount")
• C.df.withColumn("discount", df.purchase_amount * 0.1).select("discount")
• D.df.groupBy("user_id").agg(sum("purchase_amount").alias("total_purchase")).repartition(10)

A data scientist is working on a project that requires processing large amounts of structured data, performing SQL queries, and applying machine learning algorithms. The data scientist is considering using Apache Spark for this task.
Which combination of Apache Spark modules should the data scientist use in this scenario?
Options:

• A.Spark DataFrames, Structured Streaming, and GraphX
• B.Spark SQL, Pandas API on Spark, and Structured Streaming
• C.Spark Streaming, GraphX, and Pandas API on Spark
• D.Spark DataFrames, Spark SQL, and MLlib