A research group wants to test different generative AI models to create research papers. The research group has defined a prompt and needs a method to assess the models' output. The research group wants to use a team of scientists to perform the output assessments. Which solution will meet these requirements?
Correct Answer: C
The correct answer is C because Amazon Bedrock's model evaluation feature allows users to compare outputs from different foundation models using human evaluation or automatic metrics. It enables the creation of structured evaluations where human reviewers (in this case, scientists) can assess model responses based on custom criteria like relevance, coherence, or accuracy. From AWS documentation: "Amazon Bedrock provides model evaluation capabilities that support both automatic and human evaluation. You can define custom evaluation prompts and collect assessments from reviewers to compare foundation model outputs for tasks such as summarization, text generation, and more." This solution is ideal for research workflows requiring domain experts to provide feedback on LLM-generated content. Explanation of other options: A). Amazon Personalize is used for building recommendation systems, not for evaluating model output. B). Amazon Rekognition is used for analyzing images and videos (e.g., moderation, facial recognition), not textual output. D). Amazon Comprehend provides NLP services like sentiment analysis, but sentiment is not sufficient for full quality evaluation of research paper generation. Referenced AWS AI/ML Documents and Study Guides: Amazon Bedrock Developer Guide - Model Evaluation Overview AWS Generative AI Best Practices AWS ML Specialty Study Guide - Evaluation and Feedback Loops in LLMs
AIF-C01 Exam Question 137
A media company wants to analyze viewer behavior and demographics to recommend personalized content. The company wants to deploy a customized ML model in its production environment. The company also wants to observe if the model quality drifts over time. Which AWS service or feature meets these requirements?
Correct Answer: D
The requirement is to deploy a customized machine learning (ML) model and monitor its quality for potential drift over time in a production environment. Let's evaluate each option: A). Amazon Rekognition: This service is designed for image and video analysis, such as object detection, facial recognition, and text extraction. It is not suited for deploying custom ML models or monitoring model quality drift. B). Amazon SageMaker Clarify: This feature helps detect bias in ML models and explains model predictions. While it addresses fairness and interpretability, it does not specifically focus on monitoring model quality drift over time in production. C). Amazon Comprehend: This is a natural language processing (NLP) service for extracting insights from text, such as sentiment analysis or entity recognition. It does not support deploying custom ML models or monitoring model performance drift. D). Amazon SageMaker Model Monitor: This feature is part of Amazon SageMaker and is specifically designed to monitor ML models in production. It tracks metrics such as data drift, model drift, and performance degradation over time, alerting users when issues are detected. Exact Extract Reference: According to the AWS documentation on Amazon SageMaker, "Amazon SageMaker Model Monitor allows you to detect and remediate data and model quality issues in production. It continuously monitors the performance of deployed models, capturing data and model predictions to detect deviations from expected behavior, such as data drift or model performance degradation." (Source: AWS SageMaker Documentation - Model Monitoring, https://docs.aws.amazon.com/sagemaker/latest/dg/model- monitor.html). This directly aligns with the requirement to observe model quality drift, making Amazon SageMaker Model Monitor the correct choice. References: AWS SageMaker Documentation: Model Monitoring (https://docs.aws.amazon.com/sagemaker/latest/dg /model-monitor.html) AWS AI Practitioner Study Guide (conceptual alignment with monitoring deployed ML models)
AIF-C01 Exam Question 138
A hospital wants to use a generative AI solution with speech-to-text functionality to help improve employee skills in dictating clinical notes.
Correct Answer: D
AWS HealthScribe provides speech-to-text and medical documentation generation, specifically designed for healthcare applications. Amazon Polly is text-to-speech, not speech-to-text. Amazon Rekognition is computer vision. Amazon Q Developer is a generative AI assistant for developers, not healthcare. # Reference: AWS Documentation - AWS HealthScribe
AIF-C01 Exam Question 139
An AI practitioner has trained a model on a training dataset. The model performs well on the training data. However, the model does not perform well on evaluation data. What is the MOST likely cause of this issue?
Correct Answer: D
When a model performs well on training data but poorly on evaluation/test data, it indicates overfitting. Overfitting: The model memorizes the training data patterns instead of generalizing. Underfitting (A) means the model performs poorly on both training and test data. Bias (C) refers to systemic errors in predictions, not this training/test mismatch. Prompt engineering (B) applies to generative AI, not general ML training models. # Reference: AWS ML Glossary - Overfitting and Underfitting
AIF-C01 Exam Question 140
A company has developed an ML model to predict real estate sale prices. The company wants to deploy the model to make predictions without managing servers or infrastructure. Which solution meets these requirements?
Correct Answer: D
Amazon SageMaker endpoints provide fully managed, serverless model deployment for real-time and batch predictions, allowing companies to deploy ML models without handling any servers or infrastructure management. D is correct: SageMaker endpoints let you deploy, scale, and monitor ML models with no infrastructure overhead. A and B require infrastructure management. C (CloudFront/S3) is not for model deployment, but for static content delivery. "Amazon SageMaker endpoints allow you to deploy machine learning models for inference without the need to manage underlying infrastructure." (Reference: AWS SageMaker Model Deployment, AWS Certified AI Practitioner Study Guide)