The goal is to create a serverless, scalable, and asynchronous transaction processing system with minimal management overhead.
Serverless Requirement:Options involving installing Kafka on VMs (A, B) or using VM instances for processing (B, C) introduce management overhead associated with VMs (patching, scaling configuration, OS management) and Kafka cluster management, violating the serverless and minimal management criteria.
Asynchronous Requirement:Both Kafka and Pub/Sub can handle asynchronous messaging. However, Pub
/Sub is Google Cloud's fully managed, serverless messaging service, inherently minimizing management overhead compared to self-managed Kafka on VMs.
Scalability and Processing:Cloud Run is a fully managed, serverless platform that automatically scales based on traffic, suitable for processing transactions without managing underlying infrastructure. VM instances require manual scaling configuration or managed instance groups, adding overhead.
Combining Pub/Sub for asynchronous message ingestion (fully managed, serverless) and Cloud Run for processing (fully managed, serverless, scalable) directly meets all requirements: serverless, scalable, asynchronous, and minimal management overhead. Option D is the only one that uses fully serverless components for both ingestion and processing.
References:
Google Cloud Pub/Sub Overview: "Pub/Sub is an asynchronous and scalable messaging service..." -
https://cloud.google.com/pubsub/docs/overview
Google Cloud Run Overview: "Cloud Run is a managed compute platform that lets you run containers directly on top of Google's scalable infrastructure." -https://cloud.google.com/run/docs/overview/what-is- cloud-run Serverless Patterns (Pub/Sub + Cloud Run): This combination is a standard pattern for event-driven, serverless applications. -https://cloud.google.com/run/docs/triggering/pubsub-push

https://cloud.google.com/appengine/docs/standard/python/splitting-traffic#gcloud

We need to apply the GCP Best practices. roles/browser Browser Read access to browse the hierarchy for a project, including the folder, organization, and IAM policy. This role doesn't include permission to view resources in the project.https://cloud.google.com/iam/docs/understanding-roles

The best option is to attach a single service account to the compute instances and add minimal rights to the service account. Then, allow the service account to impersonate a Cloud Identity user with elevated permissions to create, update, or delete resources. This way, the service account can use short-lived access tokens to authenticate to Google Cloud APIs without needing to manage service account keys. This option follows the principle of least privilege and reduces the risk of credential leakage and misuse.
Option A is not recommended because it requires human intervention, which can slow down the CI/CD pipeline and introduce human errors. Option C is not secure because it grants all required IAM permissions to a single service account, which can increase the impact of a compromised key. Option D is not cost-effective because it requires creating and managing multiple service accounts and keys, as well as using a secret manager service.
1: https://cloud.google.com/iam/docs/impersonating-service-accounts
2: https://cloud.google.com/iam/docs/best-practices-for-managing-service-account-keys
3: https://cloud.google.com/iam/docs/understanding-service-accounts

Installing Jenkins
In this section, you use Cloud Marketplace to provision a Jenkins instance. You customize this instance to use the agent image you created in the previous section.
Go to the Cloud Marketplace solution for Jenkins.
Click Launch on Compute Engine.
Change the Machine Type field to 4 vCPUs 15 GB Memory, n1-standard-4.
Machine type selection for Jenkins deployment.
Click Deploy and wait for your Jenkins instance to finish being provisioned. When it is finished, you will see:
Jenkins has been deployed.
https://cloud.google.com/solutions/using-jenkins-for-distributed-builds-on-compute-engine#installing_jenkins Reference: https://cloud.google.com/solutions/jenkins-on-kubernetes-engine