The resource that you need to get access is in the other project.
roles/bigquery.dataViewer BigQuery Data Viewer
When applied to a table or view, this role provides permissions to:
Read data and metadata from the table or view.
This role cannot be applied to individual models or routines.
When applied to a dataset, this role provides permissions to:
Read the dataset's metadata and list tables in the dataset.
Read data and metadata from the dataset's tables.
When applied at the project or organization level, this role can also enumerate all datasets in the project.
Additional roles, however, are necessary to allow the running of jobs.

gcloud deployment-manager deployments create creates deployments based on the configuration file.
(Infrastructure as code). All the configuration related to the artifacts is in the configuration file. This command correctly creates a cluster based on the provided cluster.yaml configuration file.
Ref: https://cloud.google.com/sdk/gcloud/reference/deployment-manager/deployments/create

Create an instance template for the instances so VMs have same specs. Set the "":Automatic Restart' to on to VM automatically restarts upon crash. Set the ""On-host maintenance' to Migrate VM instance. This will take care of VM during maintenance window. It will migrate VM instance making it highly available Add the instance template to an instance group so instances can be managed.
* onHostMaintenance: Determines the behavior when a maintenance event occurs that might cause your instance to reboot.
* [Default] MIGRATE, which causes Compute Engine to live migrate an instance when there is a maintenance event.
* TERMINATE, which stops an instance instead of migrating it.
* automaticRestart: Determines the behavior when an instance crashes or is stopped by the system.
* [Default] true, so Compute Engine restarts an instance if the instance crashes or is stopped.
* false, so Compute Engine does not restart an instance if the instance crashes or is stopped.
Enabling automatic restart ensures that compute engine instances are automatically restarted when they crash.
And Enabling Migrate VM Instance enables live migrates i.e. compute instances are migrated during system maintenance and remain running during the migration.
Automatic Restart If your instance is set to terminate when there is a maintenance event, or if your instance crashes because of an underlying hardware issue, you can set up Compute Engine to automatically restart the instance by setting the automaticRestart field to true. This setting does not apply if the instance is taken offline through a user action, such as calling sudo shutdown, or during a zone outage.Ref: https://cloud.google.com/compute/docs/instances/setting-instance-scheduling-options#autorestart Enabling the Migrate VM Instance option migrates your instance away from an infrastructure maintenance event, and your instance remains running during the migration. Your instance might experience a short period of decreased performance, although generally, most instances should not notice any difference. This is ideal for instances that require constant uptime and can tolerate a short period of decreased performance.Ref: https://cloud.google.com/compute/docs/instances/setting-instance-scheduling-options#live_mi

auto-provisioning = Attaches and deletes node pools to cluster based on the requirements. Hence creating a GPU node pool, and auto-scaling would be better
https://cloud.google.com/kubernetes-engine/docs/how-to/node-auto-provisioning