MLEP C4 Assignment Troubleshooting Tips

For all Qwiklabs

1 - You get the error message “You do not have permission to view this project” or you can’t find files that you have already created earlier

If you’re sure that you’ve followed all instructions but get this type of issue, please click on the upper right icon to check if you’re using the student credentials given to you by Qwiklabs. We found that GCP sometimes switches your account to your personal gmail account especially if it opens a new popup.

Screen Shot 2021-09-27 at 8.46.35 PM

One way to avoid this is to do the Qwiklabs in an Incognito or private browsing session to make sure that GCP does not see previously logged-in Gmail accounts.

When you switch accounts, also check if the project selected is also pointed to the Project ID given to you by Qwiklabs. It has the pattern qwiklabs-gcp-<identifier> as shown in the screenshot below. In some instances, we found that this can be unselected when you switch accounts.



2 - You get the error message “Sorry, your quota has been exceeded for this lab.” or “Any credits or tokens you used to start this lab have been refunded.”

Qwiklabs limits the amount of retries to the lab. Please use the Qwiklabs chat support and ask the agent to extend your quota. It is on the upper right of the Qwiklabs page. Before doing so, please make sure to ask for tips here in Discourse in case you were stuck in one of the lab instructions. That will help you resolve the issue once you get access again to the lab and you will avoid exceeding your new quota.

Week 2: Assignment: Autoscaling TensorFlow Model Deployments with TF Serving and Anthos Service Mesh

1 - When trying to create a cluster at the start of the lab, you get this error:

Solution - Please use these two commands instead:

Create the cluster with this:

gcloud container clusters create $CLUSTER_NAME \
    --cluster-version=latest \
    --machine-type=n1-standard-4 \
    --num-nodes=3

And then enable autoscaling:

gcloud container clusters update $CLUSTER_NAME \
	--enable-autoscaling \
	--min-nodes=1 \
	--max-nodes=3


----

Week 3: Assignment: TFX on Google Cloud Vortex Pipelines


1 - Some learners have reported a bug when trying to create the run from the notebook. Here is a sample error:

kfp_server_api.exceptions.ApiException: (400)
Reason: Bad Request
HTTP response headers: HTTPHeaderDict({‘Content-Type’: ‘application/json’, ‘Date’: ‘Mon, 13 Sep 2021 02:34:40 GMT’, ‘Vary’: ‘Origin’, ‘X-Content-Type-Options’: ‘nosniff’, ‘X-Frame-Options’: ‘SAMEORIGIN’, ‘X-Powered-By’: ‘Express’, ‘X-Xss-Protection’: ‘0’, ‘Transfer-Encoding’: ‘chunked’, ‘Set-Cookie’: ‘S=cloud_datalab_tunnel=SeeYrml80va1g14xinpasYetMEPbjmaVL6FiGNBaVZE; Path=/; Max-Age=3600’})
HTTP response body: {“error”:“Failed to create a new run.: Failed to fetch workflow spec.: Invalid input error: Please provide a valid pipeline spec”,“message”:“Failed to create a new run.: Failed to fetch workflow spec.: Invalid input error: Please provide a valid pipeline spec”,“code”:3,“details”:[{"@type":“type.googleapis.com/api.Error",“error_message”:"Please provide a valid pipeline spec”,“error_details”:“Failed to create a new run.: Failed to fetch workflow spec.: Invalid input error: Please provide a valid pipeline spec”}]}

Solution - This seems to be related to the automatic creation of the Kubernetes cluster. This can be resolved by doing the steps in this README. Please see the Enabling Cloud Services section before doing the notebook.



2 - There is no cluster-1 automatically created when you configure Kubeflow Pipelines.

Solution - You can create a cluster on your own instead. Pick us-central1-a as the cluster zone and tick the Allow access to the following Coud APIs... box.


1 Like