C4W3: Implementing Canary Releases of TensorFlow model deployments with Kubernetes and Istio

Course Q&A Machine Learning Engineering for Production(MLOps) Deploying Machine Learning Models in Production
22

Thank you for sharing Dom! Our partners are already aware of the issue and hopefully, it will be resolved this week. We also suggested that the lab checkpoint be more flexible with the compute zones so learners can still get the perfect score. Sorry again for the inconvenience!

23

I’m getting the same error even after changing it to us-west1-b and us-west1-a.

24

Same error using us-west1-c, us-west1-b and us-west1-a

25

I am also getting same error, tried so many zones but no luck.Can anyone please help.

26

Can we get a workaround for this please. We might have to extend the subscription or else get an option to mark this as complete to not incur extra cost.

27

Hi everyone! We just got an update from our partner and the fix is still ongoing. I’m consulting with them again regarding another workaround. Will update here as soon as I can. Sorry that this is taking a while. Rest assured that we’re doing the best we can to have this resolved.

28 
gcloud config set compute/zone ${CLUSTER_ZONE}
gcloud beta container clusters create ${CLUSTER_NAME} \
    --machine-type=n1-standard-4 \
    --num-nodes=4 \
    --workload-pool=${WORKLOAD_POOL} \
    --logging=SYSTEM,WORKLOAD \
    --monitoring=SYSTEM \
    --subnetwork=default \
    --release-channel=regular \
    --labels mesh_id=${MESH_ID}

I have changed the num-nodes to 4 while remaining the zone as us-west1-c.
It worked for me to complete the session.

2 Likes
29

This worked for me also. Was able to finish the assignment without any issues

1 Like
30

Hi! Welcome to the community and thank you for sharing the workaround. I’ve consulted with our partners and they said we could indeed do this while they’re fixing the lab. The other option is to use an older version of GKE:

gcloud config set compute/zone ${CLUSTER_ZONE}
gcloud beta container clusters create ${CLUSTER_NAME} \
    --machine-type=n1-standard-4 \
    --num-nodes=6 \
    --workload-pool=${WORKLOAD_POOL} \
    --logging=SYSTEM,WORKLOAD \
    --monitoring=SYSTEM \
    --subnetwork=default \
    --cluster-version=1.21 \
    --labels mesh_id=${MESH_ID}

The problem with this is it might stop working when GKE v1.21 eventually becomes unavailable in the Cloud Shell. So, reducing the number of nodes is the better approach.

Thanks again!

31

Hi Chris, i am having issues installing anthos service mesh. I keep getting an “error: Autopilot clusters are only supported with managed control plane.” Not sure how to proceed with debugging this.

1 Like
32

Screenshot (3)
Screenshot (3)1920×1280 398 KB

35

I’m facing the same issue as well, in addition to the issues that I’ve posted in my other post. My coursera subscription is ending in a few days and this is the only assignment that’s preventing me from completing this course, it’s so frustrating

36

The error in your other post is directly connected to this error. The Anthos service Mesh is not installed so the gateway throws the error. I have actually tried debugging this and I found that the iosctl system isn’t being created due to the failure to install.

37

Hi everyone! Thank you for reporting! We are looking into this issue and will update this thread asap. Thanks!

38

Hi everyone! I was able to get past Task 2 and install ASM in Task 3 by using an older version of GKE. Please run this instead:

gcloud config set compute/zone ${CLUSTER_ZONE}
gcloud beta container clusters create ${CLUSTER_NAME} \
    --machine-type=n1-standard-4 \
    --num-nodes=6 \
    --workload-pool=${WORKLOAD_POOL} \
    --logging=SYSTEM,WORKLOAD \
    --monitoring=SYSTEM \
    --subnetwork=default \
    --cluster-version=1.21 \
    --labels mesh_id=${MESH_ID}

Sorry for the inconvenience and I’m filing a report now to our partners. That bug is very recent and didn’t come up in my last testing with 1.24. I hope the workaround above also works for you!

temporarily marking this as the solution for visibility

1 Like
39

Thanks a lot Chris,

Unfortunately I can no longer start the lab: “Sorry, your quota has been exceeded for this lab.” after starting the labs too many times and being unable to complete it because of this error.

Any recommendation?

Best,
Max

Edit: you can get this issue removed by getting in touch with chat support, took at most 5 minutes.

3 Likes
40

Thanks for this. It worked for me and I could successfully complete the lab.

1 Like
41

Hi everyone! Unfortunately, the lab is currently unavailable and we’re coordinating with our partners so it will be back asap. Will update this thread as soon as I get more info. Thank you!

42

Due to this lab not being available at the moment, I won’t be able to finish the course in time to avoid paying for another billing cycle. Is there something that can be done about it?

43

I have the same issue here.