W5 Assignment - TensorFlow in Vertex AI

Hello Learners,

I am going through the assignment in Qwiklabs/Google cloud,

On Task 5. Deploy flights model to Vertex AI, I get an error which is not getting self-resolved.
Here is the instruciton:

The error I keep getting, is this one:

gs://qwiklabs-gcp-02-8517692a8908-dsongcp/ch9/trained_model/export/flights_20231031-111537/
Using endpoint [https://us-central1-aiplatform.googleapis.com/]
Endpoint for flights-20231031-112824 already exists
Using endpoint [https://us-central1-aiplatform.googleapis.com/]
ENDPOINT_ID=5332211381271789568
Using endpoint [https://us-central1-aiplatform.googleapis.com/]
Using endpoint [https://us-central1-aiplatform.googleapis.com/]
Waiting for operation [7706048670583488512]...
..................................................................................................................................done.
Using endpoint [https://us-central1-aiplatform.googleapis.com/]
MODEL_ID=1208572186135101440
Using endpoint [https://us-central1-aiplatform.googleapis.com/]
ERROR: (gcloud.beta.ai.endpoints.deploy-model) INVALID_ARGUMENT: Invalid image "us-docker.pkg.dev/vertex-ai/prediction/tf2-cpu.2-14:latest" for deployment. Please use a Model with a valid image.
---------------------------------------------------------------------------
CalledProcessError                        Traceback (most recent call last)
Cell In[25], line 1
----> 1 get_ipython().run_cell_magic('bash', '', '# note TF_VERSION and ENDPOINT_NAME set in 1st cell\n# TF_VERSION=2-6\n# ENDPOINT_NAME=flights\n\nTIMESTAMP=$(date +%Y%m%d-%H%M%S)\nMODEL_NAME=${ENDPOINT_NAME}-${TIMESTAMP}\nEXPORT_PATH=$(gsutil ls ${OUTDIR}/export | tail -1)\necho $EXPORT_PATH\n\n# create the model endpoint for deploying the model\nif [[ $(gcloud beta ai endpoints list --region=$REGION \\\n        --format=\'value(DISPLAY_NAME)\' --filter=display_name=${ENDPOINT_NAME}) ]]; then\n    echo "Endpoint for $MODEL_NAME already exists"\nelse\n    echo "Creating Endpoint for $MODEL_NAME"\n    gcloud beta ai endpoints create --region=${REGION} --display-name=${ENDPOINT_NAME}\nfi\n\nENDPOINT_ID=$(gcloud beta ai endpoints list --region=$REGION \\\n              --format=\'value(ENDPOINT_ID)\' --filter=display_name=${ENDPOINT_NAME})\necho "ENDPOINT_ID=$ENDPOINT_ID"\n\n\n# delete any existing models with this name\nfor MODEL_ID in $(gcloud beta ai models list --region=$REGION --format=\'value(MODEL_ID)\' --filter=display_name=${MODEL_NAME}); do\n    echo "Deleting existing $MODEL_NAME ... $MODEL_ID "\n    gcloud ai models delete --region=$REGION $MODEL_ID\ndone\n\n\n# create the model using the parameters docker conatiner image and artifact uri\ngcloud beta ai models upload --region=$REGION --display-name=$MODEL_NAME \\\n     --container-image-uri=us-docker.pkg.dev/vertex-ai/prediction/tf2-cpu.${TF_VERSION}:latest \\\n     --artifact-uri=$EXPORT_PATH\nMODEL_ID=$(gcloud beta ai models list --region=$REGION --format=\'value(MODEL_ID)\' --filter=display_name=${MODEL_NAME})\necho "MODEL_ID=$MODEL_ID"\n\n\n# deploy the model to the endpoint\ngcloud beta ai endpoints deploy-model $ENDPOINT_ID \\\n  --region=$REGION \\\n  --model=$MODEL_ID \\\n  --display-name=$MODEL_NAME \\\n  --machine-type=e2-standard-2 \\\n  --min-replica-count=1 \\\n  --max-replica-count=1 \\\n  --traffic-split=0=100\n')

File /opt/conda/lib/python3.10/site-packages/IPython/core/interactiveshell.py:2478, in InteractiveShell.run_cell_magic(self, magic_name, line, cell)
   2476 with self.builtin_trap:
   2477     args = (magic_arg_s, cell)
-> 2478     result = fn(*args, **kwargs)
   2480 # The code below prevents the output from being displayed
   2481 # when using magics with decodator @output_can_be_silenced
   2482 # when the last Python token in the expression is a ';'.
   2483 if getattr(fn, magic.MAGIC_OUTPUT_CAN_BE_SILENCED, False):

File /opt/conda/lib/python3.10/site-packages/IPython/core/magics/script.py:154, in ScriptMagics._make_script_magic.<locals>.named_script_magic(line, cell)
    152 else:
    153     line = script
--> 154 return self.shebang(line, cell)

File /opt/conda/lib/python3.10/site-packages/IPython/core/magics/script.py:314, in ScriptMagics.shebang(self, line, cell)
    309 if args.raise_error and p.returncode != 0:
    310     # If we get here and p.returncode is still None, we must have
    311     # killed it but not yet seen its return code. We don't wait for it,
    312     # in case it's stuck in uninterruptible sleep. -9 = SIGKILL
    313     rc = p.returncode or -9
--> 314     raise CalledProcessError(rc, cell)

CalledProcessError: Command 'b'# note TF_VERSION and ENDPOINT_NAME set in 1st cell\n# TF_VERSION=2-6\n# ENDPOINT_NAME=flights\n\nTIMESTAMP=$(date +%Y%m%d-%H%M%S)\nMODEL_NAME=${ENDPOINT_NAME}-${TIMESTAMP}\nEXPORT_PATH=$(gsutil ls ${OUTDIR}/export | tail -1)\necho $EXPORT_PATH\n\n# create the model endpoint for deploying the model\nif [[ $(gcloud beta ai endpoints list --region=$REGION \\\n        --format=\'value(DISPLAY_NAME)\' --filter=display_name=${ENDPOINT_NAME}) ]]; then\n    echo "Endpoint for $MODEL_NAME already exists"\nelse\n    echo "Creating Endpoint for $MODEL_NAME"\n    gcloud beta ai endpoints create --region=${REGION} --display-name=${ENDPOINT_NAME}\nfi\n\nENDPOINT_ID=$(gcloud beta ai endpoints list --region=$REGION \\\n              --format=\'value(ENDPOINT_ID)\' --filter=display_name=${ENDPOINT_NAME})\necho "ENDPOINT_ID=$ENDPOINT_ID"\n\n\n# delete any existing models with this name\nfor MODEL_ID in $(gcloud beta ai models list --region=$REGION --format=\'value(MODEL_ID)\' --filter=display_name=${MODEL_NAME}); do\n    echo "Deleting existing $MODEL_NAME ... $MODEL_ID "\n    gcloud ai models delete --region=$REGION $MODEL_ID\ndone\n\n\n# create the model using the parameters docker conatiner image and artifact uri\ngcloud beta ai models upload --region=$REGION --display-name=$MODEL_NAME \\\n     --container-image-uri=us-docker.pkg.dev/vertex-ai/prediction/tf2-cpu.${TF_VERSION}:latest \\\n     --artifact-uri=$EXPORT_PATH\nMODEL_ID=$(gcloud beta ai models list --region=$REGION --format=\'value(MODEL_ID)\' --filter=display_name=${MODEL_NAME})\necho "MODEL_ID=$MODEL_ID"\n\n\n# deploy the model to the endpoint\ngcloud beta ai endpoints deploy-model $ENDPOINT_ID \\\n  --region=$REGION \\\n  --model=$MODEL_ID \\\n  --display-name=$MODEL_NAME \\\n  --machine-type=e2-standard-2 \\\n  --min-replica-count=1 \\\n  --max-replica-count=1 \\\n  --traffic-split=0=100\n'' returned non-zero exit status 1.

Any workarounds you can suggest?
Thanks in advance

ERROR: (gcloud.beta.ai.endpoints.deploy-model) INVALID_ARGUMENT: Invalid image “us-docker.pkg.dev/vertex-ai/prediction/tf2-cpu.2-14:latest” for deployment. Please use a Model with a valid image.

See here and notice that there’s no image for tensorflow 2.14.
I just tested the lab with

  1. Environment = tensorflow enterprise 2.11
  2. Region = us-central1
  3. Zone = us-central-1-a

and everything works as expected.