Week 3 Graded External tool: Notebook stuck: No executable skaffold

About graded assignment on week 3: TFX on Google Cloud AI Platform Pipelines

I stuck in the cell:

!tfx pipeline create  \
--pipeline_path=runner.py \
--endpoint={ENDPOINT} \
--build_target_image={CUSTOM_TFX_IMAGE}

With the following error:

2021-09-11 10:32:30.876246: W tensorflow/stream_executor/platform/default/dso_loader.cc:59] Could not load dynamic library 'libcudart.so.10.1'; dlerror: libcudart.so.10.1: cannot open shared object file: No such file or directory
2021-09-11 10:32:30.876392: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
CLI
Creating pipeline
Detected Kubeflow.
Use --engine flag if you intend to use a different orchestrator.
Reading build spec from build.yaml
Target image gcr.io/qwiklabs-gcp-03-5a14fbd0fd85/tfx_covertype_continuous_training is not used. If the build spec is provided, update the target image in the build spec file build.yaml.
No executable skaffold
please refer to https://github.com/GoogleContainerTools/skaffold/releases for installation instructions.
No container image is built.
Traceback (most recent call last):
  File "/home/jupyter/.local/lib/python3.7/site-packages/tfx/tools/cli/container_builder/skaffold_cli.py", line 40, in __init__
    stdout=subprocess.DEVNULL)
  File "/opt/conda/lib/python3.7/subprocess.py", line 512, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['which', 'skaffold']' returned non-zero exit status 1.

Had the same problem too, looks like you’ll need to install skaffold using the install.sh script as mentioned in the instructions (do this in the jupyter lab terminal):

cd training-data-analyst/self-paced-labs/tfx/tfx-ai-platform
./install.sh

Then restart the kernel and re-run the cells. Should work ok after.

2 Likes

I had to install skaffold manually using

!curl -Lo skaffold https://storage.googleapis.com/skaffold/releases/v1.31.0/skaffold-linux-amd64 && chmod +x skaffold && sudo mv skaffold /usr/local/bin
7 Likes

I had the same issue because my TF version hadn’t agreed with 2.3.2 as in the comments, so I ran the optional step to install that version and restarted the kernel as suggested, which meant that that earlier cell’s update of the environment variable to add skaffold to the path had been lost. I reran that “Set PATH to include…” cell and recompiled. That worked.

2 Likes

I also had the same issue. Fortunately I read forum occasionally.:grinning: