C4W1 Lab 2 - Problems with Docker

Hello!

I am having problems with Docker. They first occurred in the first lab when I installed Docker. Docker Desktop seems to start but the Engine would not. This would manifest in “Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?” message when trying to run the waving whale. To fix this, I edited the settings json by removing everything from filesharing argument (ref: Docker is not starting · Issue #6980 · docker/for-mac · GitHub). This got the engine running and the happy whale appeared on my screen.

Now, in Lab2, I cannot proceed with mounting the temp datafile because - of course - Docker Desktop cannot see any files due to emptying the filesharing list (ref: Error: invalid mount config for type "bind": bind source path does not exist · Issue #4709 · docker/docs · GitHub). Putting stuff back to filesharing in the settings json brings us just back to the beginning.

Now, the problem seems to be Docker Desktop specific since my colleague does not have the same problems with the same version of Docker Engine. The version is quite new so might be a newly introduced bug that does not manifest in earlier versions of Docker but I haven’t bothered to try.

Anyway, since the lab is ungraded, I can just glance through everything that’s written and still pass the course but that’s kind of lame. Also, as this may be a problem to someone else in the future as well, I thought you’d like to know.

Bonus note: Is Desktop needed in some earlier assingments? Because if not, a quick fix could be to just drive people to install the CLI version (as it works normally according to my colleague) since we mostly seem to use the command line anyway?

Cheers!

What’s your OS and CPU?

Ubuntu 22.04 and Intel Core i5-8250U if I’m not mistaken. @balaji.ambresh

@lallimyl if the issue persist you can also try to reinstall the docker desktop. Also have a look on these trouble shooting blogs: How to Debug and Fix Common Docker Issues | DigitalOcean
Troubleshooting the Docker build process | by Brian Dart | IHME Tech | Medium

@Isaak_Kamau It persists. Reinstalled approximately seven times to no avail, but exactly the same symptoms every time.

Thank you for the link. Glanced it through, but unfortunately, could not locate anything related to my issues.

[PURE SPECULATION] DD might run in a VM on Mac and Ubuntu, and hence, needs the filesharing setting to set up what the VM can see on the actual machine. Now, with the new version of the engine been published earlier this month, something in the communication between the engine and the DD broke regarding the filesharing. That could explain why my colleague’s CLI engine works fine but DD version doesn’t. [PURE SPECULATION ENDS]

Could you please share your error logs? I can’t tell precisely what is causing the issue

@Isaak_Kamau

In the first phase, the engine simply won’t start. There is no error message per se, it just doesn’t do anything. The desktop gui starts and claims everywhere that the engine is stopped and I cannot get it started. Trying to give Docker a job to do simply states that it cannot be connected to, and “is it running?” (ref the same as above: Docker is not starting · Issue #6980 · docker/for-mac · GitHub)

Now, after tinkering with the settings.json, the engine does start and I can run the printing whale program from the lab. However, when in the second lab I try and spin up the TFS with docker, I get the following: “invalid mount config for type “bind”: bind source path does not exist”. Now, the statement is simply false; the path does exist. I checked multiple times. Using volume instead of mount gives the following:

“docker: Error response from daemon: Mounts denied:
The path /data
is not shared from OS X and is not known to Docker.
You can configure shared paths from Docker → Preferences… → File Sharing.
See Change Docker Desktop settings on Mac | Docker Docs for more info.”

As such, it seems tinkering with the filesharing field in settings.json prevents DD from seeing the path. Now, this wouldn’t be problem otherwise, but adding any content to the filesharing field in settings.json prevents the engine from running again and we’re back to square one.

That is all Docker tells me and I have managed to dig from the interwebs. If you’d like to have the actual log files, I’m gonna have to ask which ones you’d like? There are quite many with various names and I’m not familiar enough with Docker to tell which ones would be most useful in this situation. Quick look has errors in many of them but I’m not quite sure what each of them is trying to do.

There is a new release of DD which they say should fix the issue. Will test later and report.

1 Like

No cigar. Docker Engine still won’t start without emptying the filesharing field in settings.json. Tried digging a bit further and one other twist seems to be that in Ubuntu, the docker.sock is put in a wrong folder. I have no idea if that has something to do with all this, but added a symlink between the default location and the actual one (as per suggested here: https://youtrack.jetbrains.com/issue/IDEA-294871/Docker-desktop-for-linux-create-docker.sock-in-other-folder-instead-of-var-run-docker.sock). Did not fix the starting problem but seems to have fixed a new oddity I noticed, which is that before creating the symlink, DD sometimes slurs about not being able to fetch extensions, even though I don’t think I had any.

This is getting kind of annoying and time-consuming (e.g. the update button in DD does not do anything, so I had to uninstall and reinstall the whole thing - again…) so I really could not be bothered to test if the symlink with some magic could get mounting stuff to work even if the filesharing field is empty. Probably not though. If we cannot come up with anything here, I guess, for now, I will just hope that we don’t have to mount anything in the upcoming labs, and worry about that when it happens.

Aaaaaand it is a problem: W2 Lab 3 uses mounting, and as before, files are out of my reach:

docker compose up
[+] Building 0.0s (0/0) docker:desktop-linux
[+] Running 6/6
:heavy_check_mark: Network c4_w2_lab_3_latency_test_compose_default Created 0.0s
:heavy_check_mark: Container c4_w2_lab_3_latency_test_compose-locust-1 Created 0.0s
:heavy_check_mark: Container c4_w2_lab_3_latency_test_compose-batch-1-1 Created 0.1s
:heavy_check_mark: Container c4_w2_lab_3_latency_test_compose-no-batch-1 Created 0.1s
:heavy_check_mark: Container c4_w2_lab_3_latency_test_compose-batch-64-1 Created 0.1s
:heavy_check_mark: Container c4_w2_lab_3_latency_test_compose-batch-32-1 Created 0.1s
Attaching to c4_w2_lab_3_latency_test_compose-batch-1-1, c4_w2_lab_3_latency_test_compose-batch-32-1, c4_w2_lab_3_latency_test_compose-batch-64-1, c4_w2_lab_3_latency_test_compose-locust-1, c4_w2_lab_3_latency_test_compose-no-batch-1
Error response from daemon: Mounts denied:
The path /home/lallimyl/mlops_course/machine-learning-engineering-for-production-public/course4/week2-ungraded-labs/C4_W2_Lab_3_Latency_Test_Compose is not shared from the host and is not known to Docker.
You can configure shared paths from Docker → Preferences… → Resources → File Sharing.
See https://docs.docker.com/ for more info.

Luckily, it is ungraded, but still quite unfortunate.

Hello @lallimyl
Have you tried this:

Then add the directory:

/home/lallimyl/mlops_course/machine-learning-engineering-for-production-public/course4/week2-ungraded-labs/C4_W2_Lab_3_Latency_Test_Compose

to the list of shared paths, restart docker and try to re-run docker-compose up

Hello @Isaak_Kamau !

Yes, I have, but then we hit the wall with the very original problem, that is, the Docker Engine won’t start if there is anything in the filesharing field in the configuration files.

So, it is a circle: engine won’t start if there are shared paths → removing them starts the engine and things can be run as long as there’s no mounting involved → in order to mount anything, paths need to be added to the shared list → engine won’t start if there are shared paths.