Not able to download dataset

[Aman]

Hi,

I want to download dataset of U-Net exercise(Course4-Week3) to work locally. But I am not able to find a way to download it, I also tried to download the sub folders CameraMask and CameraRGB , still no option to download , there are 1000s of images so selecting each and downloading will not be feasible .

Any one knows how to solve this issue, please help!

[paulinpaloalto]

You’ll need to create a compound file with all the images. The Docker images include the linux tar command, so that’s the easiest way. You may end up with a tar file that’s too large to download in one shot, so you might also need the linux split command. The easiest way is either to add a new cell in the notebook and use “bang escapes” to execute shell commands as in:

!ls -ltr

Or you can click “File → Open” and then “New → Terminal” and that gives you a linux shell.

[Aman]

Hi,
I used following code in the notebook to create zip of whole assignment and then split it into sizes of 50 mb each.

!tar chvfz notebook.tar.gz *
!split -b 50m notebook.tar.gz allfiles.tar.gz.part.

Then I used 7-Zip to first combine all zip files and then extracted it.

Thanks,

[paulinpaloalto]

Great! Yes, those commands look like they should work. The only possible refinement would be to add a “–exclude” to make sure you don’t end up tarring up part of the tar file itself:

!tar czvhf notebook.tar.gz --exclude notebook.tar.gz *

As you probably already figured out, the reason the mask images show up as black is that they included the alpha channel for some reason. You can find the logic in the notebook to get the masks to render correctly.

[Marco_Nguyen]

After runnung your code, does it look like this in the folder?

[Aman]

Yes , select the one ending with aa and then first combine all zips using 7-zip and then extract it…

note:- just keep same file name throughout I have by-mistake used ‘notebook’ and ‘allfiles’ both in my above code :-

!tar chvfz allfiles.tar.gz *
!split -b 50m allfiles.tar.gz allfiles.tar.gz.part.

[AffDk]

Hi,

I follow the instructions above and downloaded all the files. it generated 6 files that I downloaded them one by one and put them in one folder, then I tried to use 7-zip to combine them but when I choose the first file (ending with aa) and run “combine file” from the File menu, I get an error message that “cannot detect file as split file”.

any suggestion?

[paulinpaloalto]

The point is that split has divided the tar file into a lot of pieces. You first need to reassemble them into a single file. I’m a linux/Mac person, so I would use this command:

cat file.parts.* > wholefile.tgz

With the appropriate name of the files that are output from split. I think the Windows equivalent of that would be:

type file.parts.* > wholefile.tgz

Then apply 7-zip to the result of that.

[AffDk]

Thanks for the feedback.
I finally managed to use 7-zip on windows to merge the files. I selected all files in windows explorer and then from 7-zip menu appearing with right click, chose Extract to “*\”.
Anyhow nice to know about the commands in windows doing that.

Best,
Aff