Has anyone been successful at running L5 on their own images on Google Colab?

Short Course Q&A Prompt Engineering for Vision Modelss
1

I’m revisiting after two months. It does not look like there has been any activity here since then. Is there a way to have the instructors know? This is one of the coolest exercises but it’s not that useful if we can’t recreate it.

Thanks in advance

1 Like
2

Well, I wasn’t able to run it in Azure. Fixed several typos and bugs in the code (like Fixed Attribute Access: Changed noise_scheduler.config_num_train_timesteps to noise_scheduler.config.num_train_timesteps.

  • Typo Correction: Fixed eduction="mean" to reduction="mean" in the MSE loss calculation.)

Then I got TypeError: argument of type ‘NoneType’ is not iterable.

Then tried to add added_conf_kwargs:
encoder_hidden_states = batch[“input_ids”]

        # Prepare additional conditioning arguments (e.g., text embeddings)
        added_cond_kwargs = {
            "text_embeds": encoder_hidden_states  # Example: You might need to generate or retrieve text embeddings
        }

model_predict = unet(
noisy_model_input,
timesteps,
encoder_hidden_states,
return_dict=False,
added_cond_kwargs=added_cond_kwargs # Pass the added condition arguments
)[0]

But it still didn’t work for me.

@Deepti_Prasad Probably you can help with this?

3

If run the codes in your local colab and made many changes, then surely one would need to see what changes you did with the codes.

As far the instructions shared here, it is stating the text embeddings would be required to retrieve or generate, means your inputs aren’t working with the codes provided.

csn you share a complete image of all the errors you got from beginning and you what changes you made for each error. I hope you have keep an error log documentation, so you have an idea from where you started.

Regards
DP

4

@Deepti_Prasad thanks for answering!
I’ve created a special public repository where I am fixing all the code:

Each commit is for fixing some next bug in the code. Some of these bugs are specific to environment, but some are real typos or bugs in the code.

The commented code in the markdown coulnd’t have been working because of the typos in it.

But most important is that even after fixing all of them I get the last error which you can see in the last commit to the repository:

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
Cell In[22], line 30
     22 noisy_model_input = noise_scheduler.add_noise(
     23     model_input,
     24     noise,
     25     timesteps
     26 )
     28 encoder_hidden_states = batch["input_ids"]
---> 30 model_predict = unet(
     31     noisy_model_input,
     32     timesteps,
     33     encoder_hidden_states,
     34     return_dict=False,
     35 )[0]
     37 target = noise
     39 model_pred, model_pred_prior = torch.chunk(model_pred, 2, dim=0)

# skipping some error lines 

File /anaconda/envs/azureml_py38_PT_TF/lib/python3.9/site-packages/diffusers/models/unets/unet_2d_condition.py:969, in UNet2DConditionModel.get_aug_embed(self, emb, encoder_hidden_states, added_cond_kwargs)
    966     aug_emb = self.add_embedding(text_embs, image_embs)
    967 elif self.config.addition_embed_type == "text_time":
    968     # SDXL - style
--> 969     if "text_embeds" not in added_cond_kwargs:
    970         raise ValueError(
    971             f"{self.__class__} has the config param `addition_embed_type` set to 'text_time' which requires the keyword argument `text_embeds` to be passed in `added_cond_kwargs`"
    972         )
    973     text_embeds = added_cond_kwargs.get("text_embeds")

TypeError: argument of type 'NoneType' is not iterable

@Deepti_Prasad I’d deeply appreciate if you could help with resolving it and update the code also in the course to help future learners.

I’ve tried to fix it but didn’t succeed, so I’m stuck with this.
If you need any help with the compute to fix it, write me, I’ll try to help.

1 Like
5

looks like it is asking for the text embs(text embeddings) for the added_cond_kwargs to work.

When you clicked File==>Open option did you not find any file with this name?

i can’t check the colab right now as responding through phone.

6

@Deepti_Prasad Please, look at the github I provided. Text embeddings and with them time embeddings should be built somewhere in the code. And if for text it’s clear that it should be just encoder_hidden_states, it’s unclear how to build time embeddings.
Do you have some connection to engineers who wrote this notebook? I think it’s worth consulting them.

1 Like
7

@Mubsi was the requirement.txt provided with this course? Can you please look at their query as they want to run the codes locally.

Regards
DP

8

Hi @Deepti_Prasad,

Yes, it was provided in the first lab workspace of the course.

The course uses Python 3.11.9, and these are the packages:

transformers==4.37.2
torch==2.2.0
comet-ml==3.37.2
einops==0.7.0
scikit-image==0.22.0
accelerate==0.27.2
diffusers==0.26.3

1 Like
9

Thank you @Mubsi.

Hey @vgorovoy check if your requirement txt packages version matches with the information provided.

I am also suspecting

That’s true Vladimir, but if you check the error, text embeddings are being used from Unet2dconditionModel which is from which unet_2d_condition.py file

so according to what just @Mubsi shared information check if your diffusers package version match with your local environment.

check another discrepancy in version package

probably that’s why you had those bugs when you changed some of the attributes

Hope this helps resolve your issue.

@Mohsin_Ansari you can also try to with the information provided, sorry for the delayed response.

Regards
DP

1 Like
10

Thanks for reporting we’re double checking with the Comet team! We’ll inform the updates!

1 Like