In ‘3.3 - Evaluate the Model Quantitatively’ when evaluating the new fine-tuned model against the reference model the evaluation metrics are deteriorated as shown in the image below: Code part: "mean_improvement = (mean_before_detoxification - mean_after_detoxification) / mean_before_detoxificatio…

Hi Marko, and welcome to the community! Thank you for the feedback. We’ll review this and update the notebook if necessary.

I get into same worse results after finetunning

I encountered the same issue and also noticed the toxicity scores would change across runs. This is because in GenerationConfig(), do_sample is set to True which causes some variation in the LLM responses. When I set do_sample to True to get consistent LLM output, I still got a slightly negative im…

I got the same issue in the section " 3.3 - Evaluate the Model Quantitatively". I checked the results in the last section 3.4 “Evaluate the Model Qualitatively” but didn’t find any meaningful improvement either.

Maybe because the text wasn’t toxic in the first place. Add to that the randomness in answers, and you can see slight variations either way. I got a slight improvement, but looking at the completions before and after, I couldn’t detect any difference to speak of. In the small sample I looked at, I …

Hi. I also notice the same issue.

Same issue here, I got Percentage improvement of toxicity score after detoxification: mean: -11.94% std: 9.89% and in the final block of code where we show answers generated by the reference model and the fine-tuned model from a dataframe, I couldn’t notice any real improvement by reading the text…

Hello Chirs, are there any updates regarding the review and potential update of the notebook? It seems that the same issue is recurring for multiple learners (e.g., Week3-Lab3-Detoxification - #2 by Anna_Kay ). Thank you!

Hi Anna. Sorry this was deprioritized a while back. We will look into it this week. I’ll update this thread by Friday. Thanks!

Lab 3, 3.3 Toxicity is worse after fine tuning according to metrics

Course Q&A Generative AI with Large Language Models

balgheh November 1, 2024, 1:34pm 13

Hi Chris,
Is this problem still there? cause I’ve run into the this report after running my notebook:

Topic		Replies	Views
Week3-Lab3-Detoxification Generative AI with Large Language Models ai-discussions	2	42	September 16, 2024
Reinforcement learning made my lab model MORE toxic Generative AI with Large Language Models lab-help	2	56	February 17, 2025
Feedback on Lab 3 Generative AI with Large Language Models week-module-3	1	421	September 15, 2023
Lab 3 - decrease in toxicity score and performance after detoxification Generative AI with Large Language Models week-module-3	4	402	March 9, 2025
Toxicity mean value increased after detoxification Generative AI with Large Language Models week-module-3	7	564	August 6, 2023

Lab 3, 3.3 Toxicity is worse after fine tuning according to metrics

Related topics