Hey there
I have pretrained a Qwen model on my custom data. I have trained it for over 10000 steps with a batch size of 4.
When I try to inference it on kaggle I am able to get the desired results at least not garbage values, but when I try to inference the same in colab , I find the results are completely garbage ( all the settings remained same including the versions as well).
Here is the code and the corresponding screenshots
from transformers import AutoTokenizer, AutoModel, AutoModelForCausalLM, AutoConfig
from transformers import AutoTokenizer
import torch
import torch.nn as nn
model_name = 'Sujithanumala/OIC_QWEN_MODEL'
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name,torch_dtype = torch.bfloat16)
model.to("cuda")
print(tokenizer.decode(model.generate(tokenizer("What are different kind of adapters that we can see in OIC",return_tensors='pt')['input_ids'].to("cuda"),max_new_tokens = 100)[0]))
print(tokenizer.decode(model.generate(tokenizer("What is Machine Learning?",return_tensors='pt')['input_ids'].to("cuda"),max_new_tokens = 100)[0]))
Kaggle output
colab output
