SFTTrainer is using wandb.ai

Short Course Q&A Post-training of LLMs
1

In L3: Supervised Fine-Tuning (SFT) notebook, I was able to run it in Google’s Colab. Note that I did use a CPU runtime only. But when I reached this part of the code, it asked for a wandb.ai account:

sft_trainer = SFTTrainer(
    model=model,
    args=sft_config,
    train_dataset=train_dataset, 
    processing_class=tokenizer,
)
sft_trainer.train()

Is it possible to do a local training only? Note that I’m using trl==0.19.1 as well.

1 Like
2

Dear @adimaunahan,

Yes, it is possible to do local training only using trl==0.19.1 and the SFTTrainer without requiring a wandb.ai account.

By default, transformers and trl can integrate with logging tools like Weights & Biases (wandb), but this is optional, and you can disable it.

If you need more information, please let me know.

3

Thanks for a quick reply! Can you provide some codes on how to do local training? Would it require other libraries or setting up of local services? I would appreciate if you can provide a working Colab notebook too. TIA!

1 Like
4

I think I found the answer: report_to="none". Actually, I already tried this parameter before I posted my question and got an error. But for some weird reason, it is now working after running the whole notebook again. All good now! But thanks again @Girijesh for confirming that my expected behavior is possible :+1:

1 Like
5

Exactly.
you can also add
save_strategy=“no”, # Optional: avoid saving checkpoints if not needed.

Let me if you need me.

Thank you.

1 Like