# Parameters for Fine-Tuning locally

**URL:** <https://community.deeplearning.ai/t/parameters-for-fine-tuning-locally/380816>\
**Category:** Generative AI with Large Language Models\
**Tags:** week-module-2\
**Created:** [July 10, 2023, 3:30pm UTC](https://community.deeplearning.ai/t/parameters-for-fine-tuning-locally/380816 "2023-07-10T15:30:15Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![Marc1n](https://avatars.discourse-cdn.com/v4/letter/m/7feea3/32.png) [@Marc1n](https://community.deeplearning.ai/u/Marc1n)\
**Post date:** [July 10, 2023, 3:30pm UTC](https://community.deeplearning.ai/t/parameters-for-fine-tuning-locally/380816/1 "2023-07-10T15:30:15Z")

</div>

I would like to see how Fine-Tuning would work on a local machine rather than using a checkpoint.

For that to work, what parameters should I use for TrainingArguments()?

Here are defaults from the lab for PEFT:

peft\_training\_args = TrainingArguments(  
output\_dir=output\_dir,  
auto\_find\_batch\_size=True,  
learning\_rate=1e-3, # Higher learning rate than full fine-tuning.  
num\_train\_epochs=1,  
logging\_steps=1,  
max\_steps=1  
)

---

<div class="post-metadata">

**Author:** ![Aiko](https://sea1.discourse-cdn.com/dlai/user_avatar/community.deeplearning.ai/aiko/32/54069_2.png) [@Aiko](https://community.deeplearning.ai/u/Aiko)\
**Post date:** [July 12, 2023, 10:24am UTC](https://community.deeplearning.ai/t/parameters-for-fine-tuning-locally/380816/2 "2023-07-12T10:24:21Z")

</div>

At the very least, I would remove the logging\_steps and max\_steps here since that limits your step total to 1 and logs every single one. I imagine this was put in for demonstration purposes and I think the defaults are quite reasonable.

For epochs, prowling the internet tells me that there’s a bit of a range of what people actually use. There are some that mention that quite often 4 is enough, while others will do 3, check the performance, and do 3 more until they reach their desired performance or hit diminishing returns or big performance loss.

I’m not so sure about the learning rate though. Is that your comment? If it’s not, then I’d try leaving it like that.