# Speeding Up Reinforcement Learning with Pre-existing Data or Hybrid Approaches

**URL:** <https://community.deeplearning.ai/t/speeding-up-reinforcement-learning-with-pre-existing-data-or-hybrid-approaches/734313>\
**Category:** AI Discussions\
**Tags:** ai-discussions\
**Created:** [December 11, 2024, 11:17pm UTC](https://community.deeplearning.ai/t/speeding-up-reinforcement-learning-with-pre-existing-data-or-hybrid-approaches/734313 "2024-12-11T23:17:24Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![Benroke](https://avatars.discourse-cdn.com/v4/letter/b/c37758/32.png) [@Benroke](https://community.deeplearning.ai/u/Benroke)\
**Post date:** [December 11, 2024, 11:17pm UTC](https://community.deeplearning.ai/t/speeding-up-reinforcement-learning-with-pre-existing-data-or-hybrid-approaches/734313/1 "2024-12-11T23:17:24Z")

</div>

I’m new to the machine learning field, so I apologize if this is a basic question.

Is it possible to feed _pre-existing data_ into a reinforcement learning model to _speed up_ its learning process? Or perhaps combine it with supervised or unsupervised learning techniques so that the reinforcement model _doesn’t start from scratch_, making random guesses during training?

For example, if you have a reinforcement learning model learning to walk, could you provide it with walking data beforehand to help it learn faster, rather than having it start from zero and figuring everything out on its own?

---

<div class="post-metadata">

**Author:** ![rmwkwok](https://sea1.discourse-cdn.com/dlai/user_avatar/community.deeplearning.ai/rmwkwok/32/126977_2.png) [@rmwkwok](https://community.deeplearning.ai/u/rmwkwok)\
**Post date:** [December 12, 2024, 2:32am UTC](https://community.deeplearning.ai/t/speeding-up-reinforcement-learning-with-pre-existing-data-or-hybrid-approaches/734313/2 "2024-12-12T02:32:25Z")

</div>

Good question, and I think Yes! The learning algorithm cannot distinguish offline data from online data. It just learns whatever you feed it!!

-Raymond

---

<div class="post-metadata">

**Author:** ![Benroke](https://avatars.discourse-cdn.com/v4/letter/b/c37758/32.png) [@Benroke](https://community.deeplearning.ai/u/Benroke)\
**Post date:** [December 12, 2024, 3:32am UTC](https://community.deeplearning.ai/t/speeding-up-reinforcement-learning-with-pre-existing-data-or-hybrid-approaches/734313/3 "2024-12-12T03:32:31Z")

</div>

Correct me if I’m wrong, but you’re saying that you can provide the model with the same type of data it would receive during training, before the training actually begins?

---

<div class="post-metadata">

**Author:** ![rmwkwok](https://sea1.discourse-cdn.com/dlai/user_avatar/community.deeplearning.ai/rmwkwok/32/126977_2.png) [@rmwkwok](https://community.deeplearning.ai/u/rmwkwok)\
**Post date:** [December 12, 2024, 7:04am UTC](https://community.deeplearning.ai/t/speeding-up-reinforcement-learning-with-pre-existing-data-or-hybrid-approaches/734313/4 "2024-12-12T07:04:35Z")

</div>

I think we can pre-train (in phase 1) a robot (i.e. model) with some (offline) walking data, and then further train that robot as it walks (in phase 2) and receives more (online) walking data.

Certainly, the robot’s behavior (at least at the beginning of phase 2) will be affected by the walking data in phase 1, and that behavior will affect what data the robot will receive in phase 2.

For example, if your pre-training data is all about walking on a flat terrain, but the actual enviornment is very bumpy, then the robot will probably falls a lot and that is the kind of data you will receive.

Perhaps you might share if there is any reason for you to be wondering if pre-training data might not be helpful?

---

<div class="post-metadata">

**Author:** ![Benroke](https://avatars.discourse-cdn.com/v4/letter/b/c37758/32.png) [@Benroke](https://community.deeplearning.ai/u/Benroke)\
**Post date:** [December 14, 2024, 4:55pm UTC](https://community.deeplearning.ai/t/speeding-up-reinforcement-learning-with-pre-existing-data-or-hybrid-approaches/734313/5 "2024-12-14T16:55:49Z")

</div>

Providing pre-training data might risk steering the AI in the wrong direction. However, if you want a model to learn walking on bumpy terrain, wouldn’t pre-training it on walking on flat surfaces speed up the process? Instead of learning to walk from scratch, it would only need to adapt to the bumpy terrain.