Hello guys, I successfully applied the current lab for MountainCar Gym project - Mountain Car - Gym Documentation
As the current code considers that the action space will always be Discrete
I would like to apply the same code to the problems where the action space is like this -
I changed the current code a bit and somehow made it render the first action but the compute_loss function is causing the issue as it is made for a Discrete example, can someone please help me adjust the current code for this project?
https://www.gymlibrary.dev/environments/box2d/bipedal_walker/
Hey @Shivang_Chauhan,
Welcome to the community. Till now, I have only worked with finite number of states and finite number of actions when it comes to RL, and this assignment was pretty much the first exercise that I solved myself dealing with infinite state space. However, as far as my understanding goes, you won’t be able to use the algorithm, i.e., Deep Q-Learning (at least the way it is implemented in the assignment) for en environment having both, an infinite state space and an infinite action space.
This can be trivially understood from the way the neural networks (NN) have been created and trained. Given a state, NN predicts the Q-value for the next state, corresponding to all the possible actions, and during the update, we simply took the maximum across these predicted values. However, if your environment has an infinite action space, your NN must essentially predict the Q-value for the next state corresponding to an infinite number of actions, which isn’t possible (at least according to the current implementation).
However, there indeed are ways to deal with environments having an infinite state space and an infinite action space. Please read more about function approximation and the various algorithms that come under the umbrella of function approximation. I hope this helps.
Cheers,
Elemento