I have a question about Q function. At first, we randomly guess Q, and we get y from this random Q function. We tried 10000 examples and paired (x,y) trying to update q to fit y, but we use original random q to get y. how is that possible we can get other q function. what is the meaning? please explained clearly thank you! really confused.

Hey there @ryan106001020

Even though the initial Q function is random and inaccurate, but it slowly improves in the process of iteratively updating the Q function.

Here’s how it works:

First, You use a random Q to estimate the target values (`y`

) based on the rewards and future Q-values. Then, you train the Q function to better approximate these targets and update it, the Q function gets closer to the true Q values, making the agent smarter in the environment.

Hope it helps! Feel free to ask if you need further assistance.

Hi~

the point i don’t understand is that:

since i use inaccurate Q to make assumption for target y, what am I updating for?

𝑄(𝑠,𝑎;𝑤)=y hat

𝑅+𝛾max𝑎′𝑄̂ (𝑠′,𝑎′;𝑤−)=y target

we are calculating the loss between y hat and y target and updating Q function

is that correct?

thank you for replying.

Yes, you are correct! You’re updating the Q by minimizing the difference (loss) between the predicted Q value and the target value (y_hat and y). The target y is based on the reward and the best estimate of future Q values (using γ max𝑎′𝑄̂(𝑠′,𝑎′)) and over n iterations, the updates improve the Q function’s ability to predict accurate values.