Anchor boxes

if we want to use anchor boxes in our algorithm, should we change our Y labels?

You could help the community help you by being a little more loquacious. What is the algorithm to which you contemplate adding anchor boxes ? What is the functional objective - is it more accurate localization predictions? Support for more predictions per image? Or are you already using an algorithm that incorporates anchor boxes and the question is more about values? In YOLO, at least the versions related to the code used in these lectures and programming exercises, anchor box values are not included within Y, though their usage influences the shape of both Y and \hat{Y} and the location within those matrices where the non-zero values appear. So adding new or changing existing anchor boxes will alter the shape of the Y matrix but not any of its labels/values.

The role of anchor boxes in YOLO is discussed in several existing threads in this forum that can be found with the search. might help clarify. HTH

EDIT: old thread that popped up in my feed. Minor edits to (hopefully) improve the post.