after further digging, it looks like the rest of the predicted bounding box scaling takes place within the helper function yolo_head() at which time the grid information is still available.
box_xy = (box_xy + conv_index) / conv_dims
box_wh = box_wh * anchors_tensor / conv_dims
So I now believe the code used for this exercise spreads out producing the bounding box in image-relative coordinates into 3 places: yolo_eval(), yolo_head(), and boxes_to_corners().