Intersection over union or Union over intersection

Dear Learners/Mentors,
I have a query on the choice of IOU as a measure for the choice of bounding box.
As per the understanding IOU is the Intersection of the union of all the bounding boxes containing the object. The choice of the bounding box is then made with IOU >=0.5. My question is why an Union over Intersection is not considered. Essentially if I integrate/combine all the Intersections, I would expect to get the tightest bounding geometry / box for an object.
I do understand that union over intersection will not guarantee me a box, but it is not difficult to construct a rectangular scaffolding or hull over this.
Is there a flaw in my argument or understanding or visualization? Kindly clarify.
Thanks!

What happens to your U/I ratio calculation when the boxes do not share any common area?