People literally hunt in orange bunny suits, just like you’re suggesting. It’s regulation in several states, especially the more densely populated states. A lot of that “camo” you see is just because people like to wear it in public to advertise that they’re outdoorsy people. Even states that allow camo hunting gear require a considerable percentage of it to be orange, which is where you get the mixed orange/camo patterns from. Either that or people wear orange vests over their camo, but the solid orange stuff is usually cheaper so most hunters just wear that.

We know deer can’t see blaze orange because we’ve dissected their eyes, not because of whatever call of duty internally strawman you’re very mad at.

https://deerassociation.com/can-deer-see-blaze-orange/

As for why hunters haven’t worn orange until recent history, remember that producing whatever color you want cheaply, on demand, and at scale is a relatively modern thing. We produce a lot of colors that simply don’t exist in nature these days.