I have only ever passed on 2 animals that I later regretted(although later was nearly immediate on both).
I had a easy to draw tag in New Mexico with local friends to help out on the hunt. We had a bit of private access, and I was young, 15 if I remember right. We saw a buck moving through some washes to keep hidden but didn't get a great look. I was able to sneak up to within 200 yards and he was broadside but looking directly away with brush behind him. In my 3-9 power scope I looked and figured he was a 22" 3 point type buck. So I put my gun down. He then ran off and I got a better look at what I was missing. Turned out he was more like a 25" wide 180" 4 point with deep forks. Don't know how I missed that but crappy glass definitely didn't help. Never saw him again after that. Tag soup.
Next I was elk hunting opening day on a general tag. At this point I had killed a few raghorns but nothing 5 points or better. The area I hunt, most mature bulls end up right at 300" and if you find one better, you better shoot it. My family has killed a ton of elk in this area over the years and up to that point, only 2 broke 300"(315 and 322). During archery we saw multiple mature bulls in an area and were able to harvest one(315 bull) and saw one more of the same caliber or better. Didn't get the greatest look but he had great top end. Opening rifle comes around and I can hear multiple bulls bugling, 1 was just grunting and sounded big, old and mean. Another was a mature bugle that was definitely rutting that morning, and a few other wimpy bugles. The grunting bull was directly down wind so I decided I would try timber hunting towards the other mature bull which the wind was right for. I keep getting closer to the bugle until I figure its about 150 yards out, and he never makes another sound. Going slowly through the timber I end up spotting a cow to my right at the exact time she sees me, only 15 yards away. I stood still as she stared and me trying to figure out what I was. After what seemed like forever, probably 1-2 minutes, she starts looking around and decides to slow trot off. I pull my scope up on her as 2 others join, a cow and calf, then a bull joins all trotting away. To me he looked like a wide rag horn, his antlers pointing directly at me with head up, and he's about 80 yards running. I decide to pass but keep my scope on him. Last second as he turns into the timber, he turned his head and all I see is a royal, and a 5th point with no idea of size. I smacked myself for passing but knew they weren't very scared. I snuck up as slow and quiet as possible. I get to where they disappear and I hear elk running from my left going right. I pull my scope on the one opening, about 6 feet wide, and wait. First, cow, second and third, cow/calf. I knew he was next and seconds later I catch antler running through the timber towards the opening. Scope still up, as soon as I saw fur I fired(any elk tag so I knew if I accidentally shot a cow, I was ok), and he dropped not 75 yards away. Figuring I shot a 200-250" bull if I was lucky, I walked up and was very surprised. It was the bull we saw during archery with great tops, but very weak brows and third. 315" in the end and he looks amazing mounted on the wall. Just over 50" beams.