I've heard of handgun irons falling off the gun. I've personally had the dots fall out of my 3-dot sights. I know of several people who have sheared off or really banged up their Glock sights. I've known of people whose iron pistol sights moved slightly and ruined their zero. There are fiber optic sights that break. Night sights that die. So on and so forth. Likewise, there have been rare but real cases where optics malfunctioned that I know of.
The thing is, these issues tend to always reveal themselves WELL before you put the gun in the holster and walk out the door. It's along the same lines as we all carry guns from trusted manufacturers. We all know that even trusted manufacturers produce some lemons. That's why we take our guns to the range as well as keep a maintenance regimen in order to avoid, find, and fix these issues before we're out in the field. The same holds true when it comes to utilizing optics on pistols and rifles.
The chances of your reputable pistol optic that is well maintained and has been tried and tested suddenly failing out of nowhere while trying to get through the 10-15 rounds in a mag aren't very probable. Just like it isn't very probable that a well-maintained and tested Glock, for example, will suddenly, without any prior indications or warning, poop the bed in a gunfight involving only ~15 or so rounds, it is also very unlikely.