While it has many unfortunate effects in society, the ability to make up your mind about something/someone in a split second is actually a survival mechanism.

It's very rare that I run in to a situation where the initial impression is accurate. How's such an inaccurate metric supposed to be a life saver?

In modern society it usually isn't, but it was a survival mechanism for early humans, and still is for animals. If you're an animal (or human) in the wild, and an unknown creature approaches you, you don't have time to think, "hm, I wonder if it's dangerous; let's not be prejudiced and wait until it gets here so I can see what it does." If you do that, and your guess was wrong, you're dead. On the other hand, if you chose to flee or hide, you'd be more likely to still be alive (and you can still choose to come out again once you've decided that the other is not a threat).

The same mechanism still exists in today's humans. Fortunately many of us are not in situations where our immediate survival is at stake. The mechanism is still there though, it just takes different forms, like having an opinion about people or things in a split second, based on what you perceive at that moment. (The reaction is likely to be different too, not fight-or-flight like in nature.) As you point out, it's often wrong, so it's much less useful to us than it used to be (although not completely useless).

it comes down to friend-or-foe, apparently. In one research I read about in the book "how we make mistakes", the split-second judgment upon seeing a photo of a political candidate for the first time was the best indicator of voting behaviour.