This is a puzzle to me, even late in my career. I think one thing is that as a company grows and matures, it gets more organized, and more formal about hiring. This makes it harder to hire misfits such as myself. HR has a hard time passing a misfit's resume along to a hiring manager. But also harder to get rid of us. ;-)

If bigger, more successful companies hire with an eye towards specialization, then the flip side is that there's a reward for workers who specialize. Also, many people like what they're doing, want to get good at it, and do more of it.

This is why, much as I love physics and don't regret studying it, I also won't tell anybody that it's a surefire career path.