Brier scoring works on questions with cheap, fast resolution; the strategic decisions you mention (hiring, equipment, big purchases) resolve over months or years, often ambiguously, and the counterfactual never resolves at all. Curious whether the calibration gains from the rapid-feedback quiz actually transfer to the slow-feedback domains the tool is designed to help with, or whether it ends up training a slightly different skill. A second thing: most of my strategic decisions weren't solo, and once one calibrated person sits in a room with two louder uncalibrated ones, the calibration math stops being load-bearing. Have you thought about a team variant?

Both really good points. The research does suggest that the core skill does transfer. The quiz can help with long horizon predictions. The mechanism itself seems to be the actual awareness of overconfidence rather than just domain-specific knowledge. With that being said, the gap between the quiz and real-world application is real, and tracking both over time is part of why I built the decision logging side. For your question about teams, that's a built-in feature already! Submissions are "sealed" so you submit before seeing others. The team feature also has a believability-weighted aggregation based on each submitter's track record, and I also built an IC mode for investment committees. The problem you describe about one calibrated person in a room with two uncalibrated ones is what the sealed model prevents. Everyone makes draws their own conclusion, then they compare!

I'm in software and if I could hire more American engineers, I would. It's not about the cost for my company, it's about the availability of skilled talent and American schools aren't producing nearly enough of it. We've been hiring overseas for nearly a decade and unfortunately I don't see that changing anytime soon. More H1Bs would at least let us bring those great guys over here, which would make my company more successful and would obviate the need for us to send millions of dollars abroad.

American colleges aren't supposed to train, they are supposed to educate. If companies had better training or mentoring programs, it wouldn't be as much of a problem.