The principal effect of karma here is not how it helps you judge your fit, but how it actively shapes the community/content. This effect was clearly visible on reddit as it was realized that the more sensational content was vastly more promoted in the reddit karma system.
So my question:
What is the value of high quality content ordering? Are the present ordering strategies on HN/reddit/[insert karma-based discovery site] good enough? Would the theoretically perfect ordering offer much more value than current systems?
Take a look at my startup, we're trying to push a different system for ranking stuff. I personally think there's a lot more value in it. But sometimes people prefer simple algorithms that are easier to understand, there's value in it too.
I like how you automatically assign a guest ID so visitors can try out the customization features right away, and think, "Hell, I'm halfway to joining -- might as well go the rest of the way now." The same thing could have been done invisibly with a cookie, but then the psychological hook wouldn't be there.
We're only talking about ditching karma, the running total for each user. We would still be able to upvote/downvote stories and comments, but those votes would only effect the individual story or comment, not the user who posted it.
If pg says anything he gets at least 50 points for being pg. The rest is because pg is actually smart and doesn't usually say dumb unthoughtful things.
More people read pg's comments, so he does get more upvotes than the same words would if coming from someone else. However, to say that you can mention pg in a comment to get more votes is neither true nor particularly funny.
For most questions I think average would be more useful. For the question of whether people upvote pg thoughtlessly or not, median is useful. If more than half of pg's comments end up with a score of 4 or less, then we can conclude that his comments that end up with 50 or more points are a select subset.
Submission scores might be a statistically different distribution then comment scores. Also while median is better than mean, it may or may not be better than mode - difficult to tell without looking at the shape of the distribution.
Median score is not what's being discussed. It's quality vs score. Those individual comments that are of completely average quality and yet have phenomenal scores are the most telling.
Also very telling: score velocity over time. For example, if `joe` makes a good point with factual grounding and is getting lots of points. Then `bob` replies in disagreement and bob provides almost nothing to back up his position. Then suddenly the points reverse (joe goes negative and bob gets lots of points) just because bob said it it must be true and everyone blindly agreed. There are ways to measure this impulse response that would provide more concrete numerical measurements.
Median score... there are so many effects that would cause this to not relate to comment quality that that is pretty useless. For example, just commenting on more active topics can give you a much better median score.
A lot (majority even?) of factories/universities do have generators which they operate at most expensive hours and feed the excess back into the grid. It only makes sense to have these on hand from a risk perspective considering the volatility of electricity costs.
[insert jealousy for company with no business model]
[troll complete]