I like this idea a lot and have been thinking about similar topics @vbuterin, in particular, those relating to peer-prediction mechanisms or “markets”. Even though we assume that participants in the market only bid on a small percentage of posts (0.01%), we can still even further build a sort of reliability metric around them. In addition, there might be other incentives to participate beyond betting, if instead the winning side (heavier weighted side) is rewarded as part of a pseudo mining process. See crowdsourcing subjective information on heterogenous tasks or more recently this.
In any scheme, we are still limited by how quickly participants show up to bet or give their belief/opinion. Then we should also be able to reach the same tallying state from either betting (putting up collateral + information) or minting currency (putting up information). The incentives might be more appealing since there is no collateral on one.
In either model and over time, we should be able to build a score about voters/validators depending not only on what sides they ended up after some threshold time but on their score as dictated by a proper scoring rule, such as from the peer-prediction papers above.
As far as plausibility, I had trouble implementing these mechanisms on Ethereum due to gas limit issues, but nonetheless had some success implementing Dasgupta’s scoring rule (might not work in its current form).
As an aside, have you thought about employing peer-prediction techniques into mining processes? Given the premise is some game theoretic notion of truthfulness, it could provide a metric over honesty in these decentralized systems.