Title: Evaluating Subjective Expert Judgment for Science Policy: An Application of the Bayesian Truth Serum Author(s): Rebecca Weiss Affiliation: Massachusetts Institute of Technology Presented At: STiS 2008 Primary Topic Area: Science and the Public
| Attributes | Values |
|---|
| rdfs:label
| - Evaluating Subjective Expert Judgment for Science Policy: An Application of the Bayesian Truth Serum
|
| rdfs:comment
| - Title: Evaluating Subjective Expert Judgment for Science Policy: An Application of the Bayesian Truth Serum Author(s): Rebecca Weiss Affiliation: Massachusetts Institute of Technology Presented At: STiS 2008 Primary Topic Area: Science and the Public
|
| abstract
| - Subjective expert judgments are essential to creating good policies; this is especially true in science and technology policy. However, it is not easy to evaluate subjective judgments in a way that accurately reflects both the honest opinion of experts as well as objective truth. Simple measures like the averaged judgment, or majority rule, can be inadequate since such information does not account for variations in the quality of expert opinion. This may occur when some judgments are unpopular or unusual.
The “Bayesian Truth Serum” or BTS is a scoring method that gives experts incentives for truthfully reporting judgments in settings where no immediate objective truth criterion is available. This is the case with long-range scientific forecasts. According to theory, the method is also able to identify the judgment with the highest probability of being objectively true provided that the participating experts are rational Bayesian decision-makers and that a single right answer exists .
This project aims to determine whether BTS can aid the science policy-making process, where there are several potential “right” answers but where recognized experts would establish one as being more truthful than others. A selection of recent scientific studies with potential impact for science policy will be presented to recognized experts. Their judgments will be assessed using BTS and compared with other statistical measures to determine whether BTS is more accurate at estimating both objective truth and the truthfulness of expert judgment.
|
| Authors
| |
| Affiliation
| - Massachusetts Institute of Technology
|
| Conference
| |
| dbkwik:scitechsoci...iPageUsesTemplate
| |
| topics
| |
| abstract
| - Title: Evaluating Subjective Expert Judgment for Science Policy: An Application of the Bayesian Truth Serum Author(s): Rebecca Weiss Affiliation: Massachusetts Institute of Technology Presented At: STiS 2008 Primary Topic Area: Science and the Public
|
![[RDF Data]](/fct/images/sw-rdf-blue.png)