This HTML5 document contains 9 embedded RDF statements represented using HTML+Microdata notation.

The embedded RDF content will be recognized by any processor of HTML5 Microdata.

Subject Item

n2:

rdfs:label

Evaluating Subjective Expert Judgment for Science Policy: An Application of the Bayesian Truth Serum

rdfs:comment

Title: Evaluating Subjective Expert Judgment for Science Policy: An Application of the Bayesian Truth Serum Author(s): Rebecca Weiss Affiliation: Massachusetts Institute of Technology Presented At: STiS 2008 Primary Topic Area: Science and the Public

n7:

Subjective expert judgments are essential to creating good policies; this is especially true in science and technology policy. However, it is not easy to evaluate subjective judgments in a way that accurately reflects both the honest opinion of experts as well as objective truth. Simple measures like the averaged judgment, or majority rule, can be inadequate since such information does not account for variations in the quality of expert opinion. This may occur when some judgments are unpopular or unusual. The “Bayesian Truth Serum” or BTS is a scoring method that gives experts incentives for truthfully reporting judgments in settings where no immediate objective truth criterion is available. This is the case with long-range scientific forecasts. According to theory, the method is also able to identify the judgment with the highest probability of being objectively true provided that the participating experts are rational Bayesian decision-makers and that a single right answer exists . This project aims to determine whether BTS can aid the science policy-making process, where there are several potential “right” answers but where recognized experts would establish one as being more truthful than others. A selection of recent scientific studies with potential impact for science policy will be presented to recognized experts. Their judgments will be assessed using BTS and compared with other statistical measures to determine whether BTS is more accurate at estimating both objective truth and the truthfulness of expert judgment.

n3:

Rebecca Weiss

n10:

Massachusetts Institute of Technology

n5:

STiS 2008

n11:wikiPageUsesTemplate

n12:

n8:

n9:

n6:abstract

Title: Evaluating Subjective Expert Judgment for Science Policy: An Application of the Bayesian Truth Serum Author(s): Rebecca Weiss Affiliation: Massachusetts Institute of Technology Presented At: STiS 2008 Primary Topic Area: Science and the Public