Full text of the second edition of Artificial Intelligence: foundations of computational agents, Cambridge University Press, 2017 is now available.

#### 7.2.3.1 Probabilities from Experts

The use of pseudocounts also gives us a way to combine **expert opinion**
and data. Often a single agent does not have good data but may have
access to multiple experts who have
varying levels of expertise and who give different probabilities.

There are a number of problems with obtaining probabilities from experts:

- experts' reluctance to give an exact probability value that cannot be refined,
- representing the uncertainty of a probability estimate,
- combining the numbers from multiple experts, and
- combining expert opinion with actual data.

Rather than expecting experts to give probabilities, the experts can
provide counts. Instead of giving a real number such as *0.667* for
the probability of *A*, an expert can give a pair of numbers as
*⟨n,m⟩* that can be interpreted as though the expert had observed *n*
*A*'s out of *m* trials.
Essentially, the experts
provide not only a probability but also an estimate of the size of the data set
on which their opinion is based.

The counts from different experts can be combined
together by adding the components to give the pseudocounts for the system.
Whereas the ratio reflects the probability,
different levels of confidence can be reflected in the absolute values:
*⟨2,3⟩* reflects extremely low
confidence that would quickly be dominated by data or other experts'
estimates. The pair *⟨20,30⟩* reflects more confidence - a few examples would not change it much, but tens of examples would. Even hundreds of
examples would have little effect on the prior counts of the
pair *⟨2000,3000⟩*. However, with millions of data points, even
these
prior counts would have little impact on the resulting probability.