2 Formalism for statistical decision models
2.1 Actions
A decision-maker must choose from exactly one action \(a\) from an action space \(\mathbb{A}\): \(a \in \mathbb{A}\)
2.1.1 Example
- The decsion-maker is a doctor.
- \(\mathbb{A} = \{\)“give medication”, “don’t give medication”\(\}\).
In some applications, each menu option \(a\) will correspond to a bundle or vector of distinct actions. In these cases, the action space \(\mathbb{A}\) will include an exhaustive list of all such bundles or vectors.
2.2 States
In our nomenclature, states refer to observable data about relevant conditions.
A state \(x\) will correspond to a single point in a state space.
A state space \(\mathbb{X}\) describes the set of all possible values that \(x\) might take: \(x \in \mathbb{X}\).
2.2.1 Example
A test can be performed to determine whether a patient has a certain medical condition.
Let \(x\) denote the test result, which may be positive (\(x =\)“Y”) or negative (\(x =\)“N”).
The state space is the set of all possible outcomes for this test: \(x \in \mathbb{X} = \{\)“N”, “Y”\(\}\).
Note: Notationally, and computationally, it will often be more convenient to use numeric values for states, e.g., \(X = \{0,1\}\), where \(x=1\) corresponds to a positive test.
2.2.2 Representing uncertain states as random variables
Before observing the actual realized value \(x\) of the system’s state, the system’s true state is unknown.
In a typical application, we will nonetheless have some idea about the likelihood that the system’s state \(x\) will attain each of the possible values in the state space \(\mathbb{X}\).
Formally, our beliefs about the system’s state can be represented by a probability distribution over the state space \(\mathbb{X}\).
2.2.2.1 Example
For \(x \in \mathbb{X} = \{0,1\}\), let \(f(1) = \Pr\{x = 1\}\) and \(f(0) = 1-f(1) = \Pr\{x = 0\}\).
2.2.3 Random variables
Prior to knowing the realized value of the system’s state, its true value is an uncertain random variable.
By convention, we will use capital letters to refer to random variables, and lower case letters to refer to corresponding realized values.
Here, \(X\) denotes a random variable that takes values from on the state space \(\mathbb{X}\), and \(x\) denotes its realized value in \(\mathbb{X}\).
Let \(f(x | \theta)\) denote the probability that state \(x\) will be realized, conditional on the value of the unknown parameter \(\theta\).
2.2.4 Random states
For \(x \in \mathbb{X}\), let \(f(x)\) define a probability distribution over \(\mathbb{X}\).
Let \(B \subset \mathbb{X}\). If \(\mathbb{X}\) is discrete, then
\[ \Pr\{x \in B\} = \sum_{x \in B} f(x)\] If \(\mathbb{X}\) is continuous, then
\[ \Pr\{x \in B\} = \int_{x \in B} f(x)\]
More generally, if \(x\) can take any of a finite number of values \(1, 2, \ldots, N\), then a probability distribution over the state space \(\mathbb{X} = \{1, \ldots, N\}\) can be represented by a vector \(\theta = <\theta_1, \theta_2, \ldots, \theta_N>\), where \(\theta_n = \Pr\{x = n\}\).
Since \(\theta\) represents a probability distribution, we must have that \(\theta_n \geq 0\) for all \(n\), and \(\sum \theta_n = 1\).
2.3 Parameters
A parameter is a variable that describes the condition of the system, that is not directly observable. An unobserved parameter \(\theta\) takes values on a parameter space \(\mathbb{\Theta}\): \(\theta \in \mathbb{\Theta}\).
2.4 Payoffs
After choosing an action, the decision-maker realizes a payoff. This payoff depends on the chosen action \(a\). Depending on the situation being modeled, it may make sense to express payoffs either in terms of the realized value of the system’s state, or on the value of the unobserved parameters.
2.4.1 Payoffs as a function of action and state
Let \(u(a,x)\) denote the payoffs from a given action \(a\) and state \(x\).
\[ E[u(a, X) |\theta] = \int_\mathbb{X} p(x) u(a,x) dx \]
Then define the expected payoff \(U(a,\theta)\) as a function of \(\theta\) in these terms:
\[ U(a,\theta) = E[u(a, X) |\theta] = \int_\mathbb{X} p(x) u(a,x) dx \]