Given some information about a situation, a probabilistic model’s partitioning of probability space should be fine enough that all possible states-of-affairs consistent with the information are distinct events in this space (sufficiency); further, the probability space should contain no events inconsistent with the information (minimality).

Here are a couple applications of the minimal sufficient model principle.

When one is told only that the urn contains some black and some white balls, the probability 1/2 for drawing black is derived from applying the indifference principle to the 2 member probability space S = { [white drawn] , [black drawn] }. When we are told the color of the balls, we receive extra information about the possible draws which is not incorporated in S. As a result, the 1/2 probability no longer applies, because our space S does not incorporate all the given information. For example, it does not distinguish between drawing ball 1 and ball 2. To incorporate the given information one must refine S to S’ = { [ball 1 drawn], [ball 2 drawn], …}. Once the space is updated for sufficiency, the indifference principle can be correctly applied to S’ and yield the probability 3/10.

Here’s another way 1/2 could be derived: applying the indifference principle to the probability space

S = { [ball 1 drawn & is white], [ball 1 drawn & is black]. [ball 2 drawn & is white], …}

One determines the probability of drawing a black ball is 1/2. When we are told the color of the balls, however, some possibilities are no longer possible and must be stricken from S, giving us S’ as in the first example. Here the space S is sufficient but must be modified for minimality before applying the indifference principle.

The minimal sufficient model principle reminds us that before any probabilistic reasoning can be done, one must create a reasonable model of the events which are possible given a body of information.

As a final comment, here’s a situation where even the sufficient model principle does not provide a unique way forward for the modeler. Suppose we are shown a jar and told that it contains some proportion of oil and some proportion of water. We cannot determine any more information from looking at the jar. What probability distribution should we assume on the proportions?

There are at least two ways to apply our principles. In the first, we could induce a probability model from the proportion of water in the jar, then apply the indifference principle to that model. The result would be a model in which every proportion of water is equally likely. In the second, we could induce a model from the ratio of water to oil in the jar and apply the indifference principle to that. Then we have a model in which every ratio of water to oil is equally likely. A little calculus will show that the resulting distributions are not the same. [NB: one could avoid calculus by discretizing. Then in the first case our model states that 0-10% water is equally likely as 10-20%, 20-30% etc; in the second case, our model states that a ratio of oil to water in the 0-10% range is equally likely to 10-20%, 20-30%, etc.]

It seems like the first model is more justified by the information than the second. But how do we know that? Are we using some hidden information to make that decision?

i’m not sure that jaynes would have endorsed this, but here’s my take on why it’s not circular (and/or silly) to learn those symmetries by Bayesian inference:

symmetries seem like a good candidate for framework-level knowledge (abstract knowledge shared across many specific systems, or domains). you can formalize this a bit with a hierarchical bayesian model in which symmetries live at an upper level — the principle of indiference gives the prescription for taking a symmetry and turning it into a prior on lower-level theories. because the high-level “symmetry theories” receive support from many specific systems it isn’t circular to use them to constrain inference in any one system. (the symmetries here act very much like (the other) goodman’s “overhypotheses”.)

-NG

NG – That’s an interesting point – certainly, finding explicit symmetries that have to be respected would put applications of the indifference principle on a better footing, but I start to worry that there might be some sort of circularity here, because I’m not sure how Jaynes would suggest that we find out about symmetries, except by using some sort of Bayesian inference.

one of the interesting things to me about jaynes’ use of the indifference principle in many places (not necesarily the book, which i haven’t read in any detail) is that he really wants indifference to do the work that symmetry often does in modern physics. indeed, he often envokes the indifference principle to extract predictions after first stating the symmetries which relate states to whose differences we should be indifferent.

it seems that this way of doing things would address your concern. in particular, the difference between your two scenarios is (i think) some background assumptions about symmetries (balls vs drawers). of course, if we need to state symmetry assumptions to apply the p.o.i., we should probably be explicit about this in stating the principle….

-NG