Probabilistic Inference Barrier

21 01 2007

Using the methods of Russell and Restall’s paper on inference barriers, I will show that one can’t derive an “is” from a “probably”. That is, no consistent set of statements expressing only relations among the probabilities of statements expressible in a non-probabilistic object language can entail anything about the actual truth-values of such object language statements, unless the statements are either tautologies or contradictions.

Let S be a (consistent) set of probabilistic statements, and let O be the (non-tautological) object language statement that is said to follow from S. To show that O does not follow from S, start with a probability space (a set of “states”, together with an algebra of subsets of this set called “events”, a real-valued function on this algebra satisfying the probability axioms, and a specification of which state is “actual”) satisfying all of S. Call this space P. Now create a space P’ by adding a single state X in which O is false to the state space. A subset of this space will be an event iff either it doesn’t contain X and was an event in P, or it does contain X and removing X gives an event in P. Define the probability function on these events by assigning the same probability to any event in P’ that either it, or it without X, had in P. Let X be the “actual” state in the new space.

By construction, every probabilistic statement will have the same truth-value in the two spaces, because every proposition has exactly the same probability (effectively, all we did was add a single state with 0 probability). In particular, all of S is true in P’. Also, by construction, O is false in P’. Thus, S does not entail O, QED.

One way to block this argument would be to require that every non-empty event have non-zero probability, but this would block a lot of interesting probability spaces (in particular, any space with uncountably many mutually incompatible events). However, a very similar argument would go through if one allowed some small tolerance of epsilon in the probabilistic statements of S (assuming none of the statements are conditional probability statements, whose value can in fact deviate by much more than epsilon due to the addition of a single state of probability epsilon).

But in some sense, this shows the weakness of these “inference barrier” results, which Gillian Russell points out should really be called “implication barriers”. Under certain conditions, it’s certainly rational to infer that it will rain, given that there’s a 99% chance of rain. This result merely shows that no amount of probabilistic evidence will ever entail anything with certainty, even though it might entail it with probability 1. The distinction between probability 1 and certainty is something I’m thinking about right now for my dissertation.




5 responses

22 01 2007
Mike Titelbaum

Hey Kenny,
Your argument requires constructing a space in which P'(X)=0 despite the fact that X is the “actual” state. This is plausible in situations with uncountably many mutually incompatible events, but do you think it’s acceptable in situations where everything is finite? If not, does your inference (really implication) barrier result apply to such finite situations?

22 01 2007

I think that infinite cases are helpful motivation for possible states of probability 0, but that once we recognize this conceptual possibility, the infinity isn’t necessary. Of course, I also think that there are infinitely many states in the space that most of us work with, because I think of states as giving maximally specific descriptions of how things might be.

But at any rate, whether X is actual or not is in some sense irrelevant to its probability – if a state couldn’t be the actual one, then it shouldn’t be in your model at all.

And even if probability 0 states aren’t allowed, the implication barrier between probabilistic strict inequalities and statements of fact will hold, because the new state can be introduced with a probability even smaller than any difference relevant to the inequalities.

Unless, by analogy to barring probability 0 unless there are infinitely many states, one bars really small probabilities unless there are enough (but still only finitely many) states.

22 01 2007

Also, I think I should specify that both sorts of arguments only work if the probability statements are unconditional, rather than conditional. Conditional probability values can change arbitrarily even with the introduction only of very improbable states. And they need even further specification if states of probability 0 exist.

24 01 2007
Mike Titelbaum

The reason I asked is that I’m not sure how your construction works. You claim that O is false in P’. Clearly if P’ designates a state as actual and that state assigns falsehood to O, O is false on P’. But in your response you wrote,

“whether X is actual or not is in some sense irrelevant to its probability – if a state couldn’t be the actual one, then it shouldn’t be in your model at all.”

That makes it sound like what’s important here is that P’ is defined over a state space one of whose states assigns falsehood to O, even though P’ assigns a probability of zero to that state. If that’s what the argument turns on (instead of the fact that P’ designates X as actual), I’m wondering why those facts make O *false* on P’.

24 08 2007

I should also compare this to a point that Malcolm Forster attributes to Karl Popper in his note on David Deutsch’s paper on decision theoretic probabilities in quantum mechanics. He suggests that Popper at some point said “no probabilities in, no probabilities out” in The Logic of Scientific Discovery. This result is some sort of converse to that.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: