author: niplav, created: 2023-11-04, modified: 2024-02-08, language: english, status: notes, importance: 8, confidence: likely
Summary of a longer investigation into inconsistent preferences and how to resolve them. I investigate two different ways of representing inconsistent preferences, two different methods for resolving them into consistent preferences, how these perform on seven criteria, and how two of those criteria are incompatible. I conclude by connecting the question to ontological crises, and offer some ideas for further research in the area.
In 1947, John von Neumann and Oskar Morgenstern published their book Theory of Games and Economic Behavior. This has made a lot of people very angry and has been widely regarded as a bad move.
Their famous theorem is deceptively simple. Given a set of options
$\Omega$ (not further elaborated on), and a preference relation
on lotteries of those options satisfying four axioms (completeness,
transitivity, continuity, indepedence), one can construct a utility
function $u: \Omega \rightarrow [0;1]$ that assigns real values to
all the options. The value of a lottery $\mathcal{L}$, then, is simply
the expected value of the elements in the lottery.
However.
There are some points of, ah, contention around the von
Neumman-Morgenstern setup. One of those is around the set $\Omega$:
the common examples include sets of fruits, or other cute edible & easily
handleable objects. Is $\Omega$ assumed to be finite? Where do we get
it from‽
However, when pressed, proponents of vNM utility theory start talking
about how $\Omega$ really is a set of universe-histories
or multiverse-histories. This introduces some problems: For
example, one can't guarantee that the resulting utility function is
computable,
and every behavior can be interpreted as being compatible with expected
utility maximization.
But even leaving those aside, there is ample evidence that humans don't fulfill the axioms stated above, at least in the case where we partition the world into objects such as distinct amounts of money. For example, humans violate the independence axiom in the Allais Paradox, and instead might have preferences with a more involved structure. Additionally, there has been some noises that markets may (or may not?) violate the completeness axiom.
All of this points into a similar direction. Agents don't spring into the world fully formed, instead they grow and develop. They might start out as collections of dumber "agents", undergo pressure from exploitation, refactor their models of the world and sometimes their preference, perform internal bargaining and more.
If we assume that there is a "natural" or "rational" structure for preferences, then I find it likely to assume that cognitive systems, before and during their development, might not have preferences that conform to that structure.
Therefore it is useful to examine procedures to transform preferences that don't have this "natural" structure into ones that do.
Since the standard model of rationality in economics is expected utility maximization with preferences that conform to the vNM axioms, I have mainly focused my efforts in that framework. This is not an endorsement of the vNM axioms as a normative ideal for rationality, I selected it purely out of convenience, and in the hope that whatever the "true" theory of rational preference turns out to be, insights from inconsistent preferences and their resolution will transfer to this "true" theory (should it exist).
This text is a distillation of work other people and I have done on the theory of value formation. The distillation is completely my own work and highly opinionated, thus all blame and some praise goes to me.
These people pushed this project forward, disagreed, were frustrated, calmed down, were confused, went silent for a while and then came back: Kaarel Hänni, Alexander Gietelink-Oldenziehl, Filip Sondej, Felix Harder.