Note on Bad Incentives
October 9, 2021
A while back I wrote a post about structuralist and humanist approaches to social theory and political thought, how each looks cynical from the point of view of the other, and the impasse this can lead to. An example of this tension then came up a few days ago in a discussion I was having about the role played by bad incentives in creating pathological discourse. The point of contention was whether such pathologies are typically necessitated by the incentive structure of a space, or whether they are the fault of bad actors within the space. This question has clear strategic significance: if the latter is true, then the discourse could in principle be de-pathologised by piling good actors into the space; but if it’s the former, then this cannot work because it fails to address the root cause of the pathology, and those good actors will likely become complicit in sustaining it.
Naturally, my position leaned towards the former: pathological discourse seems so ubiquitous to me that it cannot possibly be explained purely by the presence of bad actors (which is not to say there are no bad actors—clearly, bad actors will often come to dominate a space with bad incentives—what is at issue is which is the cause and which is the symptom). This stance got me labelled a cynic. But I felt this misunderstood the argument in important ways. This got me thinking about two possible ways of interpreting this argument about bad incentives, one of which is genuinely cynical (but rests on bad humanist assumptions) and one of which is not (and is properly structuralist).
The first (bad) argument says that bad incentives produce pathological discourse because humans are essentially self-interested, so when bad faith engagement is incentivised we simply cannot help ourselves. Obviously, what makes this argument cynical is the intrinsic selfishness it posits in human nature. In doing so it effectively excuses the bad actors, not by trying to dispute their status as bad actors but by claiming that they couldn’t fail to become bad actors, given the external conditions they were exposed to. This reasoning lies on humanist foundations: in positing intrinsic self-interest, it diagnoses the pathological discourse as stemming partially from certain psychological facts. When I was accused of cynicism, I suspect I was being interpreted as making this argument.
But actually I think this argument is pretty terrible, precisely because of its cynical humanist assumptions. I do not think that pathological discourse arises because bad incentives turn people into bad actors; I think bad incentives can produce pathological discourse directly, which is to say from perfectly good faith intentions. The reason is simply that the presence of bad incentives undermines everyone’s ability to communicate their motives to each other. Or to put it differently, bad incentives make it harder to reason about other people’s decision procedure. This can be catastrophic, because the trust required to produce healthy discourse requires that individuals be confident that the stated commitments of others can be taken as accurate representations of their motives, intentions, and bottom lines. The issue, then, is not that incentivised bad faith engagement actually produces bad faith impulses in individuals, but that wherever such incentives exist it will become systematically harder to distinguish imperfect good faith engagement from bad faith engagement, and this will in turn undermine our ability to make judgements about where to direct our own energies. None of this depends on any assumptions about human psychology. The key premise is simply that trust is an intrinsically difficult problem, and can easily be destabilised by noise introduced into signalling systems used to communicate values.
This second version of the argument is properly structuralist, and to my mind non-cynical: it suggests that we should create new discursive spaces with good incentives, rather than compete for territory in existing spaces with bad incentives (which is ultimately just to conform to those incentives). The critical project it suggests is an attempt to diagram the bad incentives and their mechanism of operation, which if carried through with sufficient clarity would provide a blueprint for building spaces which do not reproduce the same problems. To reiterate the broad conclusion of the previous post, it seems to me that the strategic impasse the left is stuck in takes the form of a deadlock between pessimistic and optimistic variants of humanism (between ‘society is screwed because people are assholes and always will be’ and ‘society is screwed because those people are assholes but we could be better’). But the apparent conflict between them is a mirage, and mislocates the stakes of the discussion—they fail not on account of being too optimistic or too pessimistic, but because their shared humanism leaves them incapable of providing meaningful responses to structural problems.