From Aligned Outcomes to Conditions of Judgment: AI, Deep Disagreement, and Democratic Public Discourse

This session will be part of ‘Truth, Democracy, and Misinformation’, the 20206 Symposium of the Australian Association for Professional and Applied Ethics.

The AAAPE exists to promote awareness of applied ethics and foster relationships between practitioners and academics.

The event will be held on online. The event is free for members and ticketed for non-members. More information, including the full program, schedule, and registration is available at: https://events.humanitix.com/aapae-misinformation-workshop

      Abstract:

AIincreasinglymediatesdemocraticdisagreement.Itrecommendswhatcitizensencounter,ranks competingclaims,moderatesspeech,andisbeginningtoparticipatedirectlyindeliberationitself. Much of the debate about these systems therefore asks how they might identify misinformation more reliably or better reflect the values of the people affected by them. These are important questions. But they begin too late.

Alignment research has often approached normative disagreement through value extraction, preference aggregation, or the search for sufficiently shared values (Christiano et al., 2017). More recent pluralistic approaches explicitly resist collapsing human values into a single aggregate and seek instead to preserve diverse perspectives (Sorensen et al., 2024; Fazelpour & Fleisher, 2025). Yet disagreement still tends to appear as something a system must represent, accommodate, aggregate,ortranslateintoausablenormativeoutput.Theunderlyingquestionremains:giventhat people disagree, what should the system do?

Deep disagreement unsettles this framing. Some political conflicts concern not only competing judgments but the standards by which those judgments are assessed: what counts as relevant evidence, a good reason, or a consideration that ought to matter. Where the evaluative framework itself is contested, there may be no neutral procedure capable of converting disagreement into a uniquely correct answer, or even an uncontested compromise. Democratic theory confronts a structurally similar problem. Persistent normative disagreement is a normal feature of pluralistic politicallife(Rawls,1993),yetcitizensmuststillmakecollectivedecisionsandjustifythemtoone another when substantive agreement remains out of reach (Gutmann & Thompson, 1996).

This matters for alignment becauseAI systems do not merely produceanswers from judgments that people have already formed. They increasingly shape the conditions under which judgment itself develops: which perspectives become visible, how a disagreement is framed, what counts as a reason worth answering, and whether people have opportunities to challenge assumptions andrevisetheirpositions.Ifdeepdisagreementcannotsimplybetransformedintothecorrectnormative output, alignment cannot be concerned only with outputs.

I therefore develop an account of alignment as normative coordination. Drawing on Bex-Priestley andShemmer's(2017)accountofdisagreementwithinsharedprojects,Iarguethattherelevantaim is to sustain practices in which people can exchange reasons, remain mutually answerable, contest one another's claims, and revise their judgments without presupposing eventual consensus. Their account does not itself resolve deep disagreement; rather, it helps identify what can remain normatively important when substantive resolution is unavailable.

ThisreframesthegovernanceofAI-mediatedpublicdiscourse.Truthstillmatters,andsomefactual claims are simply false. But correcting falsehoods cannot settle the normative disagreements through which citizens decide what matters, what follows from the facts, or what they owe one another. The paper therefore develops criteria for evaluatingAI-mediated discourse in terms of visibility, authority, mutual answerability, and opportunities for revision.Alignment should be assessed not only by the outcomes systems produce, but by the conditions of judgment they help create.

About the Author:

Yifan Li recently completed his PhD in Philosophy at the University of Essex. His research focuses on moral responsibility, agency, and the ethical and political implications of artificial intelligence. His current work examines AI alignment, distributed responsibility, democratic disagreement, and the conditions under which AI-mediated environments shape human judgment and accountability.