Delegating decisions to AI? The risks of losing human judgment

Machines are increasingly tasked with decisions that have profound implications for individuals and society. Robbins categorizes decision delegation into two types: descriptive and evaluative. Descriptive outputs, such as identifying patterns or classifying objects, can be effectively handled by machines. However, evaluative outputs - those requiring moral agency or judgment - pose significant challenges.

As artificial intelligence (AI) systems become increasingly integrated into our daily lives, the ethical implications of delegating decision-making authority to machines are gaining prominence. In his paper, “What Machines Shouldn’t Do”, published in AI & Soc (2025), Scott Robbins from the University of Bonn emphasizes the importance of meaningful human control (MHC) in AI-driven decisions. The study critiques existing frameworks for MHC, advocating for a fundamental shift in how we delegate decisions to machines.

Robbins defines meaningful human control as the capacity of humans to take moral responsibility for the decisions made by machines. This concept is particularly significant in scenarios where AI systems influence high-stakes outcomes, such as approving loans, sentencing criminals, or assessing job candidates. The current focus in AI ethics often revolves around making systems explainable or integrating humans into decision loops. However, Robbins argues that these measures address the symptoms rather than the root cause of the issue: determining what decisions should and should not be delegated to machines in the first place.

Delegating decisions: The ethical dilemma

Machines are increasingly tasked with decisions that have profound implications for individuals and society. Robbins categorizes decision delegation into two types: descriptive and evaluative. Descriptive outputs, such as identifying patterns or classifying objects, can be effectively handled by machines. However, evaluative outputs - those requiring moral agency or judgment - pose significant challenges.

For instance, a machine labeling someone as “suspicious” inherently makes an evaluative judgment based on opaque criteria. Robbins argues that machines lack the moral agency and contextual understanding to make such determinations, which should remain under human authority. Delegating evaluative decisions to AI not only risks unethical outcomes but also undermines the accountability and transparency essential to democratic societies. Such decisions require human oversight to ensure they align with societal norms and ethical principles.

Technology-centered vs. human-centered approaches

Robbins explores two dominant approaches to MHC: technology-centered and human-centered. Each has its strengths and limitations, but neither sufficiently addresses the root problem of determining what decisions AI should handle.

Technology-Centered MHC

This approach focuses on embedding ethical reasoning and explainability into AI systems. Machine ethics, for instance, attempts to program moral principles into algorithms, while explainability initiatives aim to make AI decision-making processes transparent. Robbins critiques these methods as insufficient, arguing that machines cannot possess genuine moral agency or provide meaningful justifications for their outputs. Their reasoning is inherently limited to patterns in data rather than ethical deliberation, making them ill-suited for decisions with moral implications.

Human-Centered MHC

This approach emphasizes the role of humans in overseeing and validating machine outputs, often through frameworks like “human-in-the-loop” or “human-on-the-loop.” While this method aims to maintain oversight, Robbins highlights its flaws, including automation bias and reduced situational awareness. Humans may become over-reliant on machine outputs, undermining their ability to critically evaluate decisions. This overreliance can erode human judgment, especially in complex scenarios where machine recommendations are seen as infallible.

Why machines shouldn’t handle evaluative outputs

Robbins offers three core reasons why machines should not produce evaluative outputs. Firstly, machines often operate in contexts where their outputs cannot be verified for accuracy or effectiveness. For example, hiring algorithms that recommend candidates based on ambiguous “ideal traits” lack measurable efficacy, making their judgments unreliable. Such outputs risk perpetuating biases and failing to consider nuanced human factors.

Secondly, decisions with ethical implications, such as sentencing or hiring, require moral accountability. Machines cannot possess the autonomy or understanding necessary for moral agency, rendering their judgments ethically invalid. Delegating such decisions to AI absolves humans of responsibility, creating a dangerous accountability vacuum.

Lastly, delegating evaluative decisions to machines risks ceding control over the criteria that shape societal values. As machines influence outcomes, they may inadvertently alter human norms and priorities. For instance, an AI system prioritizing efficiency over fairness could reshape how organizations value these principles, leading to long-term societal consequences.

Recommendations for retaining human authority

To address these challenges, Robbins advocates for a proactive approach to decision delegation. He calls for clear guidelines on which tasks can and cannot be entrusted to machines. By limiting AI to descriptive outputs, humans can maintain control over evaluative judgments, ensuring accountability and ethical integrity. Robbins also emphasizes the need for interdisciplinary collaboration between technologists, ethicists, and policymakers to establish robust frameworks for AI governance.

Robbins further recommends prioritizing education and training for those who interact with AI systems. By equipping users with the skills to critically assess machine outputs, organizations can reduce the risk of automation bias and ensure that human oversight remains meaningful.