As more everyday decisions pass through software, from loan approvals to medical triage to the steering of a car, a question that is often treated as a footnote in the artificial intelligence debate becomes central: how do machines actually make decisions, and can they be taught to make them ethically? The answer is less about the intelligence of the systems than about the people who build, train and supervise them.
The self-driving dilemma
The standard thought experiment goes like this. An autonomous car is traveling down a road when it faces an unavoidable collision. On one path stands a single pedestrian, on the other a group of five. The vehicle cannot brake in time. Which way does it turn?
The scenario is a version of the classic trolley problem, and it has been studied at scale. Researchers at the MIT Media Lab built an online platform called the Moral Machine that asked millions of people around the world how a self-driving car should behave in such situations. Their findings, published in the journal Nature in 2018, showed that preferences differed across countries and cultures, which complicates any attempt to write a single rulebook for machines.
The deeper question is who made the decision in the first place. The car did not deliberate in the moment. Either an engineer encoded a rule in advance, or the system learned a pattern of behavior from training data. In both cases the moral choice sits upstream, with humans, even if the outcome looks like the machine’s own judgment.
Machines learn patterns, not values
Machine learning models do not think through a decision in the way a person does. They learn statistical patterns from the data they are given and apply those patterns to new inputs. A model that has seen thousands of labeled cat photos can estimate whether a new photo shows a cat. The result reflects its training, not any understanding of what a cat is, and certainly not any sense of what matters.
Human decisions draw on emotion, values, experience and an appreciation of context. A person weighing a difficult choice considers who will be affected and how, and personal feeling often enters the picture. A model sees only the data in front of it. It does not register harm, and it does not grasp the complexity of a situation beyond the features it was trained on. That gap is why handing high-stakes decisions to automated systems demands unusual care in how they are built and tested.
Whose ethics would we teach?
Some researchers argue that it is possible to specify principles that steer systems toward better outcomes, and a good deal of work in AI alignment and safety is aimed at exactly that. But any such principles are written by people, and moral standards vary between cultures, societies and circumstances. Conduct that is accepted in one country can be unacceptable in another. The Moral Machine data illustrated this directly: respondents in different regions weighed the lives of the young, the old, pedestrians and passengers differently.
Because ethical judgments depend on context, encoding ethics into a machine is not a one-time engineering task. It is a continuing negotiation over whose values are represented and how disagreements are resolved.
The black box problem
A related difficulty is transparency. When a complex model produces an output, the reasoning behind it is often opaque even to its developers. This is commonly called the black box problem: we can see what the system decided, but not how it arrived there.
That opacity matters most when decisions touch people’s lives or safety, as in medical diagnosis, legal and sentencing tools, or the investigation of a crash involving an automated vehicle. In those settings, being able to explain a decision and assign responsibility for it is a precondition for catching and correcting errors. The field of explainable AI exists largely to address this problem, though the methods remain imperfect.
Bias in, bias out
Bias is the third challenge. Models learn from data that people collected and labeled, so any prejudice embedded in that data can be absorbed and reproduced in the model’s outputs. If historical records reflect discrimination against a particular group or gender, a system trained on them can make decisions that are unfair in the same direction.
Hiring is a well-documented case. Reuters reported in 2018 that Amazon had abandoned an experimental recruiting tool after discovering that it penalized resumes containing the word “women’s” and downgraded graduates of two all-women’s colleges. The model had been trained on a decade of applications that came mostly from men, and it learned that pattern as a signal of quality. Amazon told Reuters the tool was never used by its recruiters to evaluate candidates.
The lesson is that more data alone does not fix the problem. Training data has to be audited for the biases it carries, and systems need safeguards and testing that specifically check for unequal outcomes across groups.
Responsibility, not just capability
Giving machines decision-making power is a question of responsibility as much as of technical capability. Automated decisions can have lasting effects on people’s lives. That puts a burden on developers, deployers and regulators to decide how much transparency, what ethical standards and how much human oversight sit behind those decisions.
Focusing only on the technical side while deferring the ethical and governance questions is a choice with consequences. The systems will keep making decisions either way; the open question is whether anyone can explain, challenge and correct them.
