If you ever took philosophy class or you’re interested in ethical quandaries, you may know about the trolley problem. It’s a hypothetical situation where a runaway trolley is headed down the tracks, and you’re standing at the switch. If you do nothing, the trolley will hit and kill five people standing on the tracks. If you pull the switch, the trolley is diverted onto a spur … where it will kill a single person. This make-believe dilemma (and its many variants) is used to explore how humans approach life-and-death decisions.
Philosophy class aside, this same old ethical puzzle now regularly comes up regarding autonomous vehicles. The premise is that self-driving machines will encounter similar challenges: if a vehicle can’t stop in time, would it be acceptable to avoid a crowded intersection by steering towards a single pedestrian? Here’s the latest I’ve read: using ChatGPT to “solve” 50,000 trolley problems. And I call bullshit.
The trolley problem is easy to understand because it seems similar to a self-driving car scenario. But there are some fundamental issues with it being applied to autonomous vehicles.
Philosophy, not reality
Autonomous car “morality” is the wrong application of a philosophical thought experiment. The original trolley problem (and all the ways it has been reskinned) has been used to debate different philosophical schools of thought such as utilitarianism. It’s a tool for understanding the human mind, that is often criticized for being too simplistic and too extreme to reflect any true ethical consideration. This “test” is not useful for deciding what someone would do in reality.
While we may all feel there are “right” and “wrong” solutions to a hypothetical trolley problem, it’s because it seems to have logical appeal when weighing the impact of one death versus five. No matter what people say, utilitarianism goes out the window in true-to-life situations. If the person on the other track is someone you know or are related to; in which case you might sacrifice the many to preserve the (loved) one.
Unreasonable answers
The central conceit of people advocating or suggesting that autonomous cars should include a “trolley test” is that an OEM would in fact implement it. In other words, it assumes that an automaker would purposefully add a function to their vehicles that would scan the road ahead and make decisions about which lives are expendable and which are not in an emergency situation. And they would be morally justified to add this function.
It seems almost inconceivable that anyone would reduce human life to a formula. Towards which group would you target your car if the brakes suddenly stopped working?
- Four teenagers
- Three middle-aged people
- Two elderly men
- A pregnant woman
There is no possible way to create a “maximum utility” function that isn’t biased or unfair, or cannot be effectively argued against. There is no acceptable answer, and any attempts to start justifying the targeting of one group over another becomes inherently problematic. The ethical guilt, the brand damage, and the lawsuit potential all guarantee that no company – let alone no thinking, feeling, or empathetic being – would want to create cars that are constantly deciding which people in front of them are acceptable kill targets.
Who watches the watcher?
We train AIs to drive by feeding it hundreds of thousands of examples. We teach the car to drive “by extrapolation” so it can handle situations that may not be precisely like how it was trained. Very different from an approach encoded in logic and decision trees, this process works very much like human reasoning. In other words, it’s a poor fit for the cold, calculating logic of the trolley problem.
So when cars are trained like humans are, why should we expect an autonomous car to operate any differently than humans do when placed into demanding and unsolvable dilemmas they’ve never encountered? It will attempt to weigh the choices in front of it, and it may make mistakes or it may freeze. That’s not only expected, but it’s okay. Similar to a human driver placed into an unwinnable trolley situation, we should hope that the car cannot easily decide what to do. (We should really start to worry if it’s able to easily and confidently select which humans it should kill.)
The only way around this is to build a “morality engine” guardian that constantly second-guesses the autonomous drive system and overrides it if a “lesser death” path can be detected. But who decides when it should kick in, what values it should use, how to make it unbiased, how it is built, and how it should be tested? It doesn’t matter how deep you stack the watchers, the problem itself is unsolvable.
At what price safety?
It’s great for us all to want the safest autonomous vehicles. But at what price? We certainly don’t want to start imbuing our cars with moral calculators that say which lives are more valued.
We can stop kidding ourselves – humans don’t have a special ability to make these kinds of life-or-death calls either. When the trolley problem is put to the test in real-world scenarios where participants are unaware the situation is a test, most people actually freeze and do nothing. It turns out that it’s easier to find a rationale that lets you justify the sacrifice of five people rather than be an active killer of one.
Being a philosophical experiment, the trolley problem should not expect to deliver a reasonable answer when applied to real-world dilemmas. Let’s stop using it as one.