Automation vs Autonomy
Automation executes a fixed procedure a human specified; autonomy chooses its own actions toward a goal, and the gap is where risk lives.
Two Different Things
Automation is a machine carrying out a predetermined procedure: given these inputs, do exactly these steps. Autonomy is a machine selecting its own actions to pursue a goal under conditions its designers did not enumerate. A thermostat is automated; a system that decides how to achieve an open-ended objective is moving toward autonomy. The difference is who chooses the actions.
A Spectrum, Not a Switch
- Fixed automation: the same steps every time, no choices.
- Conditional automation: predefined responses to predefined situations.
- Adaptive systems: behavior tuned within set bounds.
- Goal-directed autonomy: the system determines means, and sometimes sub-goals, itself.
Why the Distinction Matters
An automated system fails predictably: it does the wrong specified thing, and you can trace it to the specification. An autonomous system can fail in ways no one anticipated, because it chose an action no one foresaw, including actions that satisfy the letter of a goal while violating its intent. Predictability declines as autonomy rises, and so does the difficulty of assurance.
Specification Is the Hard Part
The more autonomy a system has, the more its behavior depends on how completely and correctly its goal was specified. Underspecified goals invite unintended strategies, a machine optimizing exactly what it was told rather than what was meant. Bounding autonomy, constraining the space of allowed actions, is often safer than trying to specify a goal perfectly.
Responsible Deployment
Matching the level of autonomy to the stakes and to how well the situation can be specified is a core design decision. High-consequence settings favor automation with clear human authority over full autonomy, and any autonomous behavior should be bounded, monitored, and reversible. The aim is capability without surrendering the ability to predict and correct what the system does.