How government use of AI could hurt democracy

Many countries are exploring how artificial intelligence might help with everything from processing taxes to determining welfare benefits. But a survey shows citizens are not as enthusiastic as their governments – and this can create real risks for democracy.

“Focusing only on short-term efficiency gains and shiny technology risks triggering public backlash and contributing to a long-term decline in democratic trust and legitimacy,” says Alexander Wuttke at the Ludwig Maximilian University of Munich in Germany.

Wuttke and his colleagues asked around 1200 people in the UK to share their feelings about government actions where either a human or an AI handled the task. These hypothetical scenarios included processing tax returns, approving or rejecting welfare applications and making risk assessments about whether defendants should be eligible for bail.

Some people were only told about how AI could improve government efficiency – but others learned about both AI-related benefits and risks. These risks included difficulty in understanding how AI decisions are made, growing government dependence on AI that becomes less reversible over time and a lack of clear paths for citizens to contest and correct AI decisions.

When people became aware of AI-related risks, they reported both a significant decline in their trust in government and a feeling of losing control. For example, the percentage of participants reporting loss of democratic control in their government increased from 45 per cent to more than 81 per cent in scenarios where the government became increasingly dependent on AI for handling specific tasks.

The proportion of people demanding less AI in government also sharply increased once participants learned about the risks – rising from under 20 per cent in the baseline scenario to more than 65 per cent in any scenario where they learned about both the benefits and risks of government using AI.

Despite these results, democratic governments could make use of AI in responsible ways that retain citizens’ trust, says Hannah Quay-de la Vallee at the Center for Democracy & Technology in Washington DC. But she says there are few success stories of AI in government so far. Meanwhile,  there are already “quite a few failure cases” – and the stakes of these cases can be incredibly high.

For example, US state efforts to automate the processing of public benefits claims have led to tens of thousands of people being wrongly charged for fraud. Some of those people ended up having to file for bankruptcy or lose their homes. “Government mistakes have enormous, long-reaching impacts,” says Quay-de la Vallee.