One-third of UK councils are using computer algorithms to make crucial decisions about benefit claims and other welfare matters, despite knowing some of the systems are unreliable.
An investigation by The Guardian found that 140 councils out of 408 have invested in software contracts with private companies.
With local authorities under pressure to save money after government cuts, councils are increasingly turning to automated systems to make decisions on benefit claims, protecting vulnerable children and allocation of school places.
The Week
Escape your echo chamber. Get the facts behind the news, plus analysis from multiple perspectives.
SUBSCRIBE & SAVE
Sign up for The Week's Free Newsletters
Join more than 350,000 subscribers and keep yourself informed with a selection of The Week’s most interesting, enlightening and entertaining stories - plus daily puzzles.
Join more than 350,000 subscribers and keep yourself informed with a selection of The Week’s most interesting, enlightening and entertaining stories - plus daily puzzles.
Latest Videos FromThe Week
Firms awarded lucrative contracts to provide the decision-making software include Palantir, a data-mining firm co-founded by billionaire Trump backer Peter Thiel, and American credit-rating giants Experian and TransUnion.
However, there is evidence that the algorithms are not always reliable. North Tyneside council used TransUnion to check housing and council tax benefit claims, welfare payments to an unknown number of people were delayed when the computer’s “predictive analytics” incorrectly identified low-risk claims as high risk. The local authority has since stopped using the system.
David Spiegelhalter, a former president of the Royal Statistical Society, said there is “too much hype” about machine learning systems among local authorities. He called for councils to “demand trustworthy and transparent explanations of how any system works, why it comes to specific conclusions about individuals, whether it is fair, and whether it will actually help in practice”.
Silkie Carlo, of the campaign group Big Brother Watch, said the growing use of such software was leaving vulnerable people at the mercy of “automated decisions... they have no idea about and can’t challenge”.
There is also concern that council staff themselves might not understand how the systems work. Gwilym Morris, a management consultant who works with IT providers to the public sector, said local authority leaders “don’t really understand what is going on”.
Fears have also been raised about privacy and data security, as well as the lack of transparency for the public to challenge automated decisions on sensitive matters.
The role of technology in benefit claims has been a subject of controversy for some time. Sky News reported in July that the new universal credit system pushed claimants towards an online portal whose “harsh, inflexible rules showed a computerised system built to say ‘no’”, wrote technology correspondent Rowland Manthorpe.
–––––––––––––––––––––––––––––––
For a round-up of the most important stories from around the world - and a concise, refreshing and balanced take on the week’s news agenda - try The Week magazine. Get your first six issues for £6–––––––––––––––––––––––––––––––
