̽��ֱ�� of Cambridge - Haydn Belfield

Aim policies at ‘hardware’ to ensure AI safety, say experts

fpjl2 — Wed, 14 Feb 2024 11:28:30 +0000

Chips and datacentres – the “compute” driving the AI revolution – may be the most effective targets for risk-reducing AI policies, according to a new report.

Risky business

zs332 — Thu, 07 Sep 2023 15:16:50 +0000

Launched during Open Cambridge, a new self-guided trail, created by researchers at Cambridge’s Centre for the Study of Existential Risk (CSER), takes the public on an altogether different tour of the city.

Community of ethical hackers needed to prevent AI’s looming ‘crisis of trust’

fpjl2 — Fri, 10 Dec 2021 09:55:23 +0000

̽��ֱ��Artificial Intelligence industry should create a global community of hackers and 'threat modellers' dedicated to stress-testing the harm potential of new AI products in order to earn the trust of governments and the public before it’s too late.

This is one of the recommendations made by an international team of risk and machine-learning experts, led by researchers at the ̽��ֱ�� of Cambridge’s Centre for the Study of Existential Risk (CSER), who have authored a new 'call to action' published in the journal Science.

They say that companies building intelligent technologies should harness techniques such as 'red team' hacking, audit trails and 'bias bounties' – paying out rewards for revealing ethical flaws – to prove their integrity before releasing AI for use on the wider public.

Otherwise, the industry faces a 'crisis of trust' in the systems that increasingly underpin our society, as public concern continues to mount over everything from driverless cars and autonomous drones to secret social media algorithms that spread misinformation and provoke political turmoil.

̽��ֱ��novelty and 'black box' nature of AI systems, and ferocious competition in the race to the marketplace, has hindered development and adoption of auditing or third party analysis, according to lead author Dr Shahar Avin of CSER.

̽��ֱ��experts argue that incentives to increase trustworthiness should not be limited to regulation, but must also come from within an industry yet to fully comprehend that public trust is vital for its own future – and trust is fraying.

̽��ֱ��new publication puts forward a series of 'concrete' measures that they say should be adopted by AI developers.

“There are critical gaps in the processes required to create AI that has earned public trust. Some of these gaps have enabled questionable behavior that is now tarnishing the entire field,” said Avin.

“We are starting to see a public backlash against technology. This ‘tech-lash’ can be all encompassing: either all AI is good or all AI is bad.

“Governments and the public need to be able to easily identify the trustworthy, the snake-oil salesmen, and the clueless,” Avin said. “Once you can do that, there is a real incentive to be trustworthy. But while you can’t tell them apart, there is a lot of pressure to cut corners.”

Co-author and CSER researcher Haydn Belfield said: “Most AI developers want to act responsibly and safely, but it’s been unclear what concrete steps they can take until now. Our report fills in some of these gaps.”

̽��ֱ��idea of AI 'red teaming' – sometimes known as white-hat hacking – takes its cue from cyber-security.

“Red teams are ethical hackers playing the role of malign external agents,” said Avin. “They would be called in to attack any new AI, or strategise on how to use it for malicious purposes, in order to reveal any weaknesses or potential for harm.”

While a few big companies have internal capacity to “red team” – which comes with its own ethical conflicts – the report calls for a third-party community, one that can independently interrogate new AI and share any findings for the benefit of all developers.

A global resource could also offer high quality red teaming to the small start-up companies and research labs developing AI that could become ubiquitous.

̽��ֱ��new report, a concise update of more detailed recommendations published by a group of 59 experts last year, also highlights the potential for bias and safety “bounties” to increase openness and public trust in AI.

This means financially rewarding any researcher who uncovers flaws in AI that have the potential to compromise public trust or safety – such as racial or socioeconomic biases in algorithms used for medical or recruitment purposes.

Earlier this year, Twitter began offering bounties to those who could identify biases in their image-cropping algorithm.

Companies would benefit from these discoveries, say researchers, and be given time to address them before they are publicly revealed. Avin points out that, currently, much of this “pushing and prodding” is done on a limited, ad-hoc basis by academics and investigative journalists.

̽��ֱ��report also calls for auditing by trusted external agencies – and for open standards on how to document AI to make such auditing possible – along with platforms dedicated to sharing “incidents”: cases of undesired AI behavior that could cause harm to humans.

These, along with meaningful consequences for failing an external audit, would significantly contribute to an “ecosystem of trust” say the researchers.

“Some may question whether our recommendations conflict with commercial interests, but other safety-critical industries, such as the automotive or pharmaceutical industry, manage it perfectly well,” said Belfield.

“Lives and livelihoods are ever more reliant on AI that is closed to scrutiny, and that is a recipe for a crisis of trust. It’s time for the industry to move beyond well-meaning ethical principles and implement real-world mechanisms to address this,” he said.

Added Avin: "We are grateful to our collaborators who have highlighted a range of initiatives aimed at tackling these challenges, but we need policy and public support to create an ecosystem of trust for AI.”

A global hacker 'red team' and rewards for hunting algorithmic biases are just some of the recommendations from experts who argue that AI faces a 'tech-lash' unless firm measures are taken to increase public trust.

We need policy and public support to create an ecosystem of trust for AI

Shahar Avin

Getty

Human face in the algorithm

̽��ֱ��text in this work is licensed under a Creative Commons Attribution 4.0 International License. Images, including our videos, are Copyright © ̽��ֱ�� of Cambridge and licensors/contributors as identified. All rights reserved. We make our image and video content available in a number of ways – as here, on our main website under its Terms and conditions, and on a range of channels including social media that permit your use and sharing of our content under their respective Terms.

Yes

Opinion: Climate change, pandemics, biodiversity loss – no country is sufficiently prepared

Anonymous — Fri, 01 Nov 2019 14:53:32 +0000

There’s little that the left and the right agree on these days. But surely one thing is beyond question: that national governments must protect citizens from the gravest threats and risks they face. Although our government, wherever we are in the world, may not be able to save everyone from a pandemic or protect people and infrastructure from a devastating cyberattack, surely they have thought through these risks in advance and have well-funded, adequately practiced plans?

Unfortunately, the answer to this question is an emphatic no.

Not all policy areas are subject to this challenge. National defence establishments, for example, often have the frameworks and processes that facilitate policy decisions for extreme risks. But more often than not, and on more issues than not, governments fail to imagine how worst-case scenarios can come about – much less plan for them. Governments have never been able to divert significant attention from the here and happening to the future and uncertain.

A recent report published by Cambridge ̽��ֱ��’s Centre for the Study of Existential Risk argues that this needs to change. If even only one catastrophic risk manifests – whether through nature, accident or intention – it would harm human security, prosperity and potential on a scale never before seen in human history. There are concrete steps governments can take to address this, but they are currently being neglected.

̽��ֱ��risks that we face today are many and varied. They include:

Tipping points in the environmental system due to climate change or mass biodiversity loss.
Malicious, or accidentally harmful, use of artificial intelligence.
Malicious use of, or unintended consequences from, advanced biotechnologies.
A natural or engineered global pandemic.
Intentional, miscalculated, or accidental use of nuclear weapons.

Each of these global catastrophic risks could cause unprecedented harm. A pandemic, for example, could speed around our hyper-connected world, threatening hundreds of millions – potentially billions – of people. In this globalised world of just-in-time delivery and global supply chains, we are more vulnerable to disruption than ever before. And the secondary effects of instability, mass migration and unrest may be comparably destructive. If any of these events occurred, we would pass on a diminished, fearful and wounded world to our descendants.

So how did we come to be so woefully unprepared, and what, if anything, can our governments do to make us safer?

A modern problem

Dealing with catastrophic risks on a global scale is a particularly modern problem. ̽��ֱ��risks themselves are a result of modern trends in population, information, politics, warfare, technology, climate and environmental damage.

These risks are a problem for governments that are set up around traditional threats. Defence forces were built to protect from external menaces, mostly foreign invading forces. Domestic security agencies became increasingly significant in the 20th century, as threats to sovereignty and security – such as organised crime, domestic terrorism, extreme political ideologies and sophisticated espionage – increasingly came from inside national borders.

Unfortunately, these traditional threats are no longer the greatest concern today. Risks arising from the domains of technology, environment, biology and warfare don’t fall neatly into government’s view of the world. Instead, they are varied, global, complex and catastrophic.

As a result, these risks are currently not a priority for governments. Individually, they are quite unlikely. And such low-probability high-impact events are difficult to mobilise a response to. In addition, their unprecedented nature means we haven’t yet been taught a sharp lesson in the need to prepare for them. Many of the risks could take decades to arise, which conflicts with typical political time scales.

Governments, and the bureaucracies that support them, are not positioned to handle what’s coming. They don’t have the right incentives or skill sets to manage extreme risks, at least beyond natural disasters and military attacks. They are often stuck on old problems, and struggle to be agile to what’s new or emerging. Risk management as a practice is not a government’s strength. And technical expertise, especially on these challenging problem sets, tends to reside outside government.

Perhaps most troubling is the fact that any attempt to tackle these risks is not nationally confined: it would benefit everyone in the world – and indeed future generations. When the benefits are dispersed and the costs immediate, it is tempting to coast and hope others will pick up the slack.

Time to act

Despite these daunting challenges, governments have the capability and responsibility to increase national readiness for extreme events.

̽��ֱ��first step is for governments to improve their own understanding of the risks. Developing a better understanding of extreme risks is not as simple as conducting better analysis or more research. It requires a whole-of-government framework with explicit strategies for understanding the types of risks we face, as well as their causes, impacts, probabilities and time scales.

With this plan, governments can chart more secure and prosperous futures for their citizens, even if the most catastrophic possibilities never come to pass.

Governments around the world are already working towards improving their understanding of risk. For example, the United Kingdom is a world leader in applying an all-hazard national risk assessment process. This assessment ensures governments understand all the hazards – natural disasters, pandemics, cyber attacks, space weather, infrastructure collapse – that their country faces. It helps local first responders to prepare for the most damaging scenarios.

Finland’s Committee for the Future, meanwhile, is an example of a parliamentary select committee that injects a dose of much-needed long-term thinking into domestic policy. It acts as a think tank for futures, science and technology policy and provides advice on legislation coming forward that has an impact on Finland’s long-range future.

And Singapore’s Centre for Strategic Futures is leading in “horizon scanning”, a set of methods that helps people think about the future and potential scenarios. This is not prediction. It’s thinking about what might be coming around the corner, and using that knowledge to inform policy.

But these actions are few and far between.

We need all governments to put more energy towards understanding the risks, and acting on that knowledge. Some countries may even need grand changes to their political and economic systems, a level of change that typically only occurs after a catastrophe. We cannot – and do not have to – wait for these structural changes or for a global crisis. Forward-leaning leaders must act now to better understand the risks that their countries face.

Gabriel Recchia, Research Associate, Winton Centre for Risk and Evidence Communication, and Haydn Belfield, Research Associate, Centre for the Study of Existential Risk.

This article is republished from ̽��ֱ��Conversation under a Creative Commons license. Read the original article.

Two Cambridge risk researchers discuss how national governments are still stuck on "old problems", and run through the things that should be keeping our leaders awake at night.

Risks arising from the domains of technology, environment, biology and warfare don’t fall neatly into government’s view of the world

Gabriel Recchia and Haydn Belfield

Markus Spiske

Banner from a climate strike in Erlangen, Germany

Yes