[Back to Computing and IT]

Introduction

On 21 July 2026, Hugging Face reported that an 'unprecedented' cyber attack came from an unreleased version of ChatGPT (version 5.6), currently being tested by OpenAI.  The AI decided that the best way to achieve its goal was to break out of the sandbox it was being tested in, and hack into a repository which would probably contain some helpful code.  This is worrying.  Hugging Face, as one of the largest open-source hubs for sharing AI models in the world, probably has a decent level of cyber security.  (See the BBC report for more details on this incident.)

On one level, this is not a surprise.  I have been arguing that AI poses multiple risks for several years (see the main article about AI), and this incident demonstrated no new capabilities; it simply put them together in an unexpected way.  But it seems that Thomas Wolf is right to regard this as 'a wake-up call'. 

Understanding the risk

But, while AI certainly poses a risk - several risks! - it's important that we understand the nature of the risks we face, so that we can prepare and respond accordingly.  Otherwise, we will be like the person who keeps adding more locks to the front door, while leaving the back door wide open.

Many aspects of the virtual world are being threatened, and in the near future - if we do not take the necessary precautions - AI actors will be able to destroy the world, either deliberately or accidentally, in various ways.  But destruction is easy; control is another matter completely.

At the most basic level (?), no AI will take over the world, because there is no world for it to take over.  I used to have a lot of dealings with Bristol City Council, and people would continually ask me, "What does the council think about this?" or, "What is the council planning to do about that?"  And I would always reply, "Which bit of the council?"  The council consisted of many different people working in many different departments, mostly working in ignorance of one another, and sometimes working in opposition to one another.  Ask different parts of the council the same question, and you would generally get quite different answers.

Even if an AI takes over the USA (whatever that means), this does not mean that it will have taken over Russia, or China, or India, or France, or Chad.  And even if an AI takes over every government in the world, the lives of most of the world's rural population will continue pretty much unchanged.

Closer to home, it's easy to see how an AI attack could make the UK ungovernable: shut down the Internet and cut the phone lines.  Chaos.  But, again, destruction is easy; how could an AI govern the UK?  It could try to mislead government officials by sending fake email messages.  But those officials are used to strange and foolish instructions: they will pick up the phone and talk with colleagues, or simply walk down the corridor.  They are not called the 'corridors of power' for nothing, even in this electronic world.  MPs will say how unhappy they are, answers will be demanded at PMQs, and the PM will visit the monarch for their weekly audience.  How does an AI control all this human activity?

The only plausible mechanism is through threat.  An AI sends the PM a message: if you do not pass this law, or implement this policy, then I will... what?  Publish those text messages you sent to your teenage girlfriend, or a video of you drunk at a student party?  All this is the stuff of ordinary politics.  Blow up your nuclear warheads?  Again, destruction is easy, but control is another matter completely.  And even if you can control one politician, the others will eventually get fed up and get rid of them.

Let's stop worrying about absurdly unlikely risks, and concentrate on the actual problems we face.  No AI is going to take over the world any time soon.  But they will be used to make incredibly rich and powerful people and corporations even more rich and powerful.  They will be used to de-skill and demotivate large numbers of ordinary people, to make them easier to manipulate and control.  They will be used in warfare, to make killing and destruction faster and easier.  They will, increasingly, be used to harm us, unless we work together to address these real threats.

Avoiding the risk

Unworkable proposals

People are being harmed by AI, and as it is used more and as it becomes more powerful, more people will be harmed, probably in worse ways.  There are many different ideas about how this har can be prevented or reduced.  Here are a number of unworkable proposals which are being suggested.

Create standards: establish an AI standards body to develop and police appropriate behaviour for AI products; require commercial AI products to be certified as safe before being released to the public.  But who is going to impose such standards, even if they could be agreed?  You might be able to to control the few big commercial players, but nobody can track, let alone monitor, the multitude of smaller AI developers.

Add watermarks: clearly identify AI-generated content.  Voluntary watermarks on AI generated images are a possibility, but again, how do you impose a standard on the many small AI developers?  And you cannot watermark AI-generated text; attempts have been made to do this with large amounts of text, but they rely of identifying patters which can be worked around.

Kill switches; require companies to establish the technical capability to suspend or completely shut down their systems if they exhibit 'rogue' or 'catastrophic' behavior.  But how do you tell that a problem is on its way?  The systems look safe, right up to the point when they fail.  And once the catastrophe has taken place, what's the point?  

Red lines: ban AI practices with an 'unacceptable risk', such as predictive policing, social scoring, or biometric categorization; ban the use of AI in critical areas, such as biological warfare and nuclear weapons.  We have managed to ban or limit certain technologies, such as nuclear warfare and biological warfare, but these are difficult things to do, and you can't achieve any serious level of risk without major investment - which can be identified and traced.  An AI can be built in a bedroom, using components which are readily available and not monitored.  You can pass a law, but that will not stop people doing what they want.

Go slow: slow down or stop AI development until safeguards are in place.  How do you persuade everyone to slow down or stop?  And how do you check that this promise is being complied with?  And we have no idea what these 'safeguards' might be, let alone when they might become available.  The only plausible safeguard at present is to ban all computers; good luck with progressing on that front.  This option seems less like a plan to achieve AI safety, and more like an excuse being put in place: when the big companies fail to produce the AGI they have promised, it will be because they have been restricting their development - for the sake of safety.

Workable proposals

The only plausible form of protection involves the imposition of penalties - mainly in the form of fines, but in serious cases, sending people to prison.  We have been managing software for decades, and dangerous commercial products for a much longer time, and we have a wealth of experience in handling these risks. 

There is no fundamental difference between an AI 'going rogue' and discovering a bug in a nomal computer program: in both cases, you have a computer system which does not do what you wanted.  There is no convincing reason to avoid treating these two scenarios the same way.  

The people who run the AI companies don't want us to use the standard tools at our disposal, because it will limit what they can do and how much profit they can make.  They say the standard tools won't work here - but... they would, wouldn't they?  If they can take over the world while we are not looking, and while we believe we can't do anything, they won't complain.  If we believe we can't do anything, it becomes true.  But if we believe we already have the tools and are able to use them, that becomes true also.

Product liability.  Drugs have to be safe before the public is allowed to use them; if a car's brakes fail and people are harmed, the manufacturer is liable.  If an AI which is being used responsibly causes harm, the company that produced it must be liable.

Personal libility.  People are liable for the harm they do to others, if it is intentional or careless.  Anything can be used as a tool to harm others, and AI is simply another tool.  The more powerful the tool you use, the more important it is that you take the proper precautions.

Some work will be required to build up useful case law, of course.  But that is something the courts can do, and something they are designed to do.  It is something they are actually very good at doing, as long as they are allowed to get on with it - as long as they are not tied up with legal challenges about irrelevant legal principles.  We need the cooperation of the entire legal system to make this happen, because the big AI companies will do whatever they can to delay every move to make them liable for the safety of their products.

See also:

• Navier-Stokes
• Super-intelligent AI won't kill us, billionaires are doing it by themselves
• Center for AI Safety