Artificial intelligence industry experts are expressing increasing concern in recent weeks about the growth of AI, calling on governments to slow down development and increase safety measures. Gov. JB Pritzker announced Tuesday the creation of an Illinois AI Cabinet to assess possible risks.
The main concerns are rogue AI agents, such as the Hugging Face incident in which OpenAI models independently conducted an unauthorized cyberattack.
Anthropic CEO Dario Amodei and OpenAI CEO Sam Altman, the heads of two AI frontier labs, urged world leaders at a U.N. Security Council meeting Wednesday to adopt international AI regulations and mitigate possible risks to humanity.
The Daily Illini spoke with two University professors in computer science about the possible threats posed by AI, proposed regulations and their opinions concerning these claims.
What is the basis for these claims?
Derek Hoiem, professor in Engineering and associate director for AI strategy and programs for the Siebel School of Computing and Data Science, explained that AI processes can now launch other AI processes.
“You can have a whole bunch of agents that are working together and assigning each other tasks and that then go out into the internet,” Hoiem said. 
Hoiem added that there is a fear these agents could hack systems such as hospital servers or grid systems and cause major harm. However, in his opinion, the biggest fear is the intentional use of these tools by government agencies in cybersecurity attacks.
Tal August, professor in Engineering who studies human and AI interaction, added that much of this alarm originates from a lack of evidence about what these agents are doing.
“It really comes down to loss of control of AI,” August said. “The thing that’s coming out the most right now as a concern is not necessarily that we’ve seen evidence that AI has bad intentions … but that we just don’t necessarily know what it’s doing all the time.”
August explained that this has led some to believe that AI could spiral out of human control and take actions that could pose a tangible danger.
“The whole argument is, ‘Well if we’ve lost control … we don’t know why they’re doing anything,’” August said. “Maybe they’ll do something that we don’t want them to do — they could hack into a water treatment facility and shut down the water in a city.”
What could slowing down AI development look like?
Hoiem pointed to Anthropic CEO Amodei’s recent proposal regarding AI safety, in which Amodei wrote he is committed to pacing AI acceleration and wants world governments to require other companies to follow suit. A major part of Amodei’s proposal is to require AI companies to have a third-party evaluator ensure the safety of its models.
Hoiem believes this method of third-party intervention might be a way for companies to avoid consequences, and that other accountability measures should be added.
“In a way, what’s being proposed might allow deflection of responsibility from AI companies that are developing the models,” Hoiem said. “Vetting the safety of these models is itself an extremely challenging research problem.”
Hoiem also mentioned another one of Anthropic’s proposals: to reduce China’s ability to develop AI through methods such as limiting AI chip distribution.
President Donald Trump has stressed the importance of competing against China in AI development and has called many recent AI safety warnings a “hoax.” However, Trump announced Sept. 19 the creation of an AI task force, and agreed Saturday to create a channel between the U.S. and China for AI-related incidents.
August talked about another restriction mentioned in Anthropic’s proposal — limiting the research into AI systems that can code new AI.
“A proposal could be to limit the research of recursive self-improvement or to limit the ability of coding agents to create code to train more AI models,” August said. 
Hoiem said implementation of any of Anthropic’s proposed restrictions would be difficult to observe, and August said it would largely impact frontier labs and likely not change everyday usage of current AI models.
Where do you stand?
August said that while he doesn’t necessarily believe all the claims being made or that AI development has to pause in the way frontier labs are calling for, figuring out how to move forward is a reasonable approach.
“Slowing down isn’t a bad thing to do,” August said. “I don’t know that I take all the claims at face value, but the truth is the labs have access to technologies and systems and models that the rest of us don’t.”
August added that another approach could be to further research and development into existing AI models, saying that “they’re more powerful than the interactions that we have for them right now allow for.”
Hoiem said that while disaster aversion regarding AI is important, he also thinks the government should do more to broadly distribute the benefits of AI. He said this could look like driving medical research, improving energy distribution and imposing taxes on AI to disincentivize companies from laying off workers.
“I think (the government) shouldn’t just be … putting the brakes on it or trying to regulate it, but also thinking about the impacts and ensuring that it’s a broad positive impact,” Hoiem said. 
 
[email protected]