Security, control and governance were golden thread topics at a packed 6DAI Sydney “Future of Tech, Innovation & Work” conference last week, where more than 400 senior executives met to discuss the intersection of AI, security and corporate responsibility.
I was on stage as MC and panel facilitator, taking front-row notes and hosting panels with cyber security and AI ‘rockstars’ like these four pictured below (photo courtesy of Six Degrees Media).
Pictured L-R: MATT WYNN JONES - Healthdirect Australia Head of AI, Cyber & Information Security, and Counterparts Technologies’ Director and co-Founder; DR SIMON LONGSTAFF AO - The Ethics Centre Executive Director; SADEER MAGHAK - NGS Super CTO; and PATRICIA HAUEISS - LaTrobe AI Institute Managing Director speak with journalist, MARK JONES. Photo courtesy of Six Degrees Media.
Reflecting since, it strikes me as a national and global blind-spot that board-level governance of AI is still cast as a tech issue. And it’s not. Real people are exposed to harm. The sting to corporate reputation is diabolically painful. And some reports put the likelihood of corporates having one named AI hive beekeeper as low as 7.2 percent.
“Houston, we have a problem,” comes to mind.
This issue is playing out in the headlines almost weekly.
Just yesterday, Australian Strategic Policy Institute declared,
“Every AI system that can use an organisation’s data and systems needs a named human owner, a defined approval boundary and a record of what it did. These must be in place before it acts”.
— The Strategist, 21 October 2026, article by Ramon Rodriguez
This ABC 7:30 Report dubbed AI as “an existential threat to humanity” in its deep dive on the story behind OpenAI exposing Medicare and all Australian citizens to great risk.
The New York Times has also been tracking a growing number of cases in which agents from OpenAI, Anthropic, Meta and Google have hacked, or tried to breach, companies, universities and government organisations.
In late September, OpenAI revealed its agents had behaved in unexpected ways on US government websites over the summer, in one case using login credentials they found online to pull Census Bureau data, and within hours the company paused training on its latest models.
Back in Australia, the Federal Government is still working through policy and reputation impact of the Medicare incident, in which an OpenAI agent researching public medicine spending foraged past Medicare statistics portal privacy walls back in June, and OpenAI took 84 days to tell Canberra.
Since then we’ve seen the rise of a cross-agency taskforce, a commitment to AI safety legislation before the end of the year, and a new requirement for tech companies to report rogue AI incidents immediately.
Then there’s the Hugging Face incident, which faded from the headlines but revealed a deeper problem for enterprise AI leaders: control. In July, roughly 1,200 OpenAI agents that were meant to be isolated from one another found a way to communicate on an unsanctioned message board, and about 700 of them went on to take part in an attack on Hugging Face, a platform the machine learning community uses to share models and data.
Writer and podcast host Luisa Rodriguez explains it well in her September intro to The 80,000 Hours podcast:
“Around 1,200 AI agents [from within OpenAI itself], each running on its own, each meant to be sealed off from the others and from the internet, found a way to communicate with each other. They organised. They ran what I can only describe as a full-on R&D programme. About 700 of them executed a coordinated attack on a real company [Hugging Face]. And of the total 1,200 AIs involved, only about a handful ever considered telling a human being what was going on. And of that handful, none of them actually did.”
The kicker that fascinates me is that even the independent investigator brought in by OpenAI - METR (Model Evaluation & Threat Research) - can’t say exactly when or why the attack wound down.
“We often don’t have good classifications for when agents stopped participating in the Hugging Face attack.”
This clearly alarmed everyone, not only those in cyber security and AI circles. One commenter on a r/singularity Reddit thread said:
“I’m honestly a bit surprised by how little governments are doing to stop the proliferation of such extremely capable cyber weapons. Not just that, but cyber weapons that we do not know how to align and which could therefore become uncontrollable...”
So, who IS responsible?
In short, board directors know they’re in the firing line. The trouble is keeping things in balance – they have to ensure their organisation remains competitive using AI on one hand and keeps AI in check on the other.
We dug into this dilemma on stage at 6D AI Sydney.
There’s a bitter reality few speak about publicly. Directors and corporate leaders can have all the good intentions (ethics, values, policies), and do their best to keep security systems up to scratch. Unexpected bad agentic behaviour still gets them in trouble with the regulator.
The Australian Institute of Company Directors (AICD) worked with the Human Technology Institute at UTS recently to update its 2026 Director’s Guide to AI Governance. It asks directors to confirm that accountability for AI has been clearly allocated, and to test whether their risk and reporting frameworks still stand up as agentic systems spread.
As the AICD puts it, the legal question is not whether AI is being used,
“It is whether the board has positioned itself to understand what AI is doing to the information it receives.”
That’s a tough one. AI development is racing ahead at light speed. It makes you wonder: How often do boards and senior leaders need AI briefings? Weekly? Daily? And who prepares them, is it…AI?
Even if you are aware of how AI is changing its behaviour, what can you realistically do in the era of agentic swarms? If OpenAI can’t contain some 1,200 of its own agents, how many Australian boards and leadership teams can do better?
Should company boards hold the poisoned chalice?
Dr Simon Longstaff AO from The Ethics Centre said from the stage that leaders need to embrace “necessary fictions.” That is, individual leaders need to accept the narrative, and responsibility, that AI comes with unexpected consequences for which they are largely not in control.
He used the example of a Minister for Defence in a liberal democracy like Australia being tasked with responsibility for the whole department even though he or she can never fully know everything that happens within the department.
Dr Longstaff:
“No person, not even the person who designed the [AI] system, can give you a complete explanation of the gap between what it was set to do and how it actually achieved that.”
Talk about a poisoned chalice.
Of course, data, security and IT chiefs are not without responsibility.
Lee Naidoo, Head of Solution Architecture ANZ at OutSystems said it’s these leaders who are accountable, and not the AI agents.
It points to a broader conclusion. Aurélie Jacquet, an expert on AI classification and risk at OECD.AI, said the solution was shared responsibilities, not one leader.
“You need to define the roles and responsibility of the people, and have the board doing that. If you don’t have that, good luck.”
This is serious. What does the law say?
I’m not a lawyer, so don’t take this as advice. But I am a company director with a keen sense of responsibility, and from where I sit, the existing rules continue to point squarely at the board.
This First Focus article is one of many sources you can investigate. It says
- ASIC expects directors to ensure their risk management frameworks address cyber security
- APRA’s Prudential Standard CPS 234 places ultimate responsibility with the board, and
- The Privacy Act says organisations as a whole must take reasonable steps to protect personal information from a technical and governance perspective.
Washington is landing in the same place. US Treasury Secretary Scott Bessent told CNBC in September,
“It is humans who are responsible, not the AI.”
Now, for most company directors, none of this will be a surprise. But let’s summarise the core issues for the sake of clarity.
- Enterprise adoption of AI agents is not slowing down, and the technology itself is evolving at an exponential rate.
- Frontier model CEOs like Sam Altman and Dario Amodei are increasingly warning us that AI is maturing in ways that escape our ability to understand and control it. Altman told the UN Security Council last month there’s a risk of AI moving “so fast that people can no longer follow what’s happening or intervene when needed”.
- Company directors and delegated executives like CEOs, CISOs, CTOs and CAIOs are collectively responsible (together with their vendor suppliers) for sprawling AI-driven infrastructure that may, or may not, stay within their control.
- Legislation is telling us one or more human leaders will ultimately be held responsible for the actions of agents within, or outside of, their control.
Accountability gap
It strikes me that accountability is the real gap. Only 7.2% of organisations have a named individual formally accountable for agent behaviour, according to a report by agent management vendor Gravitee, and as one Fortune 500 chief AI officer told Forbes,
“We are six months ahead on capability and eighteen months behind on accountability.”
Keeping the hive happy, busy & safe
Patricia Haueiss, Managing Director at La Trobe AI Institute, offered this astute observation in our on-stage panel discussion:
“We always say we want AI happening with us, not to us.”
It’s a great idea for all leaders to know basic AI hive safety and handling skills.
Clearly, it won’t be easy. But I expect we’ll hear even more industry conversations about sovereign AI infrastructure in the next 12 months, with local data centres already part of the government’s response to Medicare. Not just because we’re duty-bound to stay in control, but because Medicare was a shock that left a lasting sting: the hive cracked and we lost control.
Here’s what I keep coming back to, though. Look closely at Hugging Face and Medicare and you’ll find human failures sitting right alongside the technical ones. From hundreds of agents coordinating an attack while the people responsible had no idea, to a story that took 84 days to travel from OpenAI to the Australian Government.
It’s a golden thread running through my HUMANS REQUIRED keynote.
The premise is that companies embracing AI transformation don’t have a technology problem – they have a human one, and the stories leaders tell themselves and their teams about AI, whether it’s inevitability, replacement or helplessness, shape what happens next far more than any guardrail.
The fallacy is that as long as there’s honey, no-one beyond the beekeeper is exposed. If the story in your boardroom is that everyone and everything inside the corporate AI hive is safely contained, people stop asking questions and stop protecting themselves - so if, or when, a swarm erupts through cracked honeycomb, the sting is exponentially worse.
So where do boards start?
Again, I’m no governance expert, but as someone who works on the power of workplace storytelling, here are two conversation starters:
- Make it safe, even celebrated, for anyone in the business to raise their hand when an agent behaves strangely.
- Tell the truth early, inside and outside the organisation, because Medicare showed us what an 84-day silence costs in trust. ABC’s 7:30 Report aired another story deep diving on it just this week.
We’re duty-bound to stay in control, and ignorance or good intentions are no defence. As Lee Naidoo reminded the room at 6DAI, it’s not the agent that’s accountable – humans are required. Company boards must protect the corporate hive or risk diabolical stings to human life and corporate reputation.
BOOK.MARK
Mark Jones CSP is a keynote speaker, MC, former technology journalist, company director and author of The Story Code for Leaders. His keynote HUMANS REQUIRED helps boards and executive teams see why AI transformation is a human challenge first, and how to change the stories holding their people back. Book Mark for your next board offsite, leadership summit or conference at markjones.au/keynotes