Avoiding the next technical debt: Building AI governance before it breaks

The AI rush is repeating a familiar mistake. Early in my career, a risk executive I worked with used to say, “You didn’t invite me to drink the beer; now you want me to pay the bill?” whenever problems came up because a project moved ahead without enough oversight. If someone tried to avoid explaining the details, he’d add, “I don’t know if you’re showing me the monster’s head or just its toe.”

Since 2011, I’ve watched new products, business services and innovations launch without enough security or risk checks. Cloud computing, big data, BYOD, APIs, IoT, social media and low-code are just a few examples. We usually innovate first and worry about governance later.

AI is following the same pattern. Leaders in many industries are excited about AI, just like they were with earlier technologies. But many still don’t have a clear way to track where AI is used, who owns the risks or how automated decisions could affect the business.

Ten years of fail fast have shown us the risks: more incidents, data breaches and bigger exposure. If organizations don’t build risk and accountability into AI now, they’ll face the same problems we saw with earlier innovations.

The real risk isn’t AI itself, it’s how we use it

Even with detailed frameworks like the MIT AI Risk Repository, many organizations still struggle to connect AI risks to real business problems. Everyone wants new use cases, but few are tracking where risks begin — in the data, the models or the quick decisions machines make.

In fact, AI risks aren’t just about the future — they’re already part of daily operations. These risks arise when algorithms affect business results without clear accountability, when tools collect sensitive data and when automated systems make decisions that people no longer check.

These governance gaps aren’t new. We saw the same issues with cloud, APIs, IoT and big data. The solution is also familiar: keep track, assess, control and monitor. The first step is knowing where AI is used, what data it handles and which processes it touches. With this visibility, governance becomes about managing what’s already in the business, not just fearing the unknown.

The next step is protection. We don’t need to reinvent the wheel or develop advanced new methods. Instead, we should start with the basics: with simple governance steps and then you can evolve your journey.

Borrow what already works

The good news is companies don’t have to start from scratch with AI governance. Guidelines for secure and compliant technology already exist in cybersecurity, cloud and privacy programs.

What’s needed is to apply traditional controls to this new context:

• Classification and ownership. Every model should have a clear owner, with limits on who can train, query or deploy it. Its relevance to the business should be clear by different criteria, such as regulatory, operational or revenue.
• Baseline security and non-negotiables. Access control, multifactor authentication, network segmentation and audit logging are just as important for AI environments as they are for servers or clouds.
• Continuous monitoring. Model behavior should be more than just accurate — it should be observable, traceable and accountable for any changes in purpose.
• Third-party due diligence. Contracts with AI providers should clearly define rights over training data, generated content and how to respond to incidents.
• Testing and validation. Red-teaming, AI-specific penetration testing and scenario simulations should be regular practices.

These controls aren’t new, nor is the hope of avoiding another form of technical debt. Maybe this time we can apply the secure by design approach.

The same governance principles will be tested again soon; this time by a new wave of autonomous systems.

The rise of agent AI and the accountability vacuum

A new generation of agent AI systems can act on their own across different platforms, doing tasks, making purchases or retrieving data without direct human input. This move from simple chatbots to self-directed agents creates an accountability gap that most organizations aren’t ready for.

Without the right guardrails, an agent can access systems it shouldn’t, expose confidential data, create unreliable information, start unauthorized transactions, skip established workflows or even act against company policy or ethics. These risks are made worse by how fast and independently agent AI works, which can cause big problems before people notice.

In the rush to try new things, many companies launch these agents without basic access controls or oversight. The answer is to use proven controls like least privilege, segregation of duties, monitoring and accountability.

Executives should be able to answer fundamental questions, drawn from frameworks such as NIST AI RMF, about any autonomous AI operating in their environment:

1. What governance processes are in place (policies, roles and responsibilities, oversight)?
2. Which use cases and business applicability are being leveraged?
3. Who is accountable when it goes wrong?
4. Which risks does it represent? And which controls are applied?

Building governance into the business, not around it

Effective AI governance isn’t an IT function, any more than cybersecurity is. It’s a business function with shared accountability. Forward-looking organizations are now introducing three mechanisms that embed governance into operations:

1. AI self-assessment frameworks — simple checklists that help each business unit map their AI use cases, data sources and risks.
2. Leverage governance committees — cross-functional bodies with representation from risk, compliance, cybersecurity and business leaders.
3. Corporate AI use policies — defining approved tools, contractual standards and minimum safeguards for both internal and external AI usage.

These aren’t bureaucratic layers but foundations of sustainable innovation. When the business owns the inventory, risk teams can focus on assurance rather than discovery. Modern governance should enable adoption, not inhibit or slow it down, but help scale it safely.

Don’t build another debt

The similarities to cloud adoption are clear. Ten years ago, not having early controls led to exposed data, unmonitored systems and expensive fixes. AI is showing the same pattern, but it’s happening faster and with bigger consequences.

Technical debt isn’t just about code anymore. It’s also about trusting your data, holding models accountable and protecting your brand’s reputation.

The organizations that succeed with AI will be the ones that see governance as part of the design process, not as something that causes delays. They’ll move forward with clear plans and measure value and risk together.

They’ll see that real innovation isn’t just about building smarter systems but about making them safe, accountable and trusted from the start. For technology and business leaders, this isn’t just a security imperative. It’s a strategy for sustainable innovation.

This article is published as part of the Foundry Expert Contributor Network.
Want to join?