FIELD NOTE |
October 22, 2026
The Interface Is Not the System
ChatGPT made AI feel accessible. Implementing it inside an organization reveals how much of the work was never about the model.
I think one of the more consequential things ChatGPT did was make artificial intelligence feel easy.
You can open a browser, type something in reasonably ordinary language, and get an answer. You don't have to understand the technology. You don't have to write code. You don't even have to be particularly precise about what you're asking.
And the results can be remarkable.
That experience has shaped how people think about implementing AI inside organizations.
If I can get a useful answer in thirty seconds, why should it take months to make AI useful in my business?
I've been involved in implementing the first operational AI use cases at several companies. I've worked with engineers who could make models do things I wouldn't have imagined possible a few years ago.
And some of the hardest problems had very little to do with the models.
They had to do with everything the models needed to know, everything they needed to interact with, and everything the organization hadn't yet figured out about itself, or wasn't ready or willing to acknowledge.
A Good Answer Isn't Necessarily a Reliable Operation
There's a meaningful difference between asking an AI assistant a question and giving an AI system responsibility for part of a business process.
When I'm having a conversation with ChatGPT, I can evaluate the answer. I can provide missing context, correct an assumption, or decide to ignore what it tells me.
I'm doing quite a bit of work in that interaction, even if it doesn't feel like work. I know what I meant by the question. I have context the model doesn't. I can recognize when an answer conflicts with something I know. I can decide whether the result is good enough for what I need.
Now imagine removing me from that interaction.
Instead of asking an assistant to help research a prospective customer, I want an AI system to identify prospects, research their businesses, evaluate their fit, determine which account they belong to, and initiate the appropriate workflow.
The model may be perfectly capable of researching a company.
But what counts as a company?
Is a subsidiary a separate account? What happens when two legal entities share a parent? Which account owns the relationship? What if the organization has different definitions of ownership in its CRM and its sales engagement platform?
What information can the system use to determine fit? What makes a signal relevant? When should it act, and when should it ask someone?
These are not questions that can be answered simply by choosing a better model.
They require decisions about the business.
And often, those decisions haven't been made as explicitly as we imagine.
The Organization Knows More Than Its Systems Do
One of the recurring challenges I've encountered is the difference between the information an organization possesses and the information its systems can actually represent.
An organization might have extensive customer records, documented processes, product information, contracts, and years of historical activity.
That sounds like a tremendous amount of context to give an AI system.
But the existence of information doesn't tell us whether it's accurate, consistent, current, or meaningful.
A field called annual recurring revenue might mean one thing to Finance and something slightly different to Sales.
An account hierarchy might reflect legal ownership in one system and commercial relationships in another.
A process document might describe the official approval path while everyone who actually performs the work knows that certain situations require a conversation with someone who isn't mentioned anywhere in the documentation.
People navigate these inconsistencies constantly.
They ask questions. They remember exceptions. They recognize when something looks wrong. They understand that the same word can mean different things depending on who is using it.
Sometimes they don't even recognize that they're supplying missing context. It has simply become part of doing the job.
Then we introduce AI and discover that the system needs that context to be explicit.
I've worked with AI engineers who could build sophisticated capabilities but struggled with the underlying business data. Not because they weren't capable engineers, but because the information required an understanding of the organization that wasn't contained in the technical specification.
And sometimes the people asking for the implementation couldn't fully articulate that understanding either.
They knew what they wanted the AI to accomplish.
They didn't necessarily know how to explain the judgments a person was making along the way.
Not all of these gaps exist because nobody has noticed them.
Sometimes the organization genuinely doesn't know how something works. Sometimes people recognize an inconsistency but have learned to accommodate it. And sometimes resolving the ambiguity would require a decision about authority, ownership, or accountability that nobody has been particularly eager to make.
An organization can operate with unresolved questions for a remarkably long time, especially when people are willing and able to compensate for them.
Someone knows which number to trust. Someone remembers why the exception exists. Someone has enough influence to get a decision made without formally owning it.
The work gets done.
But when we ask an AI system to perform that work, those informal arrangements become harder to rely on.
What is the system authorized to decide? Which rule takes precedence? What should happen when the usual process doesn't fit the situation?
And who is willing to own the answer?
In IBM's study of 1,700 chief data officers, 81% reported that their data strategy was integrated with their technology roadmap. Only 26% were confident their data capabilities could support new AI-enabled revenue streams.
I find that gap more interesting than either number on its own.
Having a strategy for information and having information that can reliably support a new kind of operation are different things.
Connecting the Data Isn't the Same as Understanding It
There's a lot of discussion about giving AI systems more context.
Retrieval-augmented generation, knowledge graphs, model context protocols, tool integrations, memory, and increasingly sophisticated agent architectures all offer ways to expand what a model can access or do.
These are useful capabilities, but they solve different problems.
A tool connection can let an AI system retrieve a customer record or update an opportunity.
A knowledge graph can help represent relationships between entities.
A retrieval system can supply relevant documents when they're needed.
None of these automatically establishes whether the underlying information is trustworthy or how conflicting information should be interpreted.
If two systems disagree about which account owns a customer, connecting both systems doesn't resolve the disagreement.
It makes the disagreement available to the AI.
And if the AI is expected to act without clarification, someone has to decide what should happen next.
Which source takes precedence? Does the answer depend on the situation? Is the discrepancy itself something that should trigger review?
Those are questions about authority, policy, and judgment.
We can encode answers to them in software. We can give an AI system instructions for handling them. We can design evaluations to test whether the system follows those instructions.
But we still have to make the decisions.
And not every judgment should become a rule.
Some decisions depend on circumstances that aren't fully represented in the data. Some require weighing competing interests, recognizing something genuinely new, or taking responsibility for consequences that can't be reduced to a predefined threshold.
Making more information available to AI doesn't necessarily mean giving AI responsibility for deciding what to do with all of it.
I wonder how often organizations interpret these difficulties as limitations of AI when they're actually discovering limitations of their own information architecture or disagreements about decisions that the organization has never needed to resolve formally.
The model didn't create the ambiguity.
It just became another participant expected to navigate it.
The Work Doesn't End When the Model Works
There's another distinction I think gets lost in the excitement around AI capabilities.
A demonstration shows what a system can do under particular conditions.
An operational implementation has to work across the conditions the organization actually encounters.
That includes incomplete information, conflicting records, unusual requests, changing policies, unavailable services, unexpected customer behavior, and the ordinary accumulation of exceptions that comes with running a business.
It also includes knowing when not to act.
In Gartner's January analysis, the firm reported that at least half of generative AI projects had been abandoned after proof of concept. The reasons included poor data quality, inadequate risk controls, escalating costs, and unclear business value.
None of those necessarily means the model couldn't perform the task it was asked to perform.
A system that successfully completes a task 95% of the time may be extraordinarily valuable in one context and completely unacceptable in another.
The important question isn't simply how often it succeeds.
It's what happens when it doesn't.
Can we recognize a failure? Does the system know when it lacks sufficient information? Can someone reconstruct why a decision was made? Is there a way to reverse an action? Who is responsible for resolving the consequences?
And perhaps most importantly, does the system make the work easier for the people who depend on it?
I've seen enough technology implementations to know that automating one activity can create additional work somewhere else.
We're seeing a version of this with AI slop: content, code, analysis, and other outputs that are inexpensive to generate but may require substantial effort to evaluate, correct, or make useful.
The person producing the work can generate more of it, faster. But someone still has to determine whether it's accurate, whether the assumptions hold, whether it fits the context, and whether it can actually be used.
And that responsibility often moves downstream.
A task disappears from one person's workflow, but someone else inherits the validation, correction, exception handling, or reconciliation. The cost hasn't necessarily disappeared. It has changed location, and sometimes become harder to identify.
There's an asymmetry here that I think matters. Generating an answer can be nearly instantaneous. Establishing that the answer is trustworthy may require understanding the very system, relationships, and exceptions we were hoping to automate.
The automation may still be worthwhile.
But measuring the time saved by the person generating the output doesn't tell us whether the organization has actually become more effective.
Sometimes we've removed work.
Sometimes we've moved it.
Sometimes we've made it harder to see.
What Does the Human Actually Contribute?
I also think we need to be more precise when we talk about keeping humans involved in AI systems.
A human approving an action isn't necessarily exercising meaningful judgment.
If the person doesn't have enough information to evaluate the recommendation, they're not really making a decision. They're accepting responsibility for one.
And if the system produces so many recommendations that reviewing them becomes impossible, the approval process may eventually become another automated behavior performed by a person.
The question isn't simply whether a human is in the loop.
It's what the human is there to contribute.
Are they supplying context the system doesn't have? Evaluating competing objectives? Making a decision that requires legitimate authority? Recognizing a novel situation? Challenging an assumption?
Or are they just being asked to confirm that the machine has done something?
Those are different roles, and they require different designs.
There's an opportunity here to make human judgment more consequential rather than less.
But that doesn't happen automatically when we put an approval button at the end of an automated workflow. It happens when we understand which decisions benefit from automation, which require deliberation, and how information should move between them.
It also requires recognizing that some ambiguity isn't a defect to be eliminated.
A person may need room to exercise judgment precisely because the circumstances don't fit neatly into an existing rule. We don't necessarily want to eliminate that flexibility. We want to understand when it's necessary, who should have it, and what information they need to use it responsibly.
The goal isn't to make every decision machine-readable.
It's to be deliberate about which decisions we're asking machines to make.
Maybe the Difficulty Is Telling Us Something
I don't think implementing AI needs to be unnecessarily complicated.
Some applications are relatively straightforward. Others are difficult because the consequences of getting them wrong are significant. And some are difficult because we've asked a new technology to operate inside systems whose complexity we've spent years learning to work around.
Those differences matter.
The accessibility of AI has made it possible for people to explore ideas, build things, and solve problems that previously might have required months or years of work, entire teams, specialized expertise, or substantial capital.
I think that's a good thing.
But it has also created an expectation that organizational implementation should feel similarly effortless.
I don't think that's a reasonable expectation, at least not for every kind of work.
An organization isn't a conversation.
It's a collection of relationships, responsibilities, definitions, commitments, constraints, and decisions that have accumulated over time.
Some of those arrangements are explicit. Many aren't.
When we ask AI to participate in that system, we're asking it to operate within all of those conditions.
And when the implementation becomes difficult, I don't want to assume the answer is always a better model, a longer prompt, or another integration.
Sometimes the difficulty is information.
Sometimes it's an unresolved decision.
Sometimes it's a responsibility nobody has clearly assigned.
Sometimes it's a process that only works because someone knows when not to follow it.
And sometimes the difficulty comes from asking technology to make a decision that someone in the organization is unwilling to make themselves.
Those are useful things to discover, whether or not we ultimately use AI to address them.
Making a system easier to interact with is not the same as making the system itself work better.
And perhaps one of the most valuable things AI implementation can do is reveal what was already there.