Getting out of pilot purgatory: from strategy to scaled value in the AI-powered bank

Ashwini opened with a point that will be familiar to anyone running engineering in a bank. Development is only the middle of the software lifecycle, and AI tooling has made that middle much faster. "Our bottlenecks, if you like, are moving elsewhere." Getting requirements and context into the system, validating what AI produces, deploying it and watching how it performs in production are now where the time goes. Organisations, she added, are still structured around how development used to work, and need reimagining to be properly AI native.

Dave described what separates the organisations that get out of pilot purgatory from those that stay in it. Start with a business outcome tied to an executive who cares about moving that metric. Gather your deep builders, so time to value shortens with every use case. And treat security, compliance and governance as part of the build: "Bring them into production from day one."

The technology is largely there. The work is in the organisation around it.

David was frank about why enterprise adoption lags consumer adoption. Tools are being built too narrowly, in a field moving faster than anyone can track. But "the bigger thing in the enterprise specifically is fear." Individuals fear for their jobs. Security teams face a new surface they cannot yet quantify. Heads of Governance, Risk and Compliance lie awake worrying about what an agent might do. His answer is to name the fear first, then put education, controls and governance in place to quantify it and quell it.

Ashwini made it concrete. If you are a scrum master or a product owner today, what does your role look like tomorrow? "That pathway is hard and we're all kind of learning as we go."

Dave set out how responsibility divides. The model provider supplies safe, auditable, secure AI, and keeps control of your data with you. Partners bring a deep understanding of your processes and core systems. The bank owns the decisions: what can I touch, and, in his words, "What does good look like?" Everyone has a seat at the table, and the bank holds the pen.

David offered a delineation he suggested people take back to their own organisations. Run AI through the lens of efficiency and "you are going to end up having it be a cost of doing business." Run it through the lens of growth, and efficiency comes with it, in service of doing more: new products, new markets, more capable teams. That is how you build a moat rather than join a race to the same efficient answer. It takes bravery, he said, because AI will need you to take risks.

Ashwini added the pressure from the other direction. A large share of a bank's cost is keeping the lights on, and AI is raising the bar there too. Vulnerabilities, for example, are increasingly expected to be closed on the day they are found, across estates full of legacy technology. "Now we don't have a choice. We have to become better, quicker, across the entire development lifecycle."

The panel also described AI as a five-layer cake: energy at the bottom (the biggest constraint right now), then GPUs, infrastructure, model builders, and applications on top. Banks sit in that top layer as application creators.

Some of the most interesting use cases are not the ones making headlines. Revolut applied transformer models to its transaction data and published its work in April. Lloyds followed with its own more recently, and banks have been asking since how to build their own. Not all AI is a chatbot. The future of banking may well be a series of domain-adapted models that banks own and run, supported by research labs inside the banks themselves.

There is also a change in how banks view their technology functions. As banks build AI factories, technology teams start to look like revenue-generating divisions rather than cost centres.

The audience questions went straight to the hard part. One asked how to protect outcomes from hallucination when agents do more of the work. Dave's answer was practical: you cannot promise zero. For something like a KYC rollout, start with a hundred cases and a human reviewing every one, then extend as you measure accuracy and build confidence in the platform and the process.

David’s answer was about context. A model hallucinates because it is trying to "guess the bit in the middle that you haven't told it," so well-designed context architectures that supply the right information at the right moment reduce the guessing. I added that humans make mistakes too, and we build controls to catch them. Agents deserve the same.

A senior leader from a global bank then asked the sharper version: can we trust models at all? Dave pointed to Anthropic's work with external evaluators and added safeguards, and said what every banker in the room knows: "once you lose trust, it's very hard to come back from." The panel added a useful reference point: banks already trust credit-decisioning models through measurable benchmarks such as precision and recall, and the same discipline applies to the newer ones. David raised the opposite risk. On one project the problem was over-trust: a human in the loop just clicking "approve, approve, approve." Trust has to be engineered in both directions.

When another bank leader asked who is accountable when an agent acts beyond its brief, Dave was candid that this is still being worked out with regulators and governments. It is a useful reminder that the controls you design today are your own.

The final question came from a commercial leader at a global bank, and it was the one that reframed the afternoon. Cost to serve is high, token economics are hard to justify, and most deployments are measured on productivity rather than revenue.

David's reply was a question: compared to what? Against human effort, tokens are cheap. If you are not seeing value, it is a bad use case, or one that has not been optimised, or one running on a model more powerful and expensive than it needs. Measure cost per unit of intelligence, and move down the stack to smaller models where the economics demand it.

Owning capacity changes the economics too. Once you move from paying per prompt to investing in your own capacity, the cost of tokens falls the more you use them. Ashwini noted that cost to serve has always been hard to measure, and that AI-enabled production telemetry should make it easier thereby allowing better investment decisions.

I shared one lesson from our own work at GFT. We built a centre of excellence around Claude for a US client. When it was opened up to citizen developers rather than expert developers, token costs rose fourfold, because people were not yet confident in prompting and in putting context and systems around the model. That is a skills problem as much as a technology one, and it is solvable.

1. Name the outcome and the owner. Tie every AI initiative to a business metric and an executive who is accountable for moving it.
2. Map the whole lifecycle, not just the coding. Find where the bottleneck has moved: requirements, validation, deployment, production telemetry.
3. Bring risk, compliance and model risk into the build on day one. Agree what evidence they will need before the pilot starts, not after.
4. Scale autonomy against measured accuracy. Start with human review on every case, widen as the numbers justify it, and design reviews so people are actually reviewing.
5. Put a price on a unit of intelligence. Compare it with the human baseline, right-size the model, and train people to prompt and supply context before giving them broad access.
6. Ask the growth question. Which process would you design differently if the constraint were removed? Efficiency alone becomes the cost of doing business.
7. Fix the foundations and keep your options open. 93% of the CIOs and CTOs we surveyed expect running AI on un-modernised legacy systems to trigger a security crisis, and 99% say potential government restrictions on AI access make avoiding dependence on a single provider more important.

The common thread from the panel was that the banks making progress are the ones building capability they own, with the right engineering expertise alongside them.