Why quality engineering belongs in the boardroom, not the back office

The cost of treating quality as a technical detail

A defect caught in a code review costs your firm almost nothing to fix. But if that same defect is caught by a client instead – perhaps in a statement, a transaction or a portfolio valuation – it could cost your firm a great deal more. Remediation, regulatory reporting and the hours your best engineers spend firefighting instead of building all add up, and the further a defect travels through your delivery pipeline before it’s caught, the less control your firm has over who finds it first – and the more it’s likely to cost you.

In our experience, most technology budgets are still built around the cost of testing rather than the cost of not testing enough. Most boards think that’s reasonable – but that’s because most boards treat quality as a line item. Once a board fully understands how much a single serious defect can cost in client trust alone, it tends to start to feel differently.

To make that shift, a board needs to fully understand that a defect is not simply an engineering mistake to be logged and fixed. In financial services, a defect can just as easily result in a client-facing incident, a regulatory reporting error or a long-standing client starting to wonder whether your firm is still the right custodian of their money. Framed that way, quality stops being a delivery metric and starts to look like another board-level risk indicator – which, in our experience, is a good thing. 

What quality engineering actually covers

Quality assurance and quality engineering are terms often used interchangeably, but in fact, they aren’t the same thing at all. Quality assurance tests what has already been built and reports what it finds, whereas quality engineering incorporates quality into build processes. A firm using QE practices will, for example, agree a test strategy before the first sprint, shift catching problems to the left and build automation into the delivery pipeline rather than bolt it on at the end.

In practice, quality engineering covers a wide range of disciplines: test strategy and planning, test data management, environment management, performance engineering and the quality metrics that make all of the above visible to people who aren’t in the delivery team day to day – such as board members. Your firm doesn’t need to excel at using QE within every one of these disciplines immediately. But it will certainly help you to know which of those disciplines are currently unmanaged – because when they’re left unmanaged, risk is likely to be quietly building within them.

The disciplines most often overlooked tend to be the least visible ones. Test data management and environment management rarely feature in a steering pack, yet a poorly managed test environment or an unmasked copy of client data sitting in a non-production system can create more exposure than the defect it was meant to help catch. Quality engineering treats these as first-class risks in their own right, not as background infrastructure someone else is presumably looking after.

Why your board should be asking about quality, not just delivery dates

Most steering committees want to know only two things about a delivery programme: whether it’s on time or not, and whether it’s on budget or not. Whether it’s actually ready rarely gets the same level of attention, partly because it’s harder to answer that question with a single number.

But the real-time quality metrics quality engineering can provide change that. Test coverage, defect trends and release readiness – visible as they happen rather than reported a month later in a steering pack – give your board something more useful than reassurance: an actual answer to the question of whether a release is ready to go, backed by evidence rather than confidence.

What good looks like in practice

Here at Assured Thought, we’ve worked with wealth management firms who had already lived through a failed platform replacement before bringing us in to help with a second attempt. The difference we made to those second attempts wasn’t in adding more testing for its own sake. It was in agreeing a test strategy before the project started; instituting a risk-based approach to what got tested most thoroughly; and ensuring secure, well-managed migration of client data from legacy systems. Invariably, the result was a smooth, successful go-live, and no unexpected spike in production incidents in the months that followed.

That pattern holds well beyond platform migrations too. Firms that treat test strategy, environments and test data as disciplines in their own right, rather than as afterthoughts to a delivery timeline, consistently see fewer defects reach production – and recover from the ones that do reach production more easily and quickly.

Where to start

Your firm doesn’t need to overhaul its entire approach to quality in one step. To begin with, it needs just one honest conversation at board level about where technology delivery risk currently sits, and whether anyone outside the delivery team could answer, with evidence, how confident they are about the next release.

If that conversation hasn’t happened yet, it’s worth having it before the next platform change at your firm forces it to.

Quality engineering isn’t a back-office function that keeps a delivery team honest. Done well, it’s a source of evidence your board can rely on.

If you’d like to talk through what that could look like for your firm, Assured Thought’s team is always glad to compare notes.