Architecture is dead, long live Architecture

Thursday 11th of September 2025

I've been working with a software architect recently. And I'm not impressed. Architecture was for the built environment, and software is building things too, right? Right? Building bridges is fairly understood at this stage, but the requirements are obvious, leading to only sporadically seeing re-work. Building houses, pretty similar, and whilst these grow - seldom do we see it need to expand to fit in 10x more occupants. Software exhibits an uncanny ability to make mockery of any and all assumptions that you through at a system

For example, I've been working on a DB with a fairly complicated and largish schema, 100 or so relational tables. This is built to scale to add more models, but we are now hitting some table limits about performance, especially when it comes to DDL changes, and deployments. So what does the architect do? He burns it down and starts again of course, this time it's webscale! Datalakes, warehouses, super-complexity. And it's what they are literally paid to do, so it isn't a surprise that this is what they have done. And this leads to a classic second system syndrome

We aren't doing waterfall in 2025 any more, meaning that requirements not only are good to emerge, we react to them. Architecture therefore must meet changing requirements. But most architects react poorly to changing requirements, seeking instead to solve problems both real and perceived with usually fairly large-scale changes. This yields poor deliverables, and shaky foundations for our software over the long term. Instead there is another way

Emergent Architecture

This isn't a wholly new idea. Agile has been practising emergent design since its inception, and as such I'm riffing on this. And it also isn't saying we don't need architecture, on the contrary! Architecture is so crucial to what we do in software development that abandoning it is to abandon any hope of long term sustainability of your software. But architectures must balance the needs of agility, with the needs of up-front design. And to do this we borrow from the agile ways of working by adopting short-cycle feedback and iteration loops where we test hypothesis, experiment, learn, and adapt. Meaning that wholesale re-designs are a thing of the past

To do this does require some discipline from a software team however. The ability to adapt architecture require a flexibility of domain logic unheard of in most projects, with tight coupling between interfaces, business logic, and data often yielding to systems which are hard to adapt to changing systems. But this is actually all good news. We know that adaptability, and YAGNI can kill a project, and so generic abstraction can be too unwieldly to deliver value, but there is a pattern which allows us to maintain our discipline. And its my old friend of TDD. Test Driven Development encourages clean, re-usable, testable interfaces - exactly what you need to have to allow for jumbo refactoring to occur, and hence emergent architecture to do it's thing

I recommend that an architect is embedded into a team in order to do such a function also. So those principles, staff engineers, and seniors should have a stronger hand in Architecture. After all, we're all building web services in the cloud these days using a fairly common set of services, languages, and frameworks. What do we need it for as a separate function anyway? Those in the team will know the problems, and solutions, far better than anyone outside of the team, and should be empowered to deliver

Back to my original example then. Instead of throwing things away, the architect would be the team, including PO, designers, and developers to make incremental updates to the system design. And they would probably look to any other solution that data-warehousing. Partition those difficult tables, or ask if we actually need all that data, or add a tiny bucket based solution for tiered storage - keeping the original in place which is already serving customers, adding value, and should be abandoned at your peril