Building the Research Engine by Julian Della Mattia - Review
Why the next breakthrough in product is not better research, but better plumbing
Research.
For some reason, it seems to be a bit of a triggering word these days inside and outside of product divisions. Sometimes we don’t do enough, and it gets blamed for bad decisions. Sometimes we do too much, and it gets blamed for making no decisions. And sometimes it gets used to validate an existing idea, and gets blamed when the research doesn’t support the crusade.
And then there is typically a big gap in understanding how research actually works, the mechanics and logistics of spinning up interviews, surveys, tests, and analysis.
I’m gonna say it. Research itself is not really the hard part. Interview techniques are well documented. Frameworks and textbooks on how to recruit users, ask non-leading questions, or run a decent usability test are sitting on shelves (virtual and IRL!) for anyone willing to crack them open.
What kills most research practices is everything that surrounds the research. Who knew to ask the question. Whether the findings reached the right room. Whether the answer arrived before someone in leadership had already quietly made the call. That is where good work quietly evaporates, and that is exactly where Julian Della Mattia has chosen to speak out.
Julian is the host of the Finders to Builders podcast, renowned user researcher, and author of Building the Research Engine; his codified version of years spent walking into companies where research barely existed and trying to leave behind something durable enough to outlast him.
Antonia and I recently chatted with Julian about his book and ResearchOps overall on the Product Opscast:
Best practice as a system, not a checklist
Write a good interview guide. Pick the right sample size. Validate before you build. That is Research 101, right? Do these and off you pop. But Julian asks a more useful question. What does it look like when research becomes a practice, not an activity that some keen designer holds together with personal heroics?
His answer is the CIC framework. Culture (do people here actually value evidence?), Infrastructure (do the systems exist to do the work?), Communication (do insights reach the people making decisions?). Every organisation is weak in at least one. Most are weak in two. These pillars stay stable across the entire book, and he uses them to walk readers through three stages of maturity (Build, Connect, Trust) without ever sounding like he has invented something nobody noticed before. It is best practice reframed as a system rather than a checklist. For anyone in Product Ops who has spent more time auditing process compliance than asking whether the process actually produced a better decision, this reframing alone is worth the price of the book.
And let’s pause on personal heroics for a moment, because this is where ops roles (Product, Research, Engineering, Design, Rev…) both uncover the fragility of businesses and create a robust solution that stops relying on individuals, who are fragile (sorry, I couldn’t find a better term!) and put in place independent systems. Heroes are created out of necessity, when things just needed to get done, when money was tight or speed was paramount. Nothing wrong with this, but it is not a strategy.
Strategic clarity through stages and gears
Julian frames research practice as a manual transmission with four gears (Interface, Feature, Product, Strategic) and is incredibly clear that the gear you can drive in depends on the political capital you have actually earned. At Stage 1 you stay in Interface and Feature work, because nobody has given you permission to touch existential questions yet. By Stage 3 you can shift up to Strategic, because by then the Engine has compounded enough trust to be allowed near the big decisions.
Strategic work is what you are building toward. Tactical work is what gets you there. The Engine, not the practitioner, decides when it is safe to shift up. That distinction is going to save a lot of careers from the trap of trying to be strategic in week one.
Tactical work often gets a rolled eye or a little sigh within the ops communities, and with good reason when professionals are permanently weighed down in the trenches of the tactical, the reactionary, sometimes the ‘busy’ work. But for most of us in teams of 1, or teams of few, the tactical is just necessary, particularly in the early weeks and months of tenure, almost exclusively, and then a mixture after that. And it is that work that earns the right, the political capital to do more, do bigger.
Minimum Viable Rigor and the Decision + 20 percent rule
Two ideas I will steal immediately. The first, credited to Carl Pearson, is Minimum Viable Rigor. Your research rigour should match the risk of the decision it informs. A quick scrappy test is fine for an icon choice. A market entry call needs evidence that holds up to scrutiny. Julian uses this to dismantle the false binary between proper research and good enough research, and lands a perfect line about not killing a fly with a cannon (as much as I now want to try this!).
The second is what he calls the Decision + 20 percent rule. When you scope a study, the decision it serves is your 100 percent. You deliver on that. But if you leave a small aperture, one or two extra questions in an interview, one extra thing you watch for during analysis, you will almost always uncover something nobody asked about but everybody needs. That is the difference between answering questions and raising the questions worth answering.
The Verdict
Building the Research Engine is practical without being prescriptive. Opinionated without being dogmatic. Structured without ever feeling rigid. Which is the sweet spot for Ops roles in the modern workplace. The focus is not on doing the research, but what comes next to ensure the findings are shared where they can provide the most (or any) value.
The most expensive thing in your organisation is not the headcount or the tooling, but the assumptions you keep building on without ever testing. This is the leadership hook. For ops folks, read this book twice. The first time as a researcher. The second time as the architect of a system the company will one day quietly forget you built, which is, after all, the point. And the norm for us ops folks. Us heroes. And yet, the best research practices do not look like heroics. They look like infrastructure. An engine.
Great work Julian!
Practical Applications for Product Ops
Product Operations spends a lot of time talking about insights, data, and decisions (Just listen to Antonia on our podcasts!) and not nearly enough time talking about the machinery that turns one into the others. Julian is not writing for us specifically. He is writing for designers, PMs, founders, and first researchers. But almost every chapter contains a sentence that could have been lifted out of a Product Ops job description. Build the systems. Connect the signals. Become the organisational sense maker.
By Stage 3 of his model, the researcher is no longer running studies. They are the playmaker who sees the whole field. They connect analytics, support tickets, sales calls, NPS, and qualitative work into a coherent picture nobody else is positioned to see. That is, line for line, the senior Product Operations job most of us are quietly grow into.
The reality is, many businesses are not mature enough to have built/be building ResearchOps roles just yet, and so it is likely much of this engine building falls on us in Product Ops… and so ta daa, this IS for us. And is a great playbook to help extend our skills to an area not typically in our wheelhouse.
Graham