Using Probity to enforce TDD guardrails for AI coding agents – Q&A with Nizar Selander

During the interview, Nizar showed a kata in which he used Probity together with Claude Code to get the agent to work according to his wishes. You can watch the full interview on the MSE YouTube channel. And if you would like to see the entire raw recording of the kata session, you’ll find the link at the end of this article.

But first, we gave Jimmy, CTO and co-founder of factor10, the opportunity to ask Nizar a few more questions about Probity and agentic coding that he was personally curious about. Hopefully, you are too!

Q&A with Nizar

Jimmy: As is shown in the video, Probity works like a charm in a kata. Does it work in the real world, too?

Nizar: Yes, it works with real-world projects, and that’s how I dogfood it.

What the kata demo showed well was the autonomy that Probity provides when enforcing TDD. My only input was an opening “Let’s get to work”, and the guardrails supported the agent over the next 45 minutes until it was done.

Katas, however, usually have very clear seams, and there is usually a very obvious way to test behaviour. This doesn’t leave much room for agents to game the tests. In real-world projects, behaviour can usually be tested in several ways and at different depths, and the honest test sometimes needs more setup or more digging first. That’s where agents try to cut corners or settle on their own definition of “good enough”. Probity calls them out on it and does a lot of heavy lifting on that front.

Jimmy: Are there any differences between what we see in the demo run and how you normally work?

Nizar: I intentionally left out a lot of my own practices so viewers could see how the whole thing plays out on its own. So it’s very bare-bones in that sense, whereas I’m normally much more involved in design and strategy, and use several workflows.

What’s very similar is that I don’t like to do the same thing manually over and over again. You can think of it as a city planner who notices car accidents piling up at an intersection. I would rather rework the intersection or the rules around it than guide cars through it myself one by one. I look for common agent failure modes and adjust the rules and guardrails accordingly. This way, my effort also applies across sessions and frees up my future attention.

Jimmy: When I use Probity, I notice that the agent is not just reactive but also very proactive in avoiding hitting Probity’s rules. However, I also notice that the agent sometimes decides to run a script to avoid hitting Probity’s rules. How do I close that kitchen door?

Nizar: I use the forbidCommandPattern rule to block agents from making changes through command-line tools. I simply forbid certain patterns and include a message that redirects agents back to the tools I want them to use.

Speaking of kitchen doors, did you know that the find command can also be used to modify files and execute code? I couldn’t find a clean way to allow its use while blocking unwanted patterns, so I blocked it too.

Jimmy: What are the plans for the next features of Probity?

Nizar: I’d love to explore how Probity can help with Domain-Driven Design aspects. The rules already work on writes and commands, so a next step would be enforcing boundaries, for example, blocking a change that reaches across a bounded context or makes one context dependent on another’s model, even though you have decided on Anti-Corruption Layer rather than Conformist. Agents can be quick to take shortcuts that erode boundaries over time, and guardrails can keep them honest at that.

Jimmy: A concern I hear more and more is that the divide between those who benefit from agentic coding and those who haven’t yet started using it is rapidly growing. Do you have any tips for getting the second group inspired to start trying it out and reap the benefits?

Nizar: The best advice I can give is to remain curious, and I don’t mean that as a euphemism for “get on board”. The objections I frequently hear are that typing code faster was never the real bottleneck or that the tools just get in the way. Some people also simply prefer to have their own hands on the work. I can relate to those frustrations and understand where they’re coming from. But the answer is rarely – if ever – fully binary. There’s a lot more to this space than initially meets the eye, and usually a lot of value between the two extremes.

The fastest way I’ve found to see this is to get people together in a room: engineers, testers, a domain expert or someone from the business. We look at the real picture together and ask what’s slowing us down, what pain we’ve been living with, and what’s actually worth our time. It only works when everyone feels welcome and included, regardless of background. There are no wrong questions.

What I enjoy most is when someone raises an issue that others hadn’t considered or weren’t even aware of, and it gets both the enthusiasts and the sceptics equally excited about solving it. What started as simple curiosity has turned into engagement across the organisation.

Jimmy: I know you started using LLMs in the early days. Any learnings from back then that you still adhere to?

Nizar: The one that stuck with me is thinking in orchestration. It comes from working with Erik Meijer on a number of innovative applications that were quite ahead of their time. Some of them have become commonplace since, like RAG, but others I don’t think have been fully realised yet, like orchestrating agents in serverless environments with humans in the loop. We’re seeing people talk more about those things now, though, especially with topics like loop engineering. The TDD cycle itself is one such loop if you think about it, so a lot of what I do is inspired by the things we worked on.

Watch the full demo

You can watch the full raw recording of the agent solving the kata with Probity enforcing TDD here. Please note that this video has no audio.

Learn more about Probity

Probity is open source and works with most agents. You can find instructions on how to get started here.

Get to know Nizar a bit

Nizar Selander is a software architect and consultant at factor10 who brings applied AI to business-critical work in innovative ways, without compromising on craft. His open-source tools help developers keep their code clean and maintainable when working with AI agents. He also engineered several of Sweden’s top-ranked websites in performance, security, sustainability, and accessibility. Nizar enjoys sharing what he learns through talks and writing, please connect with him on LinkedIn or GitHub if these topics interest you!

Looking for more inspiration?

If you would like to inspire your team with bleeding-edge agentic coding at the start of autumn, please get in touch, and we will share a few suggestions for presentations and workshops popular among progressive companies! Another popular option is to collaborate with us and run, for example, a bi-weekly brown-bag lunch with your teams to inspire and help them to get rid of friction.