alternative workflow

at work, we have started experimenting with new ways of working, from ai-assisted development to automated collaborative tools. this has been interesting in so many ways, considering how generative ai and llms have gotten really good over the years and especially these past months, and how much more willing we have become to trust them directly inside our workflow.

one of the things we've introduced is an ai bot reviewer that assimilates the role of a human reviewer, with a touch of quality assurance persona. it reads the ticket, looks at the code change, and can either block or accept a merge.

and at the risk of sounding cliche, working with this has greatly influenced how i think about my own work. i gave a presentation on this experience last week at work as part of a professional development rotation and thought the findings & lessons learnt will make for a good entry for the blog.

1. scope is a feature

by virtue of self-aggrandizement, I started out wanting the first-pass approval. no comments, no rework, nothing like the annoying "oh, i missed that." that llms are now notorious for. this meant i inadvertently began thinking about the reviewer as something to get past—which was the wrong framing. the more useful way to think about it was as a demanding collaborator whose job is to find the places where my change and my claims disagree. This also means accepting scope as a feature, as the reviewer was just as likely to flag doing too much as doing too little. the little "while i'm here" changes that seemed harmless to me could easily turn a focused change into something harder to review and harder to trust. so i adopted a simple rule: fix what the finding requires, and leave the rest alone. a diff that matches the ticket exactly is easier to reason about, easier to review, and easier to trust.

2. review your own pr first

every developer knows they should review their own code before opening a PR. most of us do a quick sanity check; skimming the diff, removing a rogue console.log (or the emojis from your copy and pasted slop), checking that the branch is clean which isn't necessarily review but mere housekeeping. i started doing something much slower: auditing my own work against the acceptance criteria. for every criterion, i ask myself two questions: how is this actually met, and is it structurally guaranteed or merely tested? there is a meaningful difference between the two. "it should work" is not the same as "it is verified." and writing this down forces me to confront the assumptions sitting underneath the implementation rather than relying on the fact that the tests are green or that the code looks right.

3. passing tests can still hide a wrong mental model

this was probably the most useful part of the whole exercise. the reviewer caught issues i could very easily have shipped. in one case, i had a delete operation cascading across a tree of records inside a transaction. the tests passed & the implementation looked airtight. except one of the saves wasn't actually attached to the transaction. the ORM did not magically do that for me.

a failure halfway through could therefore have left the tree partially deleted. in another case, a delete confirmation told the user how many sub-items would also be deleted. the number came from lazily-loaded data, meaning anything that had not already been opened could be silently left out of the count. the problem wasn't just that the code was wrong. i had built the wrong mental model of what the code was doing. that is the uncomfortable thing about good review: sometimes the tests pass because you tested the wrong thing.

4. know when to push back

Llms are good at making mistakes (read as hallucination) and the reviewer is not always right, and learning that is part of the workflow too. there was a finding around a simultaneous edit that could introduce a new child while a delete operation was taking place. the concern was valid, but fixing it properly would have meant making changes across unrelated parts of the system. this was where i had to push back. not with "the bot is wrong", but with evidence. i explained the scope of the problem, softened an over-strong claim in the PR, and opened a follow-up ticket for the broader issue. the reviewer accepted it. that was useful because it reinforced something i think is easy to forget with automated systems: "the reviewer said so" is not, by itself, a reason to expand a change indefinitely.

the real change was in how i define done and the biggest lesson wasn't really about the bot. it made me more rigorous even when i disagreed with it, because i had to explain why i disagreed. vague confidence slowly became reasoning i could defend. and that changed what "done" meant to me. a clean first-pass approval is not necessarily evidence that the code was perfect. it is more often the result of having already questioned your assumptions, checked the implementation against the ticket, reasoned about failure, and kept the scope honest. the bot was right most of the time. But the ultimate skill was learning to tell which time it wasn't.