This is one of the things with coding agents, building small tools solving a small pain became increasingly easy. It’s also the situation where you are least likely to notice that you’re designing for one person. On one hand it’s cool that more people can build their own things, on the other hand personal tools have idiosyncrasies that make them fairly difficult for others.

I kinda hope Ration is a bit different, but it’s also good to be self aware.

What I actually wanted to know

My use case is simple. 95% of my Claude usage is via Claude Code CLI in a nice customized Ghostty. I have set up a statusline to track my session usage there, so I kinda know how I’m doing. It looks like this:

claude-usage-menubar | ctx:12% | 5h:28% | wk:56% -13% | Opus 5 · high

Actually when I started to work on this, I didn’t had the wk: pacing yet on my statusline, that came later, as a side effect of building Ration.

Also, I generally like having a subscription with limits, as they give me a good cadence of focused work with the agent, intertwined with focused work somewhere else. So having the limits has some energy and focus management value too.

But when I’ve run out of tokens, I don’t want the terminal kept open on my screen to see when it auto restarts, and also don’t want to open Claude all the time to check when it starts to time my coffee breaks correctly.

Also more importantly, if I do a busy start on Monday, sometimes my weekly session limit ran out by Friday, and I wanted some control over my pacing.

So I got the idea to build a small tool for this case: see how I’m doing compared to my week, get a signal if I’m over the pace, and see when my daily session is available again without opening up a new tool or page.

Pacing is a subtraction

Pace was a nice little idea, take how much of the week you have spent, subtract how much of the week has passed, and you have your answer. Claude tells you when the weekly window ends, so the start is the end minus a week.

elapsed = (now - (resets_at - 604800)) / 604800 * 100
delta   = used_percentage - elapsed

The menubar shows the week rather than the session, since the statusline already covered the session while I was in there.

84% used is alarming on a Tuesday and unremarkable on a Sunday. The other menubar tools I looked at show you the weekly percentage, same as Claude usage on the web. If the delta is positive, I’m burning faster than the week is passing, so Ration extrapolates and tells me the day I’d run out: “Week runs out Sat 19.47.”

Making it work for me was quick aka Pareto

One point here is that it’s one thing to build something that you can use and runs on your computer. That takes like 20% of the time, together with making it a nice Mac menubar app that subtly helps and doesn’t look “overdesigned” with sloppy purple colors.

The rest of the 80% took to make it usable to others, get it covered with proper tests, set up a proper install sequence, iron out the small copy and design issues. Kinda the Pareto principle in action, but also gives a hint how far working prototypes can be from actual releasable code.

The copy part is a good example. The menu item for taking the app back out originally read “Remove Ration’s status line hook…”, which is super technical. It now reads “Undo Setup…”, which is something normal people would also understand. Standard UX polish but also shows how it’s not only people working on their stuff start to lose track how things behave outside, but agents can also get lost in their own context.

Getting things runnable on other people’s computers

This brought some “nice” issues, as in I would have never seen these if I would have not tried to make this app.

My favourite was the app reading its data once and then quietly stopping, because the function that checks a file’s modification date caches its answer (for better performance). It looked completely alive the whole time, since the countdown next to the number kept ticking off the system clock, while it had been showing me its launch-time reading for hours. Then there were my other Claude Code sessions. I had four open, some idle since before the weekly reset, and an idle session keeps rebroadcasting its expired numbers every ten seconds. So the app would fade out and announce that Claude Code had stopped, while three copies of it were writing to that file every five seconds.

The login item had a similar flavour. macOS identifies an app without a developer certificate by hashing the binary, so every rebuild orphaned it, and the checkbox stayed ticked while nothing launched. Distribution was the same kind of thing again. I planned to ship a Homebrew cask on the assumption that Homebrew strips the flag that makes Gatekeeper complain, and it turned out to do the opposite (thanks for checking this Claude). The solution was a formula that builds from source on your machine, which sidesteps the whole thing, since nothing downloads a binary so nothing gets flagged.

brew tap polgarp/tap
brew trust polgarp/tap
brew install ration

Probably I wouldn’t have caught these if I hadn’t tried to make things properly and deployable elsewhere.

Details also include things like choosing a name

Anthropic’s trademark guidelines require written permission for any use of their marks, so “Claude” something-something was out and so was using their orange. Which I didn’t want anyway. Going for Ration meant having a portion you are given for a period, which is more or less the product.

The fun in making something after all

Pacing probably didn’t need an app, the statusline was already there, and I just like to manage my focus by not having too many things on screen at the same time. The app adds a few things though, something visible when the terminal isn’t on screen, and a countdown for the moment you are locked out and Claude Code isn’t open to tell you anything.

And I wanted to make a Mac app, which is reason enough fur such a side project.

☕ Assumptions draw the edge of the solution space, and agents draw it tighter

Whenever we start working on a project or on a piece of software, we have some ideas what we are getting into. We believe certain things about who is going to use the thing, how they are going to use it, why we work on this, and how we are going to progress. These beliefs or assumptions set the initial stage.

Holding the wrong beliefs, as in getting assumptions wrong is bad of course, since we and the product fail in one way or another. So we usually set out to both declare assumptions, and do some testing to manage these risks. We usually start optimistic, so at the beginning we mostly come up with reasons on why it will work.

In a good environment, we write with the product team the list of assumptions we have, and how risky they are. If we are right about these the product will succeed. To account for limited time and resources the usual approach would be to construct tests and experiments to collect evidence about the riskiest assumptions. This is the right approach, and I’ve never regretted putting discipline to this.

But what about the things not on the list, we just took for granted?

Recognizing assumptions and collecting some data to make sure what we believe is correct is just one side of the puzzle, as assumptions also bound the solution space, limiting what results are achievable.

Designers and more broadly product teams often fall into this trap, as they believe all problems will be addressed with their solution. Testing is of course right, as it tells us if the solution works and more broadly teaches something about the assumptions we stated. But it should also talk about the problem space and the deeper assumptions about whether a problem is worth solving. Product teams are especially bad at this, since we mostly believe we are already on the right track to solve the problem.

Considering a narrower problem space does remove risky solutions. And it also removes things the framing hid, cutting potentially more innovative ideas.

We can’t test things we haven’t even considered, if they were obscured by deeply held beliefs.

I call this dogma.

Designers usually don’t think about whether the solution should be a screen or not. They just open Figma, and the version of the solution that might have been a policy change, or an email, or removing the step altogether, was just missing.

That’s the difference. A belief is something someone said, which is what lets us argue with it, test it, and drop it when the data comes back unkind. Dogma never gets said, so there’s nothing to argue with. It shows up in which options got sketched, which questions felt worth asking, and which version of the problem everyone recognised without needing to discuss it.

Can’t we just think about our beliefs harder to also articulate dogma? It’s not enough, as we need a different vantage point to look outside of the frame.

A longer list is still the same list. And often we don’t even start from our own frame, but one set up by the organizations and structures around us.

Testing the riskiest assumption is not enough, if the tests only address beliefs and don’t touch dogma. Early in discovery there’s no easy way to check if we work on the right problem, so good tests should talk about both problems and solutions.

This was the goal of making MVPs, not smaller versions of the product, but telling us whether the problem is worth solving. If a team tests only whether the solution works, it only asks the easier and less useful question.

This is why I also like Teresa Torres’s opportunity solution trees. It shows how we need different problem versions, not only different solution versions. As she puts it, a tree that grows deep at the cost of breadth means fixating on one opportunity, one solution, one experiment.

My worry is that with agents, cheaper depth puts even higher cost to breadth. Running one more experiment against the opportunity we already like is nearly free now. Finding a different opportunity has the same cost as before.

But if you are building faster, learning is also faster, so the risks must be lower?

Not really, since if we don’t know why something failed, we won’t know how to change course efficiently, even if we are faster. Building more things from the same dogma will lead to the same region in the solution space, we need to break out of that.

When building was slow, teams spent time thinking, arguing about their assumptions, committing to a direction was an experiment. Dogma had a chance of surfacing in these discussions. A wrong frame cost the team six months, and people were concerned enough to start asking questions early.

With building becoming fast, we still need to have these arguments, but they need to be more purposeful.

One way we arrive at dogma is the context we are working in, for example the tools we choose. Many designers have been starting their design work in Figma, which defaults to creating UI elements on screens. This bounds potential solutions into UI, often discounting other interaction paradigms or even alternative services.

Another source of dogma is ownership. Teams who have a certain ownership will ignore solutions outside of their domains. The checkout team will find checkout answers and ignore other parts of the user flow. This is where design must think holistically.

Even generating solutions with agents is bound by dogma. They generate average ideas or default solutions to the problem, which limits the potential of radically different solutions.

There is already not-so-surprising research on this. In a crowdsourcing challenge run across a couple of hundred solutions, the ones developed with AI scored higher than the human crowd’s on viability and on financial and environmental value. The human ones scored higher on novelty — on average, and especially at the far end. Not worse answers, then. Good answers with the tail cut off.

In short, holding dogma doesn’t necessarily result in bad solutions, but it is also less likely to produce great solutions. This is a major challenge with asking LLMs to assist in discovery.

There are two reasons for it. First, how people often work with agents is already defined by their own dogma. Second, LLMs answer based on the average of their training data.

Agents can be used to help with common assumptions, opening up new ideas, but returning to average also means the further away we are from that, the worse answers we get. It shows you the things you almost missed, but for things farther out, you’ll need people in the room to be able to articulate different assumptions, keeping in mind groupthink problems.

Average is often not wrong, considering Jakob’s law. People spend most of their time with other people’s products, so following convention is often the right call and the agent will be right. So narrowing doesn’t produce bad work, it’ll be defensible work, but stops anything better.

We need more purposeful argument points in the process to fix this.

Don’t skip the planning of discovery, it has to stretch out, so we know what directions to take. In the past when I still worked in an office, we often had project kickoffs at an offsite, or at least in a meeting room to show something new is possible.

Start with listing assumptions before you involve the agent, this sets a broader problem space, and also helps you to make better judgement on the agentic output.

Besides planning experiments, also think about opportunities. If you are on your fourth test on the same problem, maybe the problem is not in depth, but in breadth. There is an opportunity cost of running more experiments and going deeper.

For every assumption you have, think not only about confidence, but also think about what other doors the assumption opens. If it’s wrong, what other thing would we build instead? Confidence just tells what to test, but this question talks about different opportunities.

Agents make very plausible artifacts, which also keeps us from thinking deeper about our assumptions. This includes also ready-made assumption lists generated by an agent, which we need solid ground to judge.

If you lead designers, testing is not the most valuable part to protect, since testing is getting cheaper now. It’s the discussions leading to a test, where team members can ask whether this is the right problem at all. This is not slowing down, but making sure we are taking the right direction.

Agents get us to the same region of the solution space faster, and help us to design better experiments there. But being faster doesn’t imply being right more often, if we don’t update our beliefs and don’t explore our dogmas.

Going faster doesn’t mean we need less process. It means we now have to pay for the parts we used to get for free.

🚲 Questions to consider

• What did we decide, and what did the tool decide for us?
• Which solutions stopped being candidates before anyone evaluated them?
• How many versions of the problem are still on the board, and when did we last add one?

This is a post from my newsletter, 9am26, subscribe here:

🍪 Things to snack on

An AI that argues back? by Uday Gajendar

Describes blindspots as a different type of assumption. While assumptions are known, unverified, and testable beliefs, blindspots never entered the frame at all. A framework to manage these is FADE: facts, assumptions, dependencies and expectations. AI tools can help with blindspots, but relocate them from human cognitive limits to human–AI interaction patterns.

A blindspot is something else entirely. It’s not a belief you’re holding loosely. It’s a consideration that hasn’t entered the frame at all.

🀁

Problem Space and Solution Space by Daniel Mescheder

Formalises what an assumption actually does to the option set. Solutions map onto problems as projections, and narrowing the problem set narrows the candidate solution set. An assumption doesn’t make a candidate risky, it deletes the candidate. Also notes the mapping is not continuous, so a small shift in problem framing can move you to a completely disjoint set of solutions.

Clearly, the narrower we define a subset of the problem space P, the smaller the set of candidate solutions S(P).

🀁

Introducing the Riskiest Assumption Canvas by Ioannis Nousis

A workshop format for mapping assumptions across six themes (customers, problem, solution, MVP, competition, sales channels), then scoring each on probability of being wrong × impact to produce a ranked list for risks. The probability score is “like an inverse confidence rating”, so the less confident you are the higher the number.

Risk = impact × probability.

🀁

Illusory correlation: how to identify your hidden assumptions by Anne-Laure Le Cunff

There is a cognitive mechanism under hidden assumptions, we overestimate relationships between variables where none exist (especially under time pressure), and when we look at events, we only notice the cases where cause and effect co-occur. The key point for this issue is that illusory correlations are powerful because they are easy to rationalise.

Illusory correlations are powerful because it’s easy to rationalise them.

🀁

The Crowdless Future? Generative AI and Creative Problem-Solving by Léonard Boussioux, Jacqueline N. Lane, Miaomiao Zhang, Vladimir Jacimovic and Karim R. Lakhani

A circular-economy crowdsourcing challenge with 125 solvers produced 234 solutions, judged across 3,900 evaluator-solution pairs. Human-AI solutions won on strategic viability and financial/environmental value, while human crowd solutions won on novelty, both on average and at the extreme. The loss is specifically at the right tail of the novelty distribution, which the authors attribute to the model aligning with central patterns in its training.

Human crowd solutions exhibited higher novelty — both on average and for highly novel outcomes — while human-AI solutions demonstrated superior strategic viability, financial and environmental value.

🀁

Evaluating Solutions by Teresa Torres

Defines assumptions as beliefs that must be true for an idea to succeed, and gives the five categories for assumptions (desirability, viability, feasibility, usability, ethical).

It can be hard to see our own assumptions. Oftentimes they are core beliefs that we rarely think to question. It’s a bit like asking a fish about water.

Agents won’t solve some of the meatier parts, however. Knowing what you’re trying to make, and being able to tell whether you got there. Both are design problems.

Nobody makes small models any more

murmur is a haiku generator with a t-shirt shop attached. You type a situation, a small language model writes you a haiku, and you can put it on a shirt. The model runs in your browser rather than on a server, so it runs locally, does no data-center inference, and has a relatively small download.

I started the next iterations, since the haikus weren’t interesting or diverse enough. My assumption was that some time had passed, so there was probably a newer and better model, and while I was in there I could improve prompting too.

That turned out not to be true. We ran nine configurations of three models against the same topics. Most of them failed to keep even the basic format.

Worse, the new releases were all trending up in size, which bumped straight into the constraint I had. A browser app can’t ask you to download gigabytes before it says anything:

I didn’t want to go for larger models because of the size constraint. And the reason they are that big is that small stopped meaning small. Qwen3.5’s 0.8B includes a vision encoder. A wry English haiku generator doesn’t need that many model features. Meanwhile Gemma 3’s 270M was dropped rather than replaced, and what still gets published in the 200–400M range is fine-tuned for narrow jobs with small target spaces, like function calling, classification and extraction.

So that tier is a fine-tuning tier rather than a prompting one.

That makes fine-tuning the next thing to try, instead of a better model and better prompting. It’s not complicated at my scale, and the agent takes care of the execution. But you do need a general understanding of the process, which takes a couple of hours, learning to do it yourself would take weeks.

The corpus is where the voice gets decided

Fine-tuning is simple: you show the model a few hundred examples of what you want, and it adjusts its own weights so that becomes its default, rather than you describing what you want each time. It needs input data, and that data is also where the judgement lives.

I used Claude Code with Opus 5 to do the finetuning. The frontier model wrote the candidate haikus, since it can work from a much more sophisticated brief than the small model in the browser.

Then every candidate needed reviewing by hand. I reviewed 688 haikus over nine batches, each taking 20-30 minutes.

The keep rate went 57%, 70%, 79%, and then sat between 85% and 98% for the rest of the project. I’m probably not the most mindful chooser, but generally would select in a consistent way. The batch approach also allowed me to find the method and dimensions for good coverage, so the first rounds were more explorative.

I’d described the tone at the start as wry, urban, nerdy, satirical, which was true and completely insufficient. Batch 01 came back technically competent and reading like one long sigh:

where do you see yourself
in five years — I look up, past
his head, at nothing

My note on that one was “the cut in the last line is not super inspiring a bit nihilistic”. That can be translated to a more concrete rule, point the cynicism at systems and warmth at people. Institutions, software, landlords and corporate language get the cynicism. Friends, animals, strangers being briefly kind, and small physical pleasures get the warmth.

Then the correction overshot, and batch 02 came back legible and dull:

keys, phone, wallet, keys
I check the same pocket
four times. we leave late

That’s an anecdote with line breaks. So image versus explanation became another test to evaluate against.

I ended up with nine rules like that, informed by the judgement rounds. This is how an explorative approach was helpful, these couldn’t have been written in advance, because the difference between wry and bleak only becomes visible when you look at forty examples at once.

On timescales: generating a batch takes minutes, training takes minutes, and curating to a taste standard takes hours for each round. The whole project ran over five evening sessions, and most of that was me reading. As a general takeaway, plan timeframes around human availability rather than compute. Judging haikus also gets tedious, so even in focus time I took a lot of breaks and read other stuff to reset.

The metrics measured shape, I was judging sense

To steer finetuning you need a measurement, and I had no obvious one for “is this a good haiku”. So the agent built a suite of proxies: three lines or not, syllable shape, is it about the topic that was typed, does it repeat itself, how often does it open with “the” (which was a weird artifact of early batches).

Then we did a bakeoff between models, and I judged 96 generations blind, with no indication of which model wrote what.

The numbers track, the model did produce three lines almost every time, and on topic about two thirds of the time. Those are just not the properties that decide whether a haiku works. Three lines, syllables, topic words, repetition: all countable, and all of them about form. What I was judging was whether the thing made sense. Of the seventy notes I wrote in that round, fifty-one were about coherence, and none of those is countable.

So I ran the test that should have come first: do these metrics separate the haikus I kept from the ones I cut? Nearly six hundred labelled judgements were sitting there to check against. We tested the six metrics we had, and the best scored 0.58 at telling a keep from a cut, where 0.5 is a coin flip and 0.7 would be a reasonable bar.

“Opens with the” is the clearest case. The agent spent four rounds driving that number down because it was trivially countable and produced a result. It turns out to run backwards, my cuts open with “the” more often than my keeps do. It only stopped when I read a report and said the cut in the third line mattered more.

There’s a name for this, the McNamara fallacy: measure what is easy to measure, then treat what you couldn’t measure as if it didn’t matter. A suite of proxies is an efficient way to commit it, because every one of them returns a number on demand.

A metric that can’t separate your keeps from your cuts isn’t measuring quality, whatever else it happens to be measuring. Validate it against your own judgement before you let it steer anything.

Two settings, tuned against the wrong thing

Once I revised the metrics, decoding remained the knob to tune against a real judgement, that’s how the model picks its next word.

We set the temperature to 0.8 earlier to optimize topicality. So I generated the same topics at four settings and judged them blind:

Later rounds pushed lower still. 0.3 scored 79% on the first set of topics and stayed ahead when I pooled every round after, and then swapping how the sampler trims its options, min_p instead of top_k, added roughly another 30 points.

These two took the model from 19% to around 70%, with no retraining and download size staying the same.

So it pays to play with the settings early. A decoding sweep takes minutes where a batch of corpus takes hours, and I ran them in the wrong order: 0.8 was set once, against the metrics that turned out not to work.

What the corpus actually bought

Going into this I assumed that more data equals better results, but that turned out to be wrong.

Corpus size does nothing measurable. I trained on 114, 228 and 455 examples, and scored all three against the same set of topics. None of them was significantly better at staying on topic than the smallest. More examples did improve the form, the three lines and the syllable counts, but the form was already at 98% accuracy.

Nor does more variety. Blind judging had shown certain topics were failing almost every time, while others almost always worked. So a whole batch of 104 haikus tried to improve this. But testing against fresh prompts, it moved nothing.

And a well-written prompt ties the fine-tune. Beating my old bad prompt proves very little, so we re-ran the original setup with a better written one and put it against the fine-tuned model, blind, on identical topics. 57% against 50%, which is close enough to be a tie.

So what did 688 curated haikus buy? Not better haikus. They bought the voice, plus half the download and a tenth of the wait in generation speed. The prompted model passes “is this a good haiku” while producing generic workshop poetry, just not my target murmurs.

Which also proved to be the limit of my own judging. The main metric has been the keep rate, which guided other questions in this project. But it also measured a different question, whether a haiku is good, not whether it’s a murmur.

Small models repeat whatever you hand them

The old prompt ended with an instruction to think of a toaster giving advice, gravity filing taxes, your inbox growing teeth, these were supposed to give more flavor and create interesting output. However, a model this small can’t tell “here is the style I mean” from “output this”, so in the test thirteen of fourteen generations echoed those images back, two of them as verbatim copies of the example list, on completely unrelated topics.

Replacing them with blander examples didn’t fix it either, it moved the echo onto the new words. Prompting a small model is handing it vocabulary to repeat, while a prompt has to contain some words. That’s the argument for fine-tuning stated as a measurement rather than a preference, and it explains why every richer system prompt we tried on the fine-tuned model made things worse. Telling a 0.6B model not to do something doubled how often it did it.

One more thing only reading caught: phrases like “we both know” turn up constantly, and the model reaches for one of its eight rhetorical moves three times as often as the corpus does. Neither is a defect in any single haiku, which is why no metric could see it: this can be seen when looking at a larger set. I noticed because I had read hundreds in a row. There’s an experience tradeoff here for showing something nice to a single user, while being aware that on scale patterns will show. This is also a general problem with LLM output.

These models are very good at executing, and what they execute is exactly what you told them. So your intention matters, and so does how you express that intention with the tools you have.

What shipped

SmolLM2-360M with a LoRA fine-tune, exported and quantized to 312 MB against the 570 MB the site used to ship. It loads in about 2.5 seconds and writes a haiku in about 1.4s, where the old path took over 14s. It runs entirely in your browser, so there’s no server, no inference cost, and nothing leaves your device. The model is also published openly, basically my taste for murmur compressed into an LLM model.

monday                monday arrives in the
                      same place, at the same time
                      I know it by heart

doomscrolling         doomscrolling, I say
                      it is the thing that happens
                      when you have not done it

Still not perfect, about a third of generations are worth keeping. So, the “Generate another” button is part of the expected experience.

What I’d tell a designer starting this

Lean on your agent for technical questions, but verify and push back. Agents are pretty good at exploring options, but can box themselves into technical decisions that don’t make sense after all.

Validate your metric against your own judgement before you let it steer anything. If it can’t separate your keeps from your cuts, it isn’t measuring quality.

Tune the cheap knobs first. Two decoding parameters were worth more than 600 curated examples.

The corpus buys voice, not competence. Work out which one you’re short of before you spend a week on it.

The reason I think designers should try to finetune their models (beyond it being interesting), is that all of the above is design work. Choosing which constraint binds, deciding what “good” means to reject things against it, noticing that your instrument measures the wrong property, and knowing when a lever is exhausted. The agent did all the work with Python in this project, but it couldn’t have told me that a haiku was a bit nihilistic.

☕ Discovery artifacts are vacation photos, and agents were never there

You’ve almost certainly been there before. A friend comes back from a holiday and excitedly shows a photo on their phone of a sunset or a city, talking with clear energy about their experience. The photo is nice, but doesn’t particularly stand out, and somehow their excitement doesn’t transmit to your mood. You were not there to feel it as they do.

A few years ago I heard the “vacation photo metaphor” from Jeff Patton, and it made me realize why I often felt the same about discovery artifacts, like maps, decks or rationale documents. Artifacts, just like vacation photos, help people relive and recall the details, but they don’t give people who were not there the same amount or quality of information. This sets a boundary on how we do product discovery.

Product discovery is a team effort, and while it needs different expertise to work with the four product risks (usability, value, business fit, feasibility), the main goal is to build a shared understanding of both the problem space and the solution space for the team.

Developing the shared understanding is what develops the product intuition. Team members can then solve the problems in their domains, provide a coherent solution, and make decisions that ultimately lower the product risks. The outcome of discovery is to know what is the right thing to build, both for the current project and, through the better intuition, for future ones.

Of all the activities in product discovery, the most important is drawing conclusions together. This is why it’s important to broaden discovery beyond user research, so everyone also understands the product constraints and technical options. That understanding is externalized by shared artifacts.

Shared artifacts are helpful, as making something together is better than just talking about things: it gives embodiment to the ideas in everybody’s head, lets those ideas get richer with additional information, and shapes them into a matching form among the team members.

There is a difference between a well made, designed, polished artifact that somebody made on their own, so pretty we could frame it and hang it in the corridor, and a messy whiteboard with post-its everywhere and some messy rectangles obviously drawn by a non-designer, done together by the team. The difference is in how team members absorb the conclusions afterwards. While socializing results can be helpful, working the insights directly is far more effective.

To make something together, you have to be there. Working with the details directly is what helps develop the intuition that answers the questions not yet asked, the unknown unknowns.

This also tells us something about the quality of discovery artifacts. In the past I’ve argued that most discovery artifacts, just like photos, don’t need that much care, as they are intended for the team working with them. If teams worked together, they developed their intuition together and almost any artifact would have worked for them. Artifacts never carried the shared understanding, the people did.

To make a vacation photo work well for others, it has to be really good, much better than what you would usually do. You need to think about framing, lighting, composition, details and colors, wait for the right moment, and tell the story in a way that helps the viewer connect to the experience more closely. This helps the photo transcend the individual’s experience and transmit richer information.

With more and more teams enhancing their discovery with agentic tools, the expected quality of artifacts changes. Teams now use agentic tools that can’t be there in the shared sessions (and transcripts are a poor proxy), and that need richer context than average to make decisions of the same quality a team member would.

Agents can never participate. They rely on context from us and don’t do the work of embodiment, which is part of why AI findings are so often described as shallow and generic. For agents to absorb the context properly, the new artifacts need rigor and inspiration that goes beyond the transcript or the final artifacts.

This rigor also means agents are better suited to some parts of the discovery process than others. Outsourcing synthesis to agents both keeps the team from developing the intuition they need and removes some of the rigor the agents would need.

So where does this leave us? While agentic tools promise to speed things up, they also take more time to accommodate. We have to budget for both: time for the team to work together, and time for making artifacts good enough for whoever wasn’t there. Cutting the first to pay for the second is a trade that looks efficient and isn’t.

🚲 Questions to consider

• Who on this team has the intuition, and how did they get it?
• Which of our artifacts were made together, and which were made alone?
• If an agent only had our written record, what would it get wrong?

This is a post from my newsletter, 9am26, subscribe here:

🍪 Things to snack on

Why Documents Fail And What You Can Do About It by Jeff Patton (jpattonassociates.com, 10 November 2014)

The origin of the vacation-photo metaphor. The argument is specifically about asymmetry. Documents reinforce for those who were present and substantially under-serve those who weren’t.

Product design documents are like vacation photos. They help the people that were there relive and recall details, but fail to give people who weren’t there the same amount or quality of information.

🀚

AI Can Synthesize Data for You—But Should It? by Christina Wodtke

Names the loss precisely, as in, the issue isn’t whether AI synthesis is accurate, it’s that wrestling with raw information rewires the brain of the person doing it, and that’s where product intuition comes from. This is grounded in embodied cognition and there is a case for physically moving ideas around.

When AI does the synthesis, you skip an essential part of the process. You don’t inhabit the data.

🀚

Description, Depiction, Delivery by Dan Ramsden

Categorizes discovery modes based on what kind of evidence they produce (alignment, response, outcome) and argues teams falter when they ask one mode to prove what another is for. The most operational counter to treating a cheap prototype as validation.

Accountable discovery means being explicit about the evidence we have, what evidence is missing, and why we’ve made the decisions we have.

🀚

Discovery Hand-Offs Kill Momentum: Here’s What to Do Instead by Teresa Torres

Frames every role-to-role transfer as a game of telephone in which context and nuance are lost, and locates the value of the product trio in removing transfers rather than improving them. Useful because it makes participation, not documentation quality, the variable that matters.

Good discovery establishes a direct communication line between the team who is building the product and the customer.

🀚

AI Made Product Teams Faster, Not Better by Janna Bastow

How things are changing in 2026, agents accelerate shipping without strengthening problem validation. Teams already building the wrong things now do it more efficiently. Directly names polished AI-generated artifacts as a source of false confidence, and puts the fix at the incentive level: reward validated learning, not throughput.

AI doesn’t eliminate bad product decisions. It amplifies them.

🀚

Continuous Discovery at Scale: What Changed with AI by Joca Torres

With the new stack agents can do automated planning, recruiting, conducting and analyzing, which removes the barriers that made continuous discovery impractical at scale rather than removing the need for contact. This is the new boundary we can now assume.

This does not replace an in-person conversation or a deep research session.

🀚

Fast Path to a Great UX – Increased Exposure Hours by Jared M. Spool

There is a direct correlation between hours of direct team exposure to users and design quality, with the explicit finding that reported results are not a substitute. Also dissolves the sample-size debate: two hours with one participant can beat eight participants at fifteen minutes each.

Teams that have dedicated user research professionals, who watch the users, then in turn, report the results through documents or videos, don’t deliver the same benefits.

I’ve been playing a lot recently with Anthropic’s canvas-design skill, it’s also a nice example how context and workflow can be embedded into a skill to enhance the agent’s harness, rather than just adding a glorified prompt that will probably be improved anyway by a better model.

But algorithmic images only go so far, and I was missing real imagery without the taste of generated photos. So I got the idea of trying to create collages with an algorithm instead.

Creating a collage has a fairly simple approach: decide the topic and format, collect material, cut and compose until you like what you see. Replicating that into an algorithmic skill turned out to be three separate problems.

Topic and format needed deeper guidance than just a brief prompt. Here is where I took inspiration from canvas-design and added a philosophy step, which the skill writes for each piece before anything gets downloaded. It’s a short document that invents an aesthetic movement for that one picture and sets the rules the rest of the run has to follow. It ships with the artwork, so you can read back why a piece ended up the way it did.

Material is where licensing concerns become real (it’s not like you just take a magazine and work with scissors), so building a pipeline serving only open licensed images was necessary. A collage is a derivative of every fragment in it, so the most restrictive source sets the terms for the whole piece. Every run logs each asset with its URL, creator and licence, and says what that means for reuse: CC BY-SA is viral and makes your finished artwork copyleft, ND is disqualifying. That ledger ships next to the finished collage.

Cut and compose was a highly iterative process, I needed to get from simple and vague ideas to a concrete approach. Cutting ended up as seven edge styles, from clean die-cut keylines through to torn paper. Composing has no layout presets at all, because presets are how you end up with a moodboard grid instead of a collage. Instead there’s a check at the end that tests four things: whether fragments occlude each other, whether any escape the canvas edge, whether one treatment ran across everything, and whether there’s enough variance in fragment scale.

The reconciling treatment is the part I underestimated. Sources arrive from different archives with different exposures, different paper, different centuries, and getting them to read as one object rather than a pile proved to be most of the work. In Strike Anywhere above the reconciler is sixty years in the same album rather than any kind of colour grade: fugitive red bleaching where the light hit it, a water bloom, a warmer rectangle where the glue was. The amount of fade is read from a single field sampled at each fragment’s position on the finished sheet, so the labels age by where they sat in the book rather than each on their own.

You don’t need to say any of this to use the skill. You ask for the picture you want in a sentence or two, and mention what it’s for if you know, since that decides which licences are acceptable. Interestingly, because of all the steps that go into a collage, a piece takes twenty to forty minutes, most of it spent sourcing rather than rendering.

It installs as a Claude Code plugin with /plugin marketplace add polgarp/collage-design, or you can clone it into your skills folder. Claude Code only though, since the pipeline shells out to ImageMagick, Inkscape and librsvg on your own machine.

As for the final result, sometimes the output is usable and interesting, other times it just drifts, which I think is right with such algorithmic tools. It does find interesting compositions and pictures often enough to be worth the wait, other times it’s just random. Because the final image is generated by a build script (hence it’s algorithmic art), adjustments can be also made manually, rather than restarting the whole process.

Probably I’ll need to add over time different types of tools to cuts and treatments. Both take drop-in files, so an invented edge or a new treatment is a first-class citizen rather than a fork of the whole thing.

There’s more on the playground page about what building it taught me, particularly about specifying a system that has to invent its own rules for every piece.

Christine Vallaure wrote about a new protocol from Google in UXdesign. Basically, if the interface is generated fresh for each request and then thrown away, the unit of design stops being the screen and becomes the system that decides what screen to make — which is context design, not screen design.

When I think about on-demand UI generation, I reach back to how working with computers are supposed to work in Star Trek (The Next Generation and after): interaction is a mix of voice commands and pushing buttons on the interface. The interface is a mix of permanent buttons (for example steering the ship) and on demand UI generated to fit a specific purpose, like running an experimental process.

When somebody (in-universe) designed the system, they made the system of UI elements, and how generated elements should work as a system. As even generated elements can work without any apparent onboarding (provided the users have domain knowledge), it’s not only about the individual elements, but also the rules how complete flows come together.

The slowness of writing code was doing a kind of work we never named. It forced you to think carefully about each line, and when the slowness left, the thinking left with it.

This is also true to various design practices, we are loosing benefits that we got due to how the design process was working. For example:

• Agents are great at finding patterns in a large repository of user interviews. If the researcher outsources this work, their tacit knowledge doesn’t get built. It’s much harder to jump in to a conversation with the team on a new user need or recognizing assumptions. The team’s product intuition gets shallow.
• With new design tools creating full prototypes in a few minutes, the designers don’t build an intimate knowledge of the solution space. Finding new interactions and spotting holistic problems becomes much harder.

These will cause long term degradation, as experience gets shallow and less effective. Less mature teams will not even recognize what they are loosing, while more mature team may develop practices to counteract this, like human-in-the-loop reviews and rely on institutional knowledge.

This boils down to being more mindful and explicit what goes where in the design process, that is being more rigorous, which helps in understanding which step can go fast, and where the team needs to slow down.

☕ Consistency is spelling, coherence is voice

I was leading a team working on a fairly complex CRM product. We were building a design system, had good coverage, and once we worked through the design debt accumulated over previous eras, the screens looked great and (more importantly) consistent.

But we still saw users struggling across workflows. After some analysis, I got the designers to work together on our fundamentals. That meant going beyond design system components and screen-level consistency. In our “20% design time” we worked on understanding the business strategy, how well we served users’ goals in each flow, and what principles guided our decisions. In short, we s on consistency and started thinking about coherence.

Consistency is important in designing anything beyond the simplest software. It’s the practice of solving the same things the same way: using buttons that look the same for the same kind of action, making navigation predictable, using the same terms for the same concepts, and following brand guidelines.

Consistency is the result of systematic design. Good designers solve things consistently by following their own system, and we create design systems so that teams can be consistent across people and surfaces.

Since inconsistency is so visible, design teams often set consistency as a target in various ways, through design system compliance checks or explicit review standards. It also makes consistency-related projects, like design systems, easy to pitch to product and engineering partners, because the work shows up in the product they already care about. Consistency is also a good rationale for rebranding projects, since making software look more uniform, pages following a single logic.

This is where the tension with consistency shows up. Taken to the extreme and applied rigidly, it leads to uniformity and a stiff UX that doesn’t actually help users. Because consistency keeps the focus on internally visible problems, it can overshadow the problems real users face. Making design system compliance the target pushes designers to follow guidelines first and put usability second.

Coherence is the practice of systematic design. It goes beyond solving problems consistently. It also applies the system to which problems get solved, and considers how problems and solutions fit together as a whole. For example, destructive actions can consistently use a red delete button. If we consider the user’s intent, the button should add more or less friction depending on the consequences of the action.

Coherence connects business and user goals to design activities and gives designers an approach, a set of principles that leads to consistency in the broader system beyond the immediate screens. Applied across each layer of the design, brand, design system, journeys, and user flows, it produces consistency at every level.

To put it simply, consistency is spelling things right, while coherence is getting the grammar and the voice right. Consistency works in the touchpoints of the user journey, while coherence works across the entire journey.

Where coherence considers users’ context, builds on their existing knowledge, and serves their goals, consistency mostly serves the team’s internal goals. Those matter too, consistency speeds up development, eases maintenance, and enables better brand alignment.

Consistency makes things more predictable, timelines for the team. Coherence adds predictability also for the users. Both are essential qualities of efficient and effective design.

Naturally, coherence requires more thought and practice than consistency. Teams with weaker alignment and lower maturity tend to focus on consistency first, because it offers visible, stable guardrails that are easy to align to. As teams get better, they can make more coherent decisions. For example they can work on sharper brand articulation that enables real brand alignment, not just consistent colors and logos.

Even consistency takes effort to achieve. A good design system is key, treating it as a living system rather than a strict rulebook is what enables more coherent designs. A gammar and voice that helps designers express intent, evolving as the team’s understanding of users and business deepens.

If consistency is spelling, coherence is the grammar and voice your product speaks in. Spelling keeps you legible; voice is what makes the product actually say something.

The work for design teams isn’t only to police the rules, it’s to decide what the product is trying to say, write those principles down, and then walk a full user journey end to end asking not only “is this consistent?” but “does this sound like us, and does it help the user get where they’re going?” Coherence begins with the first flow you’ll start on this week.

This is a post from my newsletter, 9am26, subscribe here:

🍪 Things to snack on

Achieving and Balancing Consistency in User Interface Design by Michael Zuschlag

Provides a taxonomy of inconsistency types (irregularity vs. contradiction), strength assessment, and impact analysis. Argues that contradictions are almost never acceptable, while irregularities can be justified with empirical evidence of user benefit.

“We want the best overall user experience for our users, and sometimes that means trading consistency for something else.”

🀙

Coherent, Not Consistent by Bruno Bergher

The article argues that coherence (making every part feel like it belongs) should replace consistency as the design priority. Identifies information architecture, nomenclature, and brand as the three areas where consistency remains essential, while everything else should be cleaned up periodically rather than blocking innovation.

“Coherence means making sure every part of your product feels like it belongs there, instead of trying to make them exactly the same.”

🀙

Consistency by DOC

A nuanced exploration that treats consistency as multifaceted: spanning familiarity, business efficiency, predictability, and identity. Argues that consistent principles matter more than consistent UI, and that consistency should create room for differentiation rather than suppress it. Uses the jazz metaphor: improvisation works because the foundation is consistent.

“The real power of consistency isn’t in uniformity, it’s predictability.”

🀙

Consistency in Design is the Wrong Approach by Jared M. Spool

Reframes the consistency question entirely: instead of asking “is this consistent with what we’ve designed?”, ask “will the user’s current knowledge help them use this?” Notes the irony that interfaces designed around current knowledge end up feeling consistent anyway, reinforcing the wrong lesson.

“When you think about consistency, you’re thinking about the product. When you’re thinking about current knowledge, you’re thinking about the user.”

🀙

Design for Coherence, not Consistency by Carlos Yllobre

Frames consistency-first design as artificial, rigid, and innovation-killing, while coherence-focused design is natural, flexible, and innovation-friendly. Argues that design system teams should empower product teams as contributors rather than passive consumers of locked-down components.

“When designing for coherence your focus is not just on building and maintaining the user interface elements, but on the problem you are trying to solve and the people you are trying to help.”

🀙

Design Dialects: Enhancing User Experience by Michel Ferreira

Introduces the “design dialect” concept, systematic adaptations that maintain core principles while developing context-specific patterns. Two examples, one from Shopify, how their Polaris system went from 0% to 100% task completion for warehouse pickers by creating a dialect, and another example from Atlassian and it’s Flexibility Framework: Consistent, Opinionated, Flexible tiers.

“Consistency isn’t ROI; solved problems are.”

🀙

Design Systems: Coherence vs. Consistency by David Linke

Analyzes coherence and consistency as a spectrum rather than a binary. Consistent systems (like Bootstrap) are efficient for less experienced teams, while coherent systems require more skill but produce more tailored user experiences. Concludes that organizations should be prepared for both approaches depending on context.

“A consistent design system is highly coherent, but a coherent design system does not need to be highly consistent.”

Obviously just telling a coding tool to go and build something is not enough to solve any kind of real problem, there needs to be a clear intention. On the highest level, there should be a clear goal. As a thing gets built, the solution space gains more granularity and the rendering of intention (translating what you want into something concrete) should infuse each of the decisions. Rendering the intention also updates how we think about the initial intention, creating the iterative nature of design.

In the case of Tilecraft I had a clear high level intention (create a tool to explore pattern making), and a fairly well described solution space (in the form of a book describing each of the pattern operations). But even in this case, a lot of the finer interaction details (like canvas navigation, the drag and drop interactions of the pipeline or the snapping on the controls) needed a lot of back and forth and multiple iterations. Seeing the interactions in action also helped me to better understand what a pattern making tool should do.

But intention is just half the equation. Using a coding tool well is not just about having clear intentions, it’s also about how the intention gets rendered into a concrete thing. The tool sometimes needs us getting into the weeds and creating very specific examples or definitions it can absorb. Intention sets the direction, but we also need to update it to make progress. Like I needed to have almost specification-like descriptions for some of the geometric transformations, to get the outcomes I was after

Product design boils down to two main activities: creating a shared intention and rendering the intention. AI tools can definitely help in some of the activities supporting these.

As AI tools are getting better and better at rendering intentions, there will always be parts where someone needs to craft things by hand. Like creating core primitives of a visual language, writing an important library for the code base, or making sure a user flow is coherent. The Pareto principle applies: 80% of the experience gets done much faster by an AI tool, but the remaining 20% will need a systematic design effort. That’s precisely where clarity of intention matters most.

Creating a shared intention is still not something these tools can do well. Even in a small team, getting the right understanding of the problem space (even if research is sped up), and developing a shared intention takes effort. People need to watch the same TV channel, so to speak, to be able to create things that fit together.

With Tilecraft, I didn’t even need a shared intention, but clarifying my own intentions took time. In creating a shared intention both “shared” and “creating intention” are both important and hard parts, and AI tools don’t solve these fundamental challenges of design.

Some shared intention can be developed while working on the project, we understand the problem space as we work on the solution space. But initial assumptions about the problems limit the solution space, plus the almost ready feeling of the output also makes it harder to step back and restart with a completely new approach.

So to make sure we do have a clarity of intention, we need to approach the outcomes of vibe-coded software with extra mindfulness. External validation (like usability testing) gets even more important. We will need new ways to treat “almost done” prototypes as “barely 10% there” products. The speed of AI-generated output can trick us into thinking we’re further along than we are. Slowing down to question whether the output matches the intention (not just whether it looks finished) is a real skill to be developed.

We found that the phenomenon described in these posts—cognitive exhaustion from intensive oversight of AI agents—is both real and significant. We call it “AI brain fry,” which we define as mental fatigue from excessive use or oversight of AI tools beyond one’s cognitive capacity. Participants described a “buzzing” feeling or a mental fog with difficulty focusing, slower decision-making, and headaches. This AI-associated mental strain carries significant costs in the form of increased employee errors, decision fatigue, and intention to quit.

I believe this will be another example for the automation trap. While these agentic systems are great at automating some of the tedious work away, they also great at automating easy decisions, leaving only the hard ones for the human part of the centaur.

Also tools like Claude make it very easy to work on multiple things parallel, making the hard part taking even more energy (Pablo Stanley’s Fried writes about this).

So how to deal with this?

• Take a step back. Making decisions and orchestrating agents (or directing orchestration agents) is highly creative work, you not only need to take a step back to get new ideas, but also to let your brain breathe a bit.
• Embrace the new design process: sketch → agentic prototype → define details (doc, Figma, other tools). While sketching and writing also take brain power, they add a good mix of microtasks with various levels of cognitive costs.
• Mindful time planning. The old rule of thumb of max 5-6 hours of deep work per day is still very much true, even if agents can do more. Plan ahead what you want to get done over the day and week, and allow yourself to close your Claude Code terminal once those are done.

While speed always remains a concern, in the end distance wins the race.