A picture for every card, and why it got it wrong

Until 12 May, a recipe card opened on a generic drawing picked by its kind. That afternoon it got a real photo, generated a few seconds after the card was created and pushed into the open card without reopening it. Getting there meant teaching the generator to show the whole dish, taking away the language model's right to trigger the picture itself, and forbidding it from painting people everywhere.

A drawing that said "recipe", not "sauerkraut"

Since the day before, every card declares its illustration kind in its slots: hot dish, cold dish, drink, baking, travel, idea, book, place. At the top of the card, a vector composition was drawn from that kind: some shapes, a gradient, a hint. It said "this is a recipe". It didn't say which one.

The morning had gone into making the card fixable by hand. In the afternoon, we wanted the card to look like what it holds. A choucroute recipe with a photo of choucroute. A gift idea with the object. A travel note with the city.

What follows is the story of that afternoon, and of the four ways the picture got it wrong before it got it right.

One photo, two ways to get it

The first version relied on two triggers.

The first was handed to the language model. After creating a visual memo card, it had one more tool available: request a picture, passing the card's identifier and a subject, a short phrase describing what the photo should show. Its guide told it when to use it, recipes, trips, gifts, reading notes, events at a named venue, and when to hold back: reminders, tasks, contacts, facts, meeting minutes. Meeting minutes don't have a photo.

The second was given to the user. In the small menu at the top right of the card, next to "Edit" and "Delete", a new entry: "Generate a picture". If the first photo missed, you could ask for another without going back through the conversation. During the few seconds it took, a discreet line appeared, "Photo being generated…", in blue rather than red, because it isn't an error.

Behind both paths, the same chain on the server. The subject is wrapped in a photographic style instruction, sent to an image generator, and the picture comes back as bytes. Those bytes are stored under the card's name, overwriting any previous picture, with a version number in the address so the phone doesn't pull the old one out of its cache. Finally, the address is written into the card's metadata, next to the template and the list, without touching anything else. Along the way the server checks that the card really belongs to whoever is asking, signed-in account or anonymous.

If the service key is missing, the server says so at startup and cleanly refuses the tool instead of crashing in the middle of a conversation. If the generator fails, the card keeps its drawing. The picture is an extra, never a condition.

Sauerkraut with no sauerkraut

First real test: a choucroute for six. The photo showed sausages, carrots and potatoes, nicely lit. Not a shred of cabbage.

The cause was in the subject passed along. The language model had sent something like "Choucroute, 6 people, 2 h". The image generator doesn't know choucroute the way an Alsatian reader does: to it, that's one word among others in a short phrase, and it clings to what it knows how to draw. Two fixes followed.

The style instruction, on the server, received a sentence that carries weight: the picture must depict exactly this subject, with every named element visible. Without it, the generator takes the subject as a mood. With it, as an order.

And the tool's description, on the language model side, was rewritten with examples of failed and successful subjects. "Choucroute" gives you a cabbage. "Choucroute garnie on a large stoneware dish, a mound of white fermented cabbage in the centre, smoked sausages, sliced pork knuckle and bacon, steamed potatoes, pale wooden table, warm light" gives you a choucroute. The rule fits in one question: if you removed the dish's name from the phrase, would it still describe the same picture? If yes, it's too vague. You need the dish, three to five visible elements that set it apart, and the serving context, dish, plate or board. Twenty to forty words, no verbs, no quotes.

The italic line that repeated the question

The same test exposed a flaw unrelated to the picture, fixed straight away because it lived in the same place.

Under the card's title, an italic line repeated the original sentence: "I'd like to make a choucroute this weekend, can you…". Read on the card a week later, that sentence is noise. You know you asked for it, you don't want to reread how.

That line changed nature. It became an editorial standfirst, one or two sentences that introduce the card like the opening of a magazine piece: "A generous Alsatian dish for a winter Sunday: fermented cabbage, smoked meats and steamed potatoes." The field keeps its name, it shows in the same place, only the meaning of its content changed. In the card editor, the hint went from "original sentence" to "editorial summary".

That standfirst had a happy side effect. The card's "Generate a picture" button has no language model at hand to write a subject. It assembles what the card already holds, the title and the badges. With the standfirst added, it finally has visual elements, cabbage, smoked meats, potatoes, where the original sentence carried none.

The identifier that didn't exist yet

The second flaw wasn't visible in the photo. It was in the log.

The language model, asked to call the generation "immediately after" creating the card, took the instruction literally: it called both tools at once, in the same turn. But the card's identifier only exists once creation is finished. Rather than wait, the model made one up, plausible, something like "card-tartiflette-recipe". The generation went off to a ghost card, failed, and the real card kept its drawing.

You can reword the instruction ten times. You can also take away from the model a responsibility it isn't built to hold: the order of operations. That's what we did.

From then on, the server decides. When a creation ends up as a memo card whose illustration kind is visual, hot dish, cold dish, drink, baking, travel, idea, book, place, it launches the generation itself, with the real identifier, since it has just written it. It doesn't wait for the result: the reply goes back to the model right away, and the picture is made in the background. The subject is built from the card's title, editorial standfirst and badges, the same signals the manual button uses, so both paths produce photos of the same nature. Reinforced cards, the ones where a request matched an existing memory instead of creating one, trigger nothing: they already have their picture, or never wanted one.

What remained was getting the picture into the card the user is looking at, without asking them to close and reopen it. When generation succeeds, the server sends a real-time message to every device of the owner: this identifier, this picture address. On the phone, that message lands on a tiny notice board, indexed by card identifier. An open card subscribes to its own identifier there, and swaps its drawing for the photo the moment the notice goes by, about three seconds after creation. The board also keeps the last address announced, for the card that opens after the message rather than before: it finds it on arrival.

The generation tool stayed with the language model for one use only: an explicit request from the user, "redo the tartiflette photo", "replace the picture with the plated dish". Never chained with a creation.

Two photos for a single card

The third flaw came through two complaints that had nothing in common.

The first: in the card list, the thumbnail still showed the drawing even when the card had its photo. The list built the thumbnail without passing it the picture's address. One line to forward it, and the card shows the same picture outside and inside.

The second was more interesting: some cards received two photos back to back, a poor one and then a better one. Despite the rewritten instruction, the model still sometimes called the generation after the creation, racing the server's automatic trigger that was already doing the same job. Two generations for one card, and the last to arrive won.

We stopped negotiating. The tool was removed from the list the model sees. The function still exists on the server, for the route the card's button calls, but the model can no longer trigger it. And for the case where the route is called twice in a row, from two phones for instance, the server keeps a table of generations in flight, per card: a second request for the same card latches onto the promise already under way instead of starting from scratch. Belt and braces, but the belt was the real fix.

Evening: two cards flashing, and people everywhere

At the end of the day, one last test brought up two flaws at once, again unrelated.

"Remind me tomorrow at 8 a.m. to go to the hardware store for cement and straps." On screen, a card appeared, empty, a bare appointment, then was replaced by a second one, with the list. An unpleasant flash, and a duplicate in the card list.

The log showed the sequence: a creation of type event, then a call to schedule the reminder, then a second creation of type task carrying the list. The model had split one request into two memories. Each creation opens its card automatically, and the debounce meant to absorb that kind of burst was set to three hundred milliseconds: enough for two creations back to back, not for two creations separated by another call.

Two fixes. In the model's guide, a block at the head of the memo section: one request, one main neuron. An appointment with a list is a dated task carrying its list, not an event plus a task. If in doubt between the two, it's a task as soon as there are things to tick, an event only when there's nothing to tick. And the debounce went up to two seconds: the card opens two seconds after the last creation, so if the model still splits, only the last card reaches the screen. On a simple request those two seconds go unnoticed, because the spoken reply is being prepared at the same time.

The second flaw concerned the pictures of those shopping lists: they showed women. A woman in a hardware aisle, a woman in front of bags of cement. Nobody had asked for a character.

The cause was in the first words of the style instruction: "editorial photography, magazine quality". To a generator raised on magazines, those words mean fashion, and fashion means models. The subject came after, too late to change the scene.

The instruction was re-anchored: still-life and object-scene photography, with explicit clauses, no people, no faces, no silhouettes, no portraits. The generator takes those prohibitions literally, exactly as it took "magazine" literally. Hardware aisles went back to being aisles.

What the afternoon taught

Four times the picture got it wrong, and four times for a different reason. A subject too thin. An invented identifier. A race between two triggers. Two style words that weighed more than the subject.

Two lessons stayed. The first: an image generator doesn't interpret, it executes. It doesn't know what a choucroute must contain, it doesn't know a shopping list needs no extras. Everything you don't tell it, it invents, and everything you tell it first weighs more than the rest.

The second holds beyond pictures. When an operation depends on the order of things, the identifier after the creation, a single generation per card, it isn't the language model's job to hold that order. We leave it what it does well, understanding what the user wants, and the server keeps what has to be exact. Memory stays the source, conversation the means, and the photo, from now on, looks like the dish.