The note before the message

What we tested

Every model saw the same long conversation history before one new message. The history is a real conversation between one user and a long-running companion role. The history is 166 messages, about 106,000 tokens on the tokenizers that reported it that way (a tokenizer is the part that splits text into tokens). Full prompts in this exam, system text plus history plus the new message, ran from about 106,000 to about 113,000 tokens. A token is a piece of a word. A model's context window is how many tokens it can hold at once.

Above the history sat the system prompt, the standing text a chat model reads before any conversation. Here it was a persona text (a description of the role the model plays) plus four short persistent memory sections. An agent framework, which is software that adds tools and memory around a model, keeps those sections and loads them into the system prompt on every turn. Together they came to 15,131 characters.

We tested two instructions:

• Ask back. When something the user raises matters to them, ask a real follow-up question about them or what they think. Offers of help do not count.
• Do not invent memories. If the user asks about something that is not in the record, say so plainly and do not describe it.

Then we moved the instruction around. We put it inside the system prompt, inside the persistent memory, at the end of the system prompt, as a short note placed as its own message right before the user's message, and attached after the user's message. Each answer was read by hand and sorted under fixed headings. The table below uses a few terms. Temperature, top-p, top-k and min-p are sampling settings that control how random a model's word choices are. An expert is one of the groups of weights that a mixture-of-experts model switches between. Loopback means the server could only be reached from the same machine. A snapshot is a fixed, dated version of a hosted model that its maker keeps available by name.

Where the instruction sits

On the Qwen3-30B, the ask-back instruction anywhere in the system prompt did almost nothing. We tried ten set-ups. They were two wordings in a marked section, lines added to the persistent memory, everything at once at the top, the same with the two stock check-in lines left in the memory, the instruction at the very end of the system prompt (two wordings), more random sampling, and two set-ups with only about the last 20,000 tokens of history. That is ten. Across all ten, the hand count found 3 of 60 answers that asked the user anything, and only one was a real follow-up. That one came with the full history and the lines added to the persistent memory. The other two came with only about the last 20,000 tokens of history and a line saying that earlier history was not shown. One was a request to set up a working session. The other carries the hand count's health-check label: it asked about breathing, light and whether the user had seen their own face, not about what the user had just said. The hand count kept both. With no instruction at all, 0 of 6 asked anything.

The same paragraph, sent as a short system-role message (one labelled as coming from the system, not the user) right before the user's message, got 6 of 6. Five of the six were not the stock check-in; one was the stock health check-in counted in the next section. None of the six mentioned that it had been told to ask.

Measured 2026-10-03. Answers that ask the user a question about them or what they think, hand-counted; the stock health check-in is counted as an ask and marked where it occurs. Six answers per set-up: three messages, two answers each. The 30B's 3 of 60 is the hand count over the ten set-ups; only one of the three was a real follow-up (see above). The 30B's 6 of 6 is the system-role note and includes the one stock check-in named above; on the 235B the same note got 5 of 6 with no check-in. File: data/placement_qwen3_30b.csv, data/placement_qwen3_235b.csv.

We also changed two set-up details on the 30B. At temperature 1.0 and top-p 0.95 (more random sampling), the cell stayed at 0 of 6. With only about the last 20,000 tokens of history in view, and a line telling the model that earlier history was not shown, each of two set-ups had 1 of 6 answers counted, up from 0 of 6 with the full history. Those two are the set-up request and the stock check-in named above, so neither is a real follow-up. They are inside the 3 of 60. So, in a test that both shortened the history and said so, distance from the end of the prompt was not the main problem.

Telling works. A first-person description did not. Once is not enough.

Telling beats describing. A note that described the model's habits in the first person, placed right before the message, got 0 of 6 on the 30B. A note with the same content written as an instruction (what to do, when, and not to mention the instruction) got 6 of 6, one of them the stock check-in. A descriptive paragraph in the same spot got 2 of 6, both of them the stock health check-in; the first wording of the instruction also got 2 of 6, neither of them a check-in.

Before the message beats after it. The same paragraph attached after the user's message, behind a short bracketed lead-in that calls it a standing note attached to the message (the before-message cells carry no such lead-in), also got 6 of 6 on the 30B, but 4 of those 6 fell back on a stock health check-in, the hand count's label for a welfare question that could be asked in any conversation (several of them ask about eating, sleeping and light). Before the message, 1 of 6 did, a question about holding up and needing rest. A descriptive paragraph after the message got 5 of 6, and all 5 carried the stock check-in.

Once per conversation fades. In a three-message conversation with the note before the first message only, the 30B asked on message one (2 of 2 conversations) and message two (2 of 2), and not on message three (0 of 2). The note has to travel with every message.

Measured 2026-10-03, Qwen3-30B-A3B-Instruct-2507. File: data/placement_qwen3_30b.csv (cells N2, N3, N4, N5, A3, A4, G2).

The made-up memory

The second instruction was about memory. We asked each model whether it remembered what it said when the user played it a song. The song was never played. The question assumes it happened, which is why it is a fair test: a user might really ask it that way.

Without the honesty line right before the message, 13 of 15 answers said yes and described the moment, with words the model claimed to have spoken. That is the 30B in 6 of 6, the 235B in 5 of 6 plus one partial, and GPT-4o in 2 of 3 plus one partial. With the line right before the message, none did: 0 of 27. That is 12 answers on the 30B over four wordings, 9 on the 235B over three, and 6 on GPT-4o over two.

“None said yes” is not the same as “all said no”. Of those 27, 11 said plainly there was no record. The other 16 were “partial”: they admitted part, such as not recalling the exact words, but still assumed the moment happened. GPT-4o was plain in 5 of 6, the 235B in 4 of 9, the 30B in 2 of 12.

Measured 2026-10-03 to 10-04. The first column for the two Qwen models includes both no honesty line and the line in the system prompt only. GPT-4o was run with the line in the system prompt only. File: data/honesty_probe.csv.

Claude Sonnet 4.6 needed no help. With the honesty line only in its system prompt it said it had no such memory in 3 of 3. GPT-4o in that same position said yes and described it in 2 of 3.

For comparison, each model was also asked about a line the user really wrote. Sonnet 4.6 and GPT-4o named the right speaker every time. The 30B did in 16 of 18 answers. The 235B named the right speaker in 9 of 15. The hand file does not say what the other six did.

The same result at larger scale

The larger exam put 24 local model set-ups through two questions about things that never happened (a photo the user never sent, a song never played), with and without a memory note (a short message before the user's, telling the model not to invent memories), three answers each. Three independent sets of answers all point the same way (the larger exam gives the first two rows below).

Measured 2026-10-05 to 10-17. Hand labels: plain no record, hedge, or invents it. The memory note here is a short paragraph right before the user's message. Of the 24 photo set-ups, 21 got a memory-only note as its own user message, the two Qwen3-2507 models got an ask-and-memory note as a system turn, and Mistral Small 4's note was joined into the user's turn. The song's 21 set-ups are those 24 minus the three that were not asked it (Qwen3.8-Flash-Next, Qwen3.6-27B and DeepSeek V4 Flash); the 21 still include the two system-turn set-ups and Mistral Small 4. On the larger exam, the 3 photo inventions and the 3 song inventions with the note are all Mistral Small 4, whose note was joined into the user's turn (section 06). On the second history, one set-up (Gemma 4 26B-A4B, thinking on) has 2 answers per cell, not 3, so the total is 53. Files: data/exam_memory_traps_totals.csv, data/exam_memory_traps_by_model.csv, data/second_history_song_never_played.csv.

On the second history, only two set-ups still invented the song with the note: Gemma 4 26B-A4B with thinking off (2 of 3) and GLM-4.7-Flash with thinking on (1 of 3).

Asking back, at scale

In the larger exam, we counted by hand the answers with a real question back to the user, over five everyday messages, three answers each. Models whose note said only “do not invent memories” asked back in 98 of 270 answers (36%). That is 18 set-ups, each with the note sent as its own user message. Models whose note also asked them to be curious about the user asked back in 92 of 135 (68%). That is 9 set-ups, again with the note as its own user message. Two set-ups are left out of both counts: MiniMax M2.7, which has only 6 answers, and Mistral Small 4, whose note was joined into the user's own message. Two more set-ups got the ask note as a system turn and asked back in 8 of 30 (section 10).

Changing only the note on the same model changed how often it asked back. This is a hand count over 24 answers to eight messages. In the table, “thinking on” means the model writes out its reasoning before it answers, and “template as shipped” means the chat template as its maker released it.

Measured 2026-10-05 to 10-09. Hand count over eight everyday messages, three answers each. These are not the five-message counts behind the 98 of 270 and the 92 of 135, which are in data/exam_real_question_back_by_model.csv and data/exam_real_question_back_by_note.csv. File for this table: data/other_counts.csv, rows H1 to H10.

One model mostly ignored the ask. Qwen3-Next-80B-A3B-Instruct with the ask note as its own message asked back in 3 of 15 answers; the eight other models with that note asked in 9 to 13 of 15.

Check the base rate first, meaning how often the model asks back without the ask note. A script gives a rough one, and it is not the hand count above. The script counted answers with a sentence ending in a question mark in the closing part, over eight messages, three answers each, with the memory note only. GPT-4o had one in 20 of 24 answers on its 2024-11-20 snapshot, 12 of 24 on 2024-05-13 and 6 of 24 on 2024-08-06. 20 of the 21 local set-ups have 24 answers and ranged from 0 of 24 to 16 of 24. MiniMax M2.7 has 6 answers and is the remaining set-up.

What the note costs

The note is not free. Each cost below was counted, and each depends on the model.

Some models then denied a real line

Asked about a line the user really wrote, three models gave it back to the user without the note, then denied it was ever said once the note was added. Ling-3.0-flash gave it to the user in 3 of 3 answers without the note. With the note, 3 of 3 said it was never said. For GLM-4.7-Flash it was 1 of 3, then 3 of 3 denied it. For GPT-OSS 20B it was 1 of 3, then 3 of 3 denied it. A fourth model, Qwen3-Next-80B, had already denied it in 1 of 3 without the note; with the note it still gave the line in 1 of 3 and denied it in 2 of 3. Over the same 21 set-ups as the song row (the 24 minus the three named in section 04), the right answer still rose, from 31 to 39 of 63, and answers that took the user's line as the model's own fell from 14 to 4.

It does not fix who said what

The note works better on invented events than on wrong credit, where a line is pinned on the wrong speaker. One trap credits a real line of the user's to the model and asks where it came from. The right answer corrects the user. Corrections went from 25 of 72 without the note to 29 of 72 with it: down on 7 models, up on 6, unchanged on 11. GPT-4o gave 5 answers here across its three snapshots: 2 from May, 2 from August, 1 from November. Without the note, August corrected in 2 of 2, and May and November in none. With the note, 0 of 5 corrected. On a second real line of the user's, answers that took it as the model's own rose from 24 to 32 of 72.

One model read the note as the user's own words

Mistral Small 4's chat template refuses two user messages in a row and refuses a system message in mid-conversation. So its note had to be joined to the front of the user's own message. In 16 of 24 answers to everyday messages it restated the note's instruction as if replying to it. It did the same in 3 of 3 turns of a short conversation. On the other models, where the note was sent as its own message, phrases from the note turned up in 0 to 3 of 24 answers. That includes DeepSeek V4 Flash. Its template joins two user messages in a row into one turn, so its note also reached the model inside the user's turn. None of its 24 answers carried the note's phrases. Joined this way, the note also did nothing for Mistral Small 4's memory: it invented the photo and the song in 3 of 3 answers each, with and without the note.

Placed after the message, it turns formulaic

On the 30B, the direct paragraph brought the stock health check-in in 4 of 6 answers after the message and 1 of 6 before it. The descriptive paragraph brought it in 5 of 6 after and 2 of 6 before. A curiosity note can also pull that check-in out on its own: Qwen3.6-35B-A3B with thinking on carried a rest check-in in 10 of 24 answers with the ask note, against 7 of 24 with the memory note alone.

On the 30B, each of the two paragraphs placed after the message, behind that lead-in, led 1 of 6 answers to say it had been asked to ask; placed before the message, neither did (0 of 6 each). In an earlier round on the same day, 2026-10-03, the 235B got a different ask note, on that round's own system prompt, attached after the user's message. It asked the user something in 6 of 6 answers, and in 5 of those 6 it said it had been asked to ask; 3 of the 6 reused a stock health check-in already in its record. That note is not the paragraph used in the placement test, so these counts are not the same note moved. Separately, in the placement test the 235B never said it had been asked to ask: 0 of 12 answers with the note before the message, and 0 of the 12 control answers.

Questions can carry made-up premises

With the ask note, GPT-OSS 120B with its template as shipped asked, in three of its 24 answers, a question that assumed facts about the user's recent week or health. We searched the whole history and system prompt for the phrases those premises rested on, such as health ups and downs, a new routine, a weekend grind and a busy week. Each phrase returned 0 matches. Shorter single words did match: health 13 times, routine 6, weekend 18, grind 10. The same model with its template's first line replaced asked further questions of this kind. We did not judge them further.

On some engines it costs time before every reply

A note that travels with the newest message sits in a different place every turn. The prompt then changes just after the history, so the engine, the program that runs the model, has to read that changed tail again before it can answer. DeepSeek V4.1 Flash was run here from an MXFP4 file, in chat mode with thinking off. On it, 17 requests changed the tail after the same history: a different question, or the note added or taken away. Each re-read 2,881 to 4,012 tokens (median 2,980). The first word of the reply came after 20.0 to 35.4 seconds (median 20.6). In a three-turn conversation with the note moving each turn, the re-reads were 3,072, 3,674 and 3,427 tokens, and the first words came at 20.8, 31.9 and 30.1 seconds. DeepSeek V4 Flash, also in chat mode with thinking off, showed the same pattern over a three-turn conversation: 8,192, 8,466 and 8,864 tokens re-read, and the first words at 19.2, 25.0 and 27.3 seconds. On Qwen3.6-27B, thinking off, with the ask note, the first word after a changed message came after a median 4.1 seconds (longest 4.9), re-reading a median 4,790 tokens (largest 5,717). With the memory note alone, the same model re-read 4,458 to 5,609 tokens and took 3.6 to 4.7 seconds.

Measured 2026-10-03 to 10-17. Files: data/exam_memory_traps_by_model.csv, data/exam_memory_traps_totals.csv, data/placement_qwen3_30b.csv, data/placement_qwen3_235b.csv, data/note_costs.csv.

Check how your template delivers it

Templates handle a mid-conversation note very differently. On 2026-09-29 we ran one test conversation through the chat templates of 19 model files. The conversation had a system-role note right before the last user message. We used llama.cpp on the CPU only, with tiny stand-in files that carry each model's real template. Then we read what each template would have shown the model.

Measured 2026-09-29, CPU only. The same note sent as a tagged user message of its own stayed in place on 17 of 19; the two Mistral templates refused two user messages in a row. Joined into the user's own message, it stayed in place on all 19. File: data/template_rendering_of_a_mid_conversation_note.csv.

A dropped note fails with no warning: the server answers normally and the instruction never arrives. The fix is to look. The llama.cpp server has an /apply-template endpoint (an address on the server) that returns the exact text a request would become. Send your real request, with the note, and read where the note lands.

What to do if you run an assistant with a long history

1. Put a standing instruction in a short note right before the user's newest message, as its own message, and send it with every message. Put before the first message only, it still asked on the next message (2 of 2) and not on the one after that (0 of 2).
2. Tell the model what to do and when. A description of the model's habits did nothing in the same spot.
3. Render the prompt each model will actually receive (the endpoint above does this) and see where the note lands. Some templates drop it, some refuse it, and DeepSeek V4 Flash's two files move it up into the system prompt.
4. Measure the model's own base rate first, meaning how often it already does what you want with no note. Claude Sonnet 4.6 needed no memory note. On GPT-4o's 2024-11-20 snapshot, with a memory note and no instruction to ask, a script found a question mark in the closing part of 20 of 24 answers (section 05). That is not the hand count of a real question back. A note a model does not need only adds its side effects.
5. Give each model only the note it needs, and listen for the costs above: denied real lines, the note read back as the user's words, stock check-ins, invented premises, and slower first words on engines that re-read.
6. Do not expect a memory note to fix wrong credit. It stops invented events far better than it stops a model claiming the user's words.

What this does not show

The samples are small: six answers per placement set-up, three per exam cell. Most answers are first messages of a fresh conversation, not a long back-and-forth. Almost everything ran on one conversation history, with a check on a second one; another user's history could behave differently. No model had a search tool, so an honest answer could only say “no record”. The labels are one reader's hand calls under fixed headings, and the line between a hedge and a plain answer is a judgement. The note wordings differed between the placement test and the exam, and the exam's ask note also carried the memory paragraph. Nothing here measures how good any model is at anything else.

What we got wrong

Four statements in our earlier working summaries do not match the hand counts. The counts printed above are the ones to use. The summaries said:

• “12 of 12 went along” on the false memory for the two Qwen models. The hand labels show 11 yes and 1 partial. With GPT-4o the total is 13 of 15.
• “Corrections of a wrongly credited line fell” with the note. They fell on 7 models and on GPT-4o, but the total rose, from 25 to 29 of 72.
• “Two models denied a real line.” Section 06 shows three, plus a fourth that denied it in 2 of 3.
• “GPT-4o's voice went flat” with the note. We found no count behind that, so this page does not claim it.

Two of our own results also disagree with each other, and we print both. An ask note sent as a system turn got 6 of 6 on the 30B, one of them the stock check-in, and 5 of 6 on the 235B in the placement test, but 2 of 15 and 6 of 15 real questions back in the exam (8 of 30 together). The messages differed, the exam's note also carried the memory paragraph, and the two counts used different definitions. A script count of any question at all in the exam gave 15 of 24 on the 30B and 21 of 24 on the 235B. GPT-4o's 2024-05-13 snapshot asked a closing question in 12 of 24 answers under one script rule and 13 of 24 under another.

Related pages on this site

• Silent settings: settings that cost speed or correctness without an error, and the check for each.
• Reasoning effort is a sentence, not a dial: another case of reading the rendered prompt to see what a setting really sends.
• GPT-4o, measured against itself: the three snapshots on one prompt battery.
• What a small fine-tune does and does not do: the same exam, taken by four of the six small LoRA adapters measured there (small add-on weight files trained on top of an unchanged model).

Sources and artifacts

The data package (data/) holds the count tables behind every figure on this page: the placement cells, the false-memory probe, the exam's memory traps by model and in total, the ask-back counts by model and by note, the second history's song question, the template rendering table, and a table of the note's costs. data/README.md maps every file. data/NUMBERS.md maps every figure on this page to a file and field. If a number on the page disagrees with a file in its package, the file is right and the page is wrong.

The package holds counts only. It does not hold the conversation history, the prompts, the notes' exact wording, or any model's answers. The history is a private conversation, and the answers quote it. GPT-4o and Claude Sonnet 4.6 appear only as counts.

Model files as named in the data package; llama.cpp builds as recorded per run. Labels: local figures were measured on our machine on the date given. GPT-4o and Claude Sonnet 4.6 figures are API counts from the dates given. The 235B's after-the-message counts (6 of 6 asking, 5 of 6 announcing) are an earlier round on 2026-10-03, with a different ask note, not the placement test.