Describing pictures locally

What it is

A vision model reads a picture as tokens, much as it reads words. On this desktop a 3840 x 2160 picture came to 4,154 prompt tokens with the Qwen3-VL models, against 1,974 for a 1200 x 1600 poster. Reading those tokens is prompt processing; writing the description is generation. On the graphics card both are quick. On the CPU, reading a large picture is most of the time, and two settings decide how long it takes: how many CPU threads the server uses, and how big the picture is when it arrives.

The card is fast, but the models here took 4.3 to 23 GB of memory as Ollama reported it (section 02), and a card that already holds a large model may not have that to spare. The CPU path uses system memory instead and leaves the card alone. This page measures both, with the settings that decide the CPU's speed, and checks what the fast settings cost.

What we ran, exactly

Every figure on this page is measured on this machine on 20 September 2026 unless it is marked arithmetic. The screenshot's counts in section 07 were made on 26 September from the replies recorded on 20 September, and the logo file's pixels were read on 26 September (section 08).

The models

Ollama's library tags, each a Q4_K_M file (the load log's “Q4_K - Medium”). “Memory” is what ollama ps reported with the model on the card at the 32,768-token window.

The pictures

Pictures 3 to 6 and 8 were drawn by our build scripts, which are in the package with the strings each picture is expected to yield.

Observed: six models on the card

All seven pictures at full size, each model on the card at the 32,768-token window. The server reported 4 threads for each of these runs; the request set none.

Card memory is nvidia-smi's reading in MiB, desktop included. The 8B run started with the 30B still loaded from the run before it; Ollama evicted the 30B first (its log, 08:57:36), so that 24,208 is not an empty card. Downloads ran during the Qwen3.6 and 4B runs (section 02).

Three models read everything. Qwen3-VL 30B-A3B, Qwen3.6 35B-A3B and Qwen3-VL 8B found all 39 strings, with warm pictures between 1.31 and 3.82 seconds. The first picture is where the model load shows: 36.97 and 36.11 seconds for the two large models, loaded for the first time that morning, and 3.86 to 5.16 for the others.

The misses say different things. Qwen3-VL 4B's one miss was “house” on the sideways cottage, which it called “an arrow pointing leftward”. Qwen3.5 9B's two misses were correct readings in other words: it wrote “£2/day” and “12 months” where the invoice says “£2 per day” and “twelve-month”. Qwen3.5 4B missed four poster details (the address, the price, the phone number and the contact name) and five of the six chart values, and three of its replies ran to 1,562, 2,048 and 2,048 tokens (section 08).

A smaller window saves card memory. A picture plus the longest reply our settings allow fits in 8,192 tokens (4,158 + 2,048 = 6,206; arithmetic, with the largest prompt recorded here). At num_ctx 8192, ollama ps reported 19 GB for the 30B instead of 22, and 4.2 GB for the 4B instead of 7.9. The card read 22,562 MiB with the 30B loaded (1,150 before) and 7,343 with the 4B (1,148 before), on runs of two pictures (the nebula and the invoice, 4 threads) that found all 15 of their strings. A later 30B run at 8,192 with 16 threads found all 39 strings at 1.23 to 2.95 seconds a warm picture.

For reference, two hosted services. On the same morning the same seven pictures went, with the describer settings we use, to two hosted services. Google's found 39 of 39. Mistral AI's found 22 of 39 with one model (2 of the invoice's 13) and 39 of 39 with Mistral Medium. Every hosted reply ends with a line that repeats the picture's file name, and we count the description only: score.py, which reads that line too, gives the first Mistral run 23, its extra hit the word “earth” in the file name, not in the description. The result files name the provider, not the model version, and the Medium name is the run's own label. We publish only these counts, not the replies.

On the CPU, the thread count sets the wait

Qwen3-VL 30B-A3B with num_gpu 0, the 32,768-token window, full-size pictures, one run per thread count. The first picture is the 3840 x 2160 nebula (4,154 prompt tokens), including the model load; the second is the poster (1,974 tokens).

Amber: over the 150-second limit. The 4-thread run was stopped after its first picture, so it left no reply text and no string count; its figures are from the harness's console log and the server's own timing lines. The 8, 16 and 24-thread runs found all 12 strings in their two pictures.

Where the 4 comes from. When the request set no thread count, Ollama's launch line for the server carried no thread flag, and the server reported n_threads = 4 (n_threads_batch = 4) / 24. With num_thread 8, 16 or 24 the launch line carried -t with that number and the server reported it back. Four is also what llama.cpp's own default chose on this CPU in The wrong flag, where we traced it through llama.cpp's code; we did not read the code of the server Ollama ships. The card runs that set no thread count were launched with the same 4.

Reading and writing peak in different places. Reading the nebula ran at 17.1, 33.6, 47.3 and 54.3 tokens a second with 4, 8, 16 and 24 threads (arithmetic, tokens over reading time), so it kept improving all the way to 24. Writing did not: 10.0, 14.5, 19.6 and 12.6 tokens a second on the nebula's reply, and 18.4, 23.4 and 12.5 on the poster's with 8, 16 and 24 (arithmetic). Over a whole picture, 24 threads were 3.1 seconds faster than 16 on the nebula, and 16 was 3.9 seconds faster on the poster (arithmetic). This CPU has 8 performance and 16 efficiency cores; we did not record which cores the threads ran on.

The smaller model reads faster, but not enough. Qwen3-VL 4B at 16 threads, full-size pictures: 95.69 seconds on the nebula (71.48 reading), 23.07 to 80.80 seconds on the warm pictures (the two slowest, 80.80 and 80.66, were the other 4K pictures), and 38 of 39 strings. Model downloads were running during its first two minutes.

On the CPU, the picture size sets the rest

The capped sets were made by one script (in the package): it applies each picture's rotation tag, then shrinks any picture whose longest side is over the cap. Prompt tokens as Ollama counted them for the Qwen3-VL models, instruction included:

At the 1,024 cap every picture came to 1,106 to 1,127 tokens; that run's Qwen3-VL 4B launch carried --image-min-tokens 1024, which suggests a floor there (our reading of the flag). The Qwen3.5 and Qwen3.6 models counted 4 more tokens per picture.

Then the same CPU settings as section 04 at 16 threads, 32,768-token window, all seven pictures unless stated:

The cap took the 4K picture's reading from 87.77 seconds to 18.17 on the 30B, and the whole first picture from 111.01 to 38.52. With the cap, the 30B's warm pictures took 22.64 to 38.93 seconds, close to the 4B's 19.83 to 39.36 at the same cap (174.27 against 172.17 seconds for the six warm pictures together; arithmetic). The log calls the 30B's model type 30B.A3B, a mixture of experts that by that name uses about 3B parameters per token (the name's figure, not a measurement here); on this CPU it cost little more time than the 4B while needing 22 GB of memory to the 4B's 8.0, as ollama ps reported them.

The 4B's one miss. Its only full-size miss was “house”, on the cottage. At the 1,536 cap the cottage was turned upright and not shrunk (1,138 prompt tokens both ways), and that reply contains “house”. With one run each, that is all the evidence.

On the card the cap barely matters. There the 30B read the 4K screenshot in 1.85 seconds uncapped and 0.49 at the 1,536 cap (section 07).

With the card mostly full

Four runs started while another program already held 26,390 to 26,405 MiB of the card (nvidia-smi, desktop included); Ollama's log put the card's free memory at 5.1 GiB. What that program was, and whether it was working during these runs, is not in our records. Two pictures each: the nebula first, including the load, then the poster.

Threads as the server reported them; batch is the launch line's -b and -ub. The CPU-only launch also carried --no-mmap (memory mapping off); the other three did not. All four runs found all 12 strings in their two pictures. The capped run's pictures were smaller: 1,370 and 1,802 prompt tokens, against 4,154 and 1,974 for the others.

The default request was the slowest thing we measured with a full card. Ollama fitted part of the model into the free memory, kept the picture encoder off the card (its launch line said --no-mmproj-offload, and the encoder's log line says it ran on the CPU), and ran everything left on the CPU with 4 threads. The first picture took 206.87 seconds.

Setting 16 threads and an 8,192-token window, two changes at once, brought a similar split (88 percent on the CPU) to 85.45 seconds on the nebula and 31.02 on the poster.

CPU only, with 16 threads, the 8,192 window and the cap, the nebula took 39.39 seconds, on a picture capped to a third of the tokens (1,370 against 4,154; arithmetic), and the poster 42.68, slower than the split's 31.02. Against the default request its launch also changed the batch (1,024 against 512) and turned memory mapping off, so the fall from 206.87 to 39.39 seconds is all of those changes together, not the threads and the cap alone. It is the only one of the four that took no card memory: the split runs raised the card's reading by 3,057, 3,018 and 3,425 MiB (arithmetic, after load minus before), memory the program already on the card could no longer use.

The CPU path took about as long with the card full. The capped CPU settings with the card empty (section 05; the two launches differ only in the window, 32,768 against 8,192) took 38.52 seconds on the nebula and 38.93 on the poster, against 39.39 and 42.68 here, one run each.

What the 1,536-pixel cap costs on small text

The seven pictures have fairly large text, so they did not show what shrinking does to small print. The screenshot does: 24 codes such as F12-zephyr-6823, six in each of four text sizes, on a 3840 x 2160 page. Qwen3-VL 30B-A3B on the card, 4 threads, 32,768-token window, one run per cap, each the first picture after a load (8.56 to 9.88 seconds including a 4.56 to 4.84-second load). score.py has no expected strings for this picture, so we counted the codes on 26 September with its matching rule (score_hires.py in the package).

Every missing code was replaced by a wrong one. At 2,048 the model wrote F12-zephyr-6829 for F12-zephyr-6823. At 1,536 it wrote D14-kettle-B104 for D14-kettle-8104 (28 px text), and, in the 13 px lines, F12-pebble-4823, E28-ribbon-7319 and H40-pebble-8359 for F12-zephyr-6823, E26-ribbon-7519 and H20-pebble-8359. A reader of those replies would have no way to tell the four wrong codes from the twenty right ones. At the 1,536 cap, text drawn 13 pixels high becomes 5.2 pixels high, and at 2,048 about 6.9 (arithmetic).

This ran on the card. On the CPU the model receives the same pixels, so we expect the same kind of misreading there, but we did not measure it (transferred).

What 39 strings show, and what they do not

The count shows whether a model copied specific text and named obvious subjects, and it separated the models: three found everything, and Qwen3.5 4B dropped nine strings. It is also a narrow instrument. We read the cottage replies of all eleven seven-picture runs, the chart replies of five, three logo replies, and Qwen3.5 4B's long replies; this is what the count could not see.

• A picture on its side. The cottage is stored lying on its side with a tag that says to turn it upright. All six models on the card described it as stored: a green band down the right-hand side and the caption running vertically, and the 4B called the house an arrow. Five of them scored 3 of 3. When the capped sets sent it upright, the 30B and the 4B described sky at the top and green ground at the bottom. As far as these replies show, the picture reached the models without its rotation applied.
• Wrong statements beside right numbers. On the card, the 30B listed all six chart values correctly and then wrote that “the bars increase in height from January to April”; February's 35 is lower than January's 42. The 8B called the bars “arranged in ascending order from left to right, except for the peak in April”. Both scored 9 of 9.
• Replies that run away. Qwen3.5 4B's replies to the Earth photograph, the poster and the chart ran to 2,048, 1,562 and 2,048 tokens. Within its first 1,400 characters each became one long unpunctuated stream, ending in lists of loosely related words (winemaking terms, music genres, military ranks). The Earth reply still scored 1 of 1.
• Right readings in other words. “£2/day” and “12 months” counted as misses (section 03), and so did a full-size CPU reply that called the house “a building or cottage”.
• Answers nobody asked for. Some replies, most of them the 4B's, wrote a question of their own after the description and answered it, though we sent none; the 30B did it in three replies.
• Wrong text that looks right. A misread code (section 07) scores as a miss, but in a reply it reads as a fact.
• Our own notes, wrong. Our notes and the file's name call picture 7 a white logo on a transparent background. Its pixels, read on 26 September, are all opaque: a light gray field, dark gray letters with a coloured bar under the O, and thin coloured curves in the top-right and bottom-left corners. The card replies we read (the 30B's, the 8B's and Qwen3.6's) describe that picture: a dark wordmark, a TM mark (which we did not check in the pixels) and coloured curves in those corners. “COSMIC” matched either way, so the count could not have told which account was right, and the set has no transparency test.

We did not grade the replies for invented details beyond these examples. All the local replies are in the package, with 13 prompt passages and one word removed (section 12).

What to run

From one desktop's measurements; your card, CPU and pictures will move every number. Each line says what it rests on.

1. If the card has room, use it. Qwen3-VL 30B-A3B found all 39 strings at 1.31 to 3.16 seconds a warm picture in 22 GB, as Ollama reported it (19 GB at an 8,192-token window). Qwen3-VL 8B found all 39 at 1.81 to 3.82 seconds in 10 GB. Qwen3.5 4B did worst here: 30 of 39, and three runaway replies.
2. If a large model holds the card, go to the CPU on purpose. Set num_gpu 0. Left to itself, Ollama fitted part of the model into the card's last free memory, using 3,018 to 3,425 MiB of it in our runs (arithmetic), and still ran the picture encoder and most of the model on the CPU (section 06).
3. Set the thread count. With no num_thread, the server here ran 4 threads of 24. On this CPU, 16 and 24 threads were within 4 seconds of each other on both pictures we timed (arithmetic; 24 read faster, 16 wrote faster), 16 leaves more of the CPU for everything else, and 8 missed the 150-second limit on a 4K picture. Check your own server: on every load, Ollama 0.30.10's log printed system_info: n_threads = … (on a standard Linux install, journalctl -u ollama | grep n_threads).
4. Shrink large pictures before sending them, and turn them upright first. A 1,536-pixel cap cut a 4K picture from 4,154 tokens to 1,370, and the 30B's first picture from 111.01 seconds to 38.52 at 16 threads, with all 39 strings found. Apply the rotation tag yourself: as far as the replies show, nothing on the way did.
5. Keep more pixels for small text. The same cap misread 4 of 24 codes on a 4K screenshot, where 2,560 found all 24 (one run each, on the card). For screenshots and fine print, cap at 2,560 or send them to the card.
6. Set num_ctx 8192 for single pictures. A 4K picture and the longest reply we allow need 6,206 tokens (arithmetic), and at the smaller window ollama ps put the 30B at 19 GB on the card instead of 22. Raise it if you send several pictures or a long question at once.
7. Budget the wait. With 16 threads and the 1,536 cap, the 30B took 22.64 to 38.93 seconds a warm picture on this CPU, plus 12.30 seconds of model load on the first, at the 32,768-token window; ollama ps put its memory at 22 GB there and 19 GB at 8,192. The 4B at a 1,024 cap took 18.12 to 32.64 seconds in 8.0 GB and found all 39 strings in our set, with the small-text risk of section 07.

What this does not show

• One run per configuration. No repeats, so we cannot say how much a second run would move any figure; a difference of a few seconds or one string may be noise.
• A small test. Seven pictures and 39 strings, plus one screenshot with 24 codes. Not a benchmark, and not a grade of description quality: section 08 lists what the count misses, and we did not score invented details.
• One machine and one CPU. The thread results belong to a Core Ultra 9 285K with 8 performance and 16 efficiency cores. The best thread count on another CPU may differ.
• One Ollama version, one quantization. Ollama 0.30.10, whose bundled server build is not in its log, and each model's Q4_K_M library file. What the server chose when a request said nothing (the 32,768-token window, 4 threads, the split on a full card) may differ in other versions and setups.
• First pictures include loads. The file cache was not recorded, so how much of each first-picture time was disk is unknown; the first card load of the 30B took 36.97 seconds and the later ones 4.56 to 4.86.
• The full card was a test of memory, not of shared work. We do not know what held the card or whether it was busy. A large model whose experts sit in system RAM also uses the CPU, and a describer on the CPU beside it would compete for threads and memory bandwidth; we did not measure that.
• Few pictures per CPU and full-card run. The thread runs and the full-card runs used two pictures each, the 4-thread run one.
• Overlaps. Model downloads ran during three runs, and two hosted-service runs during the 4-thread CPU run (section 02).
• The capped sets changed two things. They shrank the large pictures and turned the cottage upright, so the string counts at a cap mix both effects; for the cottage only the rotation changed (section 05).
• No transparency test. Picture 7 was meant to be one; its file has no transparent pixels (section 08).
• Sampling. The request set no sampling options, so each reply is one sample at the model's own defaults.
• The hosted counts. The result files record the provider, not the model version; they are counts only.

Related pages on this site

The wrong flag traced a 4-thread default on this CPU to llama.cpp's thread heuristic, on a 675B model whose decode it roughly halved. llama.cpp's batch size measures the reading settings of the models this desktop serves with llama.cpp; Ollama launched the describers here with batches of 1,024 or 512 tokens, and we did not vary them. Slower by default is another default measured on this desktop, and the method page explains the labels and the data packages.

Sources and artifacts

The evidence for this page ships as a data package: the files the runs produced, not a summary. If a number on this page disagrees with a file in the package, the file is right and the page is wrong; tell us and we will fix the page.

• data/README.md: every file, where the files disagree with each other, and every redaction.
• data/NUMBERS.md: every figure on this page, with its file and field.
• The runs (data/results/): one result file per run with every reply, timing and card reading, and the console log of the 4-thread run. Where a small model repeated our private prompt word for word, 13 passages in five files read [prompt wording removed], and one word in one reply reads [one word removed]; data/README.md lists both.
• The server's log (data/service-log/): Ollama's version line and, for these runs, each launch command, thread report, file type, memory and timing line.
• The pictures and the scripts: the test pictures except the third-party logo, the scripts that drew and shrank them, the harness with the prompt wording removed, score.py, score_hires.py, and tables.py, which rebuilds the derived tables in data/derived/.
• Picture credits. Nebula: NASA, ESA, M. Robberto (Space Telescope Science Institute/ESA) and the Hubble Space Telescope Orion Treasury Project Team (ESA/Hubble image heic0601a, licensed CC BY 4.0). The JPEG we ship is a 3840 x 2160 derivative taken from a desktop wallpaper set, not the release file. Earth: photograph ISS064-E-29444, image courtesy of the Earth Science and Remote Sensing Unit, NASA Johnson Space Center. Their use here does not imply endorsement by NASA or ESA.