How to mark up a screenshot for an AI model
Most of the value comes from one habit: mark the thing you are asking about, then refer to the mark in the prompt.
The three steps
- Capture and paste. Copy a screenshot to your clipboard (⌘+⇧+Ctrl+4 on macOS, Win+Shift+S on Windows) and press ⌘V in Markup.
- Mark what you are asking about. One mark per thing. Numbered pins if there is more than one.
- Copy and paste into the model. Enter or ⌘C puts the flattened PNG on your clipboard. Write the prompt around the marks.
Which mark for which job
| Use | When |
|---|---|
| Arrow | You want one element and the element is small. An arrow points without covering anything. |
| Box | You mean a region rather than an element: a whole column, a card, a block of results. |
| Numbered pin | You have more than one question about the same screenshot, or you want to talk about order. |
| Text | The intent is not obvious from the mark alone: should be 16px, wrong state. |
| Free draw | The thing you mean is not a rectangle: a curve on a chart, a run of text across two lines. |
Write the prompt around the marks
Numbered pins are worth using even for a single question, because they turn a description of a location into a name. Compare:
“The spacing under the heading in the second card on the right looks wrong, and the button below it is a different size from the others.”
with the same question over a screenshot carrying two pins:
“1: too much space under the heading. 2: this button is a different height from the others. What in the CSS explains both?”
The second version is shorter, has no ambiguity to resolve, and gives the model a vocabulary to answer in: it can reply “for 2, the padding comes from…” and you know exactly what it means.
Patterns that work
- Number the bugs. Pin every defect, then list them in the prompt in the same order. You get one reply covering all of them instead of a conversation per bug.
- Box the scope, arrow the detail. A box saying only look in here plus an arrow at the specific element keeps the model from redesigning the whole page.
- Mark the expected and the actual. Two boxes on one screenshot, labelled with the text tool, beats two separate images.
- Say what the marks are. One clause is enough: “the red pins are the elements that shift on mobile”. The model does not otherwise know whether a mark means broken, keep, or explain this.
Mistakes that cost you
- Covering the thing you are asking about. A box drawn tight around small text can sit on top of the descenders. Leave a little air.
- Marking everything. Ten marks read as noise. If the whole screen is the problem, say so in words and leave the image clean.
- Cropping the context away. The model often needs the surroundings to answer. Mark the element, keep the page.
- Relying on colour alone for meaning. If red means broken and blue means keep, write that in the prompt rather than expecting it to be inferred.
- Scaling the image down afterwards. Small text stops being legible to the model. Copy at full resolution, which is what the copy button gives you.
Every mark is drawn at the screenshot's own resolution, so a retina capture stays sharp and the annotations keep their proportions when the model reads it.