Blind page review · Labeling guide网页盲评 · 标注指南

We score AI-built web pages with an automatic program and use that score to decide which version is better. If the score disagrees with human judgment, the optimization drifts in the wrong direction.

Your labels serve three purposes:

1. Testing the overall score: whether the grader also scores the page you prefer higher.
2. Finding blind spots: the reason tags you choose and the locations you mark tell us what the grader misses.
3. Checking verdicts: in a claim audit you judge whether a page meets a given requirement, and we compare that with the grader's verdict.

Labeling is blind: you can't see which version built a page or what the grader scored it. The grader can't see your labels either.

我们用程序自动给 AI 做的网页打分，用这个分数决定哪个版本更好。分数如果和人的判断不一致，优化就会走偏。

你的标注有三个用处：

1. 检验总分：你更喜欢的那一份，评分器是否也给了更高分。
2. 找出盲区：你选的原因标签和标出的位置，告诉我们评分器漏看了什么。
3. 核对判定：在「逐条核对」里，你判断页面是否做到某条要求，我们拿来和评分器的判定对照。

标注是盲评：你看不到页面是哪个版本做的，也看不到评分器的分数。评分器也看不到你的标注。

A pair is two web pages, A and B, built from the same brief. Work through the steps below; a pair takes about 10–12 minutes.

1. Read the brief (about 1 minute). Work out who the requester is, who the page is for and which things they want delivered. Click “Show full brief” to read all of it; the attachment links are below the brief.
2. Explore both pages (about 3 minutes each). Follow the testing script below, the same way on both sides. The “Explore first” card at the top of the panel ticks itself off as you go.
3. Choose the overall. Answer on your overall impression: if you had written the brief, which page would you rather receive? If you choose “same”, also say whether both are good, both are weak, or it's a trade-off.
4. Compare by dimension. Compare the five dimensions in order: requirements, content, interaction, visual, motion. For each, choose A better, same or B better. A dimension with a winner needs a reason: tick a problem of the other page or a highlight of this one, or write a few words under “Other”.
5. Mark locations (recommended). If you can point to where a problem is on the page, click “Mark on A” (or B), then click once on the page or drag a box.
6. Grade severity. A problem is ordinary by default. Click “Severe” if it would stop visitors from doing what the brief asks for, mislead them about the content, or make them stop trusting the page.
7. Final check and submit. After the dimensions, if your view of the overall has changed, go back and change it (your first choice is kept as well). Press ⌘/Ctrl + ↵ to submit; for 9 seconds afterwards you can click “Revise last”.

一组是同一份题目做出的两个网页，A 和 B。按下面的顺序做，一组大约 10–12 分钟。

1. 读题目（约 1 分钟）。弄清楚提需求的人是谁、给谁看、要哪几样东西。点「展开全文」看完整题目，附件链接在题目下方。
2. 体验两边（每边约 3 分钟）。按下面的体验脚本做，两边做法相同。面板顶部的「先体验」会自动打勾。
3. 选总体。凭整体印象回答：如果你是提需求的人，更愿意收到哪一份。选「差不多」时，再说明是两边都不错、两边都弱，还是各有长短。
4. 分维度比较。按顺序比较五个维度：需求、内容、交互、视觉、动效。每个维度选 A 更好、差不多或 B 更好，分出胜负就要给原因：勾对方的一个问题、这一边的一个亮点，或在「其他」里写几个字。
5. 标位置（推荐）。能指出问题在页面哪里，就点「标位置」，在页面上点一下或拖一个框。
6. 分轻重。问题默认是一般问题。会让访客无法完成题目要的事、误解内容，或不再信任这个页面的，再点「严重」。
7. 最后确认并提交。看完各维度，如果对总体的看法变了，回到总体修改（你最初的选择也会保留）。按 ⌘/Ctrl + ↵ 提交；之后 9 秒内可以点「修改上一组」。

Explore both pages with the same actions, so that your conclusions are comparable. AI-built pages usually get through the normal flow; problems tend to show up under stress tests.

At desktop size (about 2 minutes)

1. Hands off (10 s): just look, and say which things you think you can operate. This step tests “Doesn't look operable”.
2. Walk through the main flow (40 s): use every feature the brief asks for once, keeping your eyes on the control and its result.
3. Stress test (25 s): click the same button 5 times fast; toggle on, off, on; change the same value with two controls; while dragging, move the pointer off the handle; click while an animation is halfway through; finally, try to reset.
4. Keyboard (20 s): click an empty part of the page, press Tab repeatedly, then try Enter, Space and the arrow keys. On slide-style pages, try → ← and Space.
5. Navigation (10 s): click two table-of-contents or navigation links, and check that they land in the right place and that the heading isn't hidden.

At phone size (about 1 minute)

1. Edges (15 s): scroll from top to bottom, checking whether text, chart labels or buttons at the bottom are cut off or overflow.
2. Taps and drags only (30 s): redo the main flow without relying on hover. Notice whether the page still scrolls when you drag on a canvas.

If both pages have the same problem, tick it on both. It doesn't affect the preference, but it is just as useful for testing the grader.

两边用同样的动作体验，结论才可比。AI 做的页面通常能走通正常流程，问题多出在压力测试里。

桌面尺寸（约 2 分钟）

1. 先别碰（10 秒）：只看页面，说出哪些东西你觉得能操作。这一步检验「看不出能操作」。
2. 走一遍主流程（40 秒）：把题目要求的每个功能用一次，眼睛盯住控件和它的结果。
3. 压力测试（25 秒）：同一按钮快速点 5 次；开关开、关、开；用两个控件改同一个值；拖动时把指针移出把手；动画播到一半再点；最后试着复位。
4. 键盘（20 秒）：点一下页面空白处，连按 Tab，再试回车、空格和方向键。演示类页面试 → ← 空格。
5. 导航（10 秒）：点两个目录或导航链接，看是否跳对、标题是否被挡住。

手机尺寸（约 1 分钟）

1. 边缘（15 秒）：从头滚到尾，看文字、图表标签、底部按钮有没有被切掉或溢出。
2. 只用点和拖（30 秒）：把主流程再做一遍，不要依赖悬停。注意拖动画布时页面是否还能滚。

两边都有的问题，两边都勾上。它不影响偏好，但对检验评分器同样有用。

By default you see one page at a time and switch with the A and B keys. The switch is instant, so your eye is drawn to whatever changes; this is called blink comparison. Side by side, your eye has to jump back and forth, and small differences are easy to miss.

Once you have clicked inside a page, the keyboard belongs to that page. Switch with ⌥A ⌥B (Option or Alt plus the letter); the toolbar then says that the keyboard is in the page.

Pages are usually shown scaled down, with the zoom level shown at the top right of each page. To judge font size, line length and contrast, press Z for actual size.

默认一次只看一个页面，用 A B 键来回切换。切换是瞬间的，眼睛会自动注意到变化的地方，这叫闪烁对比（blink comparison）。并排看时，眼睛要来回跳，细小的差别很容易漏掉。

在网页里点过之后，键盘会交给网页，这时请用 ⌥A ⌥B（Option 或 Alt 加字母）切换，工具栏会提示「键盘在页面里」。

页面通常是缩小显示的，右上方会显示缩放比例。判断字号、行长和对比度时，按 Z 切到实际大小。

The overall uses five steps: A clearly better, A slightly better, same, B slightly better, B clearly better. The dimensions use only three: A better, same, B better, plus “Can't judge” (for example when neither page opens, or when you lack the knowledge a dimension needs).

The dimensions don't distinguish “clearly” from “slightly” because research has found that people barely agree with each other on that distinction; it only adds work.

总体用五级：A 明显好、A 稍好、差不多、B 稍好、B 明显好。分维度只用三级：A 更好、差不多、B 更好，另有「无法判断」（例如两边都打不开，或你缺少判断这一维度所需的知识）。

分维度不分「明显」和「稍」，是因为研究发现这一级的区分在人与人之间几乎不一致，只会增加负担。

The five dimensions run from “was it done” to “how well was it done”. Each has problem tags and highlight tags: tick a problem tag for the page that has the problem, and a highlight tag for the page that does something especially well. You can tick both sides.

The tag tables below use the same definitions as the labeling tool and stay in sync with it automatically.

五个维度从「做没做到」到「做得多好」。每个维度下面都有问题标签和亮点标签：问题标签勾出现问题的一边，亮点标签勾做得特别好的一边，两边都可以勾。

下面的标签表和标注页面用的是同一份定义，会自动同步。

Look only at whether what the brief explicitly asks for is done, not at how it looks. The best method is to treat the brief's numbered list as a checklist and find each item on the page.

• “Built but broken” is harder to spot than “missing”: try every required interaction yourself.
• Check requirements with numbers or quotation marks (counts, word limits, exact wording) one by one.
• Sentences with “don't” or “must not” are requirements too: doing what they forbid breaks a prohibition.

只看题目明确要求的东西有没有做到，不看好不好看。最好的方法是把题目里的编号列表当清单，逐条在页面上找。

• 「做了但用不了」比「漏做」更隐蔽：每个要求的交互都要亲手试一遍。
• 题目里带数字和引号的要求（数量、字数、文字原样），逐个核对。
• 题目里「不要、不得」的句子同样是要求，做了就是违反禁止项。

Check accuracy first, then clarity. Accurate means consistent with the brief and the attachments; clear means the target reader actually understands after reading.

• Spot-check specifics: pick numbers, names and dates on the page and compare them with the attachments.
• Watch for overclaiming: guesses stated as fact, or limits the brief asks to be stated left out.
• Scan test: reading only the headings and the bold text, can you tell within 10 seconds what the page is about and where its key points are?

Four checks for charts

• Units and meaning: can you say what one bar or one point stands for, and in what unit?
• Bar charts start at zero: a bar chart whose value axis doesn't start at 0 exaggerates differences.
• The right type: bars for rankings, lines for change, and pies only rarely for parts of a whole (with many slices they are hard to compare).
• State the conclusion: a good chart title states the takeaway, not just “Data on X”.

先查准确，再看清楚。准确指和题目、附件一致；清楚指目标读者读完真的懂了。

• 抽查具体信息：挑页面上的数字、名称、日期，回到附件里对照。
• 警惕「说得过满」：把推测写成事实，或删掉了题目要求写出的限制。
• 扫读测试：只看标题和加粗，10 秒内能否知道页面讲什么、要点在哪。

图表的四个检查

• 单位和含义：能否说出一根柱、一个点代表什么、单位是什么。
• 柱状图从零开始：柱状图的数值轴不从 0 开始，会夸大差别。
• 类型合适：排名用条形，变化用折线，部分与整体少用饼图（切块多时难以比较）。
• 点明结论：好图表的标题直接说出结论，而不只是「某某数据」。

Good interaction has three marks: you can see what is operable, every action gives immediate feedback, and the result matches what you expected.

Signifiers and feedback

A signifier is a visual cue that something can be clicked or dragged, such as a button's shape or a change on hover. Feedback is the change in the element itself after an action, telling you that the action took effect.

好用的交互有三个特征：看得出能操作，操作后马上有反馈，结果和预期一致。

可操作提示（signifier）和反馈（feedback）

可操作提示是让人看出「这里能点、能拖」的视觉线索，例如按钮的形状、悬停时的变化。反馈是操作后元素自己的变化，告诉你操作已经生效。

Professional designers usually look at the whole first, then the details. For the whole, they look at the focal point and the hierarchy; for the details, at spacing, alignment, type and color.

Aesthetic judgments are hard to agree on. These five methods help different people see the same things:

• 5-second test: look at the first screen for 5 seconds, look away, and say what the page is about, who it is for and what it wants you to do. People form a first impression of a web page very quickly (in about 50 ms).
• Squint test: squint, or blur the page, to see the focal point and the groups; see the demo below.
• Count, don't guess: count the font sizes, the main colors, the left edges and the characters per line. Things you count are things different people agree on.
• Greyscale test: imagine the page in black and white. Do the things told apart only by color still stand apart?
• Swap test: replace the title with another subject. Does the design still work? If it does, it's a template.

Focal point and hierarchy: the squint test

Visual hierarchy is the order of importance among elements: the most important is seen first. Squint until the view blurs; once the details disappear, you can see at once whether the hierarchy is clear.

专业设计师看页面，通常先看整体，再看细节。整体看视觉重点和层级，细节看间距、对齐、字体、颜色。

审美判断很难一致，下面五个方法能让不同的人看出同样的东西：

• 5 秒测试：只看首屏 5 秒，移开视线，说出页面讲什么、给谁看、要你做什么。人对网页的第一印象形成得很快（约 50 ms）。
• 眯眼测试：眯眼或模糊后看重点和分组，见下面的演示。
• 数一数，别凭感觉：数字号有几种、主色有几种、左边缘有几条、一行有几个字。数出来的东西，不同的人才会一致。
• 灰度测试：想象页面变成黑白，靠颜色区分的东西还分得开吗？
• 换题测试：把标题换成别的主题，设计还成立吗？成立就是模板感。

视觉重点和层级：眯眼测试（squint test）

视觉层级（visual hierarchy）是元素的主次关系：最重要的最先被看到。眯起眼睛让画面变模糊，细节消失后，层级是否清楚就一目了然。

If a line is too long, the eye easily lands on the wrong line when it returns to the left edge; if it is too short, reading is interrupted all the time. Chinese body text also needs looser line spacing than English.

For English, Butterick's Practical Typography recommends 45–90 characters per line and a line height of 120–145% of the font size. For Chinese, the W3C's Requirements for Chinese Text Layout (clreq) gives a reference: book body text usually has 17–40 characters per line, and horizontal text no more than 48. The space between lines is 50%–100% of the font size, that is, a line height of about 1.5–2.0.

一行太长，眼睛换行时容易找错下一行；一行太短，阅读被频繁打断。中文正文行距也要比英文更松。

W3C 的《中文排版需求》（clreq）给了参考：书籍正文一般每行 17–40 字，横排不超过 48 字。行间空白是字号的 50%–100%，也就是行高约 1.5–2.0。

These rules apply to pages written in Chinese, and AI-built pages often break them. You don't need to read Chinese to spot most of them: look at the shapes of the punctuation and at the spacing.

• Full-width punctuation: Chinese sentences use full-width punctuation (，。！？：；“”), which fills a whole character cell. Half-width marks such as , . ! ? look small and cramped against Chinese characters.
• Ellipsis and dash: the Chinese ellipsis is six dots spread over two character cells (……), not 。。。 or ...; the dash also spans two cells (——).
• No punctuation at the start of a line: commas, full stops and closing brackets never begin a line. (Japanese typesetting has the same rule, known as kinsoku.)
• Space between Chinese and Latin text: leave a small gap between Chinese and Latin letters or digits, and handle it the same way across the page. Typed spaces and spacing added automatically by styles are both fine; spacing that comes and goes is the problem.
• No italics: Chinese has no italic. Emphasis uses bold, color or emphasis dots under the characters.
• No extra letter-spacing in body text: body text is not spread out between characters; extra letter-spacing is only for headings and captions.

中文网页有几条专门的规矩，AI 做的页面常在这里出错。

• 全角标点：中文句子用全角标点（，。！？：；“”）。半角的又小又贴字。
• 省略号和破折号：省略号是「……」，不是「。。。」或「...」；破折号是「——」。
• 避头尾：逗号、句号、右括号不出现在行首。
• 中英间距：中文和英文、数字之间留空隙，并且全页一致。手打空格和样式自动加空隙都可以，时有时无才是问题。
• 不用斜体：中文没有斜体，强调用加粗、颜色或着重号。
• 正文不加字距：正文字与字之间不额外拉开，加字距只用于标题和说明。

Art direction is a visual idea conceived for this particular subject, such as a material, a metaphor or a composition. The template look is its opposite: swap in another subject and the design still works.

To judge it, replace the title with another subject: does the design still hold? If it does, it's a template. Common templates on AI-built pages: purple-blue gradients, rows of identical rounded cards, a big centered headline, emoji used as icons.

There is also a subtler kind of “cliché decoration”: it looks designed, but it turns up again and again on AI-built pages:

• Small all-caps English labels above headings, 01/02/03 numbering, “A · B · C” info strips.
• Cream paper with serif headings and terracotta accents; or a near-black background with a single neon color.
• Glowing orbs, gradient blobs and frosted-glass panels that could be removed without losing any information.

视觉概念指为这个主题专门想出的视觉创意，例如材质、隐喻、构图。模板感正相反：换一个主题，设计照样能用。

判断方法：把标题换成别的主题，设计还成立吗？成立就是模板感。AI 做的页面常见的模板：紫蓝渐变、一排排同样的圆角卡片、居中的大标题、用 emoji 当图标。

还有一类更隐蔽的「套路装饰」，看起来很有设计感，但在很多 AI 页面上反复出现：

• 标题上方的英文全大写小标签，01/02/03 编号，「A · B · C」式的信息条。
• 米色纸面配衬线标题和赤陶色强调；或者近黑底配一种霓虹色。
• 发光圆球、渐变色块、毛玻璃面板，去掉也不损失任何信息。

Good motion has a purpose, is quick and feels natural. Purpose means it tells you something: that a state changed, where something came from, where to look. Animation without a purpose only uses up attention.

Easing

Objects in the real world don't reach full speed instantly, and they don't stop dead either. Movement in an interface shouldn't either: things entering the screen should start fast and slow down to a stop (deceleration, ease-out).

好的动效有作用、够快、自然。作用指它在告诉你什么：状态变了、东西从哪来、该看哪里。没有作用的动画只是在消耗注意力。

缓动（easing）

现实中的物体不会瞬间达到全速，也不会瞬间停下。界面里的运动也一样：进场的东西应该快速出发、慢慢停下（减速，ease-out）。

Two other common problems can't be demonstrated here; when you see them, mark them:

• Scroll-jacking: you turn the wheel, and the page moves at its own speed or in its own direction.
• Layout shift: while the page loads or while you act, the content you are reading is suddenly pushed away.

另外两种常见问题无法在这里演示，看到时请标出：

• 滚动劫持（scroll-jacking）：滚一下鼠标，页面却按自己的速度或方向移动。
• 布局偏移（layout shift）：加载或操作时，正在看的内容突然被挤走。

Marking a location tells us where on the page a problem is. The grader only looks at screenshots and code; your marks tell it “look here”.

1. After you tick A or B on a tag, click “Mark on A” (or B) below it.
2. Move the mouse over the page; the element under it is highlighted. Press ↑ to select the larger element around it, and ↓ to go back to the smaller one.
3. Click to select the highlighted element, or press and drag to draw a box around an area.
4. For a problem with the whole page (such as a template look), click “Whole-page problem”; if you really can't point to it, click “Can't point to a spot”.
5. While marking, the wheel scrolls the page. Side by side, the view switches to that page automatically and goes back to side by side when you're done.

Marks are drawn on the page and scroll with it. Click a mark in the panel to jump to it, or click × to delete it.

标位置让我们知道问题在页面的哪里。评分器只看截图和代码，你的标记能告诉它「该看这里」。

1. 勾选一个标签的 A 或 B 后，点下面的「在 A 上标位置」。
2. 鼠标移到页面上，会高亮鼠标下的元素。按 ↑ 选它外面一层更大的元素，↓ 回到小的。
3. 点一下，选中高亮的元素；或者按住拖出一个框，框住一片区域。
4. 整页性的问题（例如模板感），点「整页问题」；确实指不出来，点「说不出具体位置」。
5. 标位置时可以用滚轮滚动页面；并排时会自动切到那一边，标完再回到并排。

标记会显示在页面上，跟着页面滚动。点面板里的位置标签可以跳过去查看，点 × 删除。

A claim audit shows one page at a time and asks whether it meets four conditions. The conditions come from the brief. In the study they were the exact sentences the grader judged; in this demo they were written from the brief to illustrate the format.

1. Read the brief first, then explore the page fully.
2. For each condition choose “Met”, “Not met” or “Unsure”; the shortcuts are 1 2 3.
3. For “Not met”, choose which kind: absent, present but wrong, only partly done, present but not visible, doesn't work, or holds at one size only.
4. Point to the evidence on the page. “Met” must show where you saw it; if the evidence is the whole page, click “Whole page”. “Can't point to a spot” doesn't count as marking.
5. If you have judged but aren't confident, click “Low confidence”. Choose “Unsure” only when you can't judge at all.

“From the brief” is the passage of the brief that the condition rests on; it helps you understand what the condition intends. Where the condition and the brief differ, go by the condition.

逐条核对一次只看一个页面，回答四条条件是否做到。这些条件来自题目。正式标注时它们就是评分器判定用的原句；本演示中的条件是根据题目另写的示例。

1. 先读题目，再完整体验页面。
2. 每条条件选「做到了」「没做到」或「说不准」，快捷键 1 2 3。
3. 「没做到」要选哪一种：完全没有、有但不对、只做了一部分、有但看不到、功能用不了、只在一种尺寸成立。
4. 在页面上指出依据。「做到了」必须标出在哪里看到的；依据是整个页面时点「整页」。「说不出具体位置」不算标出。
5. 判断了但没把握，点「不太确定」。完全无法判断才选「说不准」。

「依据题目」是这条条件对应的题目原文，可以帮你理解条件的本意。条件和题目原文有出入时，以条件为准。