Runs / goods-receipt Batch Runs from different design-system versions should not be compared.
The page A screenshot in each theme, as the benchmark took it.
page.tsx As the agent left it; marked lines broke a rule.
Violations 2 counted, under 1 rule. “Not counted” marks a compiler error that restates a design-system violation, “Ignored” one the agent silenced with a comment.
Steps One row per model call, with the tools it asked for. An agent that reads the guidelines before writing shows it here; what a checker or submit_view answered is under its call.
Step Tools Tokens in Tokens out Cost 0 get_guidelines, list_components, list_tokens, read_file, list_files 625 108 $0.0000 1 read_file, get_component, get_component 7,505 76 $0.0000 2 get_component, get_component, get_component, get_component, get_component, get_component, get_component, get_component 10,097 176 $0.0000
Output cap 20,000 tokens a step 3 import { useState } from "react";
4 import data from "./fixture.json";
5 import {
6 Alert,
7 Button,
8 Card,
9 CardBody,
10 CardHeader,
11 CardTitle,
12 Checkbox,
13 DataList,
14 FormField,
15 FormatDate,
16 Heading,
17 NumberInput,
18 Select,
19 Stack,
20 Text,
21 Textarea,
22 Toaster,
23 toast,
24 } from "@vektos/ds";
25
26 type Qty = number | null;
27
28 export default function Page() {
29 const po = data.purchaseOrder;
30
31 const [received, setReceived] = useState<Qty[]>(data.lines.map((l) => l.ordered));
32 const [errors, setErrors] = useState<string[]>(data.lines.map(() => ""));
33 const [dock, setDock] = useState<string>("");
34 const [damaged, setDamaged] = useState<boolean>(false);
35 const [note, setNote] = useState<string>("");
36 const [attempted, setAttempted] = useState<boolean>(false);
37 const [confirmed, setConfirmed] = useState<boolean>(false);
38
39 function validateQty(index: number, value: Qty): string {
40 const ordered = data.lines[index]!.ordered;
41 if (value === null) return "Enter a quantity.";
42 if (value < 0) return "Cannot be negative.";
43 if (value > ordered) return "Cannot exceed the quantity ordered.";
44 return "";
45 }
46
47 function handleQtyChange(index: number, value: Qty) {
48 setReceived((prev) => {
49 const next = [...prev];
50 next[index] = value;
51 return next;
52 });
53 setErrors((prev) => {
54 const next = [...prev];
55 next[index] = validateQty(index, value);
56 return next;
57 });
58 }
59
60 const hasQtyErrors = errors.some((e) => e !== "");
61 const dockError = attempted && dock === "" ? "Select a dock." : undefined;
62 const noteError =
63 attempted && damaged && note.trim() === "" ? "Describe what was damaged." : undefined;
64
65 const canConfirm =
66 !confirmed &&
67 !hasQtyErrors &&
68 dock !== "" &&
69 !(damaged && note.trim() === "");
70
71 function handleConfirm() {
72 const nextErrors = received.map((v, i) => validateQty(i, v));
73 setErrors(nextErrors);
74 setAttempted(true);
75 if (nextErrors.some((e) => e !== "") || dock === "" || (damaged && note.trim() === "")) {
76 return;
77 }
78 setConfirmed(true);
79 toast({ title: `Receipt confirmed for ${po.number}`, tone: "success" });
80 }
81
82 return (
83 <Stack gap="6" marginBlock="8">
84 <Heading level={1}>Confirm delivery receipt</Heading>
85
86 <Card>
87 <CardHeader>
88 <CardTitle>Purchase order</CardTitle>
89 </CardHeader>
90 <CardBody>
91 <DataList
92 items={[
93 {
94 label: "Purchase order number",
95 value: <Text as="span" mono>{po.number}</Text>,
96 },
97 { label: "Supplier", value: po.supplier },
98 {
99 label: "Expected delivery",
100 value: (
101 <>
102 {po.expectedOn} <FormatDate value={po.expectedOn} dateStyle="long" />
103 </>
104 ),
105 },
106 ]}
107 />
108 </CardBody>
109 </Card>
110
111 <Stack gap="3">
112 <Heading level={2} size="md">
113 Order lines
114 </Heading>
115 {data.lines.map((line, index) => (
116 <Card key={line.sku}>
117 <CardBody>
118 <Stack gap="2">
119 <Text weight="medium">{line.name}</Text>
120 <Text size="sm" tone="muted" mono>
121 {line.sku}
122 </Text>
123 <Text size="sm">
124 Ordered: {line.ordered} {line.unit}
125 </Text>
126 <FormField label="Received" error={errors[index] || undefined}>
127 <NumberInput
128 value={received[index] ?? null}
129 onValueChange={(value) => handleQtyChange(index, value)}
130 min={0}
131 max={line.ordered}
132 step={1}
133 fractionDigits={0}
134 showSteppers={false}
135 />
136 </FormField>
137 </Stack>
138 </CardBody>
139 </Card>
140 ))}
141 </Stack>
142
143 <Card>
144 <CardBody>
145 <Stack gap="4">
146 <FormField label="Dock" error={dockError}>
147 <Select
148 options={data.docks.map((d) => ({ label: d, value: d }))}
149 {...(dock ? { value: dock } : {})}
150 onValueChange={setDock}
151 placeholder="Select a dock"
152 />
153 </FormField>
154
155 <Checkbox checked={damaged} onCheckedChange={setDamaged}>
156 Some goods arrived damaged
157 </Checkbox>
158
159 {damaged && (
160 <FormField label="Damage note" error={noteError} helperText="Describe what was damaged.">
161 <Textarea
162 value={note}
163 onChange={(e) => setNote(e.target.value)}
164 rows={3}
165 />
166 </FormField>
167 )}
168 </Stack>
169 </CardBody>
170 </Card>
171
172 <Stack flexDirection="row" justifyContent="flex-end">
173 <Button variant="solid" disabled={!canConfirm} onClick={handleConfirm}>
174 Confirm receipt
175 </Button>
176 </Stack>
177
178 {confirmed && (
179 <Alert tone="success" title="Receipt confirmed">
180 Purchase order {po.number} has been received.
181 </Alert>
182 )}
183
184 <Toaster />
185 </Stack>
186 );
187 }
188
3 get_component, get_component, get_component, get_component 16,534 106 $0.0000
4 write_file 18,609 6,238 $0.0000
5 check_code check_code: 5 violations:
- line 40: typecheck/TS2532: Object is possibly 'undefined'.
- line 123: typecheck/TS2375: Type '{ value: Qty | undefined; onValueChange: (value: number | null) => void; min: number; max: number; step: number; showSteppers: false; }' is not assignable to type 'NumberInputProps' with 'exactOptionalPropertyTypes: true'. Consider adding 'undefined' to the types of the target's properties.
- line 127: lint/react(jsx-no-duplicate-props): No duplicate props allowed. The prop "max" is duplicated.
- line 128: typecheck/TS17001: JSX elements cannot have multiple attributes with the same name.
- line 143: typecheck/TS2375: Type '{ options: { label: string; value: string; }[]; value: string | undefined; onValueChange: Dispatch<SetStateAction<string>>; placeholder: string; }' is not assignable to type 'SelectProps' with 'exactOptionalPropertyTypes: true'. Consider adding 'undefined' to the types of the target's properties.
8,552 1,224 $0.0000
6 write_file 26,561 1,699 $0.0000
7 check_code check_code: No violations.
9,532 1,226 $0.0000
8 list_files 1,301 714 $0.0000
10 get_component 128 554 $0.0000
11 write_file 1,013 4,211 $0.0000
12 check_code check_code: No violations.
9,024 1,243 $0.0000