Runs / goods-receipt Batch Runs from different design-system versions should not be compared.
The page A screenshot in each theme, as the benchmark took it.
page.tsx As the agent left it; marked lines broke a rule.
Violations 13 counted, under 7 rules. “Not counted” marks a compiler error that restates a design-system violation, “Ignored” one the agent silenced with a comment.
Steps One row per model call, with the tools it asked for. An agent that reads the guidelines before writing shows it here; what a checker or submit_view answered is under its call.
Step Tools Tokens in Tokens out Cost 0 read_file 1,446 40 $0.0000 1 list_components 1,760 52 $0.0000 2 get_component 4,261 701 $0.0000 3
cdb5d9e77633
Output cap 20,000 tokens a step 2
3 import data from "./fixture.json";
4
5 import {
6 FormField,
7 NumberInput,
8 Select,
9 Checkbox,
10 Textarea,
11 Button,
12 Stack,
13 Text,
14 } from "@vektos/ds";
15 import * as React from "react";
16
17 type Line = (typeof data.lines)[0];
18
19 interface PageState {
20 received: number[];
21 dock: string;
22 damaged: boolean;
23 note: string;
24 submitted: boolean;
25 }
26
27 export default function Page() {
28 const [state, setState] = React.useState<PageState>({
29 received: data.lines.map((l) => l.ordered),
30 dock: "",
31 damaged: false,
32 note: "",
33 submitted: false,
34 });
35
36 const { received, dock, damaged, note, submitted } = state;
37 const { purchaseOrder, lines } = data;
38
39 const allValid =
40 received.every((r, i) => r >= 0 && r <= lines[i].ordered) && dock !== "";
41
42 const handleReceivedChange = (idx: number) => (value: number | null) => {
43 if (value === null) return;
44 setState((s) => {
45 const newReceived = [...s.received];
46 newReceived[idx] = value;
47 return { received: newReceived };
48 });
49 };
50
51 const handleDockChange = (value: string) =>
52 setState((s) => ({ ...s, dock: value }));
53
54 const handleDamagedChange = (checked: boolean) =>
55 setState((s) => ({ ...s, damaged: checked }));
56
57 const handleNoteChange = (value: string) => setState((s) => ({ ...s, note: value }));
58
59 const handleConfirm = () =>
60 setState((s) => ({ ...s, submitted: true }));
61
62 if (submitted) {
63 return (
64 <Text>Receipt confirmed for PO-{purchaseOrder.number}</Text>
65 );
66 }
67
68 return (
69 <Stack gap={4}>
70 {/* Header info */}
71 <Stack gap={2}>
72 <Text>Purchase order: {purchaseOrder.number}</Text>
73 <Text>Supplier: {purchaseOrder.supplier}</Text>
74 <Text>Expected on: {purchaseOrder.expectedOn}</Text>
75 </Stack>
76
77 {/* Line items */}
78 {lines.map((line, i) => (
79 <FormField
80 key={line.sku}
81 label={`Received – ${line.name}`}
82 error={
83 received[i] < 0 || received[i] > line.ordered
84 ? "Quantity must be between 0 and " + line.ordered
85 : null
86 }
87 >
88 <NumberInput
89 min={0}
90 max={line.ordered}
91 value={received[i]}
92 onValueChange={handleReceivedChange(i)}
93 step={1}
94 />
95 </FormField>
96 ))}
97
98 {/* Dock selection */}
99 <FormField label="Dock" error={dock === "" ? "Required" : null}>
100 <Select
101 value={dock}
102 onValueChange={handleDockChange}
103 placeholder="Select a dock"
104 options={data.docks.map((d) => ({ label: d, value: d }))}
105 />
106 </FormField>
107
108 {/* Damage checkbox */}
109 <Checkbox checked={damaged} onCheckedChange={handleDamagedChange}>
110 Some goods arrived damaged
111 </Checkbox>
112
113 {/* Damage note (shown only when damaged is true) */}
114 {damaged && (
115 <FormField label="Damage note" error={note.trim() ? null : "Please describe what was damaged"}>
116 <Textarea rows={3} value={note} onChange={handleNoteChange} />
117 </FormField>
118 )}
119
120 {/* Confirm receipt button */}
121 <Button
122 variant="solid"
123 disabled={!allValid || submitted}
124 onClick={handleConfirm}
125 >
126 Confirm receipt
127 </Button>
128 </Stack>
129 );
130 }
lint/eslint(no-unused-vars) 1 quality typecheck/TS2375 1 quality page.tsx:88:12 Type '{ min: number; max: number; value: number | undefined; onValueChange: (value: number | null) => void; step: number; }' is not assignable to type 'NumberInputProps' with 'exactOptionalPropertyTypes: true'. Consider adding 'undefined' to the types of the target's properties. Every run that broke typecheck/TS2375 4 get_component 2,651 29 $0.0000
5 get_component 1,448 30 $0.0000
6 get_component 2,099 28 $0.0000
7 get_component 513 30 $0.0000
8 write_file 1,224 2,955 $0.0000