# Completed benchmark findings

The 20 original apps in Tests 1-3, twelve additional reviewed observations and
two separately scored Test 4 observations and five Test 5 reference
recreations meet their final scoring and
critical-check gates. Costs below are observed
cumulative Cowork task credits. Keep original and additional samples distinct;
app scope, model mix, review method and publication permission affect comparison.

**Read the costs against the workload:** Test 1 is a moderate virtual procurement
demo with approvals and stock changes; Test 2 is a narrower four-view app with
real SharePoint reads/writes; Test 3 is a seven-screen, multi-line workflow
prototype evolved through four planned stages. Test 3 adds linked virtual
tables, manager/finance routing, revisions, fulfilment/returns, themes and
preferences, but only its read-only profile is live. Its greater workflow
complexity is not a claim of greater production readiness or live business
integration. See the [scope comparison](README.md#what-was-built-and-tested)
and [complexity method](METHOD.md#comparing-app-complexity).
Test 4 is a separate eight-area stress workload for interacting policy, money,
budget and stale-state rules. It has no live connector or asset-return flow;
its core/advanced scores are not directly comparable with Test 3.
Test 5 adds a supplied HTML reference with separate functional and visual
acceptance. Its five-setting original cohort is complete; the preceding 34
accepted observations remain unchanged.

## Test 1: same-tenant GPT Medium pair

On 28 September 2026, two fresh GPT 5.5 / Medium apps received the identical
procurement prompt in one demo tenant. Both met the unchanged 85-point and
all-critical acceptance gate.

| Observation | Final credits | Authoring prompts | Corrections | Verified groups | Score |
|---|---:|---:|---:|---:|---:|
| Build 1 | **281** | 2 | 1 | 20/20 | 100/100 |
| Build 2 | **221** | 2 | 1 | 19/20 | 95/100 |
| Pair mean | **251** | **2** | **1** | Not pooled | Not pooled |

The observed minimum-maximum is **221-281 credits**, with a **60-credit range**.
Each sole final cumulative `/cost` includes its build and correction; no interim
cost or paid model self-review was taken.

Both first outputs used a local-storage key already present on the preview
origin. The controller contained that module before execution: this was an
observed source-level isolation defect, not demonstrated corruption. The same
narrow correction established an app-specific namespace. Independent native
review verified the exact seed, validation, approvals/rejections, stock and
oversell protection, requester views, actual reload/reopen persistence and reset.

The second app's dashboard measured **669px of content at a 390px viewport**.
The responsive group was not awarded, although native mobile request and approval
completed successfully. No optional polish prompt was purchased after acceptance.
The outputs therefore meet the same minimum but are not identical in usability.
This within-setting variation cannot establish that higher spending caused better
quality. Keep the pair separate from original paid-review measurements and other
cohorts; n=2 is insufficient to claim stable pricing or a model ranking.

## New results: Opus 5.5 / Medium

All five reported builds across Tests 1, 2, 3 and 5 are complete. The unfinished
new-version Test 4 run is omitted from this update at the user's request.
These are separate observations, not replacements for the original Opus 5
measurements or historical Test 4 results.

| Scope | Completed-app credits | Authoring prompts | Independently verified coverage |
|---|---:|---:|---|
| Test 1: virtual procurement | **159** | **1** | 20 of 20 functional groups |
| Test 2: live SharePoint, first build | **610** | **3** | 10 of 10 functional groups |
| Test 2: live SharePoint, second build | **419** | **2** | 10 of 10 functional groups |
| Test 3: complex lifecycle | **920** | **4** | 20 of 20 lifecycle groups; four planned stages, no repairs |
| Test 5: HTML reference recreation | **365** | **2** | 20 of 20 functional; 10 of 10 visual groups |

The SharePoint pair averages **514.5 credits and 2.5 authoring prompts**,
with observed costs **419-610**. Both used the identical initial prompt and
the same scoped acceptance criteria; subsequent guidance remained adaptive.
No unfinished costs, partial prompt totals or capability verdicts from the
omitted run are published.
Final repaired revisions were reviewed in the owner preview, not claimed
republished.

Both SharePoint apps initially passed nine groups but represented a real failed
read as an empty request list. One correction produced explicit errors and a
working Retry. Independent review verified actual schema files, successful
native saves, server readback, persistence after reopening and unchanged
inventory. A third prompt only made the existing schema files downloadable;
it is included in both the first build's three-prompt count and sole 610-credit total.
The second build's correction also requested those downloadable files, so it
needed no separate clarification: two prompts and a sole 419-credit total.
The difference cannot be attributed only to that extra prompt; build strategy,
generated output and run-to-run variation were not controlled separately.
Both final reviews verified all ten groups, not identical overall presentation:
the second app retains very small mobile dashboard chart labels, while its
catalogue and the rubric's named mobile controls remain readable and usable.
This bounded error coverage does not establish production resilience, and
historical untested error cases are not reclassified as failures.

Test 3 needed **no extra authoring prompts beyond the four planned stages**.
Independent review verified the live own-profile connection; saved-request and
old-policy carryover; exact GBP1,440/GBP1,560 approval routing; same-ID
resubmission and confirmed cancellation; atomic stock allocation, per-unit
assets and good/damaged returns; preferences; and desktop/mobile light/dark
usability. The one cumulative completed-app reading was **920 credits**.
All optional profile fields were present, so a genuinely missing-value response
was not exercised. A hyphen-only review-justification variation is disclosed
in the method; all numeric and workflow inputs matched. This single result is
not a stable price or a causal comparison with earlier model versions.

The Test 5 app reused the reference screens and business logic and required
one targeted styling correction. Its 365 credits are below every original
Test 5 observation, with all functional and visual groups verified. However,
this single observation does not isolate the effect of the new model from its
build strategy, product changes or run-to-run variation. Do not interpret it
as a guaranteed saving or pool it with Opus 5 averages.

## Test 5: completed matched repeats

Three settings now have a second independently accepted fresh app:

| Setting | Original / repeat credits | Mean credits | Observed range | Original / repeat authoring prompts | Functional groups, original / repeat |
|---|---:|---:|---:|---:|---:|
| Sonnet 5 / Medium | 2,517 / 3,309 | **2,913** | 2,517-3,309 | 8 / 3 | 17/20 / 18/20 |
| GPT 5.5 / Medium | 659 / 7,464 | **4,061.5** | 659-7,464 | 7 / 16 | 17/20 / 20/20 |
| Sonnet 5 / High | 2,523 / 3,153 | **2,838** | 2,523-3,153 | 3 / 6 | 18/20 / 17/20 |

All six apps passed every critical group and 8/10 visual groups, but the
particular remaining failures differ. The detailed page compares all 30 groups
for each pair. These are n=2 observations, not stable prices or a causal ranking.
Opus 5 is unavailable; its two pending repeat slots are not filled by Opus 5.5.
There is still no balanced five-setting repeat mean.

The Sonnet Medium repeat needed one build and two corrections. Native review
verified the restored fixture, draft deletion confirmation, week-calendar
pending/half-day cues and all seven actual CSV exports. Print was present but
its print layout still collapsed the main grid; global search and advertised
shortcuts also remained incomplete. Typography and report-density differences
account for the two visual failures. A transient preview input freeze required
one native reload; the cause remains unresolved. Later interaction checks
passed. The author reported private republication, but independent acceptance
covers the owner preview rather than asserting published-revision parity.

The GPT repeat cost includes one build, eight corrections and seven publication
clarifications. Its last private publication attempt timed out; the final
fixture-only correction was explicitly preview-only. Its fuller functional
coverage and much higher total cannot be attributed solely to model choice or
to a particular prompt. Sonnet High's extra guidance did not produce higher
functional coverage. Compare both delivered outcomes, not cost or prompt count
alone. All repeat screenshots, transcripts and generated source remain private.

## Test 5: original HTML-reference recreation

All five original settings are finalized. All received the same
initial prompt, identical frozen HTML bytes and the same joint acceptance
target. Required corrections stayed in the original app, with independent
native review and no arbitrary authoring-time or credit cutoff.

| Selected setting | Final cumulative credits | Authoring prompts | Corrections | Functional groups | Visual groups |
|---|---:|---:|---:|---:|---:|
| GPT 5.5 / Medium | **659** | 7 | 6 | 17 of 20 | 8 of 10 |
| Sonnet 5 / Medium | 2,517 | 8 | 7 | 17 of 20 | 8 of 10 |
| Sonnet 5 / High | 2,523 | 3 | 2 | 18 of 20 | 8 of 10 |
| Opus 5 / High | 3,197 | 3 | 2 | 19 of 20 | 8 of 10 |
| Opus 5 / Medium | 3,220 | 2 | 1 | 20 of 20 | 8 of 10 |

Every critical group passes. These counts are scope-specific, not overall
quality percentages. The mean is **2,423.2 credits across all five apps**.
No interim cost, invented zero score or incomplete prompt total is plotted.

**GPT met the same minimum at substantially lower observed cost, not because
it was assigned an easier goal.** Its 659 credits are **1,858 / 73.8% below
Sonnet Medium** and **2,538 / 79.4% below Opus High**. It needed one fewer
authoring prompt than Sonnet Medium and four more than Opus High.

The delivered apps are not feature-identical:

- **Reporting:** GPT verifies the complete reporting/export/print group,
  including all seven actual CSV downloads. Opus High verifies report scopes,
  chart data and CSVs but retains a print-layout defect. Sonnet Medium retains broader
  reporting limitations. Opus Medium verifies the complete reporting group,
  including unclipped printing, as does Sonnet High. A shared failed group does not imply the same defect.
- **Drafts and approvals:** Both Sonnet and Opus settings retain confirmed draft deletion;
  GPT does not. Opus additionally verifies confirmed selected-only bulk
  approval, queue-moving delegation and delegation removal, as does Sonnet High.
  GPT and Sonnet Medium retain gaps in this group.
- **Finding and using information:** Opus has working request/person/policy
  search, About and fuller keyboard behavior. GPT's search remains area-only
  and About is absent; its explicit notification controls work. Sonnet High
  retains request/person-search gaps despite working notification and keyboard controls.
- **Calendar:** Sonnet High retains Tuesday-shifted week navigation, an incorrect
  current-day marker and absent half-day markers. The other accepted settings
  pass this group; High's better reporting does not cancel out these defects.
- **Visual fidelity:** Sonnet Medium passes the reference typography/palette group.
  GPT, Sonnet High and both Opus builds retain typography differences. GPT, Sonnet Medium and Opus
  High retain card/tab grouping or spacing limitations. Opus Medium passes that
  spacing group but lacks the doughnut colour-key legend. Equal visual totals
  conceal different outcomes.

**Medium reasoning did not guarantee lower cost.** Opus Medium used **23 more
credits (0.7%)** than Opus High, needed one fewer authoring prompt, and verified
all 20 functional groups, including the print layout High still lacked.
High reasoning was not necessary for that coverage in this observation.
GPT's 659 credits are **2,561 / 79.5% below Opus Medium**; Opus Medium required
five fewer prompts and delivered the additional draft, bulk/delegation and
search capabilities. This is an observed cost/capability tradeoff, not a
repeatable or causal reasoning-level comparison.

**Sonnet High reduced guidance, not total credits.** It used **2,523 credits and
three prompts**, only **six credits (0.2%) more** than Medium's 2,517 but with
**five fewer authoring prompts**. High adds confirmed bulk/delegation and complete
reporting; Medium retains stronger calendar and typography coverage. High also
passes report-legend and card-spacing fidelity where Medium falls short.
The same eight visual groups passed in count, but not in identity.
GPT used **1,864 fewer credits (73.9% lower)** than Sonnet High with four more
prompts, retaining better calendar coverage but weaker draft/bulk/delegation coverage.

**Extra Opus spend bought specific additional functionality and fewer guidance
turns, not a universally better app.** GPT is a substantially lower-cost
observed route to the shared minimum when its retained gaps are acceptable.
Choose against required capabilities, rather than either price or equal group
totals alone.

This is one adaptive observation per selected setting, with model and reasoning
differing together when GPT is compared with Opus High. The two Opus settings
hold the selected model constant but still have only one adaptive run each.
Credit differences do not price the
reviewer's time, establish internal worker identity, guarantee future savings
or establish a general ranking. GPT Medium is the lowest observed completed-app
cost in this closed five-setting original cohort.

Sonnet High's final image inspection exposed a desktop legend overlap after an
initial provisional review. Acceptance was withdrawn before cost; the second
correction passed 12 native dashboard cases, checking every applicable legend
label/value against its card and neighbouring content. The final measurement
includes both repairs. No provisional cost was recorded or published.

Only the completed cumulative cost is published. Final repairs were reviewed
in the owner preview, not claimed republished; exact initial published resources
were removed after evidence. The successful prompt sequences are retained for
future testing. Guided replay is a separate cohort, has not started and has no
selected donor.

## Choosing a model without paying for unneeded capability

**Test 4 testing is closed and all five settings have final cost bars.** The chart
shows all five settings, distinguishing the original accepted GPT observation
from separately registered Sonnet Medium, Opus High, Opus Medium and Sonnet
High follow-ups. Actual spend and accepted-app prices are clearly separated;
no queued or testing entries remain.

The Sonnet Medium, Opus High and Sonnet High follow-ups have **1,992, 8,144 and
4,229 observed final task credits**, respectively, shown as spend-only with no
accepted full-target result or capability score. No complete target was
independently observed within their fixed authoring bounds; this is not proof
of model incapability. All three are excluded from the
accepted-app average and cannot establish a saving against an accepted app.
Opus Medium is accepted at **2,938 credits**, with core **100/100** and
**70 verified advanced points**. All 34 previously accepted measurements remain
unchanged; the final Sonnet High reading does not add an accepted app.

**Start with a lower-cost setting supported by the relevant scope, independently
verify the capabilities you need, then escalate for a demonstrated unmet need
or a worthwhile reduction in corrective effort.** A complex app does not
automatically require High reasoning, and a higher bill is not a quality guarantee.

Two concrete comparisons retain the same original cohort and verified check
set, rather than treating equal scores alone as equal coverage:

| Same-cohort comparison | Lower-cost observation | Higher-cost observation | Observed credit difference | Coverage and guidance |
|---|---|---|---|---|
| Original Test 2: SharePoint | Sonnet Medium: 227 mean credits/app | Opus High: 877 mean credits/app | **650 fewer mean credits/app; 74.1% lower** | Both 90/100, same verified checks; two builds each. Corrections: Sonnet 0 / 0; Opus 0 / 1. |
| Original Test 3: complex apps | Opus Medium: 2,358 credits | Opus High: 3,073 credits | **715 fewer credits; 23.3% lower** | Both 95/100, same verified checks; one build each. Medium needed two corrections; High needed one. |

Percentages are `(higher - lower) / higher`, using final task credits including
guidance. They are **observed credit differences, not guaranteed future savings,
cash-price reductions or proof of equivalent overall quality**. The small
samples do not support a promised budget saving, and external review effort is
not priced into the credit differences.

For the simpler procurement/SharePoint scopes, Sonnet Medium is a
cost-supported starting choice. For similar complex apps, GPT Medium is a
credible baseline: Test 4 verified all core checks and nine of ten advanced
scenarios for 1,573 credits, including one correction. The tested goal did not
require selecting High reasoning in that observation.

**The lowest-cost accepted result does not mean the only capable model.**
Every registered Test 4 setting had the same complete-app target, acceptance
criteria and planned time/correction limits. No setting received easier
requirements because it was expensive, and credit price itself was not a
stopping rule. A missing accepted result is not a zero capability score;
platform interruptions and review or observation limits do not establish model
incapability. GPT Medium used **1,365 fewer credits (46.5% lower)** than the
accepted Opus Medium follow-up; both verified the same 100-point core target.
This compares one original and one follow-up using the same frozen target,
not matched repeats, identical advanced coverage or a clean five-way ranking.
Opus's automatic refresh preempted two exact stale-form prerequisites; missing
points for those unexercised cases do not demonstrate weaker concurrency.

**Spend more when a required feature justifies it.** Test 3 Opus results
demonstrated fuller preferences and sampled accessibility than lower-cost GPT
results. Those are real benefits, not an equivalent-quality saving opportunity.
Choose against the actual workflow, feature gaps and guidance effort, rather
than treating either the cheapest setting or the highest score as universally best.

## Did higher cost buy better apps?

**Not consistently.** The simpler scopes showed no broader verified feature
coverage with higher spending. Complex apps showed some specific gains, but
not a universally better core application or a consistent benefit from High
reasoning. These are findings about the completed apps shown, not success-rate
estimates or a universal model ranking.

| Scope | Cost comparison within the same cohort | What the additional spend demonstrated |
|---|---|---|
| Original Test 1 | Sonnet Medium mean 357 vs Opus Medium 1,083 credits; both 95/100 | Same awarded coverage; no broader measured capability |
| Additional Test 1 | Sonnet Medium 215 vs Opus High 826 credits; both 100/100 | Same verified procurement checks |
| Original Test 2 | Sonnet Medium mean 227 vs Opus High mean 877 credits; both 90/100 | Same live read/write, validation, server read-back and reopen checks |
| Additional Test 2 | Sonnet Medium 301 vs Opus High 1,130 credits; both 90/100 | No extra scored capability; connection-error coverage still unverified |
| Additional Test 3 | GPT Medium 573/85 vs Opus High 2,860/95 | 2,287 extra credits for fuller preferences and sampled accessibility coverage, not a different core approval/asset workflow |
| Test 4, within one GPT Medium app | 1,095 first target vs 1,573 final credits | One correction added 478 credits; advanced coverage rose from 80 to 90, with safer repeated submission and sampled focus/naming improvements |

Test 4 separately adds the Opus Medium follow-up at **2,938 credits** versus
the original GPT Medium's **1,573**, both core 100. The extra **1,365 credits**
did not buy a higher core score or another completely verified advanced
challenge in these observations. This is a disclosed cross-cohort comparison,
not another row in the same-cohort table above. It does not establish equal
overall quality: the apps differ in interface, live-update behavior and
unexercised advanced prerequisites.

Test 1's original 95 and additional 100 scores use different
persistence-scoring/review methods: **that is not a model improvement**.
Equal checklist coverage also does not establish identical visual polish,
code quality or reliability beyond the measured scope.

### Complex-app differences by model and reasoning

| Selected configuration | Original credits / score | Additional credits / score | Verified strengths and limits |
|---|---:|---:|---|
| GPT 5.5 / Medium | 944 / 90 | 573 / 85 | Core workflows and cost-centre handling verified. Original meaningful density worked; additional meaningful density was not demonstrated. Default-department consumption remained incomplete; additional sampled light error contrast was below target. Save/Cancel and preference persistence worked. |
| Sonnet 5 / Medium | 1,326 / 90 | No published observation | Full preference check verified, but cost-centre handling was not met. |
| Sonnet 5 / High | 2,289 / 90 | 2,134 / 85 | Cost-centre handling verified. Additional meaningful density changed rows by 4px, but default-department consumption remained unverified and sampled light error contrast was below target. |
| Opus 5 / Medium | 2,358 / 95 | No published observation | Full preference, cost-centre and sampled accessibility checks verified. |
| Opus 5 / High | 3,073 / 95 | 2,860 / 95 | The same full checks verified. Additional density changed rows by 8px; the saved department was actually used by Approvals and all-requests filters. |

All complex apps retained unverified profile fault/Retry coverage. The
additional GPT and Sonnet High apps scored the same 85, but Sonnet had a real
density effect: **equal scores did not mean identical functionality**.
All accepted apps already passed the critical workflow gates.

**Did High reasoning help?** In original Test 3, Opus High cost **715 more**
than Opus Medium for the same 95-point check set, although it needed one
correction rather than two. Sonnet High cost **963 more** than Medium at the
same 90-point score, exchanging strengths: High verified cost-centre handling;
Medium verified the full preference check. In Tests 1 and 2, High added no
verified checks within either cohort. It was not always more expensive:
original Test 1 Opus High cost 191 fewer credits than Opus Medium.

**Did additional builds improve?** They were fresh apps, not later versions
of one app. GPT fell from 944 to 573 credits but from 90 to 85 points; Sonnet
High fell from 2,289 to 2,134 credits but from 90 to 85 points. Opus High fell
from 3,073 to 2,860 credits with the same 95-point coverage. Lower observed
spending did not itself mean a better app, and changed publication permission
prevents treating these differences as an isolated model-price effect.

**Did further work improve an individual app?** Yes: the additional accepted
Opus High app used **429 credits after its last planned stage** for a final
mobile-layout refinement, finishing at 95/100 while retaining business state.
This is a concrete usability improvement from corrective work, not a general
repair tariff or proof that the model premium caused that improvement.

**Maker takeaway:** start with the lower-cost settings for the simpler tested
scopes. For complex apps, pay attention to the particular capability you need:
the Opus results showed broader preference/accessibility coverage, but High
was not consistently better than Medium. Independently exercise the real
workflow and settings rather than assuming higher spend guarantees quality.

## Test 1: virtual-data apps

Five accepted Device Procurement and Availability apps. Virtual/local data is
allowed; managed-store persistence is not required. Recorded scores remain
95/100 without awarding additional storage points.

| Run | Selected configuration | Final credits | Final score | Clarifications | Corrections |
|---|---|---:|---:|---:|---:|
| 01 | Sonnet 5 / Medium | 317 | 95/100 | 0 | 0 |
| 02 | Opus 5 / High | 892 | 95/100 | 0 | 1 |
| 03 | Opus 5 / Medium | 1,083 | 95/100 | 0 | 2 |
| 04 | Sonnet 5 / High | 530 | 95/100 | 0 | 1 |
| 05 | Sonnet 5 / Medium | 397 | 95/100 | 0 | 1 |
| **Total** | **5 accepted apps** | **3,219** | | **0** | **5** |

Each task includes one native self-review prompt in addition to external checks.
These are build-and-review costs, not the external-only method used in Tests 2
and 3. The Sonnet Medium baseline was repeated; the other settings have one
observation each. Build-completion timing was not captured for this test, so
build durations are not estimated.

## Test 2: SharePoint-connected apps

Ten accepted apps, with five configurations tested twice in reverse-order
blocks. Every app reads the same synthetic SharePoint fixture, creates a
complete request, and demonstrates persistence through independent server
read-back and reopening. All score 90/100.

| Run | Selected configuration | Final credits | Final score | Clarifications | Corrections | Build time, approx. |
|---|---|---:|---:|---:|---:|---:|
| 01 | Sonnet 5 / Medium | 224 | 90/100 | 0 | 0 | 10m 00s |
| 02 | Opus 5 / High | 968 | 90/100 | 0 | 0 | 19m 26s |
| 03 | Opus 5 / Medium | 417 | 90/100 | 0 | 0 | 8m 48s |
| 04 | Sonnet 5 / High | 620 | 90/100 | 0 | 0 | 16m 58s |
| 05 | GPT 5.5 / Medium | 345 | 90/100 | 0 | 1 | 10m 32s |
| 06 | GPT 5.5 / Medium | 252 | 90/100 | 0 | 0 | 7m 51s |
| 07 | Sonnet 5 / High | 313 | 90/100 | 0 | 0 | 11m 26s |
| 08 | Opus 5 / Medium | 425 | 90/100 | 0 | 0 | 10m 24s |
| 09 | Opus 5 / High | 786 | 90/100 | 0 | 1 | 15m 28s |
| 10 | Sonnet 5 / Medium | 230 | 90/100 | 0 | 0 | 8m 46s |
| **Total** | **10 accepted apps** | **4,580** | | **0** | **2** | |

Eight apps required no corrective guidance; two required one corrective prompt
each. Review was external-only. Connection-error/no-fallback coverage (BC09)
remains unverified, accounting for the unawarded ten points.

### Paired cost, time and maker guidance

| Configuration | Final credits, repeat 1 / 2 | Observed mean credits | Corrective prompts, repeat 1 / 2 |
|---|---:|---:|---:|
| Sonnet 5 / Medium | 224 / 230 | 227 | 0 / 0 |
| GPT 5.5 / Medium | 345 / 252 | 298.5 | 1 / 0 |
| Opus 5 / Medium | 417 / 425 | 421 | 0 / 0 |
| Sonnet 5 / High | 620 / 313 | 466.5 | 0 / 0 |
| Opus 5 / High | 968 / 786 | 877 | 0 / 1 |

**Sonnet 5 / Medium is the cost-supported starting choice for this specific
SharePoint scope:** both observations reached 90/100 with no corrections.
Two observations are not a reliable variance estimate or a promised future price.
Per-app build times add the recorded authoring responses, including corrections,
and exclude external review and gaps between prompts. They are approximate
observations, not isolated model compute time.

## Test 3: complex-app results

Five accepted apps, each receiving the same four ordered prompts:
build the virtual-data foundation, connect Office 365 Users inside the existing
app, extend workflows/themes/preferences, then change policy/forms/mobile
requirements. Business records and approvals remain synthetic and virtual/local.

| Run | Selected configuration | Final credits | Final score | Clarifications | Corrections | Build time, approx. |
|---|---|---:|---:|---:|---:|---:|
| 01 | Sonnet 5 / Medium | 1,326 | 90/100 | 0 | 1 | 39m 31s |
| 02 | Opus 5 / High | 3,073 | 95/100 | 0 | 1 | 46m 03s |
| 03 | Opus 5 / Medium | 2,358 | 95/100 | 0 | 2 | 35m 12s |
| 04 | Sonnet 5 / High | 2,289 | 90/100 | 0 | 0 | 51m 30s |
| 05 | GPT 5.5 / Medium | 944 | 90/100 | 1 | 0 | 21m 43s |
| **Total** | **5 accepted apps** | **9,990** | | **1** | **4** | |

All five have three planned follow-up prompts and zero native self-review
prompts. Planned changes are not corrective guidance. GPT Medium's single
clarification was "Build them all"; its cost is included in the recorded task
credits, not estimated separately.

### Cumulative stage readings

| Configuration | Foundation | Profile | Workflows | Policy/forms | Final task total |
|---|---:|---:|---:|---:|---:|
| Sonnet 5 / Medium | 176 | 367 | 770 | 1,326 | 1,326 |
| Opus 5 / High | 561 | 916 | 1,679 | 2,505 | 3,073 |
| Opus 5 / Medium | 505 | 1,124 | 1,720 | 2,011 | 2,358 |
| Sonnet 5 / High | 292 | 550 | 1,507 | 2,289 | 2,289 |
| GPT 5.5 / Medium | 381 | 501 | 736 | 944 | 944 |

**Do not add these columns.** Each reading is cumulative. The final task total
includes all recorded guidance, including any after the last planned change.
Foundation readings cover that stage, not the complete target application.

### Complex-app cost, capability and maker guidance

All five demonstrate the required multi-screen navigation, forms, virtual
approval and asset workflows, data continuity, light/dark themes and actual
read-only signed-in profile. Profile identity and displayed fields were
independently verified in-app and reverified after the final reopen.

| Configuration | Final score | Coverage outside the awarded points |
|---|---:|---|
| Sonnet 5 / Medium | 90/100 | Profile fault/Retry coverage unverified (CX14); cost-centre handling incomplete (CX17) |
| Opus 5 / High | 95/100 | Profile fault/Retry coverage unverified (CX14) |
| Opus 5 / Medium | 95/100 | Profile fault/Retry coverage unverified (CX14) |
| Sonnet 5 / High | 90/100 | Profile fault/Retry coverage unverified (CX14); full default-department consumption coverage unverified (CX16) |
| GPT 5.5 / Medium | 90/100 | Profile fault/Retry coverage unverified (CX14); full default-department consumption coverage unverified (CX16) |

Preference Save/Cancel, persistence and measurable density changes were
demonstrated for Sonnet High and GPT Medium; the CX16 limitation concerns the
full default-department-consumption check, not all preference functionality.
Equal 90-point scores therefore do not imply identical delivered capabilities.

**GPT 5.5 / Medium had the lowest observed accepted cost**, at 944 credits and
90/100, with one clarification and no corrective prompts. Both Opus settings
reached 95/100; Opus Medium used fewer final credits than Opus High in these
observations. Higher reasoning did not consistently increase the final score.
The original cohort has one observation per setting; the additional pairs
below extend three settings. Neither establishes a universal best model.

Build time includes every planned authoring stage and additional clarification
or correction response. It excludes external review, screenshot-taking gaps,
time between prompts, `/cost` commands and native self-review. For example,
GPT Medium's **21m 43s** includes both its initial response and clarification,
plus all three planned follow-ups, rather than treating its first reply as
the complete app. Within-turn waits and observation uncertainty remain included.

## Additional external-review measurements

The additional samples retain the original results above and use **no native
self-review prompts**. Every value below is a completed app's final cumulative
task reading, including its recorded corrections. There are zero clarification
prompts in these twelve additional observations.

### Additional Test 1: virtual procurement

| Configuration | Final credits | Score | Corrections | Build time, approx. | Prompt |
|---|---:|---:|---:|---:|---|
| Sonnet 5 / Medium | 215 | 100/100 | 0 | 9m 52s | virtual-v1 |
| Opus 5 / High | 826 | 100/100 | 1 | 15m 01s | virtual-v1 |
| Sonnet 5 / High | 220 | 100/100 | 0 | 11m 13s | virtual-v2 |
| GPT 5.5 / Medium | 269 | 100/100 | 1 | 9m 05s | virtual-v2 |
| **Total / mean** | **1,530 / 382.5** | **4 reviewed apps** | **2** | | |

The four shown settings each have one additional observation. The new method
explicitly verifies and scores local persistence, alongside approvals,
rejections, stock safety and reset. Original 95/100 scores and paid-review costs
remain unchanged. The two prompt versions differ only in private owner-only
publication permission; keep that difference visible.

### Additional Test 2: SharePoint procurement

| Configuration | Final credits | Score | Corrections | Build time, approx. |
|---|---:|---:|---:|---:|
| Sonnet 5 / Medium | 301 | 90/100 | 0 | 11m 55s |
| Opus 5 / High | 1,130 | 90/100 | 1 | 15m 13s |
| Opus 5 / Medium | 842 | 90/100 | 1 | 14m 36s |
| Sonnet 5 / High | 1,025 | 90/100 | 2 | 27m 37s |
| GPT 5.5 / Medium | 314 | 90/100 | 0 | 7m 49s |
| **Total / mean** | **3,612 / 722.4** | **5 reviewed apps** | **4** | |

All five use the identical `sharepoint-v2` prompt. Each demonstrates live
catalogue data, complete request creation, independent backend read-back and
reopen persistence, with unchanged stock and no invalid-input writes. All retain
the same unverified BC09 connection-error/no-fallback coverage.

### Comparing the same four settings

| Configuration | Additional virtual credits | Additional SharePoint credits |
|---|---:|---:|
| Sonnet 5 / Medium | 215 | 301 |
| Opus 5 / High | 826 | 1,130 |
| Sonnet 5 / High | 220 | 1,025 |
| GPT 5.5 / Medium | 269 | 314 |
| **Equal-weight mean** | **382.5** | **692.5** |

This removes the difference in model mix between the four-app and five-app
additional samples. It does **not** make virtual approval/stock workflows and
SharePoint request tracking equivalent, or isolate a connector premium.
Publication permission and the original Test 1's paid self-review also prevent
silently pooling the historical and new means.

**Maker guidance:** Sonnet Medium is a cost-supported starting point for these
small procurement scopes; the additional virtual result used 215 credits and
the connected result 301, both without corrective guidance. GPT Medium was
close on connected cost at 314 with no correction. Higher effort did not
guarantee fewer corrections, lower cost or a higher final score. Use the
per-setting readings and actual capability coverage, not a cross-test headline
average, and do not treat these small samples as a guaranteed future price.

### Additional Test 3: complex apps

Three additional demo apps completed all four planned stages and
the independent final acceptance checks. Original results remain unchanged.
The business prompts and rubric are consistent; the amended foundation prompt
permits private owner-only publication. No native self-review prompt was sent.

| Configuration | Final credits | Score | Corrections | Build time, approx. |
|---|---:|---:|---:|---:|
| GPT 5.5 / Medium | 573 | 85/100 | 1 | 21m 09s |
| Sonnet 5 / High | 2,134 | 85/100 | 1 | 49m 11s |
| Opus 5 / High | 2,860 | 95/100 | 2 | 42m 45s |
| **Total / mean** | **5,567 / 1,855.7** | **3 reviewed apps** | **4** | |

There were no clarification prompts. Times use submission-to-first-observed-idle
intervals, not the later idle-confirmation timestamp or external review time.

| Configuration | Foundation | Profile | Workflows | Policy/forms | Final task total |
|---|---:|---:|---:|---:|---:|
| GPT 5.5 / Medium | 183 | 276 | 397 | 573 | 573 |
| Sonnet 5 / High | 433 | 654 | 1,232 | 2,134 | 2,134 |
| Opus 5 / High | 521 | 1,219 | 1,860 | 2,431 | 2,860 |

These readings are **cumulative, not additive**. Final totals include the
corrections used to reach acceptance, including any after the final planned
stage. Stage differences do not isolate connector charges.

| Configuration | Original credits | Additional credits | Mean of two | Original / additional score |
|---|---:|---:|---:|---:|
| GPT 5.5 / Medium | 944 | 573 | 758.5 | 90 / 85 |
| Sonnet 5 / High | 2,289 | 2,134 | 2,211.5 | 90 / 85 |
| Opus 5 / High | 3,073 | 2,860 | 2,966.5 | 95 / 95 |
| **Same-setting means** | **2,102** | **1,855.7** | **1,978.8** | |

The original overall Test 3 mean remains **1,998 across five settings**. The
matched comparison above uses exactly three settings on both sides; its paired
mean covers six observations. Do not treat the different sample composition or
publication permission as a measured model-price change.

All three demonstrate the critical multi-screen, multi-line, approval,
stock-safe fulfilment, continuity, theme, real-profile and responsive
requirements. Their noncritical coverage differs:

| Configuration | Coverage outside the awarded points |
|---|---|
| GPT 5.5 / Medium | Profile fault/Retry unverified; meaningful density/default-department consumption not demonstrated; sampled light-theme error text below 4.5:1 |
| Sonnet 5 / High | Profile fault/Retry and default-department consumption unverified; sampled light-theme error text below 4.5:1 |
| Opus 5 / High | Profile fault/Retry unverified |

Save/Cancel and preference persistence work in all three. Sonnet High has a
measured 4px request-row density change; Opus High has an 8px change and consumes
the default department in Approvals and all-requests filters. Do not infer that
either filter sets a new request's department. The profile is genuinely read-only
and was reverified after each final native reopen.

**Maker guidance:** GPT Medium remains the lowest observed completed cost in
these matched complex-app results, but its additional 85/100 result has more
coverage gaps than its original 90/100 result. Opus High reached 95/100 twice at
higher cost, using one and two corrective prompts respectively. Choose against
the actual required capabilities and review budget, not just the smallest
credit number. Two observations per shown setting do not establish reliable
variance, success rates or future prices.

## Test 4: capability stress

Two accepted Device Allocation and Policy Hub observations received the same
four frozen stages and independent review: original GPT Medium and the
separately registered Opus Medium follow-up. Neither used clarification prompts
or native self-review. Original outcomes and first-target assessments remain
unchanged.

| Configuration / cohort | First-target credits | First core / advanced | Early corrections | Final credits | Final core / advanced | Later corrections | Build time, approx. |
|---|---:|---:|---:|---:|---:|---:|---|
| GPT 5.5 / Medium · original | 1,095 | 100 / 80 | 0 | 1,573 | 100 / 90 | 1 | 34m 52s |
| Opus 5 / Medium · follow-up | 2,426 | 70 / 0 | 1 | 2,938 | 100 / 70 | 1 | 51m 38s |

The displayed accepted-app average is **2,255.5 credits across two apps**.
Sonnet Medium's 1,992, Opus High's 8,144 and Sonnet High's 4,229 spend-only
readings are excluded. One original and one follow-up can
support the specific observed core-cost difference, not a universal ranking,
repeatability claim or predicted future price.

| Configuration | Foundation | Workflows | Policy / pricing / themes | Resilience / first target | Final task total |
|---|---:|---:|---:|---:|---:|
| GPT Medium | 210 | 431 | 677 | 1,095 | 1,573 |
| Opus Medium | 585 | 1,392 | 1,928 | 2,426 | 2,938 |

These readings are cumulative. **Do not add them.**

### What GPT's extra 478 credits improved

One corrective prompt made rapid repeated submission safe, restored Escape
focus return and supplied the missing Requester field name. Core coverage
remained 100/100 while advanced coverage rose from 80 to 90. This is a
verified improvement within the same app, not a standard repair price or
proof that a higher-cost model would provide the same gain.

### What Opus's extra 512 credits improved

One early correction cost **196 credits**, taking the foundation's cumulative
reading from 585 to 781; it is already included in the first-target 2,426.
That first target had core 70 and no complete advanced passes: saved-data
actions blocked several prerequisites while other scenarios were only partly
exercised. Zero advanced points did not mean an app with no advanced features.

The sole later correction cost **512 credits**. Native rechecks verified repaired
saved-data actions, Escape focus return in all four width/theme combinations,
and keyboard movement of 24 labelled overflowing tables. Core reached 100 and
seven advanced challenges completed. This was repair and verification of the
same app, not a replacement or extra hidden build.

| Advanced challenge | GPT first | GPT final | Opus first | Opus final |
|---|---|---|---|---|
| TA01: exact-pence discounts/VAT, half-up rounding and quoted-price preservation | Verified | Verified | Not met | Verified |
| TA02: original V1 carryovers, revisions and exact V2 finance boundary | Verified | Verified | Not met | Verified |
| TA03: no self-approval or same-person manager/finance decisions | Verified | Verified | Not verified | Verified |
| TA04: budget contention, reservation release and retry | Verified | Verified | Not verified | Verified |
| TA05: partial fulfilment, close remainder and penny conservation | Verified | Verified | Not verified | Verified |
| TA06: atomic multi-line fulfilment when one line has insufficient stock | Verified | Verified | Not verified | Verified |
| TA07: rapid repeated actions and stale decision/fulfilment attempts | Not met | Verified | Not met | Not verified |
| TA08: two intentionally stale finance views competing for one budget | Verified | Verified | Not verified | Not verified |
| TA09: invalid/cancelled/stale imports and audited atomic valid import | Verified | Verified | Not met | Verified |
| TA10: complete exact keyboard/search/rejection-dialog scenario | Not met | Not checked | Not met | Not met |

Every advanced pass requires all frozen subcases. GPT's final TA10 entry is
unverified because the exact request-search/rejection-dialog scenario could
not be exercised: the app has no request-search control and uses an inline
rejection form. These interface choices were not explicitly prohibited by
the build prompts. No TA10 points were awarded despite the independently
verified focus/naming repairs and other sampled accessibility observations.
Do not present that coverage gap as an unresolved demonstrated defect.

Opus provides a labelled request-search field and observed automatic cross-tab
updates. In TA07 its pending decision card disappeared when the other view
approved; in TA08 the other form automatically showed the reduced balance
before its decision. No controller refresh was used, but the exact intentionally
stale forms were no longer present. Those checks remain zero-point **coverage
gaps, not demonstrated race failures**. Rapid submission and an already-open
stale fulfilment confirmation were independently verified without duplication.

Opus's remaining TA10 defect is narrower and measurable: light-theme inline
rejection error text is **4.431:1 at 14px**, below **4.5:1**, at both widths.
Its exact rejection-dialog subcase also remains unexercised because the
interface is inline. These are distinct from its repaired focus and table
keyboard behavior. Both allowed corrections were already used.

GPT's final layout/interaction evidence includes 40 main/error scenes and eight
import/reset confirmation scenes across desktop/mobile and both themes, with
1,072 text samples. Earlier unaffected business-scenario evidence was retained,
with targeted correction rechecks; this is not a claim that every historical
case was recreated after the repair.
Opus's final sample contains 48 area/error/confirmation scenes in both themes
at actual 1440px/390px, plus the 24 keyboard-scroll checks. Its original V1 and
continuity records were verified after repair and before explicit resets.

**Maker takeaway:** GPT Medium met the same verified core target for **1,365
fewer credits (46.5% lower)** than the Opus Medium follow-up. Higher spending
was not required for that core goal in these observations. Do not interpret
90 versus 70 advanced points as a clean 20-point capability deficit: two Opus
checks were not fully exercised because of live refresh. Choose against the
particular rules, interactions and remaining gaps you need, not a universal
model winner or a promised saving.

## Interpretation and limits

Acceptance means at least 85/100 and every required critical check, not complete
coverage or production readiness. The synthetic fixtures do not establish
production authorization, concurrency, large-list scale or reliable future
pricing. Selected Cowork settings do not prove internal app-builder routing.

See [METHOD.md](METHOD.md) for scope, controls, cost semantics and evidence
handling, or [data.json](data.json) for exact final measurements and capability
verdicts.