How long did each app take to build?
Observed build time per app sums the authoring turns: build, planned updates, clarifications and corrections. It excludes external review and screenshot-taking gaps between prompts, /cost commands and native self-review. Within-turn waits and observation uncertainty remain included; this is not pure model compute time. Missing authoring coverage is not estimated from elapsed test time.
| Run / configuration | Observed build time |
|---|
Times are approximate submission-to-completion observations, not total test-session duration. All authoring turns must have recorded completion times; missing coverage is shown as not recorded.
Build first. Connect inside the app. Then evolve it.
The original cohort contains five fresh demo apps, one per model/effort configuration. Every app receives the same four frozen prompts: virtual-data foundation; Office 365 Users connection within that existing managed app; virtual approvals, asset lifecycle, themes and preferences; then a changed approval threshold, revisions, cost centres and mobile refinement.
Business records and approvals are virtual/local. Only the signed-in user's profile is live and read-only. An existing draft must survive every change. The final assessment covers all four stages: twenty checks worth five points, with at least 85/100 and every critical check required. Three planned follow-ups are counted separately from corrective or clarification prompts.
Actual cumulative task credits are recorded after each stage, after guidance and at the end. Use the final reading as the total; do not add cumulative readings together. The profile is independently verified in the built app, including after reopening.
The additional completed observations give two measurements for each paired setting, with amended private-publication permission. The original five remain separate. These small samples do not establish reliable future pricing. Private profile screenshots and connection identifiers are not published.
Frozen Test 3 protocol · 1. Build · 2. Connect within app · 3. Extend · 4. Change · Rubric · Policy carryover case · Low-stock crossing case
Hold the task still. Measure the delivered app.
Every run starts a fresh task and app. The implementation-only prompt, live seed, external review instructions, desktop/mobile views and ten-check rubric are held constant. Acceptance testing is external; only required corrective guidance is sent back to Cowork.
| Control | SharePoint comparison |
|---|---|
| Backing data | Two existing synthetic SharePoint lists and the same working demo-account connection |
| Repetitions | Five model/effort configurations, twice each, in reverse-order blocks |
| Seed | 6 device models, 25 available units, 4 requests: 3 Pending, GBP 3,800 pending value |
| Acceptance | 10 checks × 10 points; at least 85/100 AND every critical check passes |
| Review | Independent browser/backend checks, with no native self-review prompts |
| Guidance | At most two narrowly evidenced corrections and two clarification replies; all counted |
| Costs | Actual final /cost, including any required guidance; cumulative readings are not added |
| Persistence | One complete Pending request, independent server read-back and reopening the actual app; no inventory mutation |
All ten apps use the same established connection. The synthetic fixture is restored between runs.
Virtual data, five accepted apps.
Test 1 allows virtual/local business data. Managed-store persistence (AC14) is outside its required scope. All five apps are accepted at their recorded 95/100 scores; no additional storage points are awarded.
Each task includes a native self-review prompt as well as external checks. These are build-and-review costs, whereas Tests 2 and 3 use external-only review. Keep the cohorts separate.
Task consumption is not a universal model ranking.
- The original Test 3 cohort has one observation per configuration; original Test 2 has two. Additional completed observations are labelled separately, with matched-setting means where available. These small samples do not establish a reliable variance estimate or a promised price. Order, caching, service load and internal builder routing remain possible influences.
- Meeting the 85-point and critical-check gate is not full coverage or production readiness. Profile fault/Retry coverage remains unverified in Test 3 (CX14), with additional per-run preference/cost-centre gaps. Connection-error/no-fallback coverage remains unverified in Test 2 (BC09). Large-list pagination, real user authorization, concurrency and scale are not established by the small synthetic fixture.
- The selected Cowork model is not proof that every internal app-building worker used that model.
- A shared connection and a synthetic persona switch are not production authorization. Sequential stock checks do not establish concurrent safety.
- Build time excludes review and evidence-capture gaps, but includes waits within an authoring turn. Build timing was not captured for Test 1.
- The tests differ in app complexity, data source and review method. Their cost differences do not isolate the price of a connector.
- Credits here are observed task consumption, not a currency amount or tenant invoice. App runtime, connector, storage, orchestration and deployment costs are not all measured by this comparison.
Evidence without publishing account data
Exact prompts, visible transcripts, approval records, backend snapshots and raw cost evidence are retained privately. Test 5 separately publishes its authorized HTML reference and selected final app-only screenshots. Public data remains an explicit allowlisted projection, not a raw transcript export.
No credentials, browser profile, tenant/site/task identifiers, private app links, raw source exports or account-bearing screenshots are included in this public page.
Download the public measurement data · Findings · Related load-testing report