path: root/docs/PERFORMANCE.md

<!--
  Author:  cirakg & claude ai
  Summary: Performance and memory budgets, how they are measured, and the measured numbers per milestone.
-->
# Performance

The targets are in [README section 18](../README.md#18-performance-budgets). This document records how each number is measured, what it does and doesn't tell you, and the results.

## How it is measured

- **In-app start-up** (`startup_ms`): debug builds take `System.getTimer()` when the app object is created and again after the first complete frame, and print the difference (`source/util/Diag.mc`). It doesn't include the time the watch needs to load the program before the app object exists.
- **Launch to first frame** (`first_frame_wall_ms`): `docker/scripts/app-diag.sh` measures the wall-clock time from launching `monkeydo` until the app's first-frame line appears. This includes the Java start of `monkeydo`, sending the program to the simulator and loading it, so it is mostly tool overhead; it's only useful to catch large regressions.
- **Memory** (`mem_used_fraction`): `System.getSystemStats()` after the first frame: used memory divided by the memory the device gives watch apps.
- **On the watch**: the numbers that matter are measured on your Forerunner 70 with a sideloaded debug build: menu, "Debug: benchmark" (and "Debug: overlay" for draw time and memory). The simulator runs on your PC's CPU and says nothing about watch speed.

`make perf` and `make memtest` fail when a value exceeds `tools/budgets.json`.

## Budgets in force

| Value | Budget | Why |
| --- | --- | --- |
| `startup_ms` | 100 ms | M4 measured 1 to 18 ms; the margin absorbs simulator jitter |
| `first_frame_wall_ms` | 36000 ms | Regression guard only: about 2× M4's 16-18 s, which include loading the glance and opening the app from it |
| `mem_used_fraction` | 70 % | Hard limit, leaves room for spikes on the 512 KB watches |
| `glance_mem_used_fraction` | 70 % | Hard limit on the glance's 64 KB (README 18), measured after its first draw |
| Watchdog | Each job step at most half the per-event limit | `make watchdog`: the largest inputs at two steps per event (M8) |

## Results

### Forerunner 70, 2026-10-21 (real watch, debug build 0.8.0)

The first numbers from the real watch, read from "Debug: benchmark" on the sideloaded debug build (docs/YOUR_STEPS.md part 4.2):

| Value | Forerunner 70 | Simulator (M4/M8) | Note |
| --- | --- | --- | --- |
| TOTP, SHA-1 | 18 ms | under 1 ms | One code per timer event is far inside the watchdog |
| TOTP, SHA-512 | 732 ms | 17.5 ms | About 40 times SHA-1, as the simulator ratio predicted. A list with several SHA-512 entries fills in about one per second; the detail screen shows the code within a second |
| PBKDF2 | 2,000 iterations per second | about 12,000 | Exactly the lower bound in README 18, so setting a PIN calibrates to the 2,000-iteration floor and unlocking takes about 1 s |
| Header reading | 0 entries in 1 ms | | The benchmark ran before the demo entries were added, so this says nothing yet |
| Self-test | 292 ms | | The crypto known-answer tests at start-up of the debug build |
| Memory at the benchmark | 179 of 763 KB (23 %) | 21 % at the first frame | Well under the 70 % budget |

Other timings reported from the same session: settings saved on the phone reach the watch in about 1 second (settings spike, test G). The Member QR demo card took noticeably long on its first opening and about 1 second the second time; its progress display jumped to 90 % almost at once, which is fixed in the next version (the percentage now follows the work, not the phase count).

What this means: unlocking stays at about 1 s because 2,000 iterations is also the minimum the app accepts. An attacker who copies the watch's storage still has to run those 2,000 iterations per guess, but on a PC that is microseconds, so the PIN protects mainly against someone holding the watch (lockout and the optional erase after 10 wrong PINs), not against an offline attack on a copied file system (docs/SECURITY.md).

### M8, 2026-10-21 (polish and release candidate, version 0.8.0)

- **Memory, Forerunner 255S (512 KB):**

  | | M7 | M8 | Budget |
  | --- | --- | --- | --- |
  | First frame (`make memtest`) | 31.4 % | 34.4 % | 70 % |
  | Peak during the 100-entry import (UI flow) | 56.2 % | 45.3 % | 70 % |
  | Glance, entry count / favourite code | about 19 % (12.5 KB) | 36.7 % / 42 % | 70 % |

  The import peak fell because parsed items are kept packed (an `Entry` object takes about 490 bytes, an item with its entry about 760; packed, about 110) and the pasted text is held once. The first frame rose by the code M8 added. Most of the 156 KB the app occupies before it draws anything is its own code and constant data. The glance now loads the crypto for the favourite code. The 768 KB watches: 22.9 % at the first frame, 30.3 % at the import's peak (37.6 % in M7), 37 % in the glance.
- **Import:** 401 ms of compute for 100 entries in the unit-test benchmark (332 ms in M5, before packing). In the simulated watches it took 7.65 s on the 255S and 3.8 s on the 768 KB watches, which now run two job steps per tick (7.6 s in M7).
- **QR compute** (unit-test benchmark, simulator, one step at a time):

  | QR | M6 | M8 |
  | --- | --- | --- |
  | Version 3 | 81 ms, 28 steps | 61 ms, 54 steps |
  | Version 10 | 312 ms, 105 steps | 139 ms, 106 steps |
  | Version 26 | 1.4 s, 407 steps | 482 ms, 391 steps |

  Mask scoring now works on packed rows and columns (16 modules per word). The step counts didn't fall, because M8 found that M6's steps were too large for big codes (below) and sized every step by measurement. At 50 ms per tick, a version 10 QR needs at least 5.3 s on the 255S and 2.7 s on the 768 KB watches, plus the watch's compute, which only the on-watch benchmark can tell. README 18's 3 s target is met on the 768 KB watches at best; the cache makes every later opening instant.
- **The demo cards in the simulated watches** (first encoding, `make screenshots`):

  | Code | Forerunner 255S | 768 KB watches |
  | --- | --- | --- |
  | QR version 6 | 3.9 s (2.8 s in M6) | 2.0 s |
  | Data Matrix 18 x 18 | 0.8 s | 0.4 s |
  | PDF417 | 1.3 s | 0.4 to 0.7 s |
  | Aztec 23 x 23 | 0.5 s | 0.25 s |

  The 255S is slower than in M6 because its steps are smaller: M6's were sized by estimate and only exercised with these small codes.
- **Watchdog (`make watchdog`).** M6's step sizes were never tried with large codes. With the largest inputs the app accepts, every 2D encoder tripped the watchdog on the 255S. Going by the measured costs, the biggest steps (QR version 40's single-step mask application, about 31,000 modules) were far over the 768 KB watches' limit too. A code like that, added from the phone, would have stopped the app during settings processing at every start. A loop iteration turned out to cost 50 to 70 counted bytecodes, 2 to 3 times the estimate. Every step is now sized so that two steps fit in one event. Results on the 255S (steps, and peak memory in the stress build, which uses 204 KB before the first code):

  | Input | Symbol | Steps | Peak |
  | --- | --- | --- | --- |
  | QR, 2,900 letters, level L | version 40, 177 x 177 | 1,041 | 316 KB (62 %) |
  | QR, 800 digits, level H | 93 x 93 | 255 | 237 KB |
  | Data Matrix, 1,500 letters | 144 x 144 | 446 | 276 KB (326 KB before the byte-array grid) |
  | Aztec, 1,400 letters | 95 x 95 | 254 | 231 KB |
  | PDF417, the most this screen takes | 120 x 48 | 50 | 222 KB |

  In the real app, which uses about 178 KB at the list, the largest QR would peak at about 58 %. At 50 ms per step it takes about 52 s on the 255S and 26 s on the 768 KB watches, once, and it can't be shown on the 255S's screen anyway.
- **Drawing.** Codes are drawn as runs of dark modules, one rectangle each. Up to 1,500 runs per watchdog unit are drawn directly in `onUpdate`. Larger codes (the Venu X1 fits a version 39 QR with 8,111 runs) are painted into a buffered bitmap in steps first; `make watchdog` decodes one from the screen.
- **Start-up:** `startup_ms` 1 to 17 ms and launch to first frame 16 to 18 s, unchanged.

### M7, 2026-10-20 (settings forms, version 0.7.0)

- **Memory is rising with the code.** The app's code is loaded into the same memory as its data, so every milestone adds to it:

  | Forerunner 255S (512 KB) | M5 | M6 | M7 | Budget |
  | --- | --- | --- | --- | --- |
  | First frame (`make memtest`) | | 26 % | 31.4 % | 70 % |
  | Peak during the 100-entry import (UI flow) | 41.5 % | | 56.2 % | 70 % |

  The 768 KB watches use 21 % at the first frame and about 38 % at the import's peak. The 255S still has room, but M8 should claw some back, for example by keeping rarely used encoders and the settings code out of memory until needed, before adding more.
- **Settings run.** The forms are applied in timer steps after the first frame, so start-up is unchanged (`startup_ms` 1 to 17 ms). Each form takes one step (a 2D code more, while it is encoded); the About scan reads four headers per step, so 110 entries take 28 steps, at least 1.4 s at 50 ms per step. It runs only when the entry list changed.
- **Package:** 12 MB for all 72 products (9.5 MB in M6).

### M6, 2026-10-20 (barcodes, in 0.6.0)

- **Linear codes** encode in milliseconds while the screen opens; nothing is cached.
- **2D codes are encoded as jobs.** QR works in steps sized for the Forerunner 255S's watchdog: at most 2,500 modules to place, 500 to score as rows, 1,000 as columns, or one Reed-Solomon block. Simulator compute time and steps:

  | QR | Compute | Steps | At 50 ms per step on a watch |
  | --- | --- | --- | --- |
  | Version 3 (29 x 29) | 81 ms | 28 | at least 1.4 s |
  | Version 10 (57 x 57) | 312 ms | 105 | at least 5.3 s |
  | Version 26 (121 x 121) | 1.4 s | 407 | at least 20 s |

- **First-time encoding in the simulated watches** (the UI flow's demo cards, `make screenshots` prints these times):

  | Code | Size | Time |
  | --- | --- | --- |
  | QR (version 6) | 41 x 41 | 2.8 s |
  | Data Matrix | 18 x 18 | 0.2 s |
  | PDF417 | 86 x 29 rows (round watches), 103 x 15 (rectangular) | 0.35 to 0.55 s |
  | Aztec (compact) | 23 x 23 | 0.15 s |

  These times are mostly the 50 ms pauses between steps. QR is the slow one, because of its eight mask evaluations.

  Version 10 is over README 18's 3 s target. Most of the time is the pauses between steps, and mask scoring (eight masks, every module) is most of the steps. The cache makes every later opening instant. M8 can cut the steps, for example by scoring masks on packed rows. The demo card (version 6) encodes in 2.8 s in the simulated watches (`make screenshots` prints it).

### M5, 2026-10-20 (version 0.5.0, import)

- **Compute.** An Aegis export with 100 entries (19 KB, compact) takes 332 ms of compute in the unit-test benchmark, spread over 159 job steps.
- **On the simulated watches.** Through the real path (settings form, timer-driven job, 50 ms pause between steps), the UI-test build imports 100 entries in about 7.6 s on the Forerunner 255S. Most of that is the pauses that keep each event below the watchdog limit. Peak memory during the import is 41.5 % of the 255S's 512 KB: the pasted text is held as a string and as bytes, along with the parsed items. The flow fails above 70 %.
- **Pasted text size.** Aegis exports take about 190 bytes per entry compact, and 400 to 500 pretty-printed. The multi-part buffer is capped at 32 KB, so very large pretty-printed exports must be imported in parts.
- **Watchdog, again.** Even the UI-test fixture code tripped the watchdog on the 255S, by Base32-encoding 96 secrets in one event. The importer itself works in steps of 40 JSON tokens, one URI, four headers or one stored entry.

### M4, 2026-10-19 (version 0.4.0, screens)

- **Start-up.** In-app start-up (object creation to the first complete frame) measured 1 to 18 ms in the simulator; the `startup_ms` budget is now 100 ms. Launch to first frame rose to 16-18 s, because the simulator first loads the glance and the scripts then open the app from it. Its regression guard is now 36 s.
- **Memory and size.** Memory after the first frame is 12 % on the Forerunner 255S and 8 % on the 768 KB watches. The signed package for 72 products is 4.5 MB.
- **Watchdog.** The scripted UI flow showed it is enforced in normal simulator runs, at the device's limit per event (120,000 bytecodes on the Forerunner 255S, 240,000 on the others):
  - **PBKDF2:** 200 byte-wise iterations in one step tripped it on the 255S. Rewritten to XOR 32-bit words, PBKDF2 takes 22 ms per 1,000 iterations in the simulator, down from 52 ms, and a step is now 100 iterations.
  - **SHA-512:** one SHA-512 code per event fits on the 255S; two in one draw don't. The list and the detail screen compute codes in their own timer events, at most one SHA-512 code per event.
- **On the watch.** The debug build's menu has "Debug: benchmark", which times TOTP with SHA-1 and SHA-512, PBKDF2 iterations per second, header reading and the self-test, one per timer event. Those are the numbers for README 18's watch targets; they come from your Forerunner 70 (real-watch checklist).

### M3, 2026-10-19 (version 0.3.0, storage)

`test/BenchTest.mc` with an in-memory store, simulator in Docker, Forerunner 70 profile:

| Operation | Time | Notes |
| --- | --- | --- |
| Read 20 entry headers | 6.8 ms | 0.34 ms per header, mostly TLV decoding in Monkey C |
| Open one entry (verify + decrypt) | 1.05 ms | Device-key entry |
| Add and remove one entry | 1.4 ms | Seal, two metadata writes |

Record sizes: 239 bytes for a typical OTP entry, 208 for a short code, 4,279 for the largest allowed payload.

`make perf` and `make memtest`: in-app start-up 1 to 6 ms, and launch to first frame 10.5 s, unchanged. Memory after the first frame rose to 4.0 % (768 KB watches) and 5.9 % (Forerunner 255S), because the debug program is larger; the app doesn't open the vault yet. The signed package for 72 products is 2.5 MB.

Consequence for M4: reading 150 headers would take about 50 ms in the simulator, and on the watch probably many times that. The list will cache headers and read only the visible rows when it opens, instead of reading all of them at start-up.

### M2, 2026-10-18 (version 0.2.0, crypto and OTP primitives)

`test/BenchTest.mc`, simulator in Docker, average per call (identical within noise on all four test devices):

| Operation | Time | Notes |
| --- | --- | --- |
| TOTP, SHA-1 | 0.35 ms | HMAC built on the native SHA-1 |
| TOTP, SHA-256 | 0.05 ms | Native HMAC-SHA256 |
| TOTP, SHA-512 | 17.5 ms | SHA-512 in Monkey C; was 27.9 ms before inlining the rotations |
| PBKDF2, 1000 iterations | 52 ms | Was 89 ms before removing the per-iteration wipe; native HMAC alone is about 3 ms |
| Seal + open a record | 0.25 ms | AES-256-CBC + HMAC-SHA256, two HKDF key derivations cached |
| Start-up self-test | 6.7 ms | All known-answer tests |
| PBKDF2 calibration for 1 s | about 12,000 iterations | In the simulator; the watch will choose far fewer |

`make perf` and `make memtest` with the larger app: in-app start-up 1 to 7 ms; launch to first frame rose from 6.5 s to 10.5 s, because `monkeydo` now sends a larger debug program (crypto code and debug information), still under its 15 s guard. Memory after the first frame: 2.3 % (768 KB watches) to 3.4 % (Forerunner 255S). The signed package for 72 products is 1.5 MB.

What this means for the watch, to be confirmed with the on-watch benchmark in M4: the simulator runs on a desktop CPU, so absolute times will be many times higher on the Forerunner 70. The ratios hold: SHA-512 entries cost about 50 times as much as SHA-1 entries, and Monkey C loop overhead, not the native HMAC, dominates PBKDF2. If the watch manages fewer than 2,000 PBKDF2 iterations per second, unlocking takes longer than the 1 s target (the lower bound is kept for security); M8 can move more of the XOR work into native calls.

### M0, 2026-10-18 (version 0.1.0, placeholder app)

Simulator in Docker on your PC (12 cores), debug builds:

| Device | Watch-app memory | In-app start-up | Launch to first frame | Memory used |
| --- | --- | --- | --- | --- |
| Forerunner 255S | 512 KB | 1 ms | 6.4 s | 1.3 % |
| Venu Sq 2 | 768 KB | 3 ms | 6.5 s | 0.9 % |
| Forerunner 70 | 768 KB | 0 ms | 6.7 s | 0.9 % |
| Forerunner 965 | 768 KB | 6 ms | 6.8 s | 0.9 % |

The placeholder app draws two lines of text, so these numbers only prove the measuring works. The first meaningful baseline comes with M4 (the real list and code screens). The signed package is 488 KB for all 72 products; a release build for one device is 7 to 15 KB.