1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
|
<!--
Author: cirakg & claude ai
Summary: Performance and memory budgets, how they are measured, and the measured numbers per milestone.
-->
# Performance
The targets are in [README section 18](../README.md#18-performance-budgets). This document records how each number is measured, what it does and doesn't tell you, and the results.
## How it is measured
- **In-app start-up** (`startup_ms`): debug builds take `System.getTimer()` when the app object is created and again after the first complete frame, and print the difference (`source/util/Diag.mc`). It doesn't include the time the watch needs to load the program before the app object exists.
- **Launch to first frame** (`first_frame_wall_ms`): `docker/scripts/app-diag.sh` measures the wall-clock time from launching `monkeydo` until the app's first-frame line appears. This includes the Java start of `monkeydo`, sending the program to the simulator and loading it, so it is mostly tool overhead; it's only useful to catch large regressions.
- **Memory** (`mem_used_fraction`): `System.getSystemStats()` after the first frame: used memory divided by the memory the device gives watch apps.
- **On the watch**: the numbers that matter are measured on your Forerunner 70 with a sideloaded debug build: menu, "Debug: benchmark" (and "Debug: overlay" for draw time and memory). The simulator runs on your PC's CPU and says nothing about watch speed.
`make perf` and `make memtest` fail when a value exceeds `tools/budgets.json`.
## Budgets in force
| Value | Budget | Why |
| --- | --- | --- |
| `startup_ms` | 100 ms | M4 measured 1 to 18 ms; the margin absorbs simulator jitter |
| `first_frame_wall_ms` | 36000 ms | Regression guard only: about 2× M4's 16-18 s, which include loading the glance and opening the app from it |
| `mem_used_fraction` | 70 % | Hard limit, leaves room for spikes on the 512 KB watches |
## Results
### M6, 2026-10-20 (barcodes, in 0.6.0)
- **Linear codes** encode in milliseconds while the screen opens; nothing is cached.
- **2D codes are encoded as jobs.** QR works in steps sized for the Forerunner 255S's watchdog: at most 2,500 modules to place, 500 to score as rows, 1,000 as columns, or one Reed-Solomon block. Simulator compute time and steps:
| QR | Compute | Steps | At 50 ms per step on a watch |
| --- | --- | --- | --- |
| Version 3 (29 x 29) | 81 ms | 28 | at least 1.4 s |
| Version 10 (57 x 57) | 312 ms | 105 | at least 5.3 s |
| Version 26 (121 x 121) | 1.4 s | 407 | at least 20 s |
- **First-time encoding in the simulated watches** (the UI flow's demo cards, `make screenshots` prints these times):
| Code | Size | Time |
| --- | --- | --- |
| QR (version 6) | 41 x 41 | 2.8 s |
| Data Matrix | 18 x 18 | 0.2 s |
| PDF417 | 86 x 29 rows (round watches), 103 x 15 (rectangular) | 0.35 to 0.55 s |
| Aztec (compact) | 23 x 23 | 0.15 s |
These times are mostly the 50 ms pauses between steps. QR is the slow one, because of its eight mask evaluations.
Version 10 is over README 18's 3 s target. Most of the time is the pauses between steps, and mask scoring (eight masks, every module) is most of the steps. The cache makes every later opening instant. M8 can cut the steps, for example by scoring masks on packed rows. The demo card (version 6) encodes in 2.8 s in the simulated watches (`make screenshots` prints it).
### M5, 2026-10-20 (version 0.5.0, import)
- **Compute.** An Aegis export with 100 entries (19 KB, compact) takes 332 ms of compute in the unit-test benchmark, spread over 159 job steps.
- **On the simulated watches.** Through the real path (settings form, timer-driven job, 50 ms pause between steps), the UI-test build imports 100 entries in about 7.6 s on the Forerunner 255S. Most of that is the pauses that keep each event below the watchdog limit. Peak memory during the import is 41.5 % of the 255S's 512 KB: the pasted text is held as a string and as bytes, along with the parsed items. The flow fails above 70 %.
- **Pasted text size.** Aegis exports take about 190 bytes per entry compact, and 400 to 500 pretty-printed. The multi-part buffer is capped at 32 KB, so very large pretty-printed exports must be imported in parts.
- **Watchdog, again.** Even the UI-test fixture code tripped the watchdog on the 255S, by Base32-encoding 96 secrets in one event. The importer itself works in steps of 40 JSON tokens, one URI, four headers or one stored entry.
### M4, 2026-10-19 (version 0.4.0, screens)
- **Start-up.** In-app start-up (object creation to the first complete frame) measured 1 to 18 ms in the simulator; the `startup_ms` budget is now 100 ms. Launch to first frame rose to 16-18 s, because the simulator first loads the glance and the scripts then open the app from it. Its regression guard is now 36 s.
- **Memory and size.** Memory after the first frame is 12 % on the Forerunner 255S and 8 % on the 768 KB watches. The signed package for 72 products is 4.5 MB.
- **Watchdog.** The scripted UI flow showed it is enforced in normal simulator runs, at the device's limit per event (120,000 bytecodes on the Forerunner 255S, 240,000 on the others):
- **PBKDF2:** 200 byte-wise iterations in one step tripped it on the 255S. Rewritten to XOR 32-bit words, PBKDF2 takes 22 ms per 1,000 iterations in the simulator, down from 52 ms, and a step is now 100 iterations.
- **SHA-512:** one SHA-512 code per event fits on the 255S; two in one draw don't. The list and the detail screen compute codes in their own timer events, at most one SHA-512 code per event.
- **On the watch.** The debug build's menu has "Debug: benchmark", which times TOTP with SHA-1 and SHA-512, PBKDF2 iterations per second, header reading and the self-test, one per timer event. Those are the numbers for README 18's watch targets; they come from your Forerunner 70 (real-watch checklist).
### M3, 2026-10-19 (version 0.3.0, storage)
`test/BenchTest.mc` with an in-memory store, simulator in Docker, Forerunner 70 profile:
| Operation | Time | Notes |
| --- | --- | --- |
| Read 20 entry headers | 6.8 ms | 0.34 ms per header, mostly TLV decoding in Monkey C |
| Open one entry (verify + decrypt) | 1.05 ms | Device-key entry |
| Add and remove one entry | 1.4 ms | Seal, two metadata writes |
Record sizes: 239 bytes for a typical OTP entry, 208 for a short code, 4,279 for the largest allowed payload.
`make perf` and `make memtest`: in-app start-up 1 to 6 ms, and launch to first frame 10.5 s, unchanged. Memory after the first frame rose to 4.0 % (768 KB watches) and 5.9 % (Forerunner 255S), because the debug program is larger; the app doesn't open the vault yet. The signed package for 72 products is 2.5 MB.
Consequence for M4: reading 150 headers would take about 50 ms in the simulator, and on the watch probably many times that. The list will cache headers and read only the visible rows when it opens, instead of reading all of them at start-up.
### M2, 2026-10-18 (version 0.2.0, crypto and OTP primitives)
`test/BenchTest.mc`, simulator in Docker, average per call (identical within noise on all four test devices):
| Operation | Time | Notes |
| --- | --- | --- |
| TOTP, SHA-1 | 0.35 ms | HMAC built on the native SHA-1 |
| TOTP, SHA-256 | 0.05 ms | Native HMAC-SHA256 |
| TOTP, SHA-512 | 17.5 ms | SHA-512 in Monkey C; was 27.9 ms before inlining the rotations |
| PBKDF2, 1000 iterations | 52 ms | Was 89 ms before removing the per-iteration wipe; native HMAC alone is about 3 ms |
| Seal + open a record | 0.25 ms | AES-256-CBC + HMAC-SHA256, two HKDF key derivations cached |
| Start-up self-test | 6.7 ms | All known-answer tests |
| PBKDF2 calibration for 1 s | about 12,000 iterations | In the simulator; the watch will choose far fewer |
`make perf` and `make memtest` with the larger app: in-app start-up 1 to 7 ms; launch to first frame rose from 6.5 s to 10.5 s, because `monkeydo` now sends a larger debug program (crypto code and debug information), still under its 15 s guard. Memory after the first frame: 2.3 % (768 KB watches) to 3.4 % (Forerunner 255S). The signed package for 72 products is 1.5 MB.
What this means for the watch, to be confirmed with the on-watch benchmark in M4: the simulator runs on a desktop CPU, so absolute times will be many times higher on the Forerunner 70. The ratios hold: SHA-512 entries cost about 50 times as much as SHA-1 entries, and Monkey C loop overhead, not the native HMAC, dominates PBKDF2. If the watch manages fewer than 2,000 PBKDF2 iterations per second, unlocking takes longer than the 1 s target (the lower bound is kept for security); M8 can move more of the XOR work into native calls.
### M0, 2026-10-18 (version 0.1.0, placeholder app)
Simulator in Docker on your PC (12 cores), debug builds:
| Device | Watch-app memory | In-app start-up | Launch to first frame | Memory used |
| --- | --- | --- | --- | --- |
| Forerunner 255S | 512 KB | 1 ms | 6.4 s | 1.3 % |
| Venu Sq 2 | 768 KB | 3 ms | 6.5 s | 0.9 % |
| Forerunner 70 | 768 KB | 0 ms | 6.7 s | 0.9 % |
| Forerunner 965 | 768 KB | 6 ms | 6.8 s | 0.9 % |
The placeholder app draws two lines of text, so these numbers only prove the measuring works. The first meaningful baseline comes with M4 (the real list and code screens). The signed package is 488 KB for all 72 products; a release build for one device is 7 to 15 KB.
|
