From 81a5b903297e59d599dd0d6f580b89597630909d Mon Sep 17 00:00:00 2001 From: cirakg Date: Mon, 19 Oct 2026 22:28:09 +0200 Subject: docs: record M4 results and simulator findings; bump to 0.4.0 Co-Authored-By: Claude Opus 5.5 --- docs/PERFORMANCE.md | 15 ++++++++++++--- 1 file changed, 12 insertions(+), 3 deletions(-) (limited to 'docs/PERFORMANCE.md') diff --git a/docs/PERFORMANCE.md b/docs/PERFORMANCE.md index fbc9709..8c5adb4 100644 --- a/docs/PERFORMANCE.md +++ b/docs/PERFORMANCE.md @@ -11,7 +11,7 @@ The targets are in [README section 18](../README.md#18-performance-budgets). Thi - **In-app start-up** (`startup_ms`): debug builds take `System.getTimer()` when the app object is created and again after the first complete frame, and print the difference (`source/util/Diag.mc`). It doesn't include the time the watch needs to load the program before the app object exists. - **Launch to first frame** (`first_frame_wall_ms`): `docker/scripts/app-diag.sh` measures the wall-clock time from launching `monkeydo` until the app's first-frame line appears. This includes the Java start of `monkeydo`, sending the program to the simulator and loading it, so it is mostly tool overhead; it's only useful to catch large regressions. - **Memory** (`mem_used_fraction`): `System.getSystemStats()` after the first frame: used memory divided by the memory the device gives watch apps. -- **On the watch**: the numbers that matter are measured on your Forerunner 70 with a sideloaded debug build that shows them on screen. That arrives with M4; the simulator runs on your PC's CPU and says nothing about watch speed. +- **On the watch**: the numbers that matter are measured on your Forerunner 70 with a sideloaded debug build: menu, "Debug: benchmark" (and "Debug: overlay" for draw time and memory). The simulator runs on your PC's CPU and says nothing about watch speed. `make perf` and `make memtest` fail when a value exceeds `tools/budgets.json`. @@ -19,12 +19,21 @@ The targets are in [README section 18](../README.md#18-performance-budgets). Thi | Value | Budget | Why | | --- | --- | --- | -| `startup_ms` | 3000 ms | Placeholder until M4 sets about 2× the measured baseline | -| `first_frame_wall_ms` | 15000 ms | Regression guard only (tool overhead dominates) | +| `startup_ms` | 100 ms | M4 measured 1 to 18 ms; the margin absorbs simulator jitter | +| `first_frame_wall_ms` | 36000 ms | Regression guard only: about 2× M4's 16-18 s, which include loading the glance and opening the app from it | | `mem_used_fraction` | 70 % | Hard limit, leaves room for spikes on the 512 KB watches | ## Results +### M4, 2026-10-19 (version 0.4.0, screens) + +- **Start-up.** In-app start-up (object creation to the first complete frame) measured 1 to 18 ms in the simulator; the `startup_ms` budget is now 100 ms. Launch to first frame rose to 16-18 s, because the simulator first loads the glance and the scripts then open the app from it. Its regression guard is now 36 s. +- **Memory and size.** Memory after the first frame is 12 % on the Forerunner 255S and 8 % on the 768 KB watches. The signed package for 72 products is 4.5 MB. +- **Watchdog.** The scripted UI flow showed it is enforced in normal simulator runs, at the device's limit per event (120,000 bytecodes on the Forerunner 255S, 240,000 on the others): + - **PBKDF2:** 200 byte-wise iterations in one step tripped it on the 255S. Rewritten to XOR 32-bit words, PBKDF2 takes 22 ms per 1,000 iterations in the simulator, down from 52 ms, and a step is now 100 iterations. + - **SHA-512:** one SHA-512 code per event fits on the 255S; two in one draw don't. The list and the detail screen compute codes in their own timer events, at most one SHA-512 code per event. +- **On the watch.** The debug build's menu has "Debug: benchmark", which times TOTP with SHA-1 and SHA-512, PBKDF2 iterations per second, header reading and the self-test, one per timer event. Those are the numbers for README 18's watch targets; they come from your Forerunner 70 (real-watch checklist). + ### M3, 2026-10-19 (version 0.3.0, storage) `test/BenchTest.mc` with an in-memory store, simulator in Docker, Forerunner 70 profile: -- cgit v1.2.3