From f439beec17419ccbe0dc7643cc039079f15e04a0 Mon Sep 17 00:00:00 2001 From: cirakg Date: Wed, 21 Oct 2026 03:55:15 +0200 Subject: docs: user guide, known limitations and M8 results; bump to 0.8.0 0.8.0 is the release candidate: 1.0.0 is tagged after the on-watch checklist (docs/TESTING.md). Co-Authored-By: Claude Opus 5.5 --- docs/TESTING.md | 29 ++++++++++++++++++++++++++++- 1 file changed, 28 insertions(+), 1 deletion(-) (limited to 'docs/TESTING.md') diff --git a/docs/TESTING.md b/docs/TESTING.md index b77ab5d..73f89d5 100644 --- a/docs/TESTING.md +++ b/docs/TESTING.md @@ -26,6 +26,25 @@ Test devices (`TEST_DEVICES`): Forerunner 255S (weakest: 218 px round MIP, butto ## Results +### M8, 2026-10-21 (polish and release candidate, version 0.8.0) + +- **Unit tests:** 163 per device on the Forerunner 255S, Venu Sq 2, Forerunner 70 and 965, all passing. New ones: + - **Import:** packed items round-trip every field and wipe the parsed secret (`importItemsPackAndUnpack`). + - **QR:** the packed mask scorer gives every mask the same penalty as the per-module rules, for versions from 1 to 21 and every level (`qrPackedScoresMatchReference`). segno's automatic version and mask choice still matches (`qrMatchesSegno`). + - **Cache and drawing:** the new cache format holds widths over 255; a 0.7 cache is refused (`codeCachePackedFormat`). The drawn runs are exactly the dark modules (`codeRunsCoverExactlyTheDarkModules`). + - **Glance:** + - The sealed item gives the entry's codes (TOTP with SHA-256, Steam) and refuses tampering and other keys. + - The About scan picks the first favourite TOTP or Steam entry without the PIN, and removes the item when the mark or the option goes. + - The favourite actions check the mark and queue for locked entries. +- **Barcode decoding:** 222 encoder dumps per device decode with zxing-cpp at 3 sizes. +- **Screenshots:** 21 screens on 7 watches (18 on the touch-only ones) match their goldens, including the new glance with the favourite's code (768 147 for GitHub at the frozen time, the value Python's `hmac` computes). The glance is a second launch, because the simulator doesn't return to it when the app exits. +- **Watchdog stress (`make watchdog`, new):** + - On the Forerunner 255S and 70, the largest code of every 2D symbology is encoded, packed, turned into runs and drawn with two steps per event, and the last code on the screen must decode. + - It also passed on the Forerunner 965 and the Venu X1, which have the largest screens. The X1 draws a version 39 QR from a bitmap. + - Writing it found watchdog trips in every 2D encoder with large inputs (see PERFORMANCE.md), all fixed. +- **Budgets:** start-up 1 to 17 ms; memory at the first frame 34.4 % (255S) and 22.9 % (768 KB watches); glance 37 %; the 100-entry import peaks at 45.3 % on the 255S. +- **Tools:** 75 Python tests, among them the watchdog check of the product list, the glance memory budget and barcode reading from screenshots. + ### M7, 2026-10-20 (phone settings forms, version 0.7.0) Monkey C unit tests: 154 per device (34 new, in `test/settings/`), all passing on the four test devices. @@ -205,7 +224,7 @@ Tests never use real secrets or personal data. Fixtures are generated or hand-wr ## Real-watch checklist -Things the simulator can't verify, done by you on the Forerunner 70. The checklist grows with each milestone; it starts with M1. +Things the simulator can't verify, done by you on the Forerunner 70, with fake or test accounts only. 1.0.0 is tagged once every box is ticked (or a remaining item is recorded as a known limitation). The items were added milestone by milestone; M8 adds the release checks at the end. - [ ] M1: install the settings spike from the Store beta and run tests A to H in [SETTINGS_SPIKE.md](SETTINGS_SPIKE.md); record the results there. - [ ] M4: sideload a debug build (`make build DEVICE=fr70`, then copy `build/debug/fr70.prg` to `GARMIN/APPS` over USB). Open the menu, choose "Debug: benchmark", and send me the numbers. They set the real PBKDF2 iteration count and the SHA-512 cost on your watch. @@ -224,3 +243,11 @@ Things the simulator can't verify, done by you on the Forerunner 70. The checkli - **Repeats:** send the same Add OTP form twice (turn the switch on again without changing anything). The second time the result is "Already added". - **Options:** change Theme and Code size; the watch changes as soon as the phone saves. - **Limits:** does the About catalog (up to 800 characters) show completely on the phone? +- [ ] M8, with the 0.8.0 Store beta: + - **Update keeps data:** install 0.8.0 over the 0.7 beta (or whichever you have); the entries, the PIN and the order must still be there. Open a 2D code you had opened before: it is re-encoded once (new cache format), then instant again. + - **Glance:** mark a test TOTP entry as favourite (Manage entry, "Mark as favourite (glance)"), turn on "Show the favourite code in the glance" and open the app once. The glance must show the same code as the app and count down. Turn the option off on the phone: the glance must go back to "N entries" without opening the app. Lock the entry ("Require PIN"): it must leave the glance. + - **Brightness:** open a barcode in a dim room. The screen should go to full brightness and stay bright for about a minute; a tap brightens it again. Turn "Full brightness while showing a barcode" off and check that it no longer does. + - **Large code timing:** add a QR with about 300 characters of text (Add code) and note how long the first opening takes ("Encoding…"). On the Forerunner 70 it runs two steps per tick. + - **Benchmark:** menu, "Debug: benchmark" on a sideloaded debug build; send me the numbers (they replace the simulator estimates in PERFORMANCE.md). + - **Uninstall:** remove the app from the watch, install it again: it must start empty (no entries, no PIN). + - **Scanning:** scan the demo cards (Gym card, Member QR, Parcel, Ticket, Rail pass) with your phone, indoors and outdoors; a shop scanner if you can. -- cgit v1.2.3