aboutsummaryrefslogtreecommitdiff
path: root/docs/TESTING.md
diff options
context:
space:
mode:
authorcirakg <cirakg@cirakg.xyz>2026-10-20 01:51:05 +0200
committercirakg <cirakg@cirakg.xyz>2026-10-20 01:51:05 +0200
commit0edb4bb6d76939aa456cbbc8deafeed50fd6182d (patch)
tree3f22349f6d64ff0fa33b6ff9bbcc10584f8b152c /docs/TESTING.md
parent42f9331e693f5058fcfcd7aef5a92cabc04c20ee (diff)
docs: record M5 results; bump to 0.5.0v0.5.0
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Diffstat (limited to 'docs/TESTING.md')
-rw-r--r--docs/TESTING.md34
1 files changed, 34 insertions, 0 deletions
diff --git a/docs/TESTING.md b/docs/TESTING.md
index 54a331e..593b120 100644
--- a/docs/TESTING.md
+++ b/docs/TESTING.md
@@ -26,6 +26,39 @@ Test devices (`TEST_DEVICES`): Forerunner 255S (weakest: 218 px round MIP, butto
## Results
+### M5, 2026-10-20 (version 0.5.0)
+
+Monkey C unit tests: 101 per device (29 new in `test/import/`):
+
+- **JSON tokenizer:** every token kind; escapes including surrogate pairs; integers versus other numbers; 16 kinds of invalid document; incomplete input (8 cases); the depth limit; long strings (icons, including escaped quotes) skipped unread; long keys refused.
+- **`otpauth://`:** URIs pasted run together; percent-encoded labels; a missing issuer; an issuer mismatch; `;` separators; `+` as a space; unknown parameters; HOTP with and without a counter; both Steam forms; nine error cases, each with its message.
+- **Aegis:**
+ - Database versions 1, 2 and 3, with groups listed after the entries.
+ - totp, hotp and steam imported; motp and yandex skipped.
+ - SHA-1, SHA-256 and SHA-512; 6, 7 and 8 digits; periods 20, 30, 50 and 60.
+ - Favourites, icons, notes and Unicode names.
+ - Encrypted exports refused, in both forms.
+ - Bad entries (9 digits, a duplicated `secret` field, invalid Base32, a non-integer digit count), and non-Aegis, newer-version and too-deeply-nested documents.
+ - An incomplete export.
+- **Importer:**
+ - The report text.
+ - Duplicates of an earlier import, of a locked entry while the watch is locked, and within one import.
+ - Possible replacements; "Require PIN" entries going pending.
+ - Replace mode keeping manual entries.
+ - A two-part JSON import, sealed while it waits, and the expiry of a waiting part.
+ - Unusable text: empty, unknown, Google Authenticator links, encrypted, invalid.
+ - A store that fills up mid-import, keeping only whole entries.
+ - Fingerprints.
+ - 100 entries.
+- **Fuzz corpus** (`tools/gen_fuzz_corpus.py`, seed 4242, `make corpus`): 50 mutated Aegis documents and 50 mutated URIs.
+ - The tokenizer always finishes.
+ - The Aegis reader always ends in a definite state, with every item either valid or explained.
+ - The full importer never stores an invalid entry: 21 entries were stored from the inputs that survived mutation.
+
+Python tool tests: 55. The new ones cover the corpus generator, including that the committed corpus matches it.
+
+Scripted UI flow: the UI-test build now puts an Aegis export of 100 entries into the Import form, as the phone would, and the flow starts with the results screen that opens after the import. 11 screens per watch (10 on the touch-only ones). Every watch reports the import's duration and peak memory, and the flow fails above 70 % of the watch-app memory.
+
### M4, 2026-10-19 (version 0.4.0)
`make ci` passed in full.
@@ -102,3 +135,4 @@ Things the simulator can't verify, done by you on the Forerunner 70. The checkli
- [ ] M4: sideload a debug build (`make build DEVICE=fr70`, then copy `build/debug/fr70.prg` to `GARMIN/APPS` over USB). Open the menu, choose "Debug: benchmark", and send me the numbers. They set the real PBKDF2 iteration count and the SHA-512 cost on your watch.
- [ ] M4: with "Debug: demo entries", check the list, a code, the countdown ring, HOTP "next", and the PIN keypad with your finger. Also check a wrong PIN (vibration) and the 30 s wait after five wrong PINs.
- [ ] M4: check that leaving the app and coming back asks for the PIN again for the locked demo entry. Check that the glance in the glance loop shows the number of entries.
+- [ ] M5: in Garmin Connect, paste a few `otpauth://` links of test accounts (not your real ones) into the Import fields, turn "Import now" on and save. Check the result on the watch and in the Result field. Also check that the text fields are emptied afterwards: this answers part of the M1 question.