diff options
Diffstat (limited to 'docs/TESTING.md')
| -rw-r--r-- | docs/TESTING.md | 18 |
1 files changed, 9 insertions, 9 deletions
diff --git a/docs/TESTING.md b/docs/TESTING.md index 14cd173..38bc112 100644 --- a/docs/TESTING.md +++ b/docs/TESTING.md @@ -4,7 +4,7 @@ --> # Testing -The full test plan is in [README section 17](../README.md#17-test-plan). This document tracks what exists, how to run it, and the latest results. Every check runs in Docker; `make ci` runs them all in order and stops at the first failure. +The full test plan is in [PROJECT_PLAN section 17](PROJECT_PLAN.md#17-test-plan). This document tracks what exists, how to run it, and the latest results. Every check runs in Docker; `make ci` runs them all in order and stops at the first failure. ## Test layers @@ -56,7 +56,7 @@ The UI flow now has two scenarios, 42 to 44 screens per watch: What the new flow found and what was fixed: - "Entry IDs" with 110 entries tripped the watchdog (all headers read in one event); it now reads 10 per step and shows "Reading…". The debug benchmark times at most 20 headers. -- Auto-lock didn't close a locked code card (README 7.5); it does now. +- Auto-lock didn't close a locked code card (PROJECT_PLAN 7.5); it does now. - The list said "Confirm in menu" for a queued Require PIN that applies by itself; locks are no longer counted there. - A locked code card showed the barcode icon instead of the padlock. - Without the OK key the keypad's bottom row looked like a missing key; it is now empty, 0, delete, as on a phone. @@ -92,8 +92,8 @@ Verification: `make screenshots TEST_DEVICES=fr70 SPOT_DEVICES=` passes (7 min 5 Monkey C unit tests: 154 per device (34 new, in `test/settings/`), all passing on the four test devices. -- **Add OTP:** every type (TOTP, HOTP with its counter, Steam ignoring algorithm, digits and period), and each validation message of README 10.4: Base32 character and position, `=` padding, missing secret, digits, period, algorithm, type, counter, control characters, length in bytes, a missing name, the warning below 80 bits. After a problem only the secret is cleared; after success the whole form, with the switch off. -- **Add code:** a valid payload for each of the 12 symbologies (2D codes come out cached and decodable) and an invalid one for each, with the encoder's message (check digit, length, character and position, capacity). Options and "Fix if wrong" are stored as sent, and codes too large for the screen are added with the README 12.2 warning. +- **Add OTP:** every type (TOTP, HOTP with its counter, Steam ignoring algorithm, digits and period), and each validation message of PROJECT_PLAN 10.4: Base32 character and position, `=` padding, missing secret, digits, period, algorithm, type, counter, control characters, length in bytes, a missing name, the warning below 80 bits. After a problem only the secret is cleared; after success the whole form, with the switch off. +- **Add code:** a valid payload for each of the 12 symbologies (2D codes come out cached and decodable) and an invalid one for each, with the encoder's message (check digit, length, character and position, capacity). Options and "Fix if wrong" are stored as sent, and codes too large for the screen are added with the PROJECT_PLAN 12.2 warning. - **Manage entry:** every action on unlocked entries; locked entries' changes queued even while unlocked, then applied by confirmation; "Require PIN" applied at once or queued; unknown, malformed, pending and damaged IDs; a queued change whose entry vanished; changes arriving while the queue is applied; tampered queue items dropped; the 16-item limit. - **Idempotency:** the same Add OTP, Manage and Import form twice is applied once; any change makes a new request. - **The whole run:** all forms at once in README order; the About catalog, its 800-character cut and the storage line; "Encrypt names" on and off, with locked entries converted after the PIN; work detected only when needed. @@ -111,11 +111,11 @@ The flow caught one watchdog trip on the Forerunner 255S that the unit tests can Monkey C unit tests: 120 per device. -1. **Reference comparison.** 24 Aztec symbols are identical to aztec_code_generator: compact (15 x 15 up) and full range, including sizes with reference grids, and error correction of 10, 23, 36 and 50 %. The payloads are of one character class each, for which both encoders produce the same bits; README 12.4 explains why mixed content can't be compared exactly. +1. **Reference comparison.** 24 Aztec symbols are identical to aztec_code_generator: compact (15 x 15 up) and full range, including sizes with reference grids, and error correction of 10, 23, 36 and 50 %. The payloads are of one character class each, for which both encoders produce the same bits; PROJECT_PLAN 12.4 explains why mixed content can't be compared exactly. 2. **Decoding.** 8 Aztec codes are decoded with zxing-cpp: mixed case with digits and punctuation (Binary Shift and mode latches), UTF-8, a URL, and a long payload. 3. **Screenshots.** The UI flow opens an Aztec card after the PDF417 card, and its screenshot decodes on all seven watches. -With that, all 12 symbologies of README 12.1 pass checks 1 to 3. The UI flow shows 16 screens per watch (15 on touch-only watches), five of them barcodes that must decode. +With that, all 12 symbologies of PROJECT_PLAN 12.1 pass checks 1 to 3. The UI flow shows 16 screens per watch (15 on touch-only watches), five of them barcodes that must decode. ### M6c, 2026-10-20 (Data Matrix and PDF417, in 0.6.0) @@ -131,7 +131,7 @@ Monkey C unit tests: 117 per device. Found and fixed along the way: -- **ppf padding bug:** see README 12.4. +- **ppf padding bug:** see PROJECT_PLAN 12.4. - **Compiler out of memory:** one long `&&` chain of `instanceof` tests in a test made the Monkey C compiler run out of memory (more than 6 GB), and splitting the chain fixed it. - **Watchdog on the 255S:** packing an 86 x 87 PDF417 matrix for the cache in the same event as the encoder's last step tripped the watchdog. PDF417 rows are now stored once with a row height of 3, packing does 8 modules per loop, and the cache is saved in its own timer event. @@ -141,7 +141,7 @@ Monkey C unit tests: 111 per device. 1. **Reference comparison with segno** (`make codes`). - **31 matrices identical module for module.** They cover versions 1 to 40 and levels L, M, Q and H, numeric, alphanumeric and UTF-8 byte data, forced versions and masks, and automatic choices. The watch picks the same version and mask as segno with `boost_error=False`. - - **Two deliberate differences**, listed in README 12.4: + - **Two deliberate differences**, listed in PROJECT_PLAN 12.4: - segno pads a stream that already ends on a codeword boundary with an extra zero byte; the watch follows ISO 18004. - segno uses a single mode per symbol; the watch segments optimally. 2. **Decoding.** 30 QR codes from the test run are decoded with zxing-cpp at 2 and 3 px per module. They include mixed segmented content, Unicode, emoji, a Wi-Fi string and all four levels. @@ -156,7 +156,7 @@ What the UI flow found: scoring the eight masks tripped the watchdog, first on t ### M6a, 2026-10-20 (linear barcodes, in 0.6.0) -Monkey C unit tests: 105 per device, with the checks of README 12.4. +Monkey C unit tests: 105 per device, with the checks of PROJECT_PLAN 12.4. 1. **Reference comparison.** 217 rows from zxing-cpp (`tools/gen_code_vectors.py`, `make codes`): fixed edge cases and seeded random payloads for every linear symbology. EAN-13, EAN-8, UPC-A, UPC-E, Code 39 (with and without the check character), ITF and Codabar match module for module. Code 128 matches in length. zxing-cpp 3's zint backend picks a different one of the equally short encodings; on 5,000 random strings the lengths were always equal. 2. **Decoding.** Every Code 128 reference row, plus 139 codes for seeded random payloads in all eight symbologies, is dumped by the test run. `tools/check_codes.py` renders each with its quiet zone at 1, 2 and 3 px per module and decodes it with zxing-cpp, on each test device: 169 codes, all decoded. |
