121 lines
7.0 KiB
Markdown
121 lines
7.0 KiB
Markdown
---
|
|
name: browser-test
|
|
description: >
|
|
Test and screenshot HTML/JS web apps in a headless browser (Puppeteer): catch
|
|
JavaScript exceptions, console errors and 404s, drive the UI (click, shift-click,
|
|
type, set inputs) and take screenshots to check visually. Use whenever a task
|
|
produces an HTML page, canvas/WebGL app or any browser UI that must be verified.
|
|
---
|
|
|
|
# Headless browser testing
|
|
|
|
Use the ready-made harness [shot.js](./shot.js). Do NOT write your own Puppeteer
|
|
launcher and do NOT start an HTTP server: the page is opened via `file://`,
|
|
CDN imports work from there.
|
|
|
|
## Setup (once per project)
|
|
```
|
|
cp <skill-dir>/shot.js . && (npm ls puppeteer > /dev/null 2>&1 || npm install puppeteer > npm.log 2>&1) && echo ok
|
|
```
|
|
`<skill-dir>` is the absolute skill directory shown at the top of this text.
|
|
The skill folder is read-only - you work with the copy `./shot.js`, and if the
|
|
harness lacks something, extend THAT copy with edit_file. `npm install puppeteer`
|
|
downloads a matching Chrome (~150 MB, 1-2 min). With an existing Chrome:
|
|
`CHROME_PATH=/path/to/chrome` plus `npm install puppeteer-core`.
|
|
|
|
## Run
|
|
```
|
|
node shot.js app.html # load, record errors, shots/01_initial.png
|
|
node shot.js app.html actions.js # + drive the UI
|
|
node shot.js --check # environment probe: which Chrome flags give WebGL
|
|
```
|
|
Always call it with `run_command(..., timeout=120)`: the harness has its own
|
|
watchdog (60 s, `SHOT_TIMEOUT=90 node shot.js ...` to raise it) and must be
|
|
allowed to report before the outer timeout kills it.
|
|
|
|
Exit codes: 1 = the page threw / `console.error` / failed resource / actions.js
|
|
threw (error list at the end); 2 = setup problem (puppeteer missing, no WebGL
|
|
with any flag set); 3 = watchdog - the output ends with `WATCHDOG ... stuck in
|
|
step: <helper + args>`, i.e. the last traced step is the one that hangs.
|
|
Every helper call is traced as `» name args`, so partial output always tells
|
|
you how far the test got. Screenshots go to `./shots/NN_name.png`; look at
|
|
them with `view_image`.
|
|
|
|
## Environment problems are not app bugs
|
|
- `No WebGL context with any flag set`, `WebGL context could not be created`,
|
|
a watchdog in step `goto`, or `Navigation timeout`: the headless browser or
|
|
the network is the problem. Run `node shot.js --check`, try
|
|
`CHROME_PATH=/Applications/Google Chrome.app/Contents/MacOS/Google Chrome`
|
|
(macOS) or `npm i puppeteer@latest`. NEVER restructure the app to run without
|
|
WebGL, wrap `renderer` in `if (renderer)`, or add "GL not supported" fallbacks
|
|
- that degrades the product the user asked for while the test setup is the
|
|
actual culprit. If the environment cannot be fixed, say so in the summary and
|
|
verify with `check_html.py` plus a hook-level test of the non-3D logic.
|
|
- A watchdog inside `eval`/`download`/`upload`: the app function called never
|
|
returns (infinite loop, awaiting a promise that never resolves, a download
|
|
that never starts). Read that app function; don't patch shot.js.
|
|
- Don't edit `shot.js` to work around a failing test. If a helper is really
|
|
missing, add it to the project copy once, with a clear name, and keep using
|
|
the traced helpers (`h.page` gives you the full Puppeteer API anyway).
|
|
|
|
## actions.js
|
|
```js
|
|
module.exports = async (h) => {
|
|
await h.clickEl('#add-box'); // click a button
|
|
await h.clickEl('#add-sphere', { shift: true }); // shift-click an element
|
|
await h.set('#pos-x', 12); // set an input, fires input+change
|
|
await h.click(640, 400, { shift: true }); // (shift-)click at page coordinates
|
|
await h.key('Delete');
|
|
const n = await h.eval(() => window.__app.state.objects.length); // read app state
|
|
if (n !== 1) throw new Error(`expected 1 object, got ${n}`);
|
|
await h.upload('#btn-load-svg', 'fixtures/circle.svg'); // REAL import: button -> file dialog -> app code
|
|
const f = await h.download('#btn-save-json'); // REAL export: button -> <a download> -> file
|
|
if (!f.text.includes('"position"')) throw new Error('saved file lacks position: ' + f.text);
|
|
await h.shot('final');
|
|
};
|
|
```
|
|
Helpers: `shot`, `click`, `clickEl`, `set`, `key`, `upload`, `download`, `eval`,
|
|
`wait`, `page` (full Puppeteer API), `errors`. Complete example:
|
|
[actions-example.js](./actions-example.js).
|
|
|
|
## How to test an app without going in circles
|
|
1. **First run without actions.** Fix every reported error before testing features.
|
|
2. **Drive state through the test hook, not through pixels.** The app exposes
|
|
`window.__app = { state, add, select, toJSON, ... }` (see the single-file-webapp
|
|
skill). To test "group a solid with a hole", do
|
|
`h.eval(() => { __app.add('box'); __app.add('cylinder'); __app.select([1, 2]); })`,
|
|
then click the Group button and assert on `__app.state`. Canvas picking is
|
|
tested ONCE, separately: add one object at a known position, click its
|
|
projected screen position (`__app.project(id)` or compute it), assert selection.
|
|
3. **Never change the app to make a test pass** (e.g. "buttons now select
|
|
existing objects") - change the test, or fix a real bug in the app.
|
|
4. **Two failures with the same message = stop editing the test.** Print the
|
|
relevant state in the error (`JSON.stringify(await h.eval(...))`), then read
|
|
the app function involved (`search_files` + `read_file` with a line range).
|
|
5. One actions file per feature area (`test_shapes.js`, `test_group.js`,
|
|
`test_files.js`), each under ~40 lines; short, deterministic, throws on
|
|
unexpected state. One 200-line file that runs everything means every fix
|
|
re-runs everything and one hang hides all other results. Add a `shot()`
|
|
after every visible change. Keep `h.eval` results small (ids, counts,
|
|
plain numbers) - returning meshes or whole state objects serializes
|
|
megabytes and can stall the protocol.
|
|
6. WebGL apps: check the screenshot for a rendered object, not just a dark canvas.
|
|
7. **Test the path the user takes, not a stand-in for it.** A feature wired to a
|
|
button, file dialog or download is verified ONLY by `h.clickEl`, `h.upload`
|
|
or `h.download` on that control. Calling the underlying function via
|
|
`__app`, or re-implementing the import inside the test with the library
|
|
exposed on `window`, proves nothing about the button - that is exactly how
|
|
"verified" features turn out broken for the user. Use `__app` for setup and
|
|
assertions, the real control for the action under test. Write fixture files
|
|
(`fixtures/box.stl`, `fixtures/circle.svg`, a saved `scene.json`) once and
|
|
keep them.
|
|
8. **Never delete the test setup.** `shot.js`, `test_*.js`, `fixtures/`,
|
|
`node_modules/` and `package.json` stay in the project so the user can rerun
|
|
`node shot.js app.html test_x.js` after you are done. Add `shots/` and
|
|
`node_modules/` to `.gitignore` instead of removing them.
|
|
9. Change an existing test with `edit_file`; rewriting a 5 KB test with
|
|
`write_file` for a two-line change is slow and error-prone.
|
|
10. The final summary names each feature with the test file and the kind of
|
|
check (real control / hook only / screenshot only). "Hook only" is not
|
|
"verified".
|