Development

docker/routlin-dash/GUIDE.md

@@ -21,13 +21,13 @@ app/                  Python source (baked into image)
       action.py       Flask Blueprint for POST actions on this page
       page.js?        Optional page-specific client-side JS (auto-included if present)
   sanitize.py         Input sanitization (strips dangerous characters)
-  config_utils.py     Config I/O, snapshot system, command queue
+  config_utils.py     Config I/O, change history (SQLite), command queue
   authorized_accounts.json  Web UI user accounts (separate from Routlin users)
 
-data/                 Live-mounted at runtime (./data:/data)
-  styles.css          Application stylesheet
-  common.js           Shared client-side interaction logic (all pages)
-  validation.js       Client-side field validation
+www/                  Live-mounted at runtime (./www:/www)
+  styles.css          Application stylesheet (inlined in dev, served as /www/styles.css in production)
+  common.js           Shared client-side JS (inlined in dev, served as /www/common.js in production)
+  icons/              SVG icon files (always inlined by factory.py)
 
                       # host directory mounted into container: $HOME/routlin -> /routlin_location
 routlin_location/    Routlin install dir
@@ -36,7 +36,7 @@ routlin_location/    Routlin install dir
   core.py             Apply script invoked to push config changes to the system
   ddns.log            DDNS update log
   blocklists/         Downloaded blocklist files (*.con)
-  .snapshots/         Config snapshot JSON files (one per saved change)
+  .dashboard-snapshots  SQLite database storing change history and revert records
   .health             Health status JSON written by Routlin
   .dashboard-queue    Pending commands waiting to be applied
   .dashboard-done     Record of completed commands
@@ -63,7 +63,7 @@ Files and directories under `/routlin_location` that the app reads or writes:
 | `core.py` | exec | Routlin's apply script. The app invokes it as `python3 core.py --apply` or `--update-blocklists` to push config changes to the system. |
 | `ddns.log` | read | DDNS update log shown in the DDNS page. |
 | `blocklists/` | read | Directory of downloaded blocklist files (`*.con`). The app reads them to count entries and report last-updated timestamps. |
-| `.snapshots/` | read/write | Directory of config snapshot JSON files. One file per saved change, used for the change history and revert feature. |
+| `.dashboard-snapshots` | read/write | SQLite database storing change history groups and field-level diffs, used for the change history and revert feature. |
 | `.health` | read | JSON file written by Routlin describing service and configuration health status. The dashboard reads it to display the health panel and auto-queue fixes. |
 | `.dashboard-queue` | read/write | Pending commands waiting to be applied. |
 | `.dashboard-done` | read/write | Record of completed commands. |
@@ -185,7 +185,7 @@ See `TYPES.md` for the full set of item types and their fields. See `TYPES.md#ac
 from pathlib import Path
 from flask import Blueprint, request, redirect, flash
 from auth import require_level
-from config_utils import load_config, save_config_with_snapshot, verify_config_hash
+from config_utils import load_config, record_group, diff_fields, verify_config_hash
 import sanitize
 import validation as validate
 
@@ -255,13 +255,9 @@ def addip_add():
             flash(msg, 'error')
         return redirect(f'/{_PAGE}')
 
-    # 4. Save with a snapshot entry.
-    flash(save_config_with_snapshot(
-        cfg,
-        path='banned_ips', key=ip, operation='add',
-        before=None, after=entry,
-        description=f'Added banned IP: {ip}',
-    ), 'success')
+    # 4. Record, save, and queue.
+    changes = diff_fields({}, entry)
+    flash(record_group(cfg, 'banned_ips', 'ip', ip, changes, 'core apply'), 'success')
     return redirect(f'/{_PAGE}')
 ```
 
@@ -270,7 +266,7 @@ Rules:
 - `flash()` is the only feedback mechanism. Use `'error'` or `'success'` as the category.
 - Parse all user input before checking the config hash. If input is invalid you want to bail out cheaply before acquiring anything.
 - Always call `validate.validate_config(cfg)` on the mutated config before saving, even for simple operations.
-- `save_config_with_snapshot` returns a human-readable success message string suitable for passing directly to `flash()`.
+- `record_group(cfg, parent_path, item_key, item_value, changes, cmd)` saves the config, records the change in the SQLite history, queues the command, and returns a human-readable flash message string. Use `diff_fields(before, after)` to produce the `changes` argument.
 
 ### Row index actions
 
@@ -310,10 +306,11 @@ ip = validate.banned_ip(raw)                                       # returns Non
 
 ### Checkbox fields
 
-Checkboxes are absent from the form body when unchecked. Always compare to `'on'`:
+Checkboxes are absent from the form body when unchecked. Both forms are equivalent and acceptable:
 
 ```python
-enabled = request.form.get('enabled') == 'on'
+enabled  = request.form.get('enabled') == 'on'   # explicit value comparison
+logging  = 'logging' in request.form              # presence check
 ```
 
 ### JSON fields from record_editor
@@ -358,7 +355,7 @@ Valid levels: `nothing`, `viewer`, `administrator`, `manager`. The decorator enf
 
 `load_config()` reads `config.json` fresh on every call. Do not cache it across a request.
 
-`save_config_with_snapshot(cfg, path, key, operation, before, after, description)` writes the config and records a snapshot entry for the change history. Always pass `before` and `after` as the specific sub-object that changed, not the whole config.
+`record_group(cfg, parent_path, item_key, item_value, changes, cmd)` saves the config, records a change group in the SQLite history database, and queues the apply command. `diff_fields(before, after)` computes the field-level diff to pass as `changes`. Always take a `copy.deepcopy` of the relevant sub-object before mutating so `before` is accurate.
 
 `verify_config_hash(hash)` confirms the config has not changed since the form was rendered. Always check it before mutating the config.
 
@@ -366,15 +363,19 @@ Valid levels: `nothing`, `viewer`, `administrator`, `manager`. The decorator enf
 
 ## Client-Side Validation
 
-Attach a `validate` field to any `field` item in `content.json` to enable live validation:
+Attach a `validate` field to any `field` item in `content.json` to enable live validation. The value is one or more `VALIDATION_*` flag names joined by `|`:
 
 ```json
-{ "type": "field", "name": "subnet", "validate": "ipv4" }
+{ "type": "field", "name": "nat_ip", "validate": "VALIDATION_IPV4_FORMAT|VALIDATION_UNRESTRICTED" }
 ```
 
-See `TYPES.md#validation` for the full table of validate values and what each accepts.
+Factory converts the flag names to a bitmask integer stored in `data-validate`. The client-side `bigValidate()` function reads this mask and runs only the enabled checks on every keystroke, marking the field valid (green), incomplete (yellow), or invalid (red). The form submit button stays disabled until all validated fields pass.
 
-For `record_editor` fields that need subnet-aware validation, use `valtype` instead of `validate` and wire `data-dep-subnet` / `data-dep-mask` attributes via the `attrs` field.
+See `TYPES.md#validation` for the full flag table.
+
+`VALIDATION_UNRESTRICTED` is special: when present, factory automatically reads the current restricted VLAN subnets from config and injects them into `data-existing-ids` on the input. No extra token is needed.
+
+For `record_editor` fields, use `validate` (or the legacy alias `valtype`) on each field definition. The same `VALIDATION_*` flags apply.
 
 For data that must be passed to a JS validator (such as existing VLAN IDs for uniqueness checking), use a `data-*` HTML attribute on the input rather than a global JS variable. The `existing_ids` field on a `field` item emits `data-existing-ids` on the rendered input.
 
@@ -391,7 +392,9 @@ app/pages/actions/page.js            -- history revert button state
 app/pages/networklayout/page.js      -- is_vpn -> mdns_reflection sync
 ```
 
-`page.js` runs after `common.js` and `validation.js`, so all shared utilities (`htmlEsc`, `showCard`, `tablePickerCloseAll`, etc.) are available.
+`page.js` runs after `common.js`, so all shared utilities (`htmlEsc`, `showCard`, `tablePickerCloseAll`, etc.) are available.
+
+In production mode (`PRODUCTION_MODE=1` env var), `common.js` and `styles.css` are served as separate files from `/www/` with `Cache-Control: public, max-age=86400` and an mtime-based `?v=` query string for cache busting. `page.js` and the generated `bigValidate()` function are always inlined regardless of mode.
 
 Rules for what belongs in `page.js` vs `common.js`: