Appearance
07 · Template, print, email, SMS, export config, field catalog, script runner
As-built reference for the TMS v2 phase-1 template and export part. A bare src/...:line is a path in bigc-tms-core-v2 (worktree wt-v2-prototype-core). A web:src/...:line is a path in bigc-tms-store-v2 (worktree wt-v2-prototype-store). Every core route is served under /api/v1/... (src/main.ts:32-37).
Siblings, not repeated here: 02-auth.md (guard, permission keys, store scope — on feature/auth, not merged at this sha), 03-logs-and-settings.md (the script and storage settings records, settings API, cache), 04-jobs-engine.md (the job runner an export run would use).
1. Summary
Admins keep templates (print, email, SMS) in Mongo. Each template is bound to one readable entity, holds an optional read-only data script and a Liquid body. POST /templates/render previews a saved or unsaved template against sample JSON or up to 50 real records. The data script runs in an isolated-vm isolate. It gets data, helpers and a db with three reads (find, get, count), and it has a budget: calls, documents, wall time and heap. Script reads honour the read API's @HideFields/@MaskFields, including inside filters and sorts. Print output is one HTML document with an @page rule, which the browser prints. Email returns subject + body. SMS returns text plus a GSM-7/UCS-2 segment count. Nothing is sent. The field catalog flattens the mongoose schema of every registered entity into a field list for the editors. Export configs (columns, an optional per-batch script) can be saved and validated, and a pure batch mapper exists. Running an export (job, xlsx file, storage, download) is not built.
| repo | branch | sha | commits of this part | folders |
|---|---|---|---|---|
| bigc-tms-core-v2 | origin/main-v2-prototype | 3d70475a | 65bdf22c feature · 5c467f17 merge · 1bfe8846 Liquid file-system lockdown, hidden-field filter guard, .dockerignore · 221052bd $jsonSchema/$text refused at every depth · 38dad930 spec touch · 3dd1c5a0 entity registry, limits moved to settings, /script_settings removed | src/template/, src/template_engine/, src/script_runner/, src/field_catalog/, src/export_config/, the onModuleInit registration in 6 domain modules |
| bigc-tms-store-v2 | main-v2-prototype | 0851a3a | b923247 | src/pages/setting/{templates,export_configs,script_settings}/, src/pages/api/{templates,export_configs,export_fields,script_settings}/, src/apis/{client,server}/{template,export_config,script_setting}/, src/services/{template,export_config,script_setting}/, src/components/{template,export_config,forms/CodeEditor}/ |
Template channels
| channel | enum | required extra fields | render output | page settings | send |
|---|---|---|---|---|---|
TemplateChannel.print (src/template/template.schema.ts:5) | none | html: one full HTML document; one <section class="tms-sheet"> per record, break-before: page from the 2nd one (src/template/template.render.service.ts:86-87, 121-127) | page.paper A4/A3/A5/Letter, orientation portrait/landscape, margin_mm ≥ 0, font_family, font_url → @page + @font-face (src/template_engine/template_engine.ts:92-101) | browser window.print() on the preview iframe (web:src/pages/setting/templates/[id].tsx:85-87); no server PDF | |
TemplateChannel.email (:6) | subject non-blank (src/template/template.service.ts:64-66) | subject, body (HTML) of the first record only (template.render.service.ts:88-90) | ignored | not built | |
| sms | TemplateChannel.sms (:7) | none | text of the first record, sms: { encoding, chars, segments } (template.render.service.ts:91-93, template_engine.ts:106-118) | ignored | not built |
There is no "template type" (such as task_print) with a code-declared base-data contract. A template names an entity from the script-entity registry, and a render with record_ids hands the script { record } loaded from that entity (template.render.service.ts:54-59, 112). The type contract is not built.
Registered entities (script reads, field catalog, template entity, export entity)
The registry is the one list for all four uses (src/script_runner/script_entities.registry.ts:11-33). Each domain module registers itself in onModuleInit. The field catalog is built from it in onApplicationBootstrap.
entity (name) | catalog label | registered at | read controller (hide / mask source) | hidden fields | masked fields | ref: fields in its schema |
|---|---|---|---|---|---|---|
delivery_order | Delivery Order | src/delivery_order/delivery_order.module.ts:44-45 | DeliveryOrderController | none | none | source_store_id → store (src/delivery_order/delivery_order.schema.ts:26), current_task_id → task (unregistered, :59) |
delivery_order_item | Delivery Order Item | src/delivery_order_item/delivery_order_item.module.ts:35-36 | DeliveryOrderItemController | none | none | delivery_order_id → delivery_order (src/delivery_order_item/delivery_order_item.schema.ts:16) |
plan | Plan | src/plan/plan.module.ts:47-48 | PlanController | none | none | vehicle.code → vehicle, driver.user_id / moved_by.user_id → user, at plan and stop level (src/plan/plan.schema.ts:32-33, 75, 89-90, src/common/schema/vehicle_ref.schema.ts:10, user_ref.schema.ts:10) |
store | Store | src/store/store.module.ts:33-34 | StoreController | none | none | none |
vehicle | Vehicle | src/vehicle/vehicle.module.ts:35-36 | VehicleController | none | none | none |
user | User | src/user/user.module.ts:33-34 | UserController | password (src/user/user.controller.ts:22) | none | none |
Not registered on purpose: application (masks secret_key, src/application/application.controller.ts:23), settings, storage, logs, jobs. A script or template naming one gets <name> is not available (src/script_runner/script_runner.sandbox.ts:62-63, template.service.ts:61-63).
2. File map
Core
| file | responsibility |
|---|---|
src/template/template.schema.ts | templates collection, TemplateChannel, TemplatePaper, TemplateOrientation, embedded TemplatePage, unique code |
src/template/template.dto.ts | TemplateInput, TemplateUpdateInput (+ revision), TemplatePageInput, TemplateRenderInput |
src/template/template.service.ts | Get, create (revision 0), update with revision compare-and-set, validateTemplate (entity registered, email subject), duplicate-code mapping |
src/template/template.render.service.ts | Render pipeline: source (saved or draft), bulk load, per-record script + Liquid, error shaping, print document, email/SMS result |
src/template/template.controller.ts | GET /templates, GET /templates/:id, POST /templates/render, POST /templates, PUT /templates/:id, admin only |
src/template/template.module.ts | Model + ScriptRunnerModule; exports TemplateService |
src/template_engine/template_engine.ts | The one Liquid engine: no file system, blocked include/render/layout, ownPropertyOnly, strictFilters, limits; helpers registered as filters; renderTemplate, pageRule, smsInfo, TemplateRenderError |
src/template_engine/template_engine.constants.ts | Liquid limits (env-overridable) and SMS segment sizes |
src/script_runner/script_runner.sandbox.ts | runInSandbox: isolate per run, bootstrap, db bridge with budget (DbBridge), wall timer, error classification, toPlain BSON → JSON |
src/script_runner/script_runner.service.ts | Entity readers: filter guards (server-code operators, hidden/masked paths, allowed operators), sort guard, projection, hide/mask on output; run, load, getLimits |
src/script_runner/script_runner.constants.ts | DEFAULT_SCRIPT_LIMITS, DB_FUNCTIONS, FORBIDDEN_FILTER_OPERATORS, script file name |
src/script_runner/script_runner.error.ts | ScriptRunError (script_error / budget_exceeded, line, limit, calls), ScriptCall |
src/script_runner/func.ts | SCRIPT_HELPERS: 13 self-contained helpers copied into the isolate by source and registered as Liquid filters |
src/script_runner/script_entities.registry.ts | ScriptEntitiesRegistry (register / find / has / all) and ScriptEntitiesModule |
src/script_runner/script_setting.settings.ts | Settings record script with one key per limit (detail in 03) |
src/script_runner/script_setting.service.ts | getLimits() from the settings cache |
src/script_runner/script_runner.module.ts | Wires settings refs, registry; exports runner, setting service, registry module |
src/script_runner/batch_mapper.ts | Pure export mapper: column types, readPath with [], number formats, Liquid text/format cells, running row_no, footer sums, script state cap. No caller yet |
src/field_catalog/field_catalog.service.ts | describeSchema (flatten, parent[].child, refs, system flags, hidden drop), catalog build at bootstrap, getEntities, getFields, hasPath through refs |
src/field_catalog/field_catalog.controller.ts | GET /export_fields, GET /export_fields/:entity |
src/field_catalog/field_catalog.module.ts | Imports ScriptRunnerModule for the registry |
src/export_config/export_config.schema.ts | export_configs collection with raw column subdocs |
src/export_config/export_config.dto.ts | ExportConfigInput, ExportConfigUpdateInput, ExportColumnInput |
src/export_config/export_config.constants.ts | Align values, column width 4–80 |
src/export_config/export_config.service.ts | Get, create, update with revision CAS, validateExportConfig (entity, per-column rules, path in catalog) |
src/export_config/export_config.controller.ts | GET/POST /export_configs, GET/PUT /export_configs/:id, admin only |
src/export_config/export_config.module.ts | Model + FieldCatalogModule |
.dockerignore | Drops .env, .env.* (keeps .env.example), node_modules, dist from the image |
Web (store-v2)
| file | responsibility |
|---|---|
web:src/pages/setting/templates/index.tsx | Template list: code (link), name, channel, entity; keyword search, paging |
web:src/pages/setting/templates/[id].tsx | Template editor (create = new): channel, code, name, entity, page settings, subject, script and body editors, preview source, Print, save-with-error confirm, conflict banner |
web:src/services/template/editor.ts | Editor hook: load, entity list, 600 ms debounced draft render, sample JSON check, save create/update, 409 handling |
web:src/services/template/index.ts, types.ts | List hook; TemplateSourceMode, DEFAULT_PAGE |
web:src/components/template/Preview/index.tsx | Preview: error line, email subject, SMS text + summary, sandboxed iframe for print/email, call log table, vars JSON |
web:src/components/forms/CodeEditor/index.tsx | CodeMirror 6 editor (javascript / html / json with lint) |
web:src/pages/setting/export_configs/index.tsx, [id].tsx | Export config list and editor: name, entity, script, script_reads tags, column rows, reset, JSON view |
web:src/services/export_config/editor.ts, column.ts, catalog.ts, types.ts, index.ts | Editor hook, column helpers (toConfigBody, resolvePath, columnSource), shared per-entity catalog cache, constants (TEXT_FILTERS, NUMBER_FORMATS, ROW_NO), list hook |
web:src/components/export_config/ColumnRow/index.tsx | One column row: drag handle, type, path picker or text with insert field/filter, header, source badge, move/remove, options panel (format, align, width, footer sum) |
web:src/components/export_config/FieldPicker/index.tsx | Field tree from the catalog, ref expansion up to depth 4, array group, row_no, free script path |
web:src/pages/setting/script_settings/index.tsx, web:src/services/script_setting/index.ts | Script limits form (6 positive integers) — calls a removed endpoint (§12) |
web:src/pages/api/{templates,templates/[id],templates/render,export_configs,export_configs/[id],export_fields,export_fields/[entity],script_settings} | Next.js proxy routes to core |
web:src/apis/server/{template,export_config,script_setting}/index.ts, web:src/apis/client/... | Server-side and browser fetch wrappers |
web:src/constants/menu.ts:107-121 | Setting menu items Templates, Script settings, Export configs |
3. Data model
templates (model template, src/template/template.schema.ts:40-84, strict, timestamps created_at/updated_at)
| field | type | required | default | enum / validation | index | note |
|---|---|---|---|---|---|---|
code | string | yes | — | DTO non-empty string (template.dto.ts:29-31) | unique (:84) | duplicate → 400 duplicate_code |
name | string | yes | — | non-empty | — | |
channel | string | yes | — | print | email | sms (:54-55, DTO :37-38) | — | |
entity | string | yes | — | must be a registered script entity (template.service.ts:61-63) | — | not a FK; a registry name |
page | TemplatePage subdoc (_id: false) | no | — | paper A4/A3/A5/Letter, orientation portrait/landscape, margin_mm number ≥ 0, font_url, font_family strings (:22-38, DTO :5-26) | — | stored for any channel; only print uses it |
subject | string | email only | — | email: non-blank after trim | — | Liquid |
script | string | no | — | none at save | — | data script source |
body | string | no | — | none at save (Liquid not parsed on save) | — | Liquid |
revision | number | — | 0 | DTO int ≥ 0 on PUT (template.dto.ts:62-66) | — | set 0 on create, $inc 1 per update |
created_by, updated_by | string | no | — | — | — | session profile.id |
_translates | Mixed | no | — | — | — | from BaseSchema (src/common/base/base.schema.ts:9-10), never written here |
export_configs (model export_config, src/export_config/export_config.schema.ts:16-49, strict, timestamps)
| field | type | required | default | enum / validation | index | note |
|---|---|---|---|---|---|---|
name | string | yes | — | non-empty | none (not unique) | |
entity | string | yes | — | must be a catalog entity (export_config.service.ts:53-55) | — | |
script | string | no | — | none | — | per-batch script, never run (no runner) |
script_reads | string[] | no | — | strings (export_config.dto.ts:54-57) | — | stored and returned only; nothing reads or checks it |
columns | array of raw subdocs (_id: false) | DTO yes (array, may be empty) | [] | see column table | — | order = output order |
revision | number | — | 0 | int ≥ 0 on PUT | — | CAS like templates |
created_by, updated_by | string | no | — | — | — |
Column subdoc (export_config.schema.ts:5-14, DTO export_config.dto.ts:6-39):
| field | type | validation | rule at save (export_config.service.ts:56-69) |
|---|---|---|---|
type | string | field | text | script (src/script_runner/batch_mapper.ts:8-13) | — |
path | string | optional string | field: required and must resolve in the catalog (hasPath, through refs up to depth 3); script: required, config script must be non-blank, path not checked |
text | string | optional | text column: Liquid over the row + row_no; not parsed at save |
header | string | required string (may be empty) | used in error messages |
format | string | optional | #,##0 / #,##0.00 / #,##0.000 → numeric cell with that Excel format; any other string → used as a Liquid filter expression {{ value | <format> }} (batch_mapper.ts:94-99); not checked against a list |
align | string | left | center | right (export_config.constants.ts:2) | DB has no enum |
width | number | int 4–80 (export_config.constants.ts:3-4) | DB has no bound |
footer_sum | boolean | optional | on a text column → 400 validation |
Settings used (record script in _settings; owned by 03)
| key | default | used here at |
|---|---|---|
max_calls | 20 | script_runner.sandbox.ts:66 |
max_docs | 1000 | script_runner.sandbox.ts:92-97, 128 |
wall_ms | 2000 | script_runner.sandbox.ts:199-212 |
heap_mb | 64 | script_runner.sandbox.ts:171 |
export_state_kb | 64 | nowhere (mapBatch takes stateKb from its caller, and there is no caller) |
max_bulk_records | 50 | template.render.service.ts:49-51 |
Defaults: src/script_runner/script_runner.constants.ts:2-9; declaration (int ≥ 1 each): src/script_runner/script_setting.settings.ts:6-19.
Indexes
| collection | index | source |
|---|---|---|
templates | { code: 1 } unique | template.schema.ts:84 |
export_configs | _id only | — |
Redis keys
None. Limits come from the settings cache (03). The field catalog and the reader map are per-pod memory.
In-memory state per pod
| holder | content | built | invalidated |
|---|---|---|---|
ScriptEntitiesRegistry.entities | name → {name, service, controller} | each domain module onModuleInit | never (process lifetime) |
FieldCatalogService.catalog | entity → CatalogField[] | onApplicationBootstrap (field_catalog.service.ts:77-84) | never; a schema change needs a deploy |
ScriptRunnerService.readers | entity → reader with its hidden/masked lists | first run/load call (script_runner.service.ts:131-136) | never |
Liquid engine | one module-level Liquid instance | module load (template_engine.ts:79) | never |
Relations
| from | to | how |
|---|---|---|
templates.entity, export_configs.entity | registry name | string, checked at save; a later unregistered entity leaves the row unusable (render 400 unknown_entity) |
export_configs.columns[].path | catalog field path, may cross ref: | string, checked at save only |
field ref: | registered entity name | catalog ref, expandable when the target is registered |
templates/export_configs created_by/updated_by | users._id | string, no FK |
4. API
All routes: guard @Roles([UserRole.admin]) on the class (template.controller.ts:13, field_catalog.controller.ts:6, export_config.controller.ts:12). The guard needs a Bearer token whose Redis session has user.role === 'admin' (src/common/guards/auth.guard.ts:18-48, 74-88), otherwise 403 Forbidden resource. system_admin, store_operator and driver are refused. On feature/auth these routes map to the keys setting.template.*, setting.export_config.*, setting.field_catalog.read (02-auth §4); that mapping is not on this sha.
Error body from throwError (src/common/utils/error.ts:4-19): { statusCode, code, message: [string], additional_info: {}, error: '' }. DTO failures come from the global ValidationPipe (transform, implicit conversion, no whitelist; src/main.ts:45-50) as Nest's 400 { statusCode, message: [...], error: 'Bad Request' } with no code.
Templates
| method + route | request | response | errors | code |
|---|---|---|---|---|
GET /templates | query SearchQueryDto: keywords, filter (qs string, convertFilter), sort, fields, page, page_size (default 10) | { page, page_size, total, total_page, rows: Template[] } with full script/body | 400 DTO | template.controller.ts:24-30, src/common/base/base.service.ts:90-131 |
GET /templates/:id | — | Template (id, no _id) | 404 not_found "Template not found" (bad ObjectId too) | template.controller.ts:32-37, template.service.ts:23-27 |
POST /templates | TemplateInput: code, name, channel, entity required; page, subject, script, body optional | 201 Template, revision: 0 | 400 DTO; 400 unknown_entity "<entity> is not available"; 400 subject_required; 400 duplicate_code | template.controller.ts:47-53, template.service.ts:29-36, 60-72 |
PUT /templates/:id | TemplateUpdateInput = full TemplateInput + revision int ≥ 0 | 200 Template with revision + 1 | 400 DTO (missing revision); 400 unknown_entity / subject_required / duplicate_code; 404 not_found; 409 version_conflict "Template was changed by someone else, reload and try again" | template.controller.ts:55-62, template.service.ts:38-58 |
POST /templates/render | TemplateRenderInput: template_id (MongoId) or template (object draft: channel, entity, page, subject, script, body); optional record_ids string[]; optional sample_data object | 200 TemplateRenderResult (below) | 400 validation "template or template_id is required"; 400 validation "template.channel must be print, email or sms"; 400 unknown_entity; 400 subject_required; 404 not_found (template id); 400 validation "record_ids is empty"; 400 bulk_limit_exceeded "At most N records can be rendered at once"; 404 not_found "Record <id> not found" (invalid or missing) | template.controller.ts:39-45, template.render.service.ts:40-107 |
TemplateRenderResult (template.render.service.ts:12-22):
| field | when | content |
|---|---|---|
channel | always | the source channel |
calls | always | ScriptCall[] { fn, entity, docs, ms } in order: the bulk load first (fn: 'load'), then every script find/get/count of every record, including the failing one |
vars | always | the script return value of record 1 when it is a plain object, else {} |
html | print, no error | full HTML document |
subject, body | email, no error | of record 1 |
text, sms | sms, no error | text of record 1, { encoding: 'GSM-7' | 'UCS-2', chars, segments } |
error | a script or Liquid failure | { code: 'script_error' | 'budget_exceeded' | 'render_error', message, line?, limit? }; in a bulk run the message starts record N (<id>): ; script errors add at line L. Status is still 200 and no output field is set |
Template template_id wins when both template_id and template are sent (template.render.service.ts:99). A draft is never saved.
Field catalog
| method + route | request | response | errors | code |
|---|---|---|---|---|
GET /export_fields | — | [{ entity, label }], label = _.startCase(entity) | — | field_catalog.controller.ts:14-17, field_catalog.service.ts:86-88 |
GET /export_fields/:entity | — | CatalogField[]: { name, type, ref?, array?, system? } | 404 not_found "<entity> is not available" | field_catalog.controller.ts:19-24, field_catalog.service.ts:90-93 |
type values: string, number (Number, Decimal128), date, boolean, objectid, mixed (Mixed, Map), buffer, object (subdoc or document array), or the lower-cased mongoose instance (field_catalog.service.ts:23-35).
Export configs
| method + route | request | response | errors | code |
|---|---|---|---|---|
GET /export_configs | SearchQueryDto | paged { ..., rows } with full script/columns | 400 DTO | export_config.controller.ts:20-26 |
GET /export_configs/:id | — | ExportConfig | 404 not_found "Export config not found" | export_config.controller.ts:28-33, export_config.service.ts:24-28 |
POST /export_configs | ExportConfigInput: name, entity, columns required; script, script_reads optional | 201 ExportConfig at revision 0 | 400 DTO (column type, header, align, width); 400 unknown_entity; 400 validation "<header>: footer sum needs a field column" / "<header>: a <type> column needs a path" / "<header>: a script column needs a config script"; 400 unknown_path "<path> is not a field of <entity>" | export_config.controller.ts:35-41, export_config.service.ts:30-33, 52-70 |
PUT /export_configs/:id | full ExportConfigInput + revision | 200 saved doc, revision + 1 | as POST; 404 not_found; 409 version_conflict | export_config.controller.ts:43-50, export_config.service.ts:35-50 |
There is no DELETE for templates or export configs, no export run endpoint, no export download.
Script limits
/script_settings was removed in 3dd1c5a0. The limits are read and saved through the generic settings API, GET /settings/script and PUT /settings/script with { rev, values: { max_calls, ... } } (src/settings/settings.controller.ts:21-33, src/settings/settings.dto.ts:3-10; full contract in 03-logs-and-settings §4).
5. Conditions
Templates
| when … | the system must … | enforced at | test |
|---|---|---|---|
a template is created with a code already used | answer 400 duplicate_code | template.schema.ts:84, template.service.ts:69-72 | src/template/template.service.spec.ts:66 |
| a template is created | store only the 8 editable fields, revision: 0, created_by/updated_by = caller | template.service.ts:11, 32 | template.service.spec.ts:72 |
| a PUT carries a revision that is not the stored one | change nothing and answer 409 version_conflict (one conditional findOneAndUpdate) | template.service.ts:43-54 | template.service.spec.ts:78 |
| a PUT targets a missing or malformed id | answer 404 not_found | template.service.ts:40, 55 | template.service.spec.ts:90 |
a PUT has no revision | answer 400 validation | template.dto.ts:62-66 | template.service.spec.ts:102 |
the caller's role is not admin | answer 403 and write nothing | template.controller.ts:13, auth.guard.ts:34-36 | src/template/template.controller.spec.ts:69, template.service.spec.ts:108 |
| channel is email and subject is blank | answer 400 subject_required (save and draft render) | template.service.ts:64-66 | template.service.spec.ts:112, render template.render.service.spec.ts:181 |
page.paper is not A4/A3/A5/Letter | answer 400 validation | template.dto.ts:6-8 | template.service.spec.ts:117 |
entity is not registered (e.g. storage_setting) | answer 400 unknown_entity | template.service.ts:61-63 | template.service.spec.ts:121 |
| a draft render has a channel outside the enum | answer 400 validation | template.render.service.ts:102-104 | untested |
neither template nor template_id is sent | answer 400 validation | template.render.service.ts:100 | template.render.service.spec.ts:153 |
record_ids has more unique ids than max_bulk_records | answer 400 bulk_limit_exceeded before any read | template.render.service.ts:47-51 | template.render.service.spec.ts:121 |
record_ids is [] | answer 400 validation | template.render.service.ts:48 | untested |
| an id is malformed or not found | answer 404 not_found naming it, no output | template.render.service.ts:52-58 | template.render.service.spec.ts:115 |
record_ids is given | load all records in one find {_id: {$in}} through the script read path (hidden fields dropped, masked fields masked), outside the script budget, in the order of the ids, duplicates collapsed | template.render.service.ts:47, 54-59, script_runner.service.ts:121-129 | template.render.service.spec.ts:128, 158 |
record_ids is absent | render once with sample_data (or {}) as record | template.render.service.ts:44 | template.render.service.spec.ts:181, 191 |
| a script or Liquid error happens on record N of a bulk | stop, return 200 with error naming record N and its line, the calls so far, and no partial HTML | template.render.service.ts:74-83 | template.render.service.spec.ts:137 |
| any other error happens in the loop | rethrow (500 or its HTTP status) | template.render.service.ts:75 | untested |
| the script returns a non-object | vars = {}; Liquid sees only record | template.render.service.ts:114 | untested |
| Liquid renders | context = script vars spread, then record (a script var named record is overwritten) | template.render.service.ts:69 | template.render.service.spec.ts:128 |
| print renders several records | one <section class="tms-sheet"> each, break-before: page on every one after the first | template.render.service.ts:121-127 | template.render.service.spec.ts:168 |
| email/sms render several records | run every record but return only record 1, silently | template.render.service.ts:88-93 | untested (gap §12) |
Liquid engine (template hardening)
| when … | the system must … | enforced at | test |
|---|---|---|---|
a body uses {% include %}, {% render %} or {% layout %} (also inside {% liquid %}) | fail with render_error "… is not allowed in templates" at parse and at render | template_engine.ts:38, 53-60, 75 | src/template_engine/template_engine.spec.ts:100-125 |
| anything asks Liquid to read a file | throw "templates cannot read files" (the fs stub) and never use relative references | template_engine.ts:40-51, 64-65 | template_engine.spec.ts:100 |
a body reads an inherited property (constructor, __proto__) | output nothing | ownPropertyOnly: true, template_engine.ts:66 | template_engine.spec.ts:95 |
| a body uses an unknown filter | fail with render_error | strictFilters: true, template_engine.ts:67 | template_engine.spec.ts:76 |
| a render runs past 2 s, a template is over 200,000 chars, or allocation passes 10,000,000 units | fail with render_error instead of hanging | template_engine.ts:68-70, template_engine.constants.ts:2-4 | template_engine.spec.ts:82 |
| a body holds JS-looking text | print it, never execute it | Liquid only | template_engine.spec.ts:89 |
| a filter has the name of a helper | run the helper with the piped value first and this = helpers; helpers replace Liquid's own date, default, join, where, group_by, sum | template_engine.ts:72-74 | template_engine.spec.ts:36-75 |
page values reach CSS | keep paper/orientation only if [A-Za-z0-9]+ (else A4/portrait), clamp margin to ≥ 0, strip " \ \n \r from font name and URL | template_engine.ts:89-101 | template_engine.spec.ts:135, 139 |
| SMS text has only GSM-7 basic/extended chars | count extended as 2, 1 segment ≤ 160 units else ceil(n/153) | template_engine.ts:103-117 | template_engine.spec.ts:145 |
| SMS text has any other char | UCS-2: chars = JS string length, 1 segment ≤ 70 else ceil(n/67) | template_engine.ts:112-115 | template_engine.spec.ts:145 |
Output is not HTML-escaped: Liquid's default outputEscape is off and none is set (template_engine.ts:63-71). Only barcode/qrcode escape their value (src/script_runner/func.ts:46-47).
Script runner sandbox
| when … | the system must … | enforced at | test |
|---|---|---|---|
| a script runs | run it in a fresh isolate (memoryLimit = heap_mb) as the body of async function (data, helpers, db) in strict mode; the isolate is always disposed | script_runner.sandbox.ts:154-155, 171, 221-224 | src/script_runner/script_runner.sandbox.spec.ts:42 |
the script touches require, process, an undeclared name, db.insert etc. | fail script_error "<x> is not available" | script_runner.sandbox.ts:144-150, 162-166 | script_runner.sandbox.spec.ts:196 |
the script reaches for host Function via data or a db result | find nothing: data and results enter as JSON copies | script_runner.sandbox.ts:189-205, 142-143 | script_runner.sandbox.spec.ts:210 |
a db call is started after max_calls calls | fail budget_exceeded max_calls, also for parallel calls (slot reserved before any await) | script_runner.sandbox.ts:65-67 | script_runner.sandbox.spec.ts:76, 87 |
reads would pass max_docs | fail budget_exceeded max_docs; a find fetches at most remaining + 1 rows, the probe row is never counted; parallel finds hold their reservation and wait for in-flight ones | script_runner.sandbox.ts:89-113, 126-129 | script_runner.sandbox.spec.ts:57, 99, 111, 122, 133 |
| the script catches a budget error and carries on | still fail with that budget error | this.breach sticky, script_runner.sandbox.ts:64, 131-134, 215 | script_runner.sandbox.spec.ts:69 |
the run takes longer than wall_ms (sync loop or awaited host calls) | dispose the isolate and fail budget_exceeded wall_ms | script_runner.sandbox.ts:199-213, classify :175-183 | script_runner.sandbox.spec.ts:169, 186 |
| the isolate runs out of heap | fail budget_exceeded heap_mb | script_runner.sandbox.ts:179 | script_runner.sandbox.spec.ts:180 |
| the script throws | fail script_error with the message and the script line (stack line − 1 for the wrapper) | script_runner.sandbox.ts:157-160, 182 | script_runner.sandbox.spec.ts:151 |
db.find/db.get has no fields or a field is not [A-Za-z0-9_]+(\.[A-Za-z0-9_]+)* | fail script_error "fields is required" / "invalid field" | script_runner.sandbox.ts:116-124 | script_runner.sandbox.spec.ts:51 |
db.get has no id string, or an invalid ObjectId | fail script_error | script_runner.sandbox.ts:81, script_runner.service.ts:184 | untested |
limit is not a positive integer | fail script_error | script_runner.sandbox.ts:89-90 | untested |
| a result holds Date / Decimal128 / ObjectId / other BSON | send ISO string / number / hex string / toString(); drop undefined keys; stop at depth 32 | script_runner.sandbox.ts:25-40 | script_runner.sandbox.spec.ts:158 |
| every call (also failing ones) | record { fn, entity, docs, ms } and return the list on success and on error | script_runner.sandbox.ts:71, 216-219 | script_runner.sandbox.spec.ts:43 |
| limits are needed | read them from the settings cache at the start of every run, unless the caller passes them | script_runner.service.ts:105-114 | src/script_runner/script_runner.service.spec.ts:75, script_setting.service.spec.ts:22 |
Script reads (data-script guards)
| when … | the system must … | enforced at | test |
|---|---|---|---|
| a filter is not a plain object | fail script_error | script_runner.service.ts:147 | untested |
a filter holds $where, $function, $accumulator or $expr at any depth | refuse | script_runner.service.ts:18-25, script_runner.constants.ts:21 | script_runner.service.spec.ts:69, 163 |
a filter document (top level or inside $and/$or/$nor) has a $ key other than $and $or $nor $comment | refuse "operator not allowed" | script_runner.service.ts:63-69 | script_runner.service.spec.ts:140-175 |
$jsonSchema or $text appears at any depth (inside $elemMatch, $not, logical branches) | refuse | script_runner.service.ts:31, 66-67 | script_runner.service.spec.ts:186, 191 |
a filter path equals or is under a hidden or masked field (also inside $elemMatch by full path) | refuse "field not allowed" | script_runner.service.ts:36-41, 72-76 | script_runner.service.spec.ts:91-135, 218 |
| a filter tests a parent of a hidden field | allow only $exists, $size, $type, $not of those, and $elemMatch by fields; refuse value matches | script_runner.service.ts:46-61 | script_runner.service.spec.ts:196, 223 |
a sort is not a string, has a field outside [A-Za-z0-9_.], or touches a hidden/masked field or its parent | refuse | script_runner.service.ts:81-93 | script_runner.service.spec.ts:116 |
id is used as a filter key or sort field | map it to _id | script_runner.service.ts:90, 150 | script_runner.service.spec.ts:61 |
| a filter has id strings | cast them the way a mongoose query does; a cast error is script_error "invalid filter" | script_runner.service.ts:151-155 | script_runner.service.spec.ts:61 |
| fields are projected | never project a hidden field; always return id (string) and drop _id | script_runner.service.ts:158-169 | script_runner.service.spec.ts:50, script_entities.registry.spec.ts:45 |
| a masked field has a value in the result | mask it (first 4 + * + last 2) | script_runner.service.ts:171-172, src/common/utils/mask.ts:31-60 | script_entities.registry.spec.ts:51 |
| two modules register the same name | crash the boot naming the entity | script_entities.registry.ts:16-20 | script_entities.registry.spec.ts:63 |
| an entity is not registered | fail "<entity> is not available" | script_runner.sandbox.ts:62-63, script_runner.service.ts:122-123 | script_entities.registry.spec.ts:57 |
Reads are not store-scoped at this sha: BaseService.aggregate and count apply no scope (base.service.ts:175-184, 237-239). Store scope plus AsyncResource.bind for script reads exist only on feature/auth (02-auth §5).
Field catalog
| when … | the system must … | enforced at | test |
|---|---|---|---|
| the catalog is built | include every registered entity; flatten subdocs as a.b, document arrays as a (object, array: true) plus a[].b; primitive arrays as the item type with array: true | field_catalog.service.ts:38-68 | src/field_catalog/field_catalog.service.spec.ts:63-112 |
a path is __v or _translates | leave it out | field_catalog.service.ts:21, 41 | field_catalog.service.spec.ts:112 |
a path is top-level _id, created_at, updated_at | list it with system: true | field_catalog.service.ts:20, 64 | field_catalog.service.spec.ts:112 |
a field (or array item) has ref: | list ref (target may be unregistered; expanding it is 404) | field_catalog.service.ts:55-63 | field_catalog.service.spec.ts:73, 86, 121 |
| a field is Mixed | list type: 'mixed', not expandable | field_catalog.service.ts:30 | field_catalog.service.spec.ts:97 |
| a field is hidden by the read controller | leave it and its children out (masked fields stay listed) | field_catalog.service.ts:43, 81 | field_catalog.service.spec.ts:126 |
hasPath(entity, path) | match an exact name, else follow a ref field prefix into the target, up to depth 3 | field_catalog.service.ts:96-101 | field_catalog.service.spec.ts:132 |
Export config
| when … | the system must … | enforced at | test |
|---|---|---|---|
| it is created | store the 5 editable fields at revision 0 | export_config.service.ts:12, 30-33 | src/export_config/export_config.service.spec.ts:68 |
| PUT with a stale revision | 409 version_conflict, one conditional write | export_config.service.ts:38-46 | export_config.service.spec.ts:76 |
| PUT/GET on a missing id | 404 | export_config.service.ts:24-28, 37, 47 | export_config.service.spec.ts:85, 96 |
entity is not in the catalog | 400 unknown_entity | export_config.service.ts:53-55 | export_config.service.spec.ts:135 |
a text column has footer_sum | 400 validation | export_config.service.ts:57-59 | export_config.service.spec.ts:100 |
a field/script column has no path | 400 validation | export_config.service.ts:61 | export_config.service.spec.ts:139 |
a script column exists and the config script is blank | 400 validation naming the column | export_config.service.ts:62-64 | export_config.service.spec.ts:111, 121 |
a field column path is not in the catalog | 400 unknown_path naming it | export_config.service.ts:66-68 | export_config.service.spec.ts:105, 129 |
revision missing on PUT, width outside 4–80, align outside the list | 400 DTO | export_config.dto.ts:26-34, 65-69 | export_config.service.spec.ts:144 |
Batch mapper (pure, for a future export run)
| when … | the system must … | enforced at | test |
|---|---|---|---|
| batches are mapped one after another | carry row_no, footer sums and the script state in BatchState | batch_mapper.ts:31-36, 102-140 | src/script_runner/batch_mapper.spec.ts:52 |
| a script is given | call runScript(rows, state); accept rows[] or { rows, state? }; anything else is script_error; rows may grow or shrink, row_no counts output rows | batch_mapper.ts:108-116 | batch_mapper.spec.ts:100, 107 |
the script state JSON is over stateKb KB | fail that batch with budget_exceeded | batch_mapper.ts:117-118 | batch_mapper.spec.ts:77 |
| a path is missing on a row | empty cell plus one warning per path | batch_mapper.ts:87-91, 138 | batch_mapper.spec.ts:94, 127 |
a path walks a[].b | collect across the array, flatten, drop nil; join arrays with , | batch_mapper.ts:68-75, 92 | batch_mapper.spec.ts:113 |
| a format is a number format | numeric cell + format string; a non-number becomes an empty cell | batch_mapper.ts:95-98 | batch_mapper.spec.ts:83 |
| a format is anything else | render {{ value | <format> }} | batch_mapper.ts:99 | batch_mapper.spec.ts:134 |
| a text column | render Liquid over the row plus row_no | batch_mapper.ts:84-85 | batch_mapper.spec.ts:89 |
footer_sum | add the numeric cell value, round to 10 decimals in footer | batch_mapper.ts:128, 137 | batch_mapper.spec.ts:52, 119 |
| the module is imported | import nothing from Nest or mongoose | batch_mapper.ts:1-3 | batch_mapper.spec.ts:139 |
6. States and flows
Template / export config record
Both use the same revision scheme. There is no status, no delete and no publish step: a saved row is live at once.
| state | event | next | guard |
|---|---|---|---|
| — | POST valid | revision 0 | entity known, rules pass, code unique (templates) |
revision n | PUT with revision n, valid | revision n+1 | findOneAndUpdate({_id, revision: n}) |
revision n | PUT with revision m ≠ n | revision n | 409 version_conflict |
Render
One sandbox run
Boot order (registry and catalog)
Export config save
Export run (job, batches, xlsx, storage, download): not built, so no flow.
7. Jobs, schedules, queues, events
None in this part. Template render is synchronous inside the HTTP request. The designed export run, a long-lane jobs row plus an export_runs record with progress and a download, is not built: there is no job target, no export_runs collection, no ExcelJS dependency, and no caller of mapBatch (rtk proxy grep mapBatch src finds only batch_mapper.ts and its spec). Email and SMS sending are not built, so no outbound event exists either. For the engine an export run would use, see 04-jobs-engine.md §13 A.
8. Settings and env
| key | default | effect | where read |
|---|---|---|---|
setting script.max_calls | 20 | db calls per script run | script_runner.sandbox.ts:66 |
setting script.max_docs | 1000 | docs read per run | script_runner.sandbox.ts:92-97, 128 |
setting script.wall_ms | 2000 | wall time per run (one per record in a bulk) | script_runner.sandbox.ts:199-212 |
setting script.heap_mb | 64 | isolate memory limit | script_runner.sandbox.ts:171 |
setting script.export_state_kb | 64 | none (declared, never read) | — |
setting script.max_bulk_records | 50 | max unique record_ids per render | template.render.service.ts:49 |
env TEMPLATE_RENDER_LIMIT_MS | 2000 | Liquid renderLimit per render call | template_engine.constants.ts:2 (read once at module load) |
env TEMPLATE_PARSE_LIMIT | 200000 | Liquid parseLimit (template chars) | template_engine.constants.ts:3 |
env TEMPLATE_MEMORY_LIMIT | 10000000 | Liquid memoryLimit | template_engine.constants.ts:4 |
| constant SMS sizes | 160 / 153 / 70 / 67 | segment math | template_engine.constants.ts:7-10 |
constant EXPORT_COLUMN_WIDTH_MIN/MAX | 4 / 80 | column width bound | export_config.constants.ts:3-4 |
constant NUMBER_FORMATS | #,##0, #,##0.00, #,##0.000 | numeric cell formats | batch_mapper.ts:5 |
storage.* settings | see 03 §8 | not used by this part: no export file is written | — |
Each script.* key validates as an int ≥ 1 (script_setting.settings.ts:6-10). A value of 1 for wall_ms or heap_mb is accepted and breaks every render.
9. Concurrency and recovery
| case | what the code does | where |
|---|---|---|
| two admins save the same template or export config | one conditional write wins; the other gets 409 and reloads | template.service.ts:43-54, export_config.service.ts:38-46 |
| two creates with the same template code | unique index: one wins, the other gets 400 duplicate_code | template.schema.ts:84, template.service.ts:69-72 |
| two creates of export configs with the same name | both are stored (no unique key) | export_config.schema.ts |
| several pods | every pod builds its own registry, catalog and reader map from the same code; nothing is shared but Mongo and the settings cache | §3 in-memory table |
| limits saved while a render runs | the render keeps the limits it read at its start; the next render uses the new values once the pod's settings cache reloads (03 §6.3) | template.render.service.ts:42, script_runner.service.ts:106 |
parallel db.* calls inside one script | call slots and doc reservations are taken before any await, so the budget holds under Promise.all | script_runner.sandbox.ts:65-67, 91-113 |
| a script loops forever | the wall timer disposes the isolate; the event loop of the pod is not blocked (isolated-vm runs on its own thread), so other requests continue | script_runner.sandbox.ts:207-213; spec script_runner.sandbox.spec.ts:169 |
| a Liquid body loops | renderLimit stops it with render_error; Liquid runs on the pod's main thread, so up to 2 s of CPU per render call | template_engine.ts:68 |
| one bulk render of 50 records | each record gets a full budget (20 calls, 1,000 docs, 2 s), plus the load, so one request may run about 50 × 2 s | template.render.service.ts:64-67 (gap §12) |
| the pod dies mid-render | nothing was written; the client sees a network error and retries | render is read-only |
| the same render request twice | harmless: render is read-only and has no idempotency need | — |
10. Web (store-v2)
Pages
| route | what it does | API calls (via Next proxy) | gating |
|---|---|---|---|
/setting/templates | list with code link, name, channel label, entity; debounced keyword search; paging (web:src/pages/setting/templates/index.tsx:15-91) | GET /api/templates?page&page_size&keywords | menu entry only (web:src/constants/menu.ts:107-111), no role or permission check on the page; core 403 for non-admins |
/setting/templates/create, /setting/templates/[id] | editor: channel radio, code, name, entity select (from /export_fields), revision label, page settings (print), subject (email), script editor (javascript), body editor (html), preview source "sample JSON" or "record ids"; preview on the right; Print button (print); Save; if the last preview had an error, Save asks to confirm (web:src/pages/setting/templates/[id].tsx:74-83, 233-252); 409 shows a banner and a sticky toast with Reload (:50-72, 114-121) | GET /api/templates/:id, GET /api/export_fields, POST /api/templates/render (draft only, never template_id), POST /api/templates, PUT /api/templates/:id | same |
/setting/export_configs | list (web:src/pages/setting/export_configs/index.tsx) | GET /api/export_configs | same |
/setting/export_configs/create, /[id] | editor: name, entity, script, script_reads tags, column rows with drag/up/down/remove/add, Reset (back to the saved columns), JSON view of the body to be sent; save blocks a script column with no script in the browser (web:src/services/export_config/editor.ts:97-102) | GET/POST/PUT /api/export_configs[/:id], GET /api/export_fields, GET /api/export_fields/:entity (one cached request per entity, web:src/services/export_config/catalog.ts:7-31) | same |
/setting/script_settings | 6 positive-integer fields, Save disabled when invalid, conflict banner (web:src/pages/setting/script_settings/index.tsx:14-60) | GET /api/script_settings, PUT /api/script_settings → core /script_settings (web:src/apis/server/script_setting/index.ts:3-9) | broken: core removed /script_settings in 3dd1c5a0; every load and save fails with 404. Confirmed. Core now serves the limits at GET/PUT /settings/script with { rev, values } |
Editor behaviour worth knowing
| behaviour | where |
|---|---|
preview renders the unsaved draft 600 ms after the last change; nothing renders until channel and entity are set or while the sample JSON is invalid | web:src/services/template/editor.ts:17, 134-142 |
record ids are split on spaces and commas; an empty box sends no record_ids, so the preview renders {} | web:src/services/template/editor.ts:38-41, 53-60 |
print saves page merged over DEFAULT_PAGE (A4, portrait, 10 mm); the web caps margin at 50 mm, core does not | web:src/services/template/editor.ts:62-71, web:src/services/template/types.ts:16-22, web:src/pages/setting/templates/[id].tsx:157-164 |
the preview iframe uses sandbox="allow-same-origin allow-modals" and srcDoc; email bodies show in the same iframe | web:src/components/template/Preview/index.tsx:51-57 |
the preview shows the call log (fn, entity, docs, ms) and vars JSON | web:src/components/template/Preview/index.tsx:60-68 |
| a field column badge tells field / array / script / text / loading / not found by walking the cached catalog (longest prefix, through refs) | web:src/services/export_config/column.ts:61-93 |
the field picker shows fields and arrays in two groups, expands refs to depth 4, offers row_no in text columns, and takes a free script path | web:src/components/export_config/FieldPicker/index.tsx:9-111 |
in a text column, picking a field appends {{ path }} and a filter button inserts | filter before the last }}; filter list money number thai_date date default upper barcode qrcode | web:src/components/export_config/ColumnRow/index.tsx:62-67, 142-152, web:src/services/export_config/column.ts:111-122, web:src/services/export_config/types.ts:28 |
| a format is one of the three number formats or "filter" with free text | web:src/components/export_config/ColumnRow/index.tsx:69-75, 202-206 |
Proxy routes
Each web:src/pages/api/... route forwards with the caller's headers and returns core's status. A 403 returns e.error, anything else e.data || e.error (web:src/pages/api/templates/index.ts:5-23). The browser request turns the body "Forbidden resource" into a redirect to /login (web:src/apis/client/request.ts:37-39), so a non-admin who opens these pages is sent to the login page.
11. Tests
Core (jest)
| spec | covers |
|---|---|
src/template/template.service.spec.ts | duplicate code, create at revision 0, stale revision 409, missing id 404, revision required, admin-only metadata, email subject, paper enum, unknown entity |
src/template/template.controller.spec.ts | through the real global guard: non-admin POST 403 with no write, admin PUT 200, admin POST 201 |
src/template/template.render.service.spec.ts | unknown record 404, bulk limit 400, script gets the loaded plan and DO calls [load, find], bulk error names record and line with no partial HTML, COD formatting, missing source 400, hidden user fields on load, break-before per sheet, email subject/body, SMS segments |
src/template_engine/template_engine.spec.ts | every helper as a filter, script/filter parity for sum, unknown filter, huge loop stops, no code execution, ownPropertyOnly, no file reads and blocked tags, pageRule, smsInfo |
src/script_runner/script_runner.sandbox.spec.ts | count, fields required, max_docs sequential and parallel, sticky breach, max_calls sequential and parallel, empty and small parallel finds, script error line, BSON to JSON, parallel runs vs an infinite loop, heap bomb, awaited calls count toward wall time, writes/require/unregistered entity, no host Function, the prototype print totals |
src/script_runner/script_runner.service.spec.ts | hidden fields never returned, id cast, server-code operators refused, limits read per run; hidden/masked fields in filter, $or, nested $and > $or, sort; operator allow-list ($jsonSchema, $text, nested, $comment); $elemMatch/$not on a parent of a hidden field |
src/script_runner/script_entities.registry.spec.ts | registered entity hides and masks, unregistered entity error, duplicate name fails boot |
src/script_runner/script_setting.service.spec.ts | limits come from the script settings record with defaults for the rest |
src/script_runner/batch_mapper.spec.ts | 3 batches keep row_no/sums/state, state over 64 KB, number format, text column, missing path warning, script doubling rows, no-script path, array join, script column read and sum, unfilled script column, filter-expression format, no Nest/mongoose import |
src/field_catalog/field_catalog.service.spec.ts | entity list and labels, DO fields and source_store_id → store, store fields, item/plan refs, unknown entity 404, Mixed, plan stops[], system fields and __v, ref to an unregistered entity, hidden fields dropped, hasPath |
src/export_config/export_config.service.spec.ts | create, stale revision 409, missing id 404 (PUT and GET), footer sum on text, unknown path, script column with and without script, extra_calc still unknown, unknown entity, missing path, DTO bounds and revision |
Web
No tests for these pages (the only web specs are web:src/__tests__/pages/setting-google.test.tsx and web:src/services/setting/__tests__/setting.test.ts).
12. Gaps and known issues
| # | gap | where | effect |
|---|---|---|---|
| G1 | Export run not built: no job target, no export_runs, no ExcelJS, no file to storage, no download page, no 31-day / 150k-row limits, no code-declared source (query, allowed filters, permission) | §7; design: handoff "What to build" 2 | an export config can be saved but never produces a file; mapBatch and export_state_kb are dead code until a runner exists |
| G2 | web /setting/script_settings calls the removed /script_settings | web:src/apis/server/script_setting/index.ts:3-9; removed in 3dd1c5a0 | page 404s; ops cannot change limits from the UI. Fix: point it at /settings/script with { rev, values } like the authen/google/kafka pages |
| G3 | bulk budget is per record: 50 records × full budget plus an unbudgeted load | template.render.service.ts:64-67, 54 | one request can hold a pod's isolate threads for ~100 s and read ~50k docs (review M2, still open) |
| G4 | email/SMS with several record_ids returns record 1 silently after running every script | template.render.service.ts:88-93 | wasted work and a misleading preview (review M3, still open) |
| G5 | script reads are not store-scoped at this sha | base.service.ts:175-184, 237-239 | any admin script reads every store's DOs and plans; scope lands only with feature/auth |
| G6 | __call (the host ivm.Reference) is a top-level const in the bootstrap and stays reachable from the user script, which shares the global lexical scope | script_runner.sandbox.ts:137-139, 198-200 | no escape is known (args go through DbBridge), but the design wanted it hidden (review M1, still open) |
| G7 | ref: sits on business keys (vehicle.code, user_ref.user_id, source_store_id), and the catalog accepts paths such as plan.vehicle.code.plate_number | src/common/schema/vehicle_ref.schema.ts:10, user_ref.schema.ts:10, field_catalog.service.ts:96-101 | a future _id join gives empty cells; the catalog does not say which key a ref joins on (review M4) |
| G8 | the catalog offers _id, but script reads rename it to id | field_catalog.service.ts:20, 64, script_runner.service.ts:165-169 | an _id column or {{ record._id }} is always blank (review M6) |
| G9 | text columns accept {{ items[].name }} from the picker; Liquid cannot parse it, and the body/text is not parsed at save | web:src/components/export_config/ColumnRow/index.tsx:64, export_config.service.ts:57-60 | every row of such a column would be render_error (review M5) |
| G10 | format is free text, used as a Liquid filter expression, not checked against a fixed list | batch_mapper.ts:99, export_config.dto.ts:21-24 | design said "format from a fixed filter list"; a typo is found only at run time |
| G11 | no HTML escaping of Liquid output | template_engine.ts:63-71 | record text with < or & breaks print/email HTML; stored HTML from a DO lands raw in the preview iframe (scripts stay off: no allow-scripts) |
| G12 | barcode/qrcode emit <span data-value> markup, but nothing draws them | func.ts:46-47; no drawer in web | barcodes and QR codes print blank |
| G13 | no unsaved-changes guard in the template and export editors | web:src/pages/setting/templates/[id].tsx, export_configs/[id].tsx | a menu click drops a long edit (review M7) |
| G14 | script_reads stored but never read or validated | export_config.service.ts:12 | dead field |
| G15 | thai_date calls this.date; a destructured helper in a script throws | func.ts:27-29 | const { thai_date } = helpers fails (review L4) |
| G16 | group_by/where/sum read x[key] in the host when used as Liquid filters, bypassing ownPropertyOnly | func.ts:31-42, template_engine.ts:72-74 | group_by: "constructor" returns host functions as names; no escalation known (review L2) |
| G17 | limits accept 1 (@Min(1)) for wall_ms and heap_mb | script_setting.settings.ts:6-10 | one bad save breaks every render (review L3) |
| G18 | Liquid errors carry no line | template_engine.ts:81-87 | the editor cannot mark the failing body line (review L1); web also ignores error.line for scripts (review L11) |
| G19 | lists return full script/body/columns; keywords is an unescaped $regex over every string path including body and script | base.service.ts:50-65, 90-131 | heavy lists; typing ( in search errors (review L7, L8) |
| G20 | no delete and no unique name for export configs; no delete for templates | controllers | rows can only be renamed |
| G21 | template type with a code-declared base-data contract (task_print) not built; templates bind to a raw entity | template.schema.ts:57-58 | the script must load relations itself; no fixed field list per print |
| G22 | TemplateRenderError.throwHttp and ScriptRunError.throwHttp are never called | template_engine.ts:19-21, script_runner.error.ts:39-41 | render errors are always 200 with error; nothing returns the 422 shape |
| G23 | the preview iframe keeps allow-same-origin on the admin origin | web:src/components/template/Preview/index.tsx:54 | <img src="/api/..."> in a body fires same-origin GETs; adding allow-scripts for barcodes would make it XSS (review L6) |
| G24 | Print uses the last finished render even while a new one is loading | web:src/pages/setting/templates/[id].tsx:100 | prints stale content (review L5) |
| G25 | web permission keys for these pages (setting.templates.read, setting.export_configs.read, setting.script_settings.read) do not match core's (setting.template.read, ...) | 02-auth G1 | applies once feature/auth merges |
Fixed since the design review: H1 Liquid file access (template_engine.ts:38-75, .dockerignore), H2 hidden-field oracle through filters and sorts, including $jsonSchema/$text at every depth (script_runner.service.ts:27-93).
13. How to add the next one
A. Make a new entity readable by scripts, templates and the field catalog
- In the entity's module, inject
ScriptEntitiesRegistry, importScriptEntitiesModule, and register inonModuleInit(copysrc/vehicle/vehicle.module.ts:9, 31-36):register({ name: '<entity>', service: this.service, controller: <Entity>Controller }). The service must extendBaseServiceand exposemodel. - Put
@HideFields([...])(never readable) or@MaskFields([...])(readable masked) on the read controller for every secret field. The runner and the catalog read the same metadata (script_runner.service.ts:142-144,field_catalog.service.ts:81). - Never register an entity that holds credentials or settings.
- Add
ref: '<target entity>'on fields that point at another registered entity, on its_id(G7). - Add a case to
script_entities.registry.spec.ts(hidden field never returned) andfield_catalog.service.spec.ts(fields listed).
B. A new helper / Liquid filter
- Add it to
SCRIPT_HELPERSinsrc/script_runner/func.ts. It must be self-contained (no imports, no outer variables), since it is copied into the isolate byfn.toString()(script_runner.sandbox.ts:194-196). Usethis.<other>only if a destructured call is never needed (G15). - Its first argument is the piped value in Liquid. It replaces a Liquid built-in of the same name.
- Escape any HTML it emits (copy
barcode). - Add it to the web filter list
web:src/services/export_config/types.ts:28if it is meant for export text columns. - Add a filter case to
template_engine.spec.ts, plus a script parity case if it computes values.
C. A new template channel
- Add the enum value in
template.schema.ts:4-8. - Add its save rule to
TemplateService.validateTemplate(template.service.ts:60-67). - Add its output branch and result fields in
TemplateRenderService.render(template.render.service.ts:86-94) andTemplateRenderResult. - Web: add a
CHANNEL_OPTIONSentry and its preview branch (web:src/pages/setting/templates/[id].tsx:259-263,web:src/components/template/Preview/index.tsx). - Sending is a jobs-engine target, not part of render (04 §13 A).
D. A new db read function for scripts
- Add the name to
DB_FUNCTIONS(script_runner.constants.ts:19), a branch inDbBridge.callthat takes a call slot and reserves docs before any await (script_runner.sandbox.ts:60-114), and the function to the bootstrapdbobject (:144). - Add it to
EntityReaderand toScriptRunnerService._reader. Every filter must go throughcastFilterand every sort throughbuildSort(script_runner.service.ts:146-156, 81-93). Project withproject()and output throughshape(). - Add budget cases (sequential and parallel) to
script_runner.sandbox.spec.ts, and hidden-field cases toscript_runner.service.spec.ts.
E. The export run (next build; not built)
- Register a long-lane job target
export:run(04 §13 A). Add anexport_runscollection (status, progress, row count, file key, error, actor) with an index on{created_by, created_at}. - In the handler, page the source with a stable sort on
_idper batch. Feed each batch tomapBatchwithstateKb = script.export_state_kb. Run the config script throughScriptRunnerService.run(script, { rows, state })per batch, and stream rows to ExcelJS. - Write the file to S3 through
StorageSettingService.getS3Client()(03 §8). Store the key on the run, and serve a presigned GET (copy the attachment pattern). - Enforce the 31-day range and the 150k-row cap before enqueueing. A dedupe key per (config, actor, filter hash) stops a double click from running twice.
- Expose
POST /export_configs/:id/run,GET /export_runs,GET /export_runs/:id/download, and add a run list page in store-v2.