ApexDocsOpen Apex
Developer

Experiment filesystem and QA commands

Pull experiment files, plan and apply safe changes, and record QA evidence from the Apex CLI.

The experiment filesystem workflow is for complete experiment edits. It pulls supported experiment files to disk, tracks file hashes in .apex/experiment-filesystem.json, plans changes through Apex, and applies only supported file paths.

Canonical loop

bash
apex experiments pull homepage-hero-trust-copy --out ./homepage-hero
apex experiments plan ./homepage-hero --json
apex experiments apply ./homepage-hero --json
apex experiments qa status ./homepage-hero --json

Edit only supported files in the pulled folder:

text
experiment.json
targeting.json
metrics.json
qa.json
launch.json
variations/control/variation.json
variations/variant-a/variation.json
variations/variant-a/mutations.json
variations/variant-a/index.css
variations/variant-a/index.js
variations/variant-a/reset.js
variations/variant-a/anti-flicker.json

.apex/ is CLI-owned metadata. Do not edit it directly.

Pull, plan, and apply

CommandArgumentsFlagsBehavior
apex experiments pull<id-or-slug>--out <folder>Resolves the experiment, pulls editable files, writes CLI metadata.
apex experiments plan[folder]None beyond global flagsSends local files to Apex for validation and returns changes, warnings, errors, QA invalidation, and precompile status.
apex experiments apply[folder]--forcePlans first, confirms if QA would be invalidated, applies through Apex, then refreshes local files.

apex experiments apply returns confirmation-required exit code 4 in non-interactive mode if the plan invalidates QA and --force is not present.

QA status

bash
apex experiments qa status ./homepage-hero

The target can be a pulled folder, experiment ID, or slug. When you pass a pulled folder, the CLI refreshes local files after reading the QA run.

QA checklist updates

bash
apex experiments qa check ./homepage-hero \
  --item responsive_matrix \
  --status pass \
  --notes "Desktop and mobile smoke checks passed." \
  --evidence @qa-result.json
FlagRequiredBehavior
--item <id>YesChecklist item ID.
--status <status>YesOne of pending, pass, warning, fail, accepted, or skipped.
--notes <text>NoChecklist notes.
--evidence <json-or-path>NoJSON object inline or @path/to/evidence.json.

QA evidence must be a JSON object. Keep screenshots, logs, and artifacts outside qa.json; attach structured references through the QA command.

QA notes and approval

bash
apex experiments qa notes ./homepage-hero --notes "Ready for stakeholder review."
apex experiments qa notes ./homepage-hero --acceptance "Responsive, tracking, and content checks passed."
apex experiments qa approve ./homepage-hero
apex experiments qa reopen ./homepage-hero

approve marks QA done after the server-side checklist gates are ready. reopen reopens QA and clears scheduled launch state.

Filesystem rules

  • Never hand-edit qa.json or launch.json as the source of truth.
  • Use apex experiments qa to update QA runs and approval state.
  • Use apex experiments plan before every apply.
  • If you hit a conflict or stale filesystem state, pull fresh files, reapply your edits, and plan again.

Per-URL variation blocks

variations/<variation>/url-blocks.json is optional. When the file is absent, Apex treats the variation as having no URL blocks. A treatment variation can include the file with this shape:

json
[
  {
    "id": "product-a",
    "name": "Product A treatment",
    "urlRules": [{ "type": "simple", "pattern": "https://shop.example/products/a" }],
    "mutations": [{ "action": "text", "selector": "h1", "text": "A-specific headline" }],
    "css": ".offer { color: red; }",
    "js": "window.productTreatment = 'a';"
  }
]

Blocks are ordered and first match wins. The matching block substitutes its mutations, css, and js for the variation's flat files. When no block matches, Apex falls back to mutations.json, index.css, and index.js as the implicit default block. If those flat fields are empty, rendering deliberately fails open as a no-op: the visitor remains assigned and contributes to pooled experiment results. Control variations remain control on every URL.

Blocks do not have a per-block resetJs. Write block js to be idempotent. The runtime removes block CSS and runs its normal cleanup when the URL or assignment signature changes, but JavaScript side effects such as globals or DOM changes outside declared mutations remain the author's responsibility, just as they do for flat custom JavaScript.