Skip to main content

​
Overview

A single simulation run tells you what happens with one set of inputs. An experiment tells you what happens across many. Experiments compare a model’s behavior across multiple scenarios — configurations, schedules, parameter settings — side by side, so you can read how an operational change moves your KPIs. Each scenario in the comparison is a Snapshot; the experiment is the harness that overlays their results. Combined with Monte Carlo (triggered per Snapshot), an experiment separates the signal of the change from the noise of run-to-run variability.

​
Setting Up an Experiment

An experiment lives inside a model. Open the Experiments section in the left rail; the experiment selector sits at the top with + Create new experiment. The dialog (“Give this experiment a name so you can come back to it later.”) takes a required Name and optional Description. The selector exposes hover rename (pencil) and delete (trash) icons — deletion is immediate, with no confirmation. Once an experiment is selected, the sidebar shows three sections — Snapshots, KPIs, Charts — plus a collapse chevron. Before one is selected, only the Snapshots header renders, with the helper “Select an experiment to manage its snapshots, KPIs, and charts.”

​
Adding Snapshots to Compare

Each Snapshot row in the sidebar is one configuration the comparison will simulate. Rows show the Snapshot’s color dot (its stable identity color, used everywhere the Snapshot appears), its name, and an × that removes it from this experiment — the underlying Snapshot is not deleted. The chevron on the pill container opens a popover with a search field (“Select snapshot to add…”) and the full list of Snapshots on the model; clicking a row toggles membership instantly, no confirm step. A Snapshot is an immutable capture taken in the Modeler: it freezes the model, the active schedule, and the full recursive registry — entities, lookup tables, constants, and sub-models — which is why it replays cleanly whenever the experiment runs.
  • To compare schedules, capture a Snapshot while each schedule is active and add them as separate rows.
  • Model edits do not propagate into existing Snapshots. Capture a new Snapshot after each change you want to compare — the prior one stays available so you can compare them side by side.
  • Restoring a Snapshot loads its captured state back into the Modeler (with a diff preview before applying), so Snapshots double as checkpoints, not just comparison cases. Restoring also captures an autosave Snapshot of the pre-restore state, so the load is undoable.
Deleting a Snapshot outright is discouraged. Every experiment that references it is left unusable with no recovery short of rebuilding it. To take a case out of one comparison, remove it from that experiment with the row × instead.
For example, you might compare three Snapshots:

​
Selecting KPIs and Charts

The KPIs and Charts sidebar sections are visibility checklists for the comparison view; toggles save immediately.
  • KPIs auto-fold from the model. Every KPI defined on the model is evaluated per Snapshot at view time and appears in the checklist.
  • Experiment-scoped KPIs add to that list, in two roles: overrides (an experiment KPI whose slug collides with a model KPI shadows it in this comparison, leaving the model definition untouched) and additions specific to the comparison question.
  • Charts do not auto-fold. Model charts stay on the Results page; only charts authored on the experiment (surface experiment) render in the comparison view. Empty state: “No charts available.”
Model KPI values are not stored in the experiment’s own results. To read a model KPI’s numeric value for one Snapshot, open that Snapshot’s individual Run on the Results page.

​
Comparison Chart Rules

Experiment charts overlay every selected Snapshot on a shared axis, which constrains how a chart is shaped compared to a Run chart:
  • Only three chart types render: Bar, Line, and Scatter. Any other type is silently dropped at render time — the chart simply doesn’t appear.
  • The source query must produce exactly one series. The comparison view uses that series as a template and pivots it into one series per selected Snapshot at render time. Multi-series source queries are also dropped.
  • Snapshot identity is set by the system. Each Snapshot’s stable color is applied to its chart series, KPI swatches, and sidebar chip; any color set by the chart author is overridden, and the series label is auto-set to the Snapshot’s name.
  • The x-axis is shared across cases. Snapshots with no row at a given x-value contribute a null there.
Pick the type by the question: Bar for categorical or single-value magnitude comparisons, Line for ordered or continuous axes where curve shape matters, Scatter for two-continuous-axis relationships where clustering reads across cases.

​
Running an Experiment

The top bar shows a back arrow, the model name, the experiment selector, and a green Run button (play icon). Run is always present once an experiment is selected — with zero Snapshots it errors at execution, and with two or more it runs one simulation per Snapshot that lacks a cached run. Re-runs are incremental:
  • Snapshots with a completed cached run do not resimulate — their queries recompute against the cached dataset, so picking up a new KPI or chart is fast.
  • Newly added Snapshots simulate now.
Long experiments continue in the background after kickoff; the dashboard tracks each Snapshot’s progress as its run terminates, and the full comparison renders once every Snapshot is done.

​
Empty States

  • No experiment selected — “Select an experiment above, or create a new one to get started.”
  • Fewer than two Snapshots — “Pick snapshots to compare” / “Select at least 2 snapshots on the left to compare them.”
  • All KPIs and charts toggled off — “Nothing to show.” Re-enable at least one in the sidebar.
Treat these as navigation cues: each tells you what’s missing for the comparison view to render.

​
Analyzing Results

The comparison view contains:
  • KPI comparison grid — one card per visible KPI, with a horizontal bar and numeric value for each Snapshot on a shared axis, each in its assigned color. This is the default surface for every KPI left toggled on.
  • Experiment charts — Bar, Line, or Scatter, rendered as one multi-series chart with a series per Snapshot.
It answers questions like: does adding a machine actually improve throughput, or does the bottleneck just move? How much does the peak-season schedule stress resources compared to baseline? Multi-seed distributions — histograms, box plots, statistical summary tables across seeds — are not part of the experiment comparison view. For distributional analysis of a single Snapshot, open that Snapshot’s individual Run and use Monte Carlo from there; Monte Carlo at the experiment level is not its own subview.

​
When to Use Experiments

Start with single runs while building and debugging the model. Once it’s stable and you’re making decisions, use experiments to compare options and Monte Carlo to quantify confidence — the combination turns a rough estimate into a rigorous basis for operational decisions.