Making a sequence of things happen without a human doing each step. Four layers, and the choice of layer matters more than the choice of tool within it.
| Layer | Runs in | Survives | Use for |
|---|---|---|---|
| Sequencer (SNL) | The IOC | Network loss, client crashes, everything short of IOC death | Machine sequences, startup/shutdown, state machines that must not stop |
| sscan | The IOC | Same | Multi-dimensional scans with in-IOC data collection |
| Scan server | A service | Client crashes; not its own | Experiment procedures with operator oversight |
| Bluesky | A Python session | Nothing | Experiments, where flexibility beats robustness |
The rule: the more the machine depends on it, the deeper it should live. A magnet ramp sequence belongs in the IOC. A user’s diffraction scan belongs in Python.
State machines inside the IOC, in a C-like language.
| Documentation | epics-modules.github.io/sequencer — the SNL reference and users’ guide |
| Source | github.com/epics-modules/sequencer |
| Mirror | mdavidsaver.github.io/sequencer-mirror |
| Training | Sequencer · lab (USPAS) |
State Notation Language (SNL) is compiled by snc into C, built into your IOC, and runs as a thread inside it. PVs are ordinary variables, assigned to channels and monitored:
program magnetRamp
double current_sp;
assign current_sp to "SR-C05-PS-QF-01:Current-SP";
double current_rb;
assign current_rb to "SR-C05-PS-QF-01:Current-RB";
monitor current_rb;
short permit;
assign permit to "HLS-CF-MPS-01:BeamPermit-Sts";
monitor permit;
double target = 102.5;
ss ramp {
state idle {
when (permit == 0) {
printf("Permit off, starting ramp\n");
} state ramping
}
state ramping {
when (current_rb < target - 0.1) {
current_sp = current_rb + 1.0; /* 1 A per step */
pvPut(current_sp);
} state ramping
when (delay(30.0)) {
printf("Ramp timed out\n");
} state faulted
when (current_rb >= target - 0.1) {
} state atField
when (permit != 0) {
} state faulted
}
state atField { when (permit != 0) {} state faulted }
state faulted { when (delay(5.0)) {} state idle }
}
Why it’s the right tool for machine sequences:
when (delay(30.0))), which is exactly what sequence logic is mostly made of.Costs: a compile step in your IOC build; a language most people learn only for this; and debugging is via seqShow/seqChanShow on the IOC console rather than a debugger. Also, +r (reentrant) and safe-mode semantics around variable sharing catch people out — read the manual section on that before writing anything with multiple state sets.
Typical uses: magnet cycling and degaussing, RF conditioning and startup, injection sequences, insertion device gap/taper coordination, cryogenic and vacuum pump-down sequences, automatic recovery after a trip.
Scanning inside the IOC.
| Source | github.com/epics-modules/sscan |
| Part of | synApps |
The sscan record implements a scan: up to four positioners, up to seventy detectors, up to four triggers, with NPTS points, and the collected data held in the record’s arrays. Scan records can be nested for multi-dimensional scans, and saveData writes the results to files.
Why do it in the IOC rather than a client:
Why not: expressing a complex, conditional, adaptive experiment in sscan records is painful, and that’s precisely what Bluesky is good at. Long-standing practice at APS-family beamlines, and still the right choice for straightforward scans where reliability matters more than expressiveness.
| Documentation | Scan server |
| Client | The Scan application in Phoebus |
A service that executes scans described as XML or built in the Phoebus scan editor. Commands include set, wait, loop, log, parallel, sequence, if, script, and include — so a scan is a structured, reviewable, restartable document rather than a program.
Features that matter operationally: scans queue, so submitting work while another scan runs is normal; scans can be paused, resumed and aborted from the UI; progress is visible; and a scan’s data is logged as it goes. The queue plus the visibility is the appeal — a control room can see what automation is running and stop it, which is not true of a script in somebody’s terminal.
Good middle ground: more robust and more visible than a client script, more flexible than sscan, less capable than Bluesky.
Python inside record processing.
| Source | github.com/klemenv/pyDevice |
| Training | Python IOC pyDevice (USPAS) |
Lets a record’s processing call a Python function, embedding an interpreter in the IOC. Excellent for calculations too awkward for a calc expression, for talking to a device with a Python-only SDK, and for gluing something together quickly.
Use with judgement: you have put a garbage-collected interpreter with a global lock inside a real-time-ish control process. Fine for slow, non-critical logic; a poor choice in a fast scan path or anything the machine depends on.
Related: luaEpics does the same with Lua, which is lighter-weight and embeds more naturally, and gives you a scriptable IOC shell as a side benefit.
| Tool | Purpose |
|---|---|
| sseq | Sequenced output records: write ten values, with delays and conditions, driven by one trigger. Declarative sequencing without SNL. |
seq record (Base) |
Up to sixteen values to sixteen links with delays. The simplest sequencing available. |
calcout with ODLY |
Delayed conditional output. Two records like this replace a surprising number of scripts. |
| caputRecorder | Records what an operator does as replayable Python. Good for capturing a procedure someone performs by hand and turning it into automation. |
| Bluesky | Experiment orchestration. The right answer for beamline science. |
| Plain scripts + cron | Legitimate for periodic housekeeping: nightly reports, log rotation, health checks. Not for anything the machine depends on. |
| Task | Where it belongs |
|---|---|
| Ramp magnets to a machine mode, correctly, every time | SNL sequencer — must survive everything |
| Recover automatically from an RF trip | SNL sequencer |
| Insertion device gap tracking monochromator energy | IOC: calc/transform records or a sequencer, never a client |
| Cycle a magnet to remove hysteresis | SNL sequencer |
| A 2D beamline scan producing a map | Bluesky or sscan, depending on the facility’s ecosystem |
| An operator-supervised alignment procedure | Scan server |
| Nightly archive health report | Script + cron |
| Adaptive optimisation of machine parameters | Optimisation framework driving PVs from Python |
| An interlock | Not any of these. Hardware. |
Automation that the machine depends on belongs in the IOC. If it stops when a laptop closes, it isn’t automation, it’s a person with extra steps. This is the single most important point on this page and the one most often ignored, because a Python script is so much quicker to write than an SNL program.
Make every automated action visible. A PV showing what the sequence is doing, its current state, and its last error. Automation an operator cannot see is automation they will fight.
Give an operator a way to stop it. An abort PV, honoured promptly, in every sequence. And an interlock ensuring that stopping it leaves the machine in a safe state rather than half-configured.
Design the failure paths first. The happy path is the easy 10%. What happens when a device doesn’t respond, a limit is hit, the permit drops mid-sequence, or the sequence is aborted at step 7 of 12? SNL’s when (delay(...)) syntax exists because everyone eventually learns that this is most of the work.
Log what automation did. To caPutLog automatically, and to the logbook with intent where it matters. “The magnets changed at 03:00 and nobody was there” should have an answer.