Organised by the text you’d paste into a search engine, rather than by subsystem. If you arrived here from a search, you’re in the right place.
!!! note “Exact wording varies” Messages differ between EPICS Base versions, between modules, and between platforms. Match on the distinctive keywords rather than the whole string. Where the precise text matters and isn’t reproduced here, search the tech-talk archive for it — thirty years of the same errors, with answers.
Symptom-first version of the same ground: Troubleshooting.
Channel connect timed out · ... not foundNothing answered the name search. In order of likelihood:
dbl on the IOC.EPICS_CA_AUTO_ADDR_LIST=NO EPICS_CA_ADDR_LIST=<ioc-ip> caget PV
If that works and the default doesn’t, your problem is discovery, not the IOC.
iocInit failed, so the IOC is running and serving nothing. Read its log from the top.--network host. See containers.EPICS_CA_SERVER_PORT.Virtual circuit disconnectThe TCP connection to an IOC dropped — the IOC stopped, its host went away, or the network broke. Normal and expected during an IOC restart; clients reconnect automatically. Repeated occurrences on a stable IOC point at network trouble: check switch error and discard counters.
Virtual circuit unresponsiveThe connection is open but the IOC stopped answering within EPICS_CA_CONN_TMO. Usually the IOC is alive but blocked — a scan task wedged behind slow device support, or CPU saturation. Check casr 2 and iocStats.
Identical process variable name on multiple serversTwo IOCs are serving the same PV name. This is a bug with no upside: clients bind to whichever answers first, so different clients silently get different values.
Find both servers with cainfo PV from several hosts, then kill or rename one. Prevented by an enforced naming convention plus a directory service that makes duplicates visible.
caRepeatercaRepeater is a small process that redistributes server beacons to every CA client on a host. It starts automatically and binds UDP 5065. If it can’t bind — blocked, or something else is on that port — clients still work but reconnect slowly after an IOC restarts.
Requested array size exceeds · truncated arraysArray size limits must agree at the client, every gateway, and the IOC. Raise EPICS_CA_MAX_ARRAY_BYTES on all of them. Modern Base sizes buffers dynamically, but older clients, older IOCs and gateways in the path may not — and the symptom looks like data corruption rather than a limit.
Record type ... not found · Can't find record typeThe DBD describing that record type wasn’t loaded. Either the module providing it is missing from configure/RELEASE, or its DBD isn’t included in your app’s DBD, or dbLoadDatabase didn’t run before dbLoadRecords.
Record already exists · duplicate record nameThe same record name is defined twice — usually a substitutions file instantiated twice, or two templates that overlap. Non-fatal in some Base versions and always a bug.
Illegal SCAN · Illegal ... field valueA field was given a value outside its menu. Check spelling and case: 1 second is valid, 1 Second and 1s are not.
PV Link errors · dbGetLink failures · link ... not foundA record’s INP/OUT/FLNK names a record that doesn’t exist. Frequently a macro that didn’t expand, leaving a literal $(P) in the link. Check with dbpr <record> 4, which shows resolved links.
0 with severity INVALIDProcessing happened and the read failed. This is not a magnitude — it means “unknown”. Look at the device support, not the record: dbpr <rec> 4 for STAT, and asyn tracing for the wire. A screen or script that ignores SEVR will present this as a real measurement.
UDF · UndefinedThe record has never successfully processed, so its value is undefined and it reports INVALID. Correct behaviour for a freshly loaded record; cleared by PINI or the first successful scan. If it persists, nothing is processing the record — check SCAN.
SCAN is Passive — the default — and nothing is triggering it. Check that something does: a PP input link, an FLNK, I/O Intr with driver support, or a periodic rate. dbtr <record> test-processes it; if that produces a fresh value, your trigger is the problem.
caput succeeds but the value doesn’t changeClamped by DRVL/DRVH; denied by access security (check cainfo, which reports access rights); the record is disabled via DISA/DISV; a read-only gateway is in the path; or the record is PACT and discarded the write — see busy.
st.cmd: Permission deniedchmod +x st.cmd.
envPaths: No such file or directoryRun st.cmd from its own iocBoot/ioc<name>/ directory. Relative paths resolve against the working directory.
iocInitMost often <app>_registerRecordDeviceDriver pdbbase missing or misnamed in st.cmd, or a mismatch between the loaded DBD and the built binary. Rebuild, then check st.cmd against a freshly generated one from makeBaseApp.pl.
Read the log from the top. The first error matters; the last line is usually a consequence. A failed dbLoadRecords early on often produces a cascade.
The save directory isn’t writable by the IOC’s user. Autosave logs a complaint once and then silently does nothing for the rest of the run. Check after every deployment. See autosave.
No PINI, and no autosave. Both, together, are the fix.
Wrong host/port, or the device accepts a single connection that something else holds. Verify outside EPICS with nc <host> <port>.
Terminator mismatch — the most common StreamDevice problem by a wide margin. Try Terminator = CR LF;, and InTerminator/OutTerminator separately if the device differs by direction.
Leftover bytes from a reply longer than your format string consumed. Add ExtraInput = Ignore;.
A set command was given an in line but the device replies with nothing. Remove the in.
Mismatch · protocol parse errorsThe format string doesn’t match the actual reply. Turn on tracing and compare byte for byte:
asynSetTraceIOMask("PORT", 0, 0x2) # escaped ASCII: terminators become visible
asynSetTraceMask("PORT", 0, 0x9) # errors + driver I/O
Watch for units appended to numbers and for leading spaces.
LockTimeout · intermittent failures with several records on one portRecords queue for one port, and one physical link is one resource. Reduce scan rates, or fetch several values per round trip with Separator.
perl: not found, or a failing .pl scriptPerl is a hard requirement of the EPICS build system, not an optional nicety.
readline.h: No such file or directoryInstall libreadline-dev (Debian/Ubuntu) or readline-devel (RHEL family), then rebuild.
softIocPVA after a successful buildSubmodules weren’t cloned. git submodule update --init --recursive, then rebuild.
A module is missing from configure/RELEASE, or was built for a different EPICS_HOST_ARCH.
make -jRetry with plain make. Parallel builds interleave output and hide the real error.
make clean but not otherwiseStale dependency files. make distclean and rebuild.
Unsupported class file major versionThe JDK is older than the project requires. Check the project’s README for its minimum — Phoebus in particular has raised its several times. See JDK.
NoClassDefFoundError: javafx/...JavaFX is separate from the JDK since Java 11. Use a JDK that bundles it, let Maven fetch it, or install openjfx.
The mvn clean verify -f dependencies/pom.xml step was skipped. It’s easy to miss and nothing in the error hints at it. See Phoebus.
Corrupted local cache: rm -rf ~/.m2/repository and rebuild, at the cost of a re-download.
export JAVA_HOME = /path had no effectNo spaces around = in shell assignments. Bash read that as a command with three arguments and JAVA_HOME is unset. Subsequent Maven errors mention neither.
caget works in the same shellPhoebus preferences override the environment. Check settings.ini, particularly auto_addr_list. See Phoebus.
The EPICS environment variables aren’t reaching the Tomcat process. Set them in $CATALINA_HOME/bin/setenv.sh and restart. The usual cause of a dead-looking fresh PVWS.
Read $CATALINA_HOME/logs/catalina.out. Deployment failures almost always log their reason there.
Check the disconnected PV list. A PV that stopped connecting months ago is a silent gap you’ll discover exactly when you need the data. Alarm on the count.
Suspicious rather than reassuring — a dead alarm system looks exactly like a quiet machine. Check the alarm server, Kafka broker health and consumer lag, and alarm on the rate reaching zero.
NTP drift on an IOC host. Timestamps come from IOCs, so a drifted clock corrupts the archive and makes causality analysis wrong. Treat it as a data-integrity incident, not housekeeping.
Before anything on this page, for almost any EPICS problem:
dbl on the IOCcaget -acainfoSCAN set to? — dbprThose four resolve a large fraction of everything above.
tech-talk, after searching the archive. Include your Base version, module versions, platform, the actual error text, the relevant .db and st.cmd fragments, dbpr <record> 4 output, and what you already tried. A well-formed question usually gets a useful answer within hours.