Console redundancy: what actually has to be copied to the second computer

The requirement sounds simple: have a second computer, and if the first one dies, the light stays on.

Then engineering starts, and the very first decision determines whether you get a working scheme or a beautiful one that fails at the worst moment.

The temptation to resist

The obvious thought: let's synchronise state. Let the backup hold the same cues, the same effects, the same position — and at takeover simply carry on computing.

That is a trap, and here's why.

A lighting engine is not a store of values but a machine that recomputes the picture every frame: effects with their phases, fades with their timing, layers of parallel cue lists, grand master. For the backup to "simply carry on", it must reproduce not the data but the execution phase: which millisecond a fade is in, what angle the rainbow's sine sits at, which layer overrides which.

Synchronising that over the network means writing a second lighting engine that lives inside the protocol. And that second engine will inevitably diverge from the first: a packet gets lost here, a tick arrives 3 ms late there, and the divergence accumulates right up to the moment you needed it not to.

What we transfer instead

The resulting frame. The main console sends the backup its output buffer ten times a second — exactly the bytes going to the fixtures. No mechanics, no phases: the picture as it is.

The backup does not output it. It stays silent — not a single byte to USB or the network — and simply remembers the last frame. At takeover it puts that frame out and carries on with its own logic from there.

The takeover is honest without any shared model. The audience sees exactly the picture that existed a fraction of a second before the main console died, because we transfer what the audience sees, not what it was computed from.

The heartbeat travels separately, four times a second. Silence longer than two seconds (eight missed beats) triggers takeover. The threshold is configurable, but going below a second is unwise: a network spike during a content upload will produce a false trigger, and two consoles writing to the line at once is worse than a second of delay.

What one frame isn't enough for

We transferred the picture. But imagine: the main console died on cue 47, the backup took over, the light is correct. The operator presses GO — and lands on cue 12, because the backup has been sitting there all evening.

So show position travels separately from the picture: the active cue list and the cue within it, grand master, blackout, freeze. It is applied at the moment of takeover — and applied without replaying the cue. The light is already correct; running the cue again would give the audience an extra fade for no reason.

These are two distinct data streams, honestly separated: "what is on stage" and "where we are in the show".

The rig: without this the scheme doesn't work

The most common beginner mistake is a USB-DMX interface in each computer, both feeding one line.

You can't. RS-485 is a bus on which exactly one transmitter talks at a time. Two interfaces writing at once physically fight over the level on the twisted pair; DMX512 provides no arbitration whatsoever. The result isn't "last writer wins", it's garbage on the line and fixtures that twitch.

There is exactly one working scheme: both consoles output over the network (Art-Net or sACN) to shared nodes. The backup is silent on the network, so the node sees a single sender — no fight, no arbitration needed.

If you are on USB anyway, redundancy gives you only a fast manual restart: pull the interface out of the dead computer and plug it into the live one. That is a valid plan, but it has to be rehearsed rather than discovered.

Handing control back is manual

The main console is alive again. The backup hears it — the heartbeat is back. And it does nothing.

There is no automatic handback, deliberately. The moment when the main console is "alive but still loading the show or re-opening the interface" looks identical on the network to the moment it is ready. Automation would hand control back in the first second — and you would get either a blackout or two senders at once.

So a human hands it back: satisfied that the main console has really returned, they press "Return control" on the backup. One button, but at a moment that makes sense.

What the backup can't do

Running effects and an unfinished fade will not continue after takeover. The backup takes the last frame as it is — statically. If the main console died exactly halfway through a ten-second fade, that fade will not complete: the picture stays where it was caught.

That is a direct consequence of "transfer the result, not the mechanics", and I wouldn't change it: a static correct picture beats a desynchronised dynamic one.

Show position is transferred regardless, so the next GO goes where it would have gone on the main console, and the fade into that cue runs normally.

The frame mirror, incidentally, is disabled on rigs wider than 64 universes: there the traffic grows faster than the benefit. The heartbeat keeps working — the backup will take over, but starting from its own picture rather than someone else's.

A four-step check before the show

1. Roles are set, ports match: the backup shows "Main console alive".
2. The backup shows "Standing by (silent)" — and the fixtures really do listen only to the main console. Verify it, don't assume it.
3. Switch the main console off. Within a couple of seconds the backup moves to "Outputting", and the picture on stage doesn't change. If it blinked, sort it out now.
4. Switch the main console back on and press "Return control". The light stays, the main console outputs again.

Step four is the one people skip — and then learn about the problem on the evening the main console actually falls over.