EtherCAT drive communication - reliability improvements

Two related problems, seen at a customer site and reproduced on the bench:

1. The controller could get stuck in an endless retry loop bringing a drive up. It gave the drive only 2 seconds to complete its transition to Safe-Op; a drive still finishing its own internal startup work reports "busy," not "failed," but the controller treated busy as failed, restarted the connection, and reset the drive's work each time - so a slow drive could never finish. The drive is now given up to 30 seconds, while a drive that actually rejects the transition still fails immediately. This was the endless-loop fix.

2. Separately, after a drive recovered from a fault (cable pulled, power blip), the first reconnect attempt failed with a burst of confusing error messages. Cause: the drive sends an unsolicited status announcement our software did not understand, which knocked the conversation out of step - like two people each answering the other's previous question. This recovered on its own but wasted time and filled the log with noise. The software now recognizes these announcements, double-checks every answer matches its question, and if the conversation still gets out of step, gives up on that attempt within seconds and starts fresh.

Also improved: the log now tells a readable story - which step failed, why, and what the drive said, one line each - instead of dozens of raw error codes.

Result: Pulling the drive cable and plugging it back in now recovers to full operation in about 5 seconds with a short, readable log. Verified on the bench against the failure mode the customer site is seeing.

USB Host Support for the XL200 ARM (groundwork for the USB-to-Ethernet network feature)

The controller's USB port can now act as a USB "host" - meaning the XL200 can operate USB devices plugged into it, the way a PC does - in addition to its existing role as a device (the USB console released in July). The controller decides automatically at power-up which role to take based on what kind of cable is plugged in. This is the foundation for the upcoming feature that will give EtherCAT machines a network connection through a USB-to-Ethernet adapter (remote console, and eventually Eclipse connectivity).

This has been proven on the bench: the controller powers the USB port, recognizes an attached USB flash drive, and reads information from it - all while running a live EtherCAT machine network without disturbing it.

Three pre-existing problems were found and fixed along the way:

1. Plugging a USB device into a running machine used to freeze the entire controller for about 1/7 of a second - long enough for every EtherCAT drive on the machine to fault. The USB standard requires the controller to pause through several waiting periods when a device is first plugged in (letting the connection settle, resetting the device, letting it recover) - about 150 ms of mostly just waiting. As delivered by the chip vendor, all of that waiting was done inside the "drop everything" interrupt handling that runs when a device is detected, so the whole controller stood still for the duration. The fix moves this work out of the interrupt handling and into a low-priority background job: the interrupt now only takes a note that a device arrived (well under a microsecond), and the background job does the required waiting while everything else - EtherCAT, the 386 interface, all normal operation - keeps running at full speed. The same approach the vendor already used for the chip's other USB port was applied to this one; it was simply never finished on the port our hardware uses.

2. An error in the USB startup code could leave the port dead with no indication of why. Fixed: startup failures are now reported cleanly on the console and the rest of the controller keeps running.

3. The ARM processor has been using only about a third of the working memory it actually has - half of its RAM was reserved for a purpose that doesn't apply to our hardware and sat unused since the original board bring-up. This was discovered when USB support pushed memory usage over the old limit and networking failed to start. Fixed: all memory is now available. Every current and future feature on this processor benefits from this.

Also included: better diagnostics. Failures in network startup and USB power now print exactly what failed on the console instead of failing silently.

No change to normal operation: machines without anything plugged into the USB port behave exactly as before, and the USB console feature from July is unchanged.

USB Ethernet support for EtherCAT machines

1. XL200 controllers configured for EtherCAT now get a direct, inexpensive network connection. On these machines the Ethernet port is dedicated to running the drives, so until now an Eclipse connection required routing through RS485 — which in practice usually meant purchasing an expensive RS485-to-Ethernet converter. With this change, plugging a low-cost USB-to-Ethernet adapter into the controller's USB port puts the machine on the network directly, and Eclipse communicates through it exactly as it does on a standard machine — no converter hardware required. Nothing changes for non-EtherCAT machines — their built-in Ethernet port works as it always has, and this was re-tested to confirm it.

2. Setup works the same way customers already know. The IP address, subnet, and gateway are entered on the same Ethernet settings screens used today, and changes take effect immediately without restarting the machine. The adapter should be plugged in when the machine powers up. Removing it while the machine runs will not disturb the machine or the drives; plugging it back in will usually restore the connection on its own, but a power cycle may occasionally be needed.

3. A long-standing hidden problem in the controller's USB system was found and fixed. The USB hardware was never being told to watch for a device being unplugged, so an unplug went completely unnoticed by the software — the root cause of the connection failing to recover. This flaw has been present since the original vendor software shipped and would have affected any future product using USB on this processor family. It is now fixed for all of them.

4. Several smaller reliability problems were fixed along the way, including one where an internal timing miscalculation could quietly consume most of a background task's time, and one that could have corrupted memory when an unplugged device left an unfinished request behind.

5. The controller now reports much more of what it is doing over its diagnostic output — which network mode the display processor selected, when the network settings arrive, and each step of a USB device being connected or removed. These messages were essential in finding the problems above and will make future field issues much faster to diagnose.

Supported adapters currently: USB 3.0 gigabit adapters using the AX88179 chip. Support for the AX88772 family (the originally planned adapter) is prepared and will be completed when reliable hardware is available.

Remote console access over the network (telnet)

1. The controller's diagnostic output can now be viewed over the network. The XL200's ARM processor continuously reports what it is doing — network activity, drive communication status, USB events, error conditions. Until now that output was only visible with a serial cable and a laptop physically attached at the controller. Now a technician can open a standard telnet session to the machine's IP address and watch the same output live from anywhere on the network. On EtherCAT machines this works through the USB Ethernet adapter; on standard machines it works through the built-in Ethernet port — same address the machine already uses.

2. Connecting is deliberately simple for field use. Any standard telnet program works with no special setup (PuTTY, or the Windows built-in telnet client once enabled). On connection, the session first replays the most recent couple of pages of output — so a technician arriving after a problem occurred immediately sees what led up to it — and then streams live from there. Only one session is active at a time: a new connection automatically takes over from an old one, so a dropped or forgotten session can never lock anyone out.

3. It is designed to be invisible to the machine. The console server is a small, low-priority background task that sleeps between activity. It cannot interfere with drive control, Eclipse communication, or production — and it shares the network connection with Eclipse rather than competing with it.

4. Groundwork is in place for two-way use. Keystrokes typed in the session are already received and routed into the controller's input system, ready for a future feature that mirrors the display processor's debug console — where a control character switches between watching output and issuing diagnostic commands (memory dumps, task lists, etc.). Nothing responds to input yet; this release is view-only.