Hardware Test
5.5.0
9/25/2026
SCN: 4870 — XL2; Modify the ARM code to selectively enable the TCP/IP stack or EtherCat

We only have one ethernet port to use EtherCat or TCP/IP.

This change allows the type of stack to be selected during initialization. It is only available on the V5 hardware.

SCN: 5010 — XL HWTest; Touchscreen/Mouse changes

There are two main changes.

  1. The number of Touchscreen/Mouse messages recieved is displayed as Mouse Count in the Keyboard and Touchscreen test results.
  2. Five new status bits are displayed with the Keyboard PIC software version. They are Touchscreen, PS/2 Keyboard Reset, PS/2 Mouse Reset, Master PIC, and PS/2 Port Swap. Each status bit results in a Detected or Not Detected status.
SCN: 5027 — HWTest; Prevent C0000000 Task error when calibrating touch screen

The Touch Screen Calibrate option in the hardware test had the capability of causing a divide by zero task error if somehow the exact same X-Coordinate or Y coordinate was sensed for the two points.

The Divide By Zero condition, C0000000 task error, is now prevented with a test on the divisor. An error message will now display instead.

In addition, the Two sensed points, along with their expected position, the X and Y Scale along with the X and Y Offsets are now displayed to help with troubleshooting calibration issues.

SCN: 5058 — XL2 HWTest; Add debug info to comm port test

The comm port tests report passes and failures. The test alternates between expecting to see its data echoed back or not, depending on the state of the transmitter.

The test reports passes and failures. As long as the UART hardware responds to the XL, generating the appropriate interrupts, normally every other test passes. If all tests fail, it is not possible to tell if it is bcause the 386 is recieving no interrupts or if the appropriate states are not being found in the Interrupt status register in the UART.

Three new counters were added to help determine what is going on in the interrupt. The Number of ISR events are counted. The number of ISR pending (UART says it has an interrupt) are counted. The number of TX Empty interrupts are counted.

These counters will help determine not only if interrups are being recieved by the 386 but is the correct statuses are being reported by the UART hardware within the interrupt handler.

SCN: 5129 — XL2; 8F000009 Task Error when powering up

This task error occurs if it takes too long to recieve a 16bit chunk of data from the ARM processor.

We had a 5386 board that was taking too long to initialize the Ethernet Port and wasn't ready to respond with the 16 bit status when it was requested. By increasing the time that the ARM was allowed to execute the initialization, the error went away and the ARM performed all other operations flawlessly.

This change makes the additional time a permanent change from here on out.

SCN: 5346 — XL2; ARM USB console output.

The ARM processor does the bulk of the hard real time software operations on the V5 XL200. In the past it has been very difficult to provide diagnostic information about what is going on in this processor since it has no access to the serial port that we have typically been using to provide it.

The ARM now has console access using the previously un-used USB port on the right hand side of the XL, when facing the screen. For now it will provide diagnostic printouts to help diagnose EtherCAT issues, but I can be expanded for other uses in the future.

Any free application like Tera Term or PUTTY can be used to access it. Any baud rate can be used. The port will self adjust for whatever is selected.

SCN: 5349 — XL2; Additional USB infrastructure added, with some fixes along the way.

USB Host Support for the XL200 ARM (groundwork for the USB-to-Ethernet network feature)

The controller's USB port can now act as a USB "host" - meaning the XL200 can operate USB devices plugged into it, the way a PC does - in addition to its existing role as a device (the USB console released in July). The controller decides automatically at power-up which role to take based on what kind of cable is plugged in. This is the foundation for the upcoming feature that will give EtherCAT machines a network connection through a USB-to-Ethernet adapter (remote console, and eventually Eclipse connectivity).

This has been proven on the bench: the controller powers the USB port, recognizes an attached USB flash drive, and reads information from it - all while running a live EtherCAT machine network without disturbing it.

Three pre-existing problems were found and fixed along the way:

  1. Plugging a USB device into a running machine used to freeze the entire controller for about 1/7 of a second - long enough for every EtherCAT drive on the machine to fault. The USB standard requires the controller to pause through several waiting periods when a device is first plugged in (letting the connection settle, resetting the device, letting it recover) - about 150 ms of mostly just waiting. As delivered by the chip vendor, all of that waiting was done inside the "drop everything" interrupt handling that runs when a device is detected, so the whole controller stood still for the duration. The fix moves this work out of the interrupt handling and into a low-priority background job: the interrupt now only takes a note that a device arrived (well under a microsecond), and the background job does the required waiting while everything else - EtherCAT, the 386 interface, all normal operation - keeps running at full speed. The same approach the vendor already used for the chip's other USB port was applied to this one; it was simply never finished on the port our hardware uses.

  2. An error in the USB startup code could leave the port dead with no indication of why. Fixed: startup failures are now reported cleanly on the console and the rest of the controller keeps running.

  3. The ARM processor has been using only about a third of the working memory it actually has - half of its RAM was reserved for a purpose that doesn't apply to our hardware and sat unused since the original board bring-up. This was discovered when USB support pushed memory usage over the old limit and networking failed to start. Fixed: all memory is now available. Every current and future feature on this processor benefits from this.

Also included: better diagnostics. Failures in network startup and USB power now print exactly what failed on the console instead of failing silently.

No change to normal operation: machines without anything plugged into the USB port behave exactly as before, and the USB console feature from July is unchanged.

SCN: 5350 — XL2; USB Ethernet support for EtherCAT machines

USB Ethernet support for EtherCAT machines

  1. XL200 controllers configured for EtherCAT now get a direct, inexpensive network connection. On these machines the Ethernet port is dedicated to running the drives, so until now an Eclipse connection required routing through RS485 — which in practice usually meant purchasing an expensive RS485-to-Ethernet converter. With this change, plugging a low-cost USB-to-Ethernet adapter into the controller's USB port puts the machine on the network directly, and Eclipse communicates through it exactly as it does on a standard machine — no converter hardware required. Nothing changes for non-EtherCAT machines — their built-in Ethernet port works as it always has, and this was re-tested to confirm it.

  2. Setup works the same way customers already know. The IP address, subnet, and gateway are entered on the same Ethernet settings screens used today, and changes take effect immediately without restarting the machine. The adapter should be plugged in when the machine powers up. Removing it while the machine runs will not disturb the machine or the drives; plugging it back in will usually restore the connection on its own, but a power cycle may occasionally be needed.

  3. A long-standing hidden problem in the controller's USB system was found and fixed. The USB hardware was never being told to watch for a device being unplugged, so an unplug went completely unnoticed by the software — the root cause of the connection failing to recover. This flaw has been present since the original vendor software shipped and would have affected any future product using USB on this processor family. It is now fixed for all of them.

  4. Several smaller reliability problems were fixed along the way, including one where an internal timing miscalculation could quietly consume most of a background task's time, and one that could have corrupted memory when an unplugged device left an unfinished request behind.

  5. The controller now reports much more of what it is doing over its diagnostic output — which network mode the display processor selected, when the network settings arrive, and each step of a USB device being connected or removed. These messages were essential in finding the problems above and will make future field issues much faster to diagnose.

Supported adapters currently: USB 3.0 gigabit adapters using the AX88179 chip. Support for the AX88772 family (the originally planned adapter) is prepared and will be completed when reliable hardware is available.

SCN: 5352 — XL2; Telnet Access for ARM Diagnostics and debugging

Remote console access over the network (telnet)

  1. The controller's diagnostic output can now be viewed over the network. The XL200's ARM processor continuously reports what it is doing — network activity, drive communication status, USB events, error conditions. Until now that output was only visible with a serial cable and a laptop physically attached at the controller. Now a technician can open a standard telnet session to the machine's IP address and watch the same output live from anywhere on the network. On EtherCAT machines this works through the USB Ethernet adapter; on standard machines it works through the built-in Ethernet port — same address the machine already uses.

  2. Connecting is deliberately simple for field use. Any standard telnet program works with no special setup (PuTTY, or the Windows built-in telnet client once enabled). On connection, the session first replays the most recent couple of pages of output — so a technician arriving after a problem occurred immediately sees what led up to it — and then streams live from there. Only one session is active at a time: a new connection automatically takes over from an old one, so a dropped or forgotten session can never lock anyone out.

  3. It is designed to be invisible to the machine. The console server is a small, low-priority background task that sleeps between activity. It cannot interfere with drive control, Eclipse communication, or production — and it shares the network connection with Eclipse rather than competing with it.

  4. Groundwork is in place for two-way use. Keystrokes typed in the session are already received and routed into the controller's input system, ready for a future feature that mirrors the display processor's debug console — where a control character switches between watching output and issuing diagnostic commands (memory dumps, task lists, etc.). Nothing responds to input yet; this release is view-only.

SCN: 5354 — XL2; USB Hub support

USB Hub Support for the XL200 ARM (lets several USB devices share the controller's single USB port)

The XL200's USB port could previously work with only one device at a time. It can now work through a standard USB hub, so several devices can be used at once. This was confirmed on the bench with a network adapter and a flash drive plugged into a hub at the same time - both fully working, while the machine's normal EtherCAT motion communication continued without any disturbance.

Reaching that point required fixing four long-standing defects in the USB software supplied by the chip vendor. None had ever caused trouble before, because no hub had ever been connected to this product, so these code paths had never actually run. The most significant of them quietly consumed an internal resource every time a hub was unplugged; left alone it would eventually have stopped USB working until the controller was power cycled. That path has now been unplugged and reconnected ten times in a row with no sign of degradation.

A separate and more serious problem was found and fixed along the way. The coprocessor sets aside a small workspace for handling hardware interrupts, and the extra activity a hub creates overflowed it - which then corrupted the operating system's own internal data. When that happened the coprocessor detected the fault, logged a task error with its diagnostic details, and restarted itself, which is the designed behavior. What made it hard to trace was that the corruption struck whatever the coprocessor happened to be doing next, so the details recorded pointed at an unrelated part of the system rather than at the real cause. That workspace is now four times larger, in all five controller programs.

One follow-up item came out of this: the way these faults are recorded can be improved so that the logged details identify the true source when the fault happens inside interrupt handling, which would have saved considerable time here.

Groundwork for USB keyboard and mouse support is included but is not yet active. Those devices will be recognized when plugged in, but they do not yet send keystrokes or pointer movement to the display. That work is fully specified and is the next step.

Current limitation: testing used a hub without its own power supply and two devices. How many devices the port can support on its own power has not yet been measured, so a hub with its own power supply is the safer recommendation until that is characterized.

SCN: 5357 — XL2; ARM Task Error parsing improvements.

Improved Fault Diagnosis on the XL200 ARM Coprocessor (accurate task error records, and keeping the information needed to read them later)

When the ARM coprocessor detects a fault it records a task error with diagnostic details and restarts itself, which is the designed behavior. Two problems were making those records misleading, and both are now fixed.

First, if the fault happened while the coprocessor was in the middle of handling a hardware interrupt, the recorded details described an unrelated part of the system instead of the code that actually failed. The record looked perfectly plausible, which is worse than no record at all - it points an investigation in the wrong direction. This was a significant part of why the recent USB hub fault took a full day to track down.

Second, two of the recorded values were stored under each other's labels, so anyone reading a record was told the failure was in one area when the evidence actually pointed somewhere else. That also contributed to the same wasted day.

Fault records now identify the code that actually failed, whether the fault occurred in normal program flow or inside interrupt handling, and they include which interrupt was active at the time. The record's format and size are unchanged, so everything that reads or displays these records continues to work exactly as before. Records logged before this change still have the two values reversed.

The build now also keeps a second piece of information under version control. A fault record identifies the failing code by its numeric address, which by itself means nothing - translating an address into the name of the function requires the "map file" the build produces, and that map is only valid for the exact program image it was built with. Until now only the program image was kept and the map existed solely on the machine that built it, so a fault reported from the field months later could not reliably be traced back to the code that caused it. The map is now saved alongside the program image automatically at build time, and a copy also remains on the build machine for immediate use.

Both fault cases were verified on the bench by deliberately causing each kind of fault and confirming the recorded details matched the known cause.

This is diagnostic accuracy only - it changes nothing about how the product runs, and it does not prevent faults. It makes the next one much faster to diagnose, including well after the software has shipped.

SCN: 5359 — XL2; Moved to newer version of USB stack for Keyboard and Mouse support.

The ARM coprocessor's USB software was a version the supplier had stopped developing and replaced. We have shipped releases on it and it worked — the USB serial console, telnet, and the USB-to-Ethernet adapter all functioned. The trouble only began when we started hot-plugging devices for the keyboard and mouse work: that exercises parts of the software a device present from power-up never touches, and there the old design does too much work with interrupts disabled. Long enough to interfere with the timing of machine control.

Since the supplier had already replaced that software, and the replacement is built around not doing that, we moved to it rather than patching a version with no future.

The move is now complete and tested on the bench. The USB-to-Ethernet adapter identifies itself, negotiates a network link, takes its address from the 386, and answers pings — the deciding test, since that adapter is how the controller reaches the network in EtherCAT mode, where the built-in Ethernet port is taken over by EtherCAT itself.

Two faults are fixed along the way. Plugging in a USB device could stall the ARM long enough that the 386 stopped receiving responses and reported an error, which in turn stopped EtherCAT and faulted the drive; plugging and unplugging no longer disturbs machine control. Separately, high-speed USB devices could appear to disconnect and reconnect repeatedly on their own.

The diagnostic USB serial console had to be moved to the new software at the same time, because the old and new versions could not coexist in one build. It works as before.

Keyboard and mouse support remains switched off and is the next stage.

SCN: 5363 — XL2; USB Stack now enumerates USB Keyboard and Mouse

USB keyboards and mice now work on the ARM board, and they can be unplugged and plugged back in while the machine is running. The mouse and Keyboard events still need to be wired into the the PEG task to be used. That will be the next step.

Three problems were fixed:

  1. The board could lock up when a mouse was unplugged and reconnected. Some USB devices describe themselves incorrectly on reconnect, and the manufacturer-supplied USB software got stuck in an endless loop reading that bad description. Because that software runs at a high priority, nothing else could run — the diagnostic console went silent even though the board itself was still alive and still talking to the drive. It now recognizes the bad description and stops rather than getting stuck.

  2. Keypresses were not being reported at all. Two causes: the board asked the keyboard for more data at once than a keyboard is able to send, and it never told the keyboard to use the simple standard reporting format. Both corrected.

  3. After three plug/unplug cycles, nothing further could connect. Internal resources were not being released when a device was removed, so the board gradually ran out of them. They are now released properly.

Keyboard and mouse are tracked separately, so both can be connected at once and either can be removed without disturbing the other. That separation is also the groundwork for barcode scanner support, since scanners connect to the system as keyboards.

Testing: about a dozen unplug/replug cycles with no lockup, followed by confirmed live mouse movement and keystrokes. EtherCAT communication stayed running throughout.

One note for future maintenance: two pieces of the manufacturer's USB code look like straightforward bugs but are actually load-bearing — correcting them introduces real defects. Both are now commented in place so a future cleanup doesn't reintroduce the problem.

SCN: 5368 — XL2; USB Keyboard and Mouse Support

USB keyboards and mice now work on the controller, both during startup and in normal operation.

Two paths were needed. Once the machine is running, keyboard and mouse events ride along with the data the ARM coprocessor already sends every millisecond, costing nothing extra when nobody is typing. But the machine can stop during startup and wait for a keypress, before that once-per-millisecond exchange begins — so a second path lets the startup code collect events directly until the normal one takes over, handing off without dropping or duplicating anything.

Three faults found and fixed along the way. Leftover test code on the ARM was consuming events before the main processor could read them, which made the mouse jerky. A startup flag was stored in battery-backed memory and so kept its value from the previous power-up, permanently disabling the startup path after the first boot. And diagnostic printing shared a task with keyboard handling, where it could stall it — that would have affected machines in the field with no PC attached, so it is now off by default.

Existing keyboard behaviour is untouched: the ARM produces exactly the same messages the keyboard processor has always produced, so the display software needed no changes.

SCN: 5379 — HWTest; Added USB Port Test to V5 HWTest

The V5 hardware has a USB port that we are now using, which means we needed to add a Hardware Test for it.

The hardware test can now check the USB port. With a keyboard, mouse, memory stick, scanner and network adapter plugged into the test fixture, a new screen shows what was found and whether the port passed. There is nothing to switch on — it tests itself and updates as devices are plugged and unplugged.

If something is wrong it names the part to look at rather than just reporting a failure: the power switch, the connector wiring, or the device itself. It then asks the operator to unplug and replug the adapter once, which checks a connector signal that cannot be tested any other way, and confirms every device comes back afterwards.

USB keyboards and mice also work in the existing keyboard test, which proves the whole path from the device through to the main processor.

Verified on hardware: full ladder to PASS, the remove and refit sequence, individual devices hot-plugged on the hub, and both keyboard and mouse reaching the keyboard test.

The test shows the following information, depending on what is connected to the Port. For each supported (ish) device that is detected, we will show the Vendor and Product ID. Follwing the instructions on the screen produces a Passed or Failed Result. An OTG adapter, a HUB and a Flash Drive are required for the Hardware Test to pass. The detection of these three devices fully tests the USB hardware. If a USB Key Board or Mouse is detected, they can be tested in the Keyboard Test.

F - Flash Test R - RAM Test X - Non-Volatile Ram Test V - Video Test I - IO Test E - Test Encoder H - Hole Detect Test P - Pic18F252 Test U - USB Port Test C - Comm Port Test A - Analog Test G - Port G and H Test T - Set Time & Date B - Burn In Test O - SERCOS Test Q - Return to Previous Menu Enter Command:u

USB Port Test Q - Return to Previous Menu

Role .............. HOST OTG Adapter ....... FITTED VBUS .............. GOOD Switch Fault ...... NO Port .............. CONNECTED Speed ............. HIGH Hub ............... 7 sockets, 5 used 0BDA:5411 Keyboard .......... 045E:082C Mouse ............. 10C4:8108 Flash Drive ....... 090C:1000 Scanner ........... 1FBB:3600 Ethernet .......... 0B95:1790 Errors ............ 0 Last: 00

Remove the OTG adapter <<<

SCN: 5382 — HWTest; USB Test improvements

Corrects the time limit on the flash drive check in the USB board test.

The limit was three seconds. A working flash drive that takes a little over four seconds to finish starting up was therefore reported as a failure every single time it was tested. Because the failure looked exactly like a hardware fault, it would have sent technicians to investigate boards that were perfectly good — the most expensive mistake a board test can make.

The limit is now ten seconds, more than double what the slowest drive on hand actually needs. A genuinely dead port still fails in ten seconds, which is quick enough for production use.

The test now also reports when a drive passes but took an unusually long time to answer. The healthy fixture drive completes its slowest step in a small fraction of a second, so this message only appears for a drive that is wearing out — which gives us the chance to replace the fixture drive before it starts producing confusing intermittent failures.

Finally, while a check is waiting on a drive, the test now prints a line each second saying what it is waiting for. Nothing on good hardware ever takes a whole second, so this stays silent in normal use; when it does appear, it makes the difference between "this drive is slow" and "this test has locked up" obvious immediately, rather than something that has to be worked out from scratch.

SCN: 5389 — XL2; Transition to 12bit encoder counter chip hardware.

The 5387 board, since V4, has used two 32-bit quadrature encoder counter chips to support the four incremental encoder ports. These parts are obsolete and we have no source for them any longer.

We have found a source for the 12-bit version of these chips. It is apparently a custom build of the 12-bit version that is housed in the 32-bit package. This allows us to use the chip without a board layout change. The only issue is that we have to modify our software to use the 12-bit version. We have to read the encoder often enough so that we never see more than 2048 counts between reads or our math will not work properly to calculate the change between reads.

Through empirical testing we have determined the Max Encoder rate to be 2MHz. If we read the encoder every 100usec, that calculates out to a maximum number of counts equal to 200 per read, well below the 2048 limit.

There will be a new limitation introduced. Any 5387 board with the new 12-bit chip will no longer be compatible with older software versions. The cutoff will be V4.110.0 and V5.110.0. Any software older than this will be incompatible with 5387 boards containing the new chip. Software versions newer than these will work with either version of the chip.

On the old chip we have a Bogus encoder count read of 10,0000 per read. We will be lowering that threshold to 2000 counts per read.

We have modified the Closed Loop Controller code to read at 100usec, even when there are no Open Loop Targets.

The OL gets similar treatment but with a much simpiler change since it was already written to read the encoder more than once per sample.