5024
XL2; Loss of precision when editing large unsigned integers.
XL200

The integer edit controls have to handle the data conversions that happen when the controller's display format requires conversions between its internal units and the users desired display units. This requires a conversion to floating point and then back to integer after rounding, which can result in a loss of precision, depending on the magnitude of the number involved. In most cases, the integer values are not large enough to encounter this issue.

The only case this has become an issue is when editing the controller switch setting. In this case, the integer values require all 32 bits, which cannot be converted to a float without loss of precision. This is also the only case when an unsigned integer to string conversion has been required.

A bit in the Format configuration for edit fields has been reserved as a special case to specify that a field is to be treated as an unsigned integer. In this case, no conversion to float is performed. It is expected that when this flag is used, no display unit conversion will be required.

Model Version Released
XL2OL 5.98.00 3/24/2025
XL2CL 5.98.00 3/24/2025
XL220OL 5.98.00 3/24/2025
XL220CL 5.98.00 3/24/2025
XL2OL 4.98.00 3/24/2025
XL2CL 4.98.00 3/24/2025
XL220OL 4.98.00 3/24/2025
XL220CL 4.98.00 3/24/2025