Troubleshooting

With the troubleshooting feature, the FUOTA server automatically identifies the failure reason for each device that fails to update. Generic reasons like "The device failed to set up in time" or "Final version check fail" usually require a human to inspect the log stream and gateway status. Instead, when a device fails, the FUOTA server runs a list of automated checks, records each check's status, and summarizes the real failure reason. This guide describes those checks.

Before You Begin

You need:

• A campaign that has run and produced at least one failed device. See Quick Start Guide.
• $MQ_TOKEN exported from the authentication step. See Quick Start Guide.

How Checks Run

The checks run as soon as a device transitions into the FAIL ("stage": 0) stage. If one check fails, the FUOTA server may stop processing the rest of the checks. Each check has a stable id, while its name property may change.

Note: The list of checks may change as the feature is developed.

Checks Before Fragment Setup

The following checks may run before the fragment setup. They usually describe why a device ended up with "Failed to set up in time".

Check	Description
Device Online	Verifies the device was online since the campaign started (were there any uplinks?). Also checks whether the device rejoined during setup, which can cause previously set up data to be lost on the device side.
Uplink Frequency	Verifies the uplink frequency was sufficient to communicate the multicast session and fragmentation setups.
Downlink Sent	Verifies the setup downlinks were sent by the gateway. When gateway capacity is exceeded, downlinks may fail to send.
Link Budget	Checks whether the device RSSI is < -90. If downlinks are being sent but the device is not responding, low RSSI can indicate the device is not receiving the downlink, because the downlink link budget and device receiver sensitivity are lower than the uplink.
Device PER	Checks whether the device PER is > 10%. With enough uplinks, the device may have sent the response, but high PER means it never reached the FUOTA server.

Checks After Fragmentation

The following checks usually run after the fragmentation session is complete, when a device responds with a version that does not match the expected version.

Check	Description
Multicast Downlinks Sent	Verifies the multicast downlinks were sent as expected and the gateway was online.
Rejoin During Multicast	If the device rejoined before the main fragmentation session was over (before the redundancy packets), it was likely reset and lost the multicast session. This is device/application specific, such as a dying battery or a manual reset, and the server cannot correct it at runtime.
Excessive Uplinks	Sending uplinks during the campaign greatly reduces the time slots available to receive multicast. Each uplink is followed by two receive windows, usually on different frequencies than the multicast group, so a single uplink could miss between 1 and 3 multicast downlinks.

Post Campaign Version Check

When a device fails the update after the multicast session, the FUOTA server checks a few conditions:

• Did the device rejoin? A rejoin event often signals that the update was received and the device restarted to apply it.
• Did the device send any uplinks post-update? If the device is not sending uplinks, it may be updated but restricted in uplink capacity or gateway availability.
• Does the device answer DevVersionReq messages? The device may be online and communicating but, for some reason, not responding to the outgoing DevVersionReq messages.

What's Next?

• Choosing Multicast Setup Time: Avoid "failed to set up in time" errors by allowing enough time.
• Gateway Selection: Improve link budget and downlink delivery with the right gateways.
• Managed Campaigns: See how repeatedly failing devices become problematic devices.