Operations, Monitoring & Best Practices

Bridge the gap between a working integration and a production integration with three disciplines: observability, error handling, and operational rigor. This guide covers fleet monitoring, error taxonomy, security hardening, and the patterns that keep an integration running when something goes wrong at 3 AM.

Observability

The platform exposes three layers of observability.

Fleet and Device Metrics

Pull aggregate metrics from the platform on a schedule to understand fleet health:

• Device activity: join rate, uplink frequency, last-seen distribution
• Gateway health: connection state, packet forwarder version, uplink/downlink throughput
• Signal quality: RSSI, SNR trends per gateway and per device

Feed these into your existing observability stack (Datadog, Grafana, New Relic). Don't build a parallel dashboard inside MQcentral; use it for fleet-level views, but business metrics belong in your standard tools.

Building Dashboards

Two patterns work well for production deployments:

Periodic poll → metrics stack. Create a job that polls device and gateway statistics, schedule it to run on a fixed interval, and push the results as gauges to your observability platform. Best for rolled-up fleet metrics.

Event stream → time-series database (TSDB). Output Profiles deliver every uplink to a collector that writes to InfluxDB, Timescale, or a similar TSDB. Best for per-device telemetry with retention policies.

Most production deployments use both: high-cardinality per-device data in the TSDB, low-cardinality fleet rollups in their primary observability tool.

Alerting Thresholds

Alert recipes that catch real problems without paging on noise:

• Gateway disconnected for > 15 minutes: page immediately. Multi-gateway sites tolerate one drop; single-gateway sites are offline until it reconnects.
• Device hasn't reported in 3× its expected interval: investigate. Tune the multiplier per device class.
• Webhook delivery failure rate > 5% over 15 minutes: page immediately. Your endpoint is failing and data is accumulating in the retry queue.
• Join rate spike: investigate. Repeated joins usually indicate a connectivity or key-management problem.
• Sustained increase in payload error rate: investigate. RF conditions may have degraded or a firmware push went wrong.

Avoid per-device alerts. At fleet scale, per-device thresholds generate noise that trains on-call to ignore pages.

Error Taxonomy

Category	HTTP	Action
Authentication failed	401	Refresh token; retry once. Alert if still 401.
Permission denied	403	Do not retry. Alert: likely a configuration issue.
Validation error	400	Do not retry. Alert: almost always a code defect.
Not found	404	Do not retry. May be expected (idempotent delete) or alert.
Conflict	409	Read current state and retry with merge logic.
Server error	5xx	Retry with exponential backoff. Alert on sustained 5xx.

Build this categorization into your client library, not into every call site. A consistent retry-and-error policy across the codebase is one of the highest-leverage improvements you can make.

Security Hardening

Use scoped credentials. Do not share a single credential across every service. Issue per-service credentials with the minimum permissions each service needs. When one leaks, the blast radius is contained.

Rotate credentials on a schedule. Quarterly is a reasonable default. Tie rotation to a calendar reminder; don't rely on remembering to do it.

Verify webhook delivery authenticity. Validate that incoming HTTP deliveries originate from MachineQ using the credential mechanism configured on your Output Destination. See Data Delivery: Webhooks & Real-Time Events.

Multi-Tenant and Multi-Region Patterns

One application per tenant. Don't multiplex tenants into a single application via naming conventions. The blast radius of a misconfigured delivery integration is the entire application.

Region-specific gateways and devices. LoRaWAN frequency plans are regional. A US915 device cannot join an EU868 gateway. Enforce region consistency in your provisioning pipeline rather than assuming it.

Cross-region data plane. If you operate sites in multiple regions, run separate webhook receivers per region for latency and data residency, then aggregate downstream.

Upgrade Delegation for managed services. If you run a managed offering on top of MQcentral, use the Upgrade Delegation feature to manage firmware on customer-owned devices without taking ownership of their account.

Capacity Planning

Track and project three numbers:

Uplinks per device per day. Drives storage cost and delivery volume. At 96 uplinks/day (15-minute reporting), 10,000 devices generates approximately 960k events per day. That's within most receiver budgets, but verify your infrastructure can absorb it before launch.

Downlinks per device per day. Drives gateway air-time consumption. LoRaWAN downlink capacity is regulated by duty cycle, especially in EU868. A fleet requiring frequent downlinks may saturate a gateway's duty cycle well before it saturates the device.

Concurrent FUOTA campaigns per gateway. Only one multicast session per gateway at a time. If you operate a small number of gateways against a large update schedule, plan campaign timing carefully to avoid ERROR_SETTING_UP failures.

For sustained scale beyond a few thousand devices per gateway or a few hundred thousand events per day, engage your account team for throughput planning before hitting a limit mid-deployment.

Production Go-Live Checklist

• Authentication uses service-account credentials, not personal credentials.
• All credentials have minimum-required permissions.
• Delivery authenticity is validated on every incoming webhook or HTTP delivery.
• Webhook receiver is idempotent on (DevEUI, FCnt).
• Webhook receiver acknowledges within 1 second and processes async.
• Errors are categorized (retry, alert, ignore) consistently across the codebase.
• Fleet metrics are flowing into your observability stack.
• Alerts are configured for gateway disconnection, delivery failure rate, and join-storm anomalies.
• FUOTA campaigns have been canary-tested on at least one device.
• On-call runbook covers the top failure modes identified in staging.
• Rollback plan exists for any device-side firmware change.
• Customer-facing teams know what is launching and when.

What's Next?

• Revisit specific topics as your integration grows using the navigation above.
• Consult the API references for endpoint-level detail: MachineQ API v1, MachineQ API v2, and FUOTA API.
• Engage your account team for architecture reviews on large deployments.