STATERA Library · Series 08 · Volume 02 · Chapter 10 · EN
Source: https://github.com/Statera-Library/08_Vol_02_Embedded_Software_Engineering/blob/main/02_10_OTA_Firmware_Update/02_10_OTA_Firmware_Update_EN.md

Volume 02 Embedded Software Engineering

10. OTA Firmware Update

10.01 OTA Update Architecture Overview: System Perspective

An over-the-air update architecture for embedded robotic systems is not simply a mechanism for transferring a new firmware image to a device. It is a coordinated system that connects software development, release management, update infrastructure, communication networks, embedded update agents, bootloaders, persistent storage, runtime supervision, and recovery mechanisms. The architecture must preserve system availability and safety while allowing software to evolve throughout the operational life of the robot.

From a system perspective, the OTA lifecycle begins in the development and release environment. Firmware, operating-system components, device drivers, configuration data, middleware, application software, and sometimes AI-related runtime packages are built through controlled pipelines. Each release receives explicit version information and compatibility metadata so that the infrastructure can determine which hardware revisions, bootloader versions, operating environments, and dependent components are permitted to install it.

Before deployment, an update artifact should be transformed into a trusted release package rather than distributed as an arbitrary binary file. The package normally contains the payload together with metadata describing its version, target platform, dependencies, required storage, installation procedure, and compatibility constraints. Cryptographic hashes and digital signatures establish integrity and authenticity, allowing the robot to reject packages that are corrupted, modified, unsigned, or issued by an unauthorized release authority.

The server side forms the management plane of the OTA architecture. It maintains software artifacts, device identities, release policies, deployment status, and update history. Robots periodically communicate their hardware identity, installed software versions, operational state, and update capabilities. The management system compares this inventory with deployment policies and determines whether a particular release is applicable, enabling updates to be controlled by robot model, hardware generation, software baseline, operational group, or fleet segment.

Communication between the update infrastructure and the robot must be treated as an unreliable distributed-system boundary. Wireless connectivity may disappear while a robot is moving, packets may be delayed, and a large image may require several sessions to download. Consequently, robust OTA implementations support resumable transfer, local caching, integrity checks, retry policies, bandwidth control, and timeout handling. Download failure must remain fundamentally different from installation failure so that interrupted communication does not unnecessarily affect the running system.

Inside the robot, an OTA update agent provides the primary interface between remote management and local embedded software. The agent receives deployment instructions, evaluates prerequisites, downloads the artifact, verifies metadata and signatures, and stages the package in persistent storage. It should also examine battery level, available disk space, network condition, current mission state, temperature, and other platform-specific constraints before permitting installation. This separates update policy from the low-level mechanisms that actually replace software.

Installation must be isolated from normal execution as far as practical. Updating an executable image while it is actively controlling motors, sensors, communication buses, or safety-related functions can produce undefined behavior. The OTA architecture therefore coordinates application shutdown, service transitions, storage synchronization, and reboot sequencing. For robots, the update manager may additionally require docking, charging, mission completion, or entry into a maintenance state before modification of critical firmware is allowed.

A/B partitioning provides an important architectural pattern for high-reliability updates and is explicitly separated as the next detailed topic in the chapter structure. The currently active software executes from one partition while a new image is written to the inactive partition. The original image remains untouched during installation. After verification, the boot configuration selects the candidate partition for the next startup, creating a transactional boundary between the known working software and the proposed replacement.

The bootloader is therefore part of the OTA trust and recovery chain rather than merely an initialization component. Earlier chapters establish secure boot, dual-partition boot design, and bootloader-based recovery as related architectural foundations. At startup, the boot process identifies the selected image, validates required security properties, loads the candidate software, and participates in determining whether the new version has successfully reached an acceptable operational state.

Successful booting alone should not automatically mean that an update succeeded. A newly installed image may initialize the kernel but fail to communicate with motor controllers, load configuration data, start ROS 2 services, access sensors, or establish required network interfaces. A health-monitoring mechanism should therefore evaluate post-update readiness. Only after essential services and platform functions satisfy defined acceptance criteria should the candidate image be committed as the new known-good software baseline.

If post-update validation fails, the recovery architecture must restore a previously verified configuration. Rollback may be triggered by repeated boot failures, watchdog resets, application crashes, health-check failures, or expiration of a confirmation timer. The detailed rollback mechanism is treated separately later in the chapter, but at the architectural level it closes the OTA transaction: deployment is incomplete until the system can either confirm the new software or reliably return to a recoverable state.

Security must span the complete update chain. Transport encryption protects communication, but secure transport alone cannot establish that firmware itself is trustworthy. Release credentials, signing keys, update servers, device identities, boot verification, and local storage form a chain of trust. The chapter structure consequently extends OTA into firmware signing, integrity verification, supply-chain attack defense, and hardware security mechanisms, showing that update security is a lifecycle property rather than a single cryptographic check.

OTA architecture must also account for heterogeneous software boundaries. A robot may contain an embedded Linux computer, GPU edge processor, safety MCU, motor controllers, sensor firmware, and peripheral microcontrollers, each with different update mechanisms. Updating these components independently can create incompatible intermediate states. A system-level updater therefore needs dependency rules, compatibility matrices, ordered installation, and coordinated reboot behavior so that the complete robot reaches a validated software configuration rather than merely a collection of individually updated devices.

Operational policy becomes especially important when the robot cannot immediately stop working. The chapter structure explicitly distinguishes partial and deferred updates during robot operation. Noncritical packages may sometimes be downloaded or staged while missions continue, whereas safety-related firmware can require a controlled maintenance window. Separating download, verification, staging, activation, and confirmation allows the architecture to minimize downtime without sacrificing deterministic transition into a known software state.

At fleet scale, OTA becomes an orchestration problem rather than a device-level installation problem. The release should normally progress through controlled populations such as development units, laboratory robots, validation fleets, canary devices, and progressively larger production groups. Telemetry from each stage provides evidence about boot success, crashes, resource consumption, communication stability, and mission behavior. The chapter therefore concludes the OTA sequence with fleet orchestration and staged rollout strategy as a distinct system-level concern.

A complete OTA architecture can consequently be understood as a closed lifecycle: build, package, sign, publish, assign, download, verify, stage, activate, reboot, validate, commit, monitor, and recover when necessary. Its primary objective is not simply remote software delivery, but controlled transformation from one known system configuration to another. In robotics, where embedded software interacts directly with physical motion and safety mechanisms, that distinction makes OTA an integral element of reliability, security, maintainability, and lifecycle engineering.

10.02 Safe Update Design with A/B Partition

A/B partitioning is a fault-tolerant update architecture designed to prevent a failed software installation from making an embedded system permanently unavailable. Instead of replacing the currently running system image, storage is divided into two bootable slots, commonly called Slot A and Slot B. One slot remains active while the other becomes the update target, preserving a known-good software environment throughout most of the OTA process.

During normal operation, the bootloader selects one partition as the active slot and the operating system, middleware, drivers, and robot applications execute from that image. The inactive slot is not required for current operation and can therefore receive a new software release without modifying the running image. This separation transforms an OTA update from an in-place overwrite into a controlled transition between two independently bootable system states.

A typical update begins when the OTA agent receives an authorized release and determines that the device satisfies installation prerequisites. After download and cryptographic verification, the new image is written to the inactive partition. If Slot A is currently active, Slot B becomes the installation destination, and vice versa. The active partition should remain unchanged so that interruption during writing does not destroy the software configuration currently responsible for operating the robot.

The inactive image must be completely validated before it becomes a boot candidate. Validation may include checking package metadata, image size, cryptographic hashes, digital signatures, filesystem consistency, version compatibility, and hardware compatibility. The updater should also confirm that all required blocks were written successfully. Only after these checks pass should boot metadata be modified to request startup from the newly installed slot during the next controlled reboot.

Boot metadata is a critical element of the design because it records which slot should start, which image is known to be good, and whether a candidate image is still undergoing validation. Implementations may additionally maintain boot-attempt counters, update states, rollback flags, and version information. These records must themselves be protected against corruption because incorrect boot metadata can prevent recovery even when both partition images remain physically intact.

The transition to the new partition should normally use a trial-boot state rather than immediately declaring the update successful. After installation into the inactive slot, the bootloader marks that slot as the next candidate and reboots the device. The candidate image then starts under probation. Until runtime validation confirms acceptable behavior, the previous active partition remains the known-good image and provides the recovery path if the candidate fails.

A robust bootloader therefore distinguishes between bootable, candidate, confirmed, and failed software states. When starting a candidate partition, it may decrement an allowed boot-attempt counter or start a confirmation mechanism. If the system repeatedly resets before successful confirmation, the bootloader interprets this behavior as evidence of update failure. It can then select the previous known-good partition automatically instead of repeatedly attempting to boot defective software.

Post-boot validation extends the safety mechanism beyond simple kernel startup. The candidate system should demonstrate that critical services initialize correctly and that required hardware interfaces remain available. In a robot, validation may examine communication with motor controllers, safety controllers, sensors, network interfaces, storage, middleware, and essential applications. The update is confirmed only when predefined health criteria indicate that the robot has reached a stable operational state.

The confirmation operation represents the commit point of the A/B update transaction. Once the candidate image passes required checks, the system records the new partition as known-good and clears temporary trial information. The former active slot can then remain available as a backup until a later update cycle replaces it. Consequently, the two slots alternate roles over successive releases, with the active image protected while its counterpart receives the next candidate version.

Failure handling must be deterministic. If power disappears while the inactive slot is being written, the existing active slot should still be bootable. If verification fails, activation should not occur. If the candidate fails during boot or runtime validation, boot control should return to the previous known-good image. These behaviors are the central reason for using A/B partitioning: failure at different update stages should lead to a defined recovery state rather than an unusable device.

Power-loss resilience also requires careful handling of metadata and persistent configuration. Updating the image and changing the boot-selection state must not be treated as one uncontrolled operation. Critical metadata should be written using atomic or otherwise recoverable techniques so that unexpected reset cannot leave ambiguous slot selection. The design should always provide enough persistent information for the bootloader to determine which image is trusted and what recovery action should be taken.

Not every form of persistent data should be duplicated together with the system partitions. Device identity, calibration parameters, security credentials, robot configuration, operational logs, and user-generated data may need dedicated persistent partitions that survive switching between Slot A and Slot B. Their schemas must remain compatible with both the old and candidate releases during the transition, because an automatic rollback is ineffective if the new software irreversibly modifies shared data required by the previous version.

Partition sizing must account for the complete update lifecycle rather than only the current image size. Both slots require enough capacity for future operating-system and application growth, while additional storage may be required for downloaded packages, temporary files, logs, and recovery information. Embedded platforms with limited flash therefore face a tradeoff between the strong recovery properties of complete A/B images and the storage overhead created by maintaining two bootable software environments.

Security controls remain active throughout the slot transition. A bootloader should not boot an image merely because it resides in the selected partition. The candidate must satisfy the platform\'s secure-boot and signature-verification requirements, while rollback protection may prevent unauthorized installation of vulnerable historical versions. The boot-selection mechanism, version metadata, signing keys, and recovery logic therefore belong to the same chain of trust as the firmware image itself.

In robotic systems, partition switching should also be coordinated with physical operating state. Activation may be prohibited while the robot is navigating, manipulating an object, controlling safety-critical equipment, or operating with insufficient battery capacity. The image can be downloaded and prepared beforehand, but reboot and slot transition can be deferred until the robot reaches a safe maintenance condition, such as mission completion, docking, charging, or an explicitly authorized service window.

A/B partitioning ultimately provides a transactional model for embedded software evolution. The old image represents the committed state, the inactive partition receives the candidate state, trial boot performs controlled execution, health validation determines acceptance, and confirmation commits the new state. If any critical stage fails, rollback restores the previous configuration. This architecture provides the foundation for reliable OTA deployment where recoverability is more important than simply installing software as quickly as possible.

10.03 Delta Update Algorithms: bsdiff / xdelta [w/Code]

Delta updating reduces OTA transmission size by distributing only the information required to transform an installed software image into a newer version. Instead of downloading a complete firmware or filesystem image for every release, the update system compares an old version with a new version and generates a compact delta package. On the target device, this package is applied to the verified base image to reconstruct the exact new software image before activation.

The basic delta-update workflow contains two computationally different stages. During release preparation, a server-side tool receives the old image and the new image and calculates their differences. The resulting patch contains instructions and changed data rather than a complete duplicate of the target image. During installation, the embedded device combines the original image with the downloaded patch, producing a reconstructed candidate image that can subsequently enter the normal OTA verification and activation process.

Delta updating is particularly valuable when communication bandwidth, download time, or data cost is constrained. A small firmware modification may change only a limited portion of the logical program, while a full-image update would transmit hundreds of megabytes or several gigabytes. When an efficient differencing algorithm identifies reusable content from the previous release, the transferred artifact can become substantially smaller, reducing network utilization and shortening the package acquisition phase of an OTA deployment.

The effectiveness of delta generation depends strongly on how software images change between builds. Small source-code modifications do not always produce small binary differences because compiler output, link addresses, compressed filesystems, timestamps, metadata, or reordered blocks can propagate changes across a binary image. Consequently, OTA engineering should evaluate actual generated patch sizes rather than assuming that a minor source change automatically creates a correspondingly small delta package.

bsdiff is a binary differencing approach designed to construct compact patches by finding relationships between an old binary and a new binary. Conceptually, it analyzes matching and differing regions, then encodes how bytes from the old file can be reused or adjusted to reproduce the new file. The generated patch separates transformation information from additional data, allowing substantial portions of an existing image to contribute to reconstruction instead of being retransmitted.

A typical bsdiff workflow can be represented conceptually as bsdiff old.bin new.bin update.patch\, followed on the target side by bspatch old.bin reconstructed.bin update.patch\. The first operation is normally performed in the build or OTA backend, where computational resources are comparatively abundant. The second operation reconstructs the target image on the embedded platform. Production implementations should wrap these operations with version checks, signature verification, storage protection, and failure handling rather than treating patch application as an isolated command.

bsdiff can achieve attractive patch sizes for binaries with significant similarity, but its resource requirements must be considered when targeting constrained embedded devices. Patch creation can require substantial processing and memory, while reconstruction also needs sufficient workspace and storage for the base data, patch, and resulting image. For this reason, delta efficiency cannot be evaluated only by network savings; RAM consumption, flash capacity, execution time, power-loss behavior, and target hardware capability must also be included.

xdelta represents another practical family of binary delta techniques and is commonly associated with VCDIFF-style encoding. Rather than requiring the entire target software to be transmitted, it encodes instructions describing how portions of source data can be copied and how new data should be inserted. This copy-and-add model is well suited to general binary content and provides a useful foundation for distributing differences between related software artifacts while preserving exact reconstruction of the target version.

Conceptually, an xdelta workflow uses an existing release as the source dictionary and generates a delta against the new release. The device later decodes that delta using exactly the expected source version. This dependency is fundamental: a patch produced for version A to version B should not normally be applied to version C. The OTA metadata must therefore identify the required base version, target version, patch format, algorithm version, and integrity information before reconstruction is attempted.

Choosing between bsdiff, xdelta, and a complete-image update should be based on system constraints rather than patch size alone. bsdiff may be attractive when minimizing binary difference size is important, whereas xdelta-style approaches can provide practical streaming-oriented behavior and controlled resource usage depending on implementation. A full image remains valuable when versions differ substantially, when reconstruction resources are limited, or when maintaining many version-specific delta paths becomes operationally inefficient.

A production OTA backend may therefore generate several artifacts for the same release. For example, devices running the immediately previous version may receive a small delta, while older installations receive another compatible delta or fall back to the complete image. The deployment service selects an artifact according to reported device state. This requires an explicit update graph in which software versions are nodes and valid delta transformations are directed edges connecting supported source and destination versions.

The target must verify its base image before applying a delta. If the installed source differs from the exact image assumed during patch generation, reconstruction can fail or produce incorrect output. The updater should compare trusted version metadata and preferably a cryptographic digest of the source image. After reconstruction, the complete candidate image must also be hashed and verified against the expected target digest, ensuring that delta processing produced exactly the release generated by the build system.

Cryptographic signing should protect the delta package and its metadata in the same way that a complete OTA artifact is protected. A small patch is still executable software transformation data and therefore belongs inside the software supply-chain trust boundary. Authentication should occur before patch application, while final-image integrity verification should occur after reconstruction. Secure boot can then independently verify the resulting candidate during startup, maintaining trust across download, reconstruction, storage, and execution.

Delta reconstruction integrates naturally with A/B partition architecture. The robot can continue operating from the active slot while the updater reads the required base content and constructs the new image in the inactive slot. The active known-good image remains protected during the transformation. Once reconstruction and integrity verification succeed, the inactive slot becomes a boot candidate and follows the trial boot, health validation, confirmation, and rollback process used for safe A/B updates.

Power interruption must be assumed during patch application. The updater should never leave the only bootable image dependent on a partially applied transformation. Temporary output, transactional metadata, checkpoints where appropriate, and inactive-slot reconstruction can prevent interrupted patching from damaging the known-good state. After restart, the update agent should be able to determine whether reconstruction can safely resume, must restart, or should discard incomplete output and continue operating from the existing release.

Delta updates also introduce lifecycle complexity because every supported source version can potentially require a different patch. Maintaining deltas from all historical releases quickly becomes expensive in storage, testing, and deployment logic. Practical systems often support deltas only across a limited number of recent versions and use a full-image fallback for older or unexpected installations. This keeps the update graph manageable while preserving bandwidth savings for the majority of regularly maintained devices.

For robotic fleets, the strongest design combines delta algorithms with release orchestration rather than treating them as standalone compression techniques. The backend determines the correct source-to-target path, the robot validates its current state, the delta reconstructs an exact candidate image, and A/B protection preserves recoverability. bsdiff and xdelta therefore optimize the transfer stage of OTA, while signing, integrity verification, partition safety, rollback, and fleet policy provide the larger architecture required for dependable embedded deployment.

10.04 Firmware Signing and Integrity Verification [w/Code]

Firmware signing establishes a cryptographic trust relationship between the organization that produces embedded software and the device that executes it. An OTA system should never assume that a firmware image is trustworthy merely because it arrived from an expected server. The device must independently verify that the software was authorized by a trusted release authority and that its contents have not changed since the approved package was created.

Integrity verification and authenticity verification address different security questions. A cryptographic hash such as SHA-256 can detect whether firmware bytes have changed, but an attacker who replaces the firmware may also replace an unprotected hash. A digital signature binds the expected digest to a trusted private signing key. Verification with the corresponding public key therefore determines both whether the image is intact and whether it originated from an authorized signing process.

The signing process normally occurs after a firmware image has passed the required build, integration, and release tests. A cryptographic digest is calculated over the defined firmware content, and the digest or a structured representation containing it is signed using a protected private key. The resulting signature, version information, hardware compatibility data, image length, algorithm identifiers, and other release attributes can then be stored in authenticated firmware metadata accompanying the update artifact.

Public-key cryptography enables signing authority to remain separated from deployed devices. The private key used to authorize production firmware should remain inside controlled signing infrastructure and should never be distributed to robots. Devices require only the corresponding public key or a trusted certificate chain for verification. Compromise of a robot therefore does not automatically provide the attacker with the capability to generate firmware that other robots will accept as authentic.

Algorithms such as RSA and elliptic-curve digital signature schemes can be used for embedded firmware signing, with the exact choice determined by platform requirements, cryptographic libraries, hardware acceleration, certification constraints, and lifecycle expectations. The security architecture should explicitly identify supported algorithms, key sizes, signature formats, and hash algorithms. Algorithm selection must also consider long-lived devices that may require secure software maintenance for many years.

The private signing key is one of the most security-sensitive assets in the OTA supply chain. Storing it as an ordinary file on a developer workstation or CI server creates an unnecessary compromise path. Production systems should isolate signing operations using tightly controlled infrastructure and, where appropriate, a hardware security module. Access control, audit logging, separation of duties, key backup, rotation, revocation, and incident-response procedures should be defined as part of key lifecycle management.

Firmware verification begins before installation. After the OTA agent receives a package, it should validate package structure and authenticated metadata, identify the expected signing authority, calculate the required cryptographic digest, and verify the digital signature. If any check fails, the update must be rejected before it modifies the bootable software state. Download completion therefore represents only acquisition of an artifact, not authorization to execute or activate that artifact.

Verification should cover exactly the content whose integrity is required. Ambiguity about which bytes are signed can create serious implementation weaknesses. Firmware format, header fields, executable regions, configuration elements, version metadata, and optional components must have clearly defined authentication boundaries. Canonical metadata encoding is useful because both the signing environment and embedded verifier must calculate the signature over precisely the same logical representation.

A second integrity check is important after the firmware has been written or reconstructed. Flash programming errors, interrupted writes, storage corruption, or defects during delta reconstruction can produce an image different from the signed target even when the downloaded package itself was valid. Calculating and comparing the final image digest before activation confirms that the candidate stored in the target partition is exactly the software release expected by the deployment system.

Delta updates require particularly careful integrity handling because the downloaded patch and the final executable image are different artifacts. The patch should be authenticated before application, the installed base image should match the required source version, and the reconstructed image should match the expected target digest. This creates a verification chain from trusted base image through authenticated transformation to the exact target image rather than assuming successful patch execution implies correct firmware.

A/B partition architecture provides a natural location for secure verification. The active slot continues running as the known-good system while the authenticated update is written to the inactive slot. Signature and integrity checks are completed before that slot becomes a boot candidate. If verification fails, the boot configuration remains unchanged. This preserves the currently trusted software while preventing an invalid candidate from crossing the activation boundary.

Secure boot extends firmware authentication into the startup sequence. OTA verification confirms an update before activation, while the bootloader independently verifies executable software when the device starts. The root public key or a hash of trusted key material may be anchored in protected hardware, immutable boot ROM, one-time-programmable memory, TPM, or another root-of-trust mechanism. This prevents an attacker from bypassing the OTA agent and directly installing unauthorized executable content.

Version control must also participate in firmware authentication because a correctly signed image may still be unsafe if it contains an old vulnerability. Anti-rollback protection associates trusted firmware with a monotonic security version or comparable policy. The bootloader or secure update mechanism rejects images below the permitted security level even when their signatures are mathematically valid, preventing legitimate historical firmware from being abused as a downgrade attack.

Key rotation is essential for long-lived embedded products. A signing key may expire, be replaced according to policy, or require emergency revocation after suspected compromise. Devices therefore need a controlled mechanism for updating trusted verification keys without allowing arbitrary trust changes. A root key can authorize intermediate release keys, allowing operational keys to rotate while a more strongly protected root of trust remains stable across the device lifecycle.

Certificate-based architectures can extend this hierarchy by allowing firmware signatures to be validated through a chain of trust. A device may trust a manufacturer root while individual products, environments, or release organizations use subordinate certificates. Such separation reduces the impact of a single operational key compromise and supports distinct development, testing, and production authorities. Trust policies must nevertheless remain simple enough for reliable implementation in constrained embedded software.

Failure behavior should always default toward preservation of the known-good state. Invalid signatures, mismatched hashes, unsupported algorithms, malformed metadata, expired credentials, incompatible hardware identifiers, or prohibited versions should stop activation and generate diagnostic evidence. The robot may continue operating with its current trusted image while reporting the failure to the OTA management system, avoiding unnecessary service interruption caused by rejection of an untrusted update.

For robotic fleets, signing and integrity verification ultimately form the security gate between software delivery and physical execution. Release pipelines establish what software is authorized, cryptographic signatures preserve that authorization across untrusted networks and storage, embedded verification checks it before installation, and secure boot enforces it again at startup. Combined with A/B partitions, rollback control, protected keys, and audited release processes, this creates a continuous chain of trust from firmware creation to robot operation.

10.05 RAUC Framework Configuration and Bundle Creation [w/Code]

RAUC is an embedded Linux update framework designed to install authenticated software bundles while preserving a recoverable system state. Rather than treating an OTA update as a simple file replacement, RAUC models the device as a set of managed installation slots and applies a signed bundle according to a predefined system configuration. This makes it especially suitable for robots and industrial devices that require controlled A/B updates, bootloader coordination, and deterministic recovery behavior.

A typical RAUC deployment separates the currently running root filesystem from an inactive update target. Two root filesystem slots, commonly represented as rootfs.0 and rootfs.1, alternate between active and inactive roles across successive releases. Additional slots can represent boot partitions, application data, or other updateable components. RAUC understands these relationships through its system configuration and uses them to determine where bundle images should be installed.

The central RAUC configuration is normally defined in system.conf\. This file describes the compatible device identity, bootloader integration, keyring location, and available slots. Each slot definition identifies properties such as device path, slot type, boot name, and parent relationships where applicable. The configuration therefore translates the physical storage layout of an embedded target into a logical update model that RAUC can safely manage during installation.

The compatible\ value provides an important compatibility boundary between update bundles and target hardware. A bundle declares the system compatibility for which it was produced, while the target configuration declares its own compatible identity. RAUC compares these values before installation. This prevents a correctly signed but technically incompatible bundle intended for another board, product generation, or robot controller from being installed accidentally.

Bootloader integration allows RAUC to connect software installation with the actual boot decision. Depending on the platform, RAUC can cooperate with supported boot mechanisms so that a newly written slot becomes the next boot candidate. The bootloader still performs the startup transition, while RAUC maintains information about slot state and update results. This separation enables installation logic and low-level boot control to participate in a common A/B recovery strategy.

Slot definitions must correspond accurately to the target\'s real partition layout. For example, rootfs.0 may reference one block device partition while rootfs.1 references another partition of equal or sufficient capacity. Incorrect device mappings can overwrite the wrong storage area and defeat the safety properties of A/B updating. Partition tables, bootloader configuration, filesystem generation, and RAUC configuration should therefore be designed and validated as one integrated storage architecture.

RAUC distributes update content using bundles. A bundle is a structured update artifact containing one or more software images together with a manifest and cryptographic authentication information. Instead of independently transferring an arbitrary root filesystem image and metadata, the release pipeline packages them into a unit that RAUC understands. The bundle becomes the controlled interface between software production infrastructure and the update mechanism running on the embedded target.

The bundle manifest describes what the package contains and where its images logically belong. It can identify bundle compatibility, version information, descriptive metadata, and image entries corresponding to configured slot classes. An image intended for the root filesystem, for example, is associated with the appropriate rootfs slot class rather than a hard-coded active partition. RAUC then resolves the correct inactive installation target according to the device\'s current slot state.

Bundle creation begins with reproducible software artifacts generated by the embedded Linux build process. A root filesystem image may originate from Yocto, Buildroot, or another controlled image-generation pipeline. The release process places the required image and manifest into a bundle input directory and then invokes the RAUC bundle-generation mechanism. The resulting .raucb\ artifact packages the release into a form suitable for authenticated deployment to compatible devices.

A simplified release environment can conceptually execute rauc bundle bundle-input update.raucb\ after the manifest, filesystem image, certificate, and signing-key configuration have been prepared. In production, the signing key should not be treated as an ordinary development asset. Bundle creation should be integrated with a protected release pipeline in which signing authority, access permissions, audit records, and key-management procedures are explicitly controlled.

RAUC uses cryptographic signing to ensure that a bundle originates from an accepted release authority and has not been modified after signing. The target device is configured with trusted verification material through its keyring, while the private signing key remains in the release environment. When an installation is requested, RAUC verifies the bundle before writing its contents into managed slots. An untrusted or corrupted bundle is therefore rejected before it crosses the installation boundary.

Installation can be initiated conceptually with a command such as rauc install update.raucb\. RAUC first examines the bundle, checks compatibility and authentication, determines the current booted slot, selects an appropriate inactive target, and installs the associated image. The active root filesystem remains protected during this process. Successful writing alone does not necessarily mean that the new software has been permanently accepted, because activation and post-boot confirmation remain separate lifecycle stages.

Slot status provides the operational view required to understand the device\'s update state. Commands such as rauc status\ can expose information about configured slots, the currently booted system, and relevant slot conditions. This information is valuable for local diagnostics and remote OTA management because the fleet backend needs to distinguish devices that merely downloaded an update from those that installed, booted, confirmed, failed, or rolled back a release.

After installation, the bootloader can start the newly selected slot as a candidate system. The updated software should then perform application-specific health checks before being considered reliable. A robot may verify middleware startup, network connectivity, motor-controller communication, safety interfaces, sensor availability, and required services. Once these checks succeed, the update process can mark the candidate as good according to the boot and RAUC integration strategy used by the platform.

If the candidate repeatedly fails to boot or cannot satisfy the required health criteria, the system should preserve or return to the previous known-good slot. RAUC does not eliminate the need for carefully designed bootloader recovery semantics; instead, it participates in a broader architecture containing A/B partitions, boot-attempt control, watchdog supervision, application health validation, and rollback policy. Reliable OTA behavior therefore depends on the complete platform integration rather than the update framework alone.

For robotic products, RAUC is most effective when incorporated into the build and deployment lifecycle rather than added only as a device-side utility. The build system creates deterministic images, the release pipeline constructs and signs bundles, the OTA service distributes them, RAUC verifies and installs them into controlled slots, and the bootloader performs the transition. Runtime validation then determines whether the new software is committed or the previous system is restored.

A well-designed RAUC implementation consequently connects storage architecture, release packaging, cryptographic trust, boot control, and operational recovery into one update workflow. Its bundle model provides a repeatable deployment unit, while slot management protects the running system from direct replacement. Combined with A/B partitioning, secure signing, compatibility checks, health monitoring, and rollback logic, RAUC provides a practical foundation for maintainable and recoverable embedded Linux OTA updates.

10.06 Mender Platform Integration: Server/Client Setup [w/Code]

Mender is an OTA software update platform designed to coordinate artifact creation, device management, deployment control, and reliable installation across embedded Linux systems. Its architecture separates server-side fleet orchestration from client-side update execution. This separation allows a central service to manage software releases across many robots while each embedded device independently performs authentication, download, installation, reboot, and update-state reporting.

The server side acts as the management plane for the OTA lifecycle. It maintains device identities, software artifacts, deployment definitions, inventory information, and update status. Operators upload approved software releases and select which devices or groups should receive them. The server then coordinates deployment while collecting progress information from clients, providing centralized visibility into heterogeneous robots operating with different software versions and update histories.

A Mender client runs on each managed embedded Linux device and communicates with the server through authenticated network connections. The client periodically checks whether a deployment applies to the device, reports inventory and software state, downloads assigned artifacts, and executes the corresponding installation workflow. Because the device initiates normal update communication, the architecture works well for robots located behind firewalls or network address translation where inbound management connections may be undesirable.

Device onboarding establishes the trust relationship between a new client and the management service. When the client first contacts the server, it presents identity information that allows the device to be recognized and authorized according to the configured deployment model. After acceptance, the server can associate the device with inventory attributes and deployment groups. This creates a managed identity that connects the physical robot with its software state and OTA lifecycle records.

A Mender Artifact is the controlled software delivery unit used by the platform. Rather than distributing an arbitrary filesystem image, the release pipeline packages update content together with metadata describing artifact identity, compatible device types, and payload information. The artifact format al...