Fixing Zephyr's Deferred Init: Power Domain Priority Issues

Hey everyone! Ever hit a wall while working with Zephyr RTOS, especially when trying to get fancy with power management and device initialization? Well, guys, you're not alone! Today, we're diving deep into a specific head-scratcher: the Zephyr device initialization priority sequence check for deferred-init nodes and how it clashes with power-domain-gpio setups. This isn't just some abstract coding problem; it's a real issue that can prevent your cool power-saving features from working as intended on boards like the STM32H7RS. We're talking about a situation where Zephyr's internal checks, designed to keep things orderly, actually trip up your system because they don't quite understand the special case of deferred initialization. When you're trying to gracefully power down parts of your board, say, switching between a default and sleep pincontrol state, you rely on drivers like power_domain_gpio or power_domain_gpio_monitor to manage those GPIO states perfectly. The idea is simple: tell certain peripherals, like UARTs, SPIs, or Ethernet controllers, that they belong to a specific power domain and should only initialize later when that power rail is good to go. You do this by assigning properties like power-domains = <&power_domain_3v3_line>; and zephyr,deferred-init; in your device tree. Sounds logical, right? Yet, this is exactly where the system can throw some confusing errors our way. We'll explore why this happens, how it manifests, and what the Zephyr community might need to consider to iron out these kinks. So, buckle up, because we're about to demystify this complex Zephyr interaction and hopefully pave the way for smoother, more power-efficient embedded designs!

Unraveling the Zephyr Device Initialization Puzzle: When Deferred Init Meets Power Domains

Alright, team, let's get into the nitty-gritty of why the Zephyr device initialization priority sequence check for deferred-init nodes can be such a tricky beast. At its heart, Zephyr RTOS has a super robust system for initializing all the different devices and drivers on your board. This system uses init priorities, like PRE_KERNEL_1, POST_KERNEL, and so on, to ensure that everything comes online in a logical order. For example, a basic GPIO driver needs to be up and running before a peripheral that relies on a specific GPIO pin can even think about initializing. This is usually awesome because it prevents all sorts of obscure runtime errors. However, things get a bit complicated when we introduce advanced power management features, specifically with the power-domain-gpio driver and the zephyr,deferred-init property. The power-domain-gpio driver is a crucial component when you want to dynamically control the power to certain parts of your board via GPIO pins. Imagine you have a 3.3V rail that you want to switch off when not in use to save power; this driver manages the GPIO line that controls that power rail. Naturally, this power domain controller needs to be initialized early so it can manage power before any dependent devices try to come alive. Makes total sense, right? Now, enter zephyr,deferred-init. This property is a fantastic tool for optimizing power consumption and startup times. It tells Zephyr, "Hey, don't initialize this device right away; wait until it's explicitly enabled or its power domain is active." This is especially useful for peripherals that might not be needed immediately or reside on a power rail that's initially off. The intended behavior is that devices marked with zephyr,deferred-init would effectively be skipped during the initial sequence validation because their actual initialization is, well, deferred! But here's the kicker, folks: Zephyr's current validation logic, designed to catch improper initialization sequences, doesn't seem to make this distinction. It still checks devices marked zephyr,deferred-init against the strict, immediate initialization rules, even though they're supposed to initialize later. This leads to a false-positive error where your power-domain-gpio driver, which might be initialized at POST_KERNEL+15, appears to be initialized after its dependent peripherals (like UARTs at PRE_KERNEL_1+24 or POST_KERNEL+8), even though those peripherals shouldn't be initializing yet due to the zephyr,deferred-init flag. It's a classic case of a good safety mechanism (the validation check) inadvertently clashing with an advanced feature (deferred initialization), causing headaches for developers trying to implement sophisticated power management strategies. This conflict creates a linking-stage error, preventing the application from even building, which is a major bummer when you're trying to leverage Zephyr's full power-saving potential. Understanding this fundamental disconnect is key to troubleshooting and, ultimately, fixing this issue within the Zephyr ecosystem.

Why Device Initialization Order Matters (and Why It's Breaking Things!)

Let's really dig into why device initialization order matters in an RTOS like Zephyr, and more importantly, why the current implementation is causing a breakdown when zephyr,deferred-init is thrown into the mix. In any complex embedded system, especially one as feature-rich as Zephyr, the sequence in which components and drivers are brought online is absolutely critical. Think of it like building a house, guys: you wouldn't lay the roof before the walls are up, right? Similarly, certain foundational drivers, like those managing power domains, need to be fully operational before anything that relies on that power can even begin to function. Zephyr addresses this with its granular initialization priorities: PRE_KERNEL_1, PRE_KERNEL_2, POST_KERNEL, APPLICATION, and so on, each with numerical sub-priorities. Drivers with lower priority numbers (like PRE_KERNEL_1+20) initialize earlier than those with higher numbers (like POST_KERNEL+10). This system works beautifully for most scenarios, ensuring dependencies are met. Now, let's zoom in on the power-domain-gpio driver. This driver is designed to control GPIO pins that manage the power state of entire sections of your board. For instance, if you have a 3.3V power rail that you can toggle on and off using a specific GPIO pin, the power-domain-gpio driver is the brains behind that operation. It must be initialized early enough, typically in POST_KERNEL, to be ready to assert or de-assert those GPIO lines when the system needs to switch power states. If this driver isn't ready, any device trying to use that power domain will simply fail, or worse, cause unpredictable behavior. This is precisely why it’s assigned a priority like POST_KERNEL+15. Now, here's where the plot thickens with zephyr,deferred-init. This property is a powerful optimization flag. When you add zephyr,deferred-init; to a peripheral's devicetree node, you're essentially telling Zephyr, "Hey, I know this device exists, but don't bother initializing its driver during the main boot sequence. I'll handle its initialization later, usually when its power domain is active or when my application explicitly requests it." The intended purpose here is to delay initialization, making the initial boot faster and allowing for dynamic power management. For example, if your UART is on a power rail that's initially off, marking it deferred-init means Zephyr won't even try to set it up until you turn that power rail on and trigger its initialization. The critical conflict arises because Zephyr's device initialization priority validation logic, as it stands, seems to ignore the zephyr,deferred-init flag during its dependency checks. It sees that a device (like your UART) is declared in the devicetree and has a power-domains dependency on power_domain_3v3_line. It then looks at the default initialization priority of the UART driver (e.g., PRE_KERNEL_1+24) and compares it to the power-domain-gpio driver's priority (POST_KERNEL+15). Because PRE_KERNEL_1+24 is numerically lower (meaning earlier) than POST_KERNEL+15, the validator incorrectly flags this as an error, stating that the UART is initializing before its power domain controller. This is fundamentally flawed because the zephyr,deferred-init flag should mean the UART is not initializing at PRE_KERNEL_1+24 at all; it's waiting! The validation check is essentially penalizing a perfectly valid, power-optimized configuration because it's not accounting for the deferred nature of the initialization. This isn't just a minor warning; it's an ERROR that stops the build process dead in its tracks. This impact is significant: you can't implement sophisticated power management strategies with power-domain-gpio and zephyr,deferred-init for these devices, limiting the functionality and power efficiency of your system. It forces developers to either abandon deferred initialization or find awkward workarounds, undermining the very flexibility Zephyr aims to provide.

The Specifics: STM32H7RS, UART, SPI, and Ethernet Woes

Let's get down to the very specific scenario that's causing this grief, pals, particularly for those working with an STM32H7RS controller. Our current problem revolves around trying to use the power_domain_gpio driver to intelligently manage the GPIO state of peripherals on the board. The goal is noble: to unpower certain parts of the board when they're not needed, switching between different pin control states like default and sleep to save precious milliamps. For this to work, we've identified the affected peripherals: UART, SPI, and Ethernet. These are common communication interfaces, and being able to power them down when inactive is a significant power-saving opportunity. To achieve this, we assign two crucial properties to these peripheral nodes in the device tree: power-domains = <&power_domain_3v3_line>; and zephyr,deferred-init;. The power-domains property tells the peripheral that its power is controlled by a specific power domain controller, in this case, power_domain_3v3_line. The zephyr,deferred-init; property is the instruction to Zephyr to delay the initialization of this peripheral until that power domain is active and good to go. This all makes perfect sense architecturally. However, this perfectly logical setup leads to a frustrating series of errors during the linking stage. The Zephyr build system throws up a wall of ERROR: Device initialization priority validation failed, the sequence of initialization calls does not match the devicetree dependencies. messages. Let's break down the specific errors we're seeing. For instance, you'll see lines like ERROR: /soc/serial@40004400 <uart_stm32_init> is initialized before its dependency /power-domain-3v3-line <pd_gpio_init> (PRE_KERNEL_1+24 < POST_KERNEL+15). What this is telling us is that the uart_stm32_init function, which handles the initialization of a specific UART (identified by its devicetree path /soc/serial@40004400), has a default initialization priority of PRE_KERNEL_1+24. On the other hand, our pd_gpio_init function, responsible for the power-domain-3v3-line controller, is set to initialize at POST_KERNEL+15. In Zephyr's priority scheme, a lower number means earlier initialization. So, PRE_KERNEL_1+24 is indeed earlier than POST_KERNEL+15. The validator sees this and flags it as an error, assuming the UART is actually initializing earlier than its power domain controller. This is the core of the problem: the validator doesn't take into account the zephyr,deferred-init flag. It should understand that because the UART is deferred, its PRE_KERNEL_1+24 priority is, in essence, irrelevant for the initial boot sequence. The same pattern repeats for other peripherals: ERROR: /soc/serial@40004c00 <uart_stm32_init> is initialized before its dependency /power-domain-3v3-line <pd_gpio_init> (PRE_KERNEL_1+28 < POST_KERNEL+15), ERROR: /soc/spi@40003800 <spi_stm32_init> is initialized before its dependency /power-domain-3v3-line <pd_gpio_init> (POST_KERNEL+8 < POST_KERNEL+15), and even for network components like ERROR: /soc/ethernet@40028000/mdio <mdio_stm32_init> is initialized before its dependency /power-domain-3v3-line <pd_gpio_init> (POST_KERNEL+13 < POST_KERNEL+15). In each case, a peripheral with zephyr,deferred-init and a dependency on our power_domain_3v3_line is being flagged for initializing before the power domain controller, even though the deferred-init flag should mean its initial setup is delayed. This isn't just an inconvenience; it's a functional limitation. It means we can't properly use power-domain-gpio with zephyr,deferred-init for these critical peripherals, hindering our ability to implement advanced power management features on boards like the STM32H7RS. The system becomes usable only if we remove these power management optimizations, which defeats the purpose of using them in the first place.

The Core Problem: Zephyr's Validation Logic Needs a Tweak

The core problem we've identified, friends, is that Zephyr's device initialization priority validation logic, while generally very useful, currently needs a critical tweak. It's simply not accounting for the zephyr,deferred-init property when performing its dependency checks. This oversight creates a build-time error for what is otherwise a perfectly valid and well-intentioned power management configuration. The validation system is designed to catch scenarios where a driver tries to use a resource before that resource's own driver has been initialized. For example, if a UART driver (priority X) needs a GPIO controller (priority Y), the validator ensures that Y happens before X. This is a good thing for system stability and correctness. However, when a device is marked zephyr,deferred-init, we are explicitly telling Zephyr: "Hey, this device's actual initialization routine won't run at its default priority X; it will run later, on demand." This means that for the initial boot sequence, its dependencies should essentially be ignored by the priority validator because the device itself isn't participating in that immediate initialization phase. The logical argument here is clear: nodes with the zephyr,deferred-init property should be exempt from this specific initialization priority validation check. Their initialization sequence will be managed dynamically, outside of the static boot-time priority ordering. Validating their static priority against their dependencies at compile time, when we know their initialization is deferred, leads to false errors and unnecessary roadblocks. The implications of this oversight are quite significant, especially for complex power management scenarios on modern microcontrollers like the STM32H7RS. Without this exemption, developers are stuck in a dilemma: either they sacrifice dynamic power management by not using zephyr,deferred-init (which means these peripherals are always powered on, negating power savings), or they can't build their application because of these validation errors. This directly impacts the functional limitation of the system – some features simply won't work as expected, or you're forced to implement clumsy workarounds that might reduce code clarity or introduce other issues. It prevents Zephyr users from fully leveraging the power-saving capabilities that are built into the RTOS and its power management framework. For instance, imagine a battery-powered device where a specific communication module (SPI or Ethernet) is only active intermittently. Using zephyr,deferred-init alongside a power-domain-gpio allows you to keep that module completely unpowered until it's needed, drastically extending battery life. But if the current validation prevents you from even compiling such a setup, then that power-saving potential remains untapped. Therefore, addressing this specific interaction between zephyr,deferred-init and the initialization priority validator is not just about fixing a bug; it's about unlocking Zephyr's full potential for efficient and flexible embedded system design. It's a crucial step towards making Zephyr even more robust and developer-friendly for advanced power management. This change would allow developers to confidently use these powerful features without encountering these frustrating and misleading build errors, leading to more optimized and functional products. The community's focus should be on ensuring that the tools designed for flexibility don't inadvertently create rigidity in other areas.

How to Recreate This Bug (and What You've Tried So Far)

Alright, everyone, let's walk through exactly how to recreate this bug so you can see it for yourselves and understand the precise configuration that triggers these errors. This isn't some phantom issue; it's reproducible with a specific set of Zephyr configurations and device tree entries. To get started, you'll need a Zephyr project, and for the purposes of this discussion, we're operating in an environment using Windows 11, with Zephyr SDK 0.17.3 and Zephyr v4.3.0-1747-g26721f667672. The first step is to enable a few crucial configuration options in your prj.conf file. These flags tell Zephyr that you want to use its power management capabilities, including device runtime power management and power domains. You'll need to add these lines:

CONFIG_PM_DEVICE=y
CONFIG_PM_DEVICE_RUNTIME=y
CONFIG_PM_DEVICE_POWER_DOMAIN=y
CONFIG_POWER_DOMAIN=y
CONFIG_POWER_DOMAIN_GPIO=y
CONFIG_PINCTRL_DYNAMIC=y

Let's quickly break down what these mean, guys: CONFIG_PM_DEVICE=y enables the core device power management framework. CONFIG_PM_DEVICE_RUNTIME=y allows for runtime power management, meaning devices can be powered on/off dynamically. CONFIG_PM_DEVICE_POWER_DOMAIN=y and CONFIG_POWER_DOMAIN=y enable the power domain infrastructure itself, which is what we're leveraging to group devices by their power source. CONFIG_POWER_DOMAIN_GPIO=y is the star of the show here, enabling the specific driver that uses GPIO pins to control power domains. Finally, CONFIG_PINCTRL_DYNAMIC=y enables dynamic pin control, which is often used in conjunction with power management to switch pin configurations (e.g., to a low-power state) when a device is off. Once these are set, the next critical piece is defining your power-domain-gpio node in your board's device tree (.dts or .overlay file). This node tells Zephyr how to control your 3.3V power line. Here’s an example:

power_domain_3v3_line: power-domain-3v3-line {
		compatible = "power-domain-gpio";
		enable-gpios = <&gpiof 14 GPIO_ACTIVE_LOW>;
		startup-delay-us = <0>;
		off-on-delay-us = <0>;
		status = "okay";

		#power-domain-cells = <0>;
};

In this snippet, power_domain_3v3_line is the label for our power domain. The compatible = "power-domain-gpio"; line tells Zephyr to use the GPIO-controlled power domain driver. The enable-gpios = <&gpiof 14 GPIO_ACTIVE_LOW>; part is crucial; it specifies that GPIOF pin 14, active low, controls this 3.3V line. The startup-delay-us and off-on-delay-us are set to zero for simplicity, though in a real application, you might need delays for power rail stabilization. status = "okay"; enables it, and #power-domain-cells = <0>; is a standard property for power domains. Finally, and this is where the conflict truly arises, you need to add this power domain to some peripheral nodes that you want to be conditionally powered and deferred. Let's take a UART as an example, but remember this applies to SPI and Ethernet nodes too:

&usart2 {
	[...]
	power-domains = <&power_domain_3v3_line>;
	zephyr,deferred-init;
	status = "okay";
};

Here, we're taking an existing UART (&usart2) and adding power-domains = <&power_domain_3v3_line>; to link it to our GPIO-controlled power supply. The key, however, is the zephyr,deferred-init; property. This is the instruction that should tell Zephyr to hold off on initializing usart2 until its power domain is active. Once you have this configuration in place and try to build your Zephyr application, you'll be greeted by the ERROR: Device initialization priority validation failed messages we discussed earlier. The build output will explicitly state that uart_stm32_init (or spi_stm32_init, mdio_stm32_init) is initialized before its dependency, /power-domain-3v3-line <pd_gpio_init>, despite the zephyr,deferred-init flag. We've tried implementing this exactly as per Zephyr's power management documentation, expecting the deferred initialization to bypass the static priority checks. However, the current validation logic doesn't seem to recognize this nuanced behavior, leading to the build failure. This is why we believe the validation test itself needs to be smarter about zephyr,deferred-init nodes.

The Path Forward: A Call for Zephyr's Community

So, what's the path forward for this particular conundrum in Zephyr? It's clear that the current Zephyr device initialization priority sequence check for deferred-init nodes is causing an unintended bottleneck for advanced power management implementations. The solution, guys, likely lies within the Zephyr kernel's build system, specifically in the logic that performs the device initialization priority validation. The core idea, as we've hammered home, is that this validation test should simply be skipped for any device node that has the zephyr,deferred-init; property. If a device's initialization is explicitly deferred, its default static initialization priority becomes irrelevant for the initial boot-time dependency checks. Its true initialization will be handled dynamically, outside of this static ordering. Therefore, the validation mechanism needs to be updated to recognize this special case and adjust its checks accordingly. One potential solution could involve modifying the C++ code or Python scripts responsible for generating the initialization sequence and performing these checks. A conditional statement could be added to ignore nodes marked zephyr,deferred-init when assessing build-time initialization priorities. This would allow developers to use power-domain-gpio and zephyr,deferred-init in harmony, unlocking the full power-saving potential of their applications without facing frustrating build errors. This isn't just about fixing a bug; it's about making Zephyr's power management framework even more robust and user-friendly. Robust power management is absolutely crucial in today's embedded systems, especially for battery-powered devices or those with strict energy efficiency requirements. Features like power-domain-gpio and zephyr,deferred-init are powerful tools for achieving these goals, and it's essential that they work seamlessly together. This is where the amazing Zephyr community comes in! We need to collectively encourage discussion and contributions from core Zephyr developers and power management experts. Identifying the exact location of this validation logic and proposing a well-thought-out patch would be a significant step forward. Perhaps others in the community have already encountered similar issues or have insights into the best way to implement this exemption without introducing other unintended side effects. Sharing experiences, code snippets, and potential solutions through Zephyr's mailing lists, GitHub issues, and Discord channels will accelerate the resolution process. Let's make sure that Zephyr continues to be at the forefront of embedded development, offering flexible and powerful solutions for even the most demanding applications. This minor adjustment would make a huge difference for developers striving to build truly energy-efficient and dynamic embedded products. Ultimately, by addressing this interaction, we can reinforce the importance of robust power management in Zephyr and make it even easier for everyone to build amazing, power-optimized devices. With collaborative effort, we can iron out these wrinkles and ensure that Zephyr remains a leading choice for cutting-edge embedded projects. Here's to a brighter, more power-efficient future with Zephyr!