Skip to content

arch/arm/stm32h7: Fail a serial open when no DMA stream is free. - #22022

Open
maxikrie wants to merge 1 commit into
apache:masterfrom
maxikrie:stm32h7-serial-dma-ebusy
Open

maxikrie wants to merge 1 commit into
apache:masterfrom
maxikrie:stm32h7-serial-dma-ebusy

Conversation

@maxikrie

Copy link
Copy Markdown
Contributor

Summary

On STM32H7, opening a serial port with RX or TX DMA crashes the system if its DMA controller has no free stream left.

stm32_dmachannel() on the H7 (DMAMUX) hands out the first free stream of the requested controller and returns NULL when there is none; it does not block as on F4/F7. up_dma_setup() did not check the handle and passed it to stm32_dmasetup(), which asserts (DEBUGASSERT(handle != NULL)) or, without assertions, dereferences NULL.

This PR makes up_dma_setup() return -EBUSY in that case, so open() fails with EBUSY. On failure it releases a TX stream that was already taken (when only the RX stream is missing) and undoes up_setup() for a non-console port (clock, pins), so a later open() works once a stream is free. The H5 driver already returns -EBUSY here.

This is item 3 of #20244, which says it "can land first", independent of the build-time budget proposed there.

Not changed here: the H7 SPI driver also only DEBUGASSERTs its DMA handles in spi_bus_initialize(). With assertions off it already falls back to spi_exchange_nodma() when a handle is NULL, so it does not crash; with assertions on it panics at boot. Its comment still says stm32_dmachannel() blocks, which is the F4/F7 behaviour.

Impact

STM32H7 U(S)ARTs with CONFIG_USARTn_RXDMA/CONFIG_UARTn_TXDMA. Only the path where no DMA stream is available changes: open() now returns EBUSY instead of crashing. No change when a stream is available. No Kconfig, API or build changes.

Testing

NUCLEO-H753ZI, master fc8d354, nucleo-h753zi:nsh + CONFIG_STM32_USART6, CONFIG_STM32_DMA1, CONFIG_USART6_RXDMA, CONFIG_USART6_TXDMA, CONFIG_DEBUG_ASSERTIONS. For the test, board.h maps USART6 to DMA1 (DMAMAP_USART6_RX/TX = DMAMAP_DMA12_USART6xX_0), and a test-only NSH app dmahog [n] takes n (default: all) DMA1 streams with stm32_dmachannel() and dmahog free releases them. Neither is part of this PR.
Toolchain: Arm GNU Toolchain 13.3.Rel1. tools/checkpatch.sh -g HEAD passes.

step master this PR
echo hello > /dev/ttyS1 (USART6) ok ok
dmahog (all 8 DMA1 streams), then echo hello > /dev/ttyS1 assertion in stm32_dmasetup(), system panics open failed: 16 (EBUSY), repeatable
dmahog free, then echo hello > /dev/ttyS1 - ok
dmahog 7 (TX gets the last stream, RX fails), echo x > /dev/ttyS1 - open failed: 16; afterwards dmahog again gets all 8 streams (TX stream released), RCC_APB2ENR.USART6EN = 0 (port shut down)

Logs

master
nsh> dmahog
dmahog: holding 8 DMA1 streams
nsh> echo hello > /dev/ttyS1
dump_assert_info: Current Version: NuttX  13.1.0 fc8d354f9a-dirty Nov 18 2026 17:58:01 arm
dump_assert_info: Assertion failed panic: at file: :0 task: <noname> process: <noname> 0x8011c4d
up_dump_register: R0: 00000000 R1: 38001354 R2: 38001354  R3: 00000000
...
up_dump_register: IP: 00000000 SP: 38001330 LR: 08000eb3  PC: 0800227a

PC 0x0800227a is stm32_dmasetup(), LR in up_dma_setup() (stm32_serial.c:2261), called from uart_open().

this PR
nsh> dmahog
dmahog: holding 8 DMA1 streams
nsh> echo hello > /dev/ttyS1
nsh: echo: open failed: 16
nsh> echo again > /dev/ttyS1
nsh: echo: open failed: 16
nsh> dmahog free
dmahog: released all
nsh> echo hello > /dev/ttyS1
nsh>
nsh> dmahog 7
dmahog: holding 7 DMA1 streams
nsh> echo x > /dev/ttyS1
nsh: echo: open failed: 16
nsh> dmahog free
dmahog: released all
nsh> dmahog
dmahog: holding 8 DMA1 streams

up_dma_setup() passed the handle from stm32_dmachannel() straight to
stm32_dmasetup().  On the H7, stm32_dmachannel() does not wait for a
stream: it returns NULL when every stream of the controller is taken.
The open() then asserted in stm32_dmasetup(), or with assertions off
dereferenced NULL.

Return -EBUSY instead.  If the RX stream is the one that is missing,
release the TX stream that was already taken, and undo up_setup() for
a port that is not the console, so a later open() can succeed once a
stream is free.

Assisted-by: Claude:claude-opus-5-5
Signed-off-by: Max Kriegleder <max.kriegleder@gmail.com>
@github-actions github-actions Bot added Arch: arm Issues related to ARM (32-bit) architecture Size: S The size of the change in this PR is small labels Nov 18, 2026
@github-actions

Copy link
Copy Markdown

MemBrowse Memory Report · Explore what changed →

No memory changes detected for:


Catch memory regressions in CI with MemBrowse. Free for open source.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Arch: arm Issues related to ARM (32-bit) architecture Size: S The size of the change in this PR is small

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant