wifi: brcmfmac: firmware crash when using WPA3/SAE

[bnicolae]

Describe the bug

The wifi firmware of rpi4 (and likely rpi5, being based on the same chip) occasionally crashes when using the integrated BCM43455 wifi chip as an AP with WPA3 SAE external authentication (the only SAE mode supported by the latest firmware). It's completely random, but usually takes several hours of continuous operation for the crash to manifest.

The original driver did not provide a trap dump. I've instrumented the code to do so:
b82a9e1

Maybe this dump is something Infineon can have a look at? Unfortunately the bug does not happen with debugging messages on, which makes it difficult to track.

Steps to reproduce the behaviour

Start the wifi as a AP (e.g. using hostapd). Configure pure WPA3/SAE:

wpa_key_mgmt=SAE
sae_password=<pwd>
sae_pwe=1
sae_groups=19
transition_disable=1

Observe the crash after a while.

Device (s)

Raspberry Pi 4 Mod. B

System

Wifi firmware version: BCM4345/6 wl0: Oct 28 2024 23:27:00 version 7.45.286 (be70ab3 CY) FWID 01-95efe7fa
Kernel version: 6.18.39

Logs

kernel: brcmfmac: brcmf_sdio_hostmail: mailbox indicates firmware halted
kernel: brcmfmac: brcmf_sdio_dump_trap: dump_trap: sh.flags=0x401 trap_addr=0x25fec8
kernel: brcmfmac: brcmf_sdio_dump_trap: firmware trap: type 0x4 epc 0x0019a330 pc 0x0019a330 lr 0x0019a30d sp 0x0025ff20
                                        r0 0x002036d8 r1 0xd110b560 r2 0xd110b530 r3 0x00199084
                                        r4 0x00000028 r5 0x00000030 r6 0x00000000 r7 0x00198718
kernel: ieee80211 phy0: brcmf_fw_crashed: Firmware has halted or crashed
kernel: ieee80211 phy0: brcmf_netdev_start_xmit: xmit rejected state=0
kernel: ieee80211 phy0: brcmf_fil_cmd_data: bus is down. we have nothing to do.
kernel: ieee80211 phy0: brcmf_fil_cmd_data: bus is down. we have nothing to do.
kernel: ieee80211 phy0: brcmf_cfg80211_stop_ap: SET SSID error (-5)
kernel: ieee80211 phy0: brcmf_fil_cmd_data: bus is down. we have nothing to do.
kernel: ieee80211 phy0: brcmf_cfg80211_stop_ap: BRCMF_C_DOWN error -5
systemd-networkd[416]: wlan0: Link DOWN
kernel: ieee80211 phy0: brcmf_fil_cmd_data: bus is down. we have nothing to do.
systemd-networkd[416]: wlan0: Lost carrier
kernel: ieee80211 phy0: brcmf_cfg80211_stop_ap: setting AP mode failed -5
kernel: ieee80211 phy0: brcmf_fil_cmd_data: bus is down. we have nothing to do.
kernel: ieee80211 phy0: brcmf_fil_cmd_data: bus is down. we have nothing to do.
kernel: ieee80211 phy0: brcmf_cfg80211_stop_ap: BRCMF_C_UP error -5
kernel: ieee80211 phy0: brcmf_fil_cmd_data: bus is down. we have nothing to do.
kernel: ieee80211 phy0: brcmf_fil_cmd_data: bus is down. we have nothing to do. 

Additional context

I decoded the firmware trap against the actual firmware image using AI (cyfmac43455-sdio.bin, rambase 0x198000):

Type 0x4 = ARM Data Abort. Faulting instruction at PC 0x19a330 = STR r3, [r4, #4], with r4 = 0x28 → the firmware tried to write to address 0x2c (a near-null/garbage pointer).

Disassembling the surrounding block (0x19a320–0x19a34e) shows it's the firmware's heap allocator free-block coalescing routine:
- it walks free-block headers reading size/next (LDR [r3,#0], LDR [r3,#4]),
- accumulates size + 8 (8-byte block header),
- merges adjacent blocks by rewriting next pointers (STR r3,[r4,#4], STR r3,[r0,#4]).
The register values confirm heap free-list corruption:
- r1 = 0xd110b560, r2 = 0xd110b530 — impossible addresses (poison), 0x30 apart.
- r4 = 0x28 — a block-header pointer that's been overwritten with the integer 40.

So the crash seems to be a firmware heap corruption, detected when the allocator next coalesces free blocks.

[bnicolae]

The culprit is likely the mainline code never receiving at least a minimal set of patches to work with the latest firmware. In particular, there are several new firmware iovars like "bss" and "extsae_pwe" that are left uninitialized. I suspect this probably skips the creation of some internal data structures on the heap and later operations trigger illegal memory overwrites (assuming these data structures exist) that eventually corrupt the heap.

In any case, I am tracking a potential fix here and will report later if it works long-term: https://github.com/bnicolae/linux-rpi/tree/wpa3

[pelwell]

That patch looks good - if it works, I'll happily merge it here.

[bnicolae]

The patch did not ultimately help. There is a clear pattern: after exactly 10h of operation, some timer-triggered cleanup is happening. I've also captured some memory contents around the trap:

kernel: brcmfmac: brcmf_sdio_hostmail: mailbox indicates firmware halted
kernel: brcmfmac: brcmf_sdio_dump_trap: dump_trap: sh.flags=0x401 trap_addr=0x25fec8
kernel: brcmfmac: brcmf_sdio_dump_trap: firmware trap: type 0x4 epc 0x0019a330 cpsr 0x0000019f spsr 0x000001bf
                                     pc 0x0019a330 lr 0x0019a30d sp 0x0025ff20 ip 0x00000000 fp 0x11cf8869
                                     r0 0x002036d8 r1 0x99b83efa r2 0x99b83ef1 r3 0x00203e98
                                     r4 0x00000001 r5 0x00000009 r6 0x00000000 r7 0x00198718
                                     r8 0x18002000 r9 0x18102000 r10 0x221b3981
kernel: brcmfmac: brcmf_sdio_dump_trap_mem: dump_trap: memory at r0=0x002036d8:
kernel: brcmfmac: brcmf_sdio_dump_trap_mem:   +0x00: 00 00 00 00 98 3e 20 00 0c 8d 04 00 29 ef 01 00
kernel: brcmfmac: brcmf_sdio_dump_trap_mem:   +0x10: 54 aa 25 00 6c ff 25 00 00 00 00 00 00 00 00 00
kernel: brcmfmac: brcmf_sdio_dump_trap_mem:   +0x20: 00 00 00 00 01 00 00 00 01 04 00 00 c8 fe 25 00
kernel: brcmfmac: brcmf_sdio_dump_trap_mem:   +0x30: 00 00 00 00 00 00 00 00 00 00 00 00 bc de 25 00
kernel: brcmfmac: brcmf_sdio_dump_trap_mem: dump_trap: memory at sp=0x0025ff20:
kernel: brcmfmac: brcmf_sdio_dump_trap_mem:   +0x00: f8 9d 25 00 44 8d 21 00 00 00 00 00 fb 91 1c 00
kernel: brcmfmac: brcmf_sdio_dump_trap_mem:   +0x10: f8 9d 25 00 44 8d 21 00 90 78 24 00 39 93 1c 00
kernel: brcmfmac: brcmf_sdio_dump_trap_mem:   +0x20: 44 57 24 00 00 00 00 00 0c 87 19 00 21 f6 01 00
kernel: brcmfmac: brcmf_sdio_dump_trap_mem:   +0x30: 19 f6 01 00 43 7f 00 00 19 7f 00 00 30 da 25 00
kernel: ieee80211 phy0: brcmf_fw_crashed: Firmware has halted or crashed

Further AI analysis reveals the following:

free()'s saved LR in trap #5's stack dump is 0x001c91fb — Tracing that address:
0x1c91e8: push {r3,r4,r5,lr}
          cbz r1, +end                  ; if (obj == NULL) return
          r0 = *(obj + 0)               ; <-- the value that becomes free()'s argument
          *(obj + 20) = 0
          if (r0 != 0) bl free(r0)      ; <-- CRASHES HERE in both traps: obj->field_0 held 7 or 9
          *(obj + 0) = 0
          *(obj + 8) = 0
          r0 = obj
          [restore lr] tail-call free(obj)   ; then frees the container itself

This helper is called from exactly 3 places, all part of a linked-list-of-entries manager (fields at struct offsets 0x430/0x434/0x438: head pointer, active-count byte, etc.) with lookup-by-MAC (memcmp at node+14, length 6) and lookup-by-handle variants, plus a "flush all" variant. The firmware's own string table confirms the subsystem: wlc_scb_cubby_reserve, wlc_scb_attach, and specifically "STA DISASSOC from non-AP SCB" — this is Broadcom's SCB (Station Control Block, the per-connected-client structure), and this code path runs on STA disassociation/removal.

I am documenting this here because Infineon (in particular @gokulkumar-ifx) is working on an upstream patch and it may be helpful to them.