PVE 6 - bnx2x driver issue with pve-kernel-5.4.140-1-pve

Eneko Lacunza elacunza at binovo.es
Wed Oct 6 12:29:19 CEST 2021 

Hi all,

We're upgrading our office cluster to latest PVE 6, and seems that bnx2x 
driver in pve-kernel-5.4.140-1-pve has some kind of bug, interfaces 
don't show after boot.

If we revert back to our previous working kernel, 
pve-kernel-5.4.124-1-pve, issue disappears and network card works as 
expected.

Network card is:

06:00.0 Ethernet controller: Broadcom Limited NetXtreme II BCM57810 10 
Gigabit Ethernet (rev 10)
     Subsystem: Broadcom Limited NetXtreme II BCM57810 10 Gigabit Ethernet
     Flags: bus master, fast devsel, latency 0, IRQ 38
     Memory at f2000000 (64-bit, prefetchable) [size=8M]
     Memory at f1800000 (64-bit, prefetchable) [size=8M]
     Memory at f2870000 (64-bit, prefetchable) [size=64K]
     Expansion ROM at f7980000 [disabled] [size=512K]
     Capabilities: [48] Power Management version 3
     Capabilities: [50] Vital Product Data
     Capabilities: [a0] MSI-X: Enable+ Count=18 Masked-
     Capabilities: [ac] Express Endpoint, MSI 00
     Capabilities: [100] Advanced Error Reporting
     Capabilities: [13c] Device Serial Number 6c-b3-11-ff-fe-4e-b1-28
     Capabilities: [150] Power Budgeting <?>
     Capabilities: [160] Virtual Channel
     Capabilities: [1b8] Alternative Routing-ID Interpretation (ARI)
     Capabilities: [1c0] Single Root I/O Virtualization (SR-IOV)
     Capabilities: [220] #15
     Capabilities: [300] #19
     Kernel driver in use: bnx2x
     Kernel modules: bnx2x

(8 instances in lspci, but only 2 physical ports)

Bad kernel startup:
Oct  6 11:50:42 amaiur kernel: [    1.117525] bnx2x: QLogic 5771x/578xx 
10/20-Gigabit Ethernet Driver bnx2x 1.713.36-0 (2014/02/10)
Oct  6 11:50:42 amaiur kernel: [    1.117744] bnx2x 0000:06:00.0: msix 
capability found
Oct  6 11:50:42 amaiur kernel: [    1.117945] bnx2x 0000:06:00.0: part 
number 0-0-0-0
Oct  6 11:50:42 amaiur kernel: [    1.192271] bnx2x: probe of 
0000:06:00.0 failed with error -22
Oct  6 11:50:42 amaiur kernel: [    1.192279] bnx2x 0000:06:00.1: msix 
capability found
Oct  6 11:50:42 amaiur kernel: [    1.192411] bnx2x 0000:06:00.1: part 
number 0-0-0-0
Oct  6 11:50:42 amaiur kernel: [    1.268397] bnx2x: probe of 
0000:06:00.1 failed with error -22
Oct  6 11:50:42 amaiur kernel: [    1.268409] bnx2x 0000:06:00.2: msix 
capability found
Oct  6 11:50:42 amaiur kernel: [    1.268546] bnx2x 0000:06:00.2: part 
number 0-0-0-0
Oct  6 11:50:42 amaiur kernel: [    1.268674] bnx2x 0000:06:00.2: VN 1 
is in a single function mode, aborting
Oct  6 11:50:42 amaiur kernel: [    1.269442] bnx2x: probe of 
0000:06:00.2 failed with error -1
Oct  6 11:50:42 amaiur kernel: [    1.269531] bnx2x 0000:06:00.3: msix 
capability found
Oct  6 11:50:42 amaiur kernel: [    1.270096] bnx2x 0000:06:00.3: part 
number 0-0-0-0
Oct  6 11:50:42 amaiur kernel: [    1.270223] bnx2x 0000:06:00.3: VN 1 
is in a single function mode, aborting
Oct  6 11:50:42 amaiur kernel: [    1.270282] bnx2x: probe of 
0000:06:00.3 failed with error -1
Oct  6 11:50:42 amaiur kernel: [    1.270287] bnx2x 0000:06:00.4: msix 
capability found
Oct  6 11:50:42 amaiur kernel: [    1.270395] bnx2x 0000:06:00.4: part 
number 0-0-0-0
Oct  6 11:50:42 amaiur kernel: [    1.270522] bnx2x 0000:06:00.4: VN 2 
is in a single function mode, aborting
Oct  6 11:50:42 amaiur kernel: [    1.270578] bnx2x: probe of 
0000:06:00.4 failed with error -1
Oct  6 11:50:42 amaiur kernel: [    1.270583] bnx2x 0000:06:00.5: msix 
capability found
Oct  6 11:50:42 amaiur kernel: [    1.270697] bnx2x 0000:06:00.5: part 
number 0-0-0-0
Oct  6 11:50:42 amaiur kernel: [    1.270823] bnx2x 0000:06:00.5: VN 2 
is in a single function mode, aborting
Oct  6 11:50:42 amaiur kernel: [    1.270880] bnx2x: probe of 
0000:06:00.5 failed with error -1
Oct  6 11:50:42 amaiur kernel: [    1.270884] bnx2x 0000:06:00.6: msix 
capability found
Oct  6 11:50:42 amaiur kernel: [    1.270992] bnx2x 0000:06:00.6: part 
number 0-0-0-0
Oct  6 11:50:42 amaiur kernel: [    1.271119] bnx2x 0000:06:00.6: VN 3 
is in a single function mode, aborting
Oct  6 11:50:42 amaiur kernel: [    1.271174] bnx2x: probe of 
0000:06:00.6 failed with error -1
Oct  6 11:50:42 amaiur kernel: [    1.271179] bnx2x 0000:06:00.7: msix 
capability found
Oct  6 11:50:42 amaiur kernel: [    1.271291] bnx2x 0000:06:00.7: part 
number 0-0-0-0
Oct  6 11:50:42 amaiur kernel: [    1.271417] bnx2x 0000:06:00.7: VN 3 
is in a single function mode, aborting
Oct  6 11:50:42 amaiur kernel: [    1.271473] bnx2x: probe of 
0000:06:00.7 failed with error -1

Working kernel startup:
Oct  6 12:19:21 amaiur kernel: [    0.994653] bnx2x: QLogic 5771x/578xx 
10/20-Gigabit Ethernet Driver bnx2x 1.713.36-0 (2014/02/10)
Oct  6 12:19:21 amaiur kernel: [    0.994862] bnx2x 0000:06:00.0: msix 
capability found
Oct  6 12:19:21 amaiur kernel: [    0.995082] bnx2x 0000:06:00.0: part 
number 0-0-0-0
Oct  6 12:19:21 amaiur kernel: [    1.096763] bnx2x 0000:06:00.0: 32.000 
Gb/s available PCIe bandwidth (5 GT/s x8 link)
Oct  6 12:19:21 amaiur kernel: [    1.096784] bnx2x 0000:06:00.1: msix 
capability found
Oct  6 12:19:21 amaiur kernel: [    1.096951] bnx2x 0000:06:00.1: part 
number 0-0-0-0
Oct  6 12:19:21 amaiur kernel: [    1.172648] bnx2x 0000:06:00.1: 32.000 
Gb/s available PCIe bandwidth (5 GT/s x8 link)
Oct  6 12:19:21 amaiur kernel: [    1.172672] bnx2x 0000:06:00.2: msix 
capability found
Oct  6 12:19:21 amaiur kernel: [    1.172831] bnx2x 0000:06:00.2: part 
number 0-0-0-0
Oct  6 12:19:21 amaiur kernel: [    1.172963] bnx2x 0000:06:00.2: VN 1 
is in a single function mode, aborting
Oct  6 12:19:21 amaiur kernel: [    1.175139] bnx2x: probe of 
0000:06:00.2 failed with error -1
Oct  6 12:19:21 amaiur kernel: [    1.175143] bnx2x 0000:06:00.3: msix 
capability found
Oct  6 12:19:21 amaiur kernel: [    1.175258] bnx2x 0000:06:00.3: part 
number 0-0-0-0
Oct  6 12:19:21 amaiur kernel: [    1.175388] bnx2x 0000:06:00.3: VN 1 
is in a single function mode, aborting
Oct  6 12:19:21 amaiur kernel: [    1.175446] bnx2x: probe of 
0000:06:00.3 failed with error -1
Oct  6 12:19:21 amaiur kernel: [    1.175450] bnx2x 0000:06:00.4: msix 
capability found
Oct  6 12:19:21 amaiur kernel: [    1.175562] bnx2x 0000:06:00.4: part 
number 0-0-0-0
Oct  6 12:19:21 amaiur kernel: [    1.175693] bnx2x 0000:06:00.4: VN 2 
is in a single function mode, aborting
Oct  6 12:19:21 amaiur kernel: [    1.175749] bnx2x: probe of 
0000:06:00.4 failed with error -1
Oct  6 12:19:21 amaiur kernel: [    1.175753] bnx2x 0000:06:00.5: msix 
capability found
Oct  6 12:19:21 amaiur kernel: [    1.175863] bnx2x 0000:06:00.5: part 
number 0-0-0-0
Oct  6 12:19:21 amaiur kernel: [    1.175993] bnx2x 0000:06:00.5: VN 2 
is in a single function mode, aborting
Oct  6 12:19:21 amaiur kernel: [    1.176058] bnx2x: probe of 
0000:06:00.5 failed with error -1
Oct  6 12:19:21 amaiur kernel: [    1.176062] bnx2x 0000:06:00.6: msix 
capability found
Oct  6 12:19:21 amaiur kernel: [    1.176183] bnx2x 0000:06:00.6: part 
number 0-0-0-0
Oct  6 12:19:21 amaiur kernel: [    1.176315] bnx2x 0000:06:00.6: VN 3 
is in a single function mode, aborting
Oct  6 12:19:21 amaiur kernel: [    1.176375] bnx2x: probe of 
0000:06:00.6 failed with error -1
Oct  6 12:19:21 amaiur kernel: [    1.176380] bnx2x 0000:06:00.7: msix 
capability found
Oct  6 12:19:21 amaiur kernel: [    1.176491] bnx2x 0000:06:00.7: part 
number 0-0-0-0
Oct  6 12:19:21 amaiur kernel: [    1.176618] bnx2x 0000:06:00.7: VN 3 
is in a single function mode, aborting
Oct  6 12:19:21 amaiur kernel: [    1.176671] bnx2x: probe of 
0000:06:00.7 failed with error -1
Oct  6 12:19:21 amaiur kernel: [    1.181002] bnx2x 0000:06:00.0 
enp6s0f0: renamed from eth0
Oct  6 12:19:21 amaiur kernel: [    1.204325] bnx2x 0000:06:00.1 
enp6s0f1: renamed from eth1
Oct  6 12:19:22 amaiur kernel: [    6.824472] bnx2x 0000:06:00.0 
enp6s0f0: using MSI-X  IRQs: sp 60  fp[0] 62 ... fp[7] 69
Oct  6 12:19:22 amaiur kernel: [    7.012029] bnx2x 0000:06:00.0 
enp6s0f0: NIC Link is Up, 10000 Mbps full duplex, Flow control: ON - 
receive & transmit

This was a previous step for upgrading to PVE 7, so not a very big issue 
if this is working well on PVE 7... :)

Thanks

Eneko Lacunza
Zuzendari teknikoa | Director técnico
Binovo IT Human Project

Tel. +34 943 569 206 | https://www.binovo.es
Astigarragako Bidea, 2 - 2º izda. Oficina 10-11, 20180 Oiartzun

https://www.youtube.com/user/CANALBINOVO
https://www.linkedin.com/company/37269706/

More information about the pve-user mailing list