Core: ix0 no carrier

[abplfab]

After upgrading to opnsense 18.7 the ix NIC (attached with a DAC to a switch) reports "media: no carrier". Setting the media to fixed 10Gbase-Twinax doesn't help...

[fichtner]

the joys of intel driver updates :(

run this and reboot

# opnsense-update -kr 18.1.11 -n "18.1\/dummy"

[fichtner]

ps booting from kernel.old should also work via boot menu

[abplfab]

Thanks

[fichtner]

if this indeed works we need to take this to FreeBSD soon if 11.2 has the same defect

[abplfab]

Doesn't help :(

[fichtner]

makes no sense at all ?!

what's the output of:

 # uname -a

[abplfab]

FreeBSD asterix2.lan.neratec.com 11.1-RELEASE-p11 FreeBSD 11.1-RELEASE-p11 21b4c8ea1d5(stable/18.7) amd64

[abplfab]

Did the opnsense-update -kr 18.1.11 -n "18.1\/dummy again, now:
root@asterix2:~ # uname -a
FreeBSD asterix2.lan.neratec.com 11.1-RELEASE-p11 FreeBSD 11.1-RELEASE-p11 116e406d37f(stable/18.1) amd64
but still:
ix0: flags=8943 metric 0 mtu 1500
options=e507bb
ether 0c:c4:7a:97:ee:ee
hwaddr 0c:c4:7a:97:ee:ee
inet6 fe80::ec4:7aff:fe97:eeee%ix0 prefixlen 64 scopeid 0x1
inet6 2a02:aa08:e000:902::253 prefixlen 64
inet6 2a02:aa08:e000:902::10 prefixlen 64 vhid 18
inet 192.168.11.253 netmask 0xffffff00 broadcast 192.168.11.255
inet 192.168.11.10 netmask 0xffffff00 broadcast 192.168.11.255 vhid 11
nd6 options=21
media: Ethernet autoselect
status: no carrier
carp: INIT vhid 11 advbase 1 advskew 100
carp: INIT vhid 18 advbase 1 advskew 100

[fichtner]

ok, kernels are correctly replaced. I'm unsure how any other change would relate to the reported issue of the driver reporting no carrier anymore.

[abplfab]

Would it make sense to try install new with 18.7 and import the config?
This system is the slave of a carp cluster, so currently no hurry...

[fichtner]

going back to 18.1.13 would make more sense than trying again with 18.7 (18.1.6 config import + install and update back to 18.1.13). but the chances for ix0 finding its carrier is not more than 50%. it could be hardware related.

[abplfab]

• Live booting 18.1 -> no link
• Booting a live linux -> no link
  Will go check hardware in the next hours (where's my 1. August Party?!)

[fichtner]

Could it be that FreeBSD 11.2 ships with a new binary firmware blob that bricked your NIC? That's the only theory I have besides it's fully the hardware's fault.

[abplfab]

No idea, the server has a second NIC (ix1), will try this one and report...

[abplfab]

• ix1 works
• Completely removed power and reboot -> ix0 has link (FreeBSD asterix2.lan.neratec.com 11.1-RELEASE-p11 FreeBSD 11.1-RELEASE-p11 116e406d37f(stable/18.1) amd64)
• Updated Kernel to 18.7 -> ix0 dead (FreeBSD asterix2.lan.neratec.com 11.1-RELEASE-p11 FreeBSD 11.1-RELEASE-p11 21b4c8ea1d5(stable/18.7) amd64)
• Completely removed power and reboot -> ix0 still dead
• Go back to Kernel 18.1.11 and remove power -> ix0 has link

So with ix NICs better not update to 18.7.

System
Supermicro SYS-5018D-FN8T

Mainboard
X10SDV-TP8F

Firmware versions
BIOS 1.3,
IPMI/BMC: 3.68
Redfish Version : 1.0.1

NIC
ix0@pci0:4:0:0: class=0x020000 card=0x15ac15d9 chip=0x15ac8086 rev=0x00 hdr=0x00
vendor = 'Intel Corporation'
device = 'Ethernet Connection X552 10 GbE SFP+'
class = network
subclass = ethernet

CPU
Intel(R) Xeon(R) CPU D-1518 @ 2.20GHz (8 cores)

[mimugmail]

@fichtner I have two identical boxed like above configured but not in production, ping me the usual way when you need access.

[abplfab]

FYI: a server reset or power off/on by ipmi/bmc doesn't "fix" the NIC hang. Had to remove power from the box.

[Tsuroerusu]

I am seeing something that could well be related to this issue. After upgrading my company setup (A HA setup of two firewalls) to 18.7, the result was that none of my VLAN interfaces on ix1 activate properly, however the non-VLAN ix0 works fine. On the main page, the VLAN interfaces are marked with red saying "Ethernet autoselect", and under System --> Interfaces --> Overview their status says "no carrier". All of my igb* interfaces come up without any issue. I have tried changing the VLAN configuration a bit hoping that a re-write of the configuration would solve it, but to no avail.

Update: I just tried booting kernel.old, as was suggested earlier in this thread, and just doing that has resolved the problem on both of my nodes! I did not have to do any actual power cycling, just booting into kernel.old did it.

[fichtner]

@Tsuroerusu Can you try the 18.1.11 kernel as well?

# opnsense-update -kr 18.1.11 -n "18.1\/dummy"

(reboot)

@abplfab said you need to remove the power, otherwise the carrier will not come up on a quick reboot.

[fichtner]

Relevant driver update was reverted and will be gone from 18.7.1. It's unclear if the issue exists in FreeBSD 11.2 but we'll find out soon (@mimugmail could you check this with your test system).

[mimugmail]

@fichtner Update is in progress .. need to figure out what exactly happens with and without VLANs.

[fichtner]

@mimugmail thanks a lot!

[abplfab]

Just started a live CD of FreeBSD 11.2 -> ix0 status: no carrier

[fichtner]

@abplfab thanks for confirming. this is really bad :(

[mimugmail]

Wow .. really noone using FreeBSD 11.2 with VLANs in production? Not even a user on pfsense beta, real hard tested, hail to the Netgate-Team, environment???

[mimugmail]

@fichtner I cannot confirm that I'm affected:

18.7 without VLANs:

ix1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=c400b8<VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,VLAN_HWTSO,TXCSUM_IPV6>
        ether ac:1f:6b:65:a5:a5
        hwaddr ac:1f:6b:65:a5:a5
        inet 10.12.12.1 netmask 0xffffff00 broadcast 10.12.12.255
        inet6 fe80::ae1f:6bff:fe65:a5a5%ix1 prefixlen 64 scopeid 0x2
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
        media: Ethernet autoselect (10Gbase-SR <full-duplex,rxpause,txpause>)
        status: active

18.7 with VLANs:

ix1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=c500b8<VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,VLAN_HWFILTER,VLAN_HWTSO,TXCSUM_IPV6>
        ether ac:1f:6b:65:a5:a5
        hwaddr ac:1f:6b:65:a5:a5
        inet6 fe80::ae1f:6bff:fe65:a5a5%ix1 prefixlen 64 scopeid 0x2
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
        media: Ethernet autoselect (10Gbase-SR <full-duplex,rxpause,txpause>)
        status: active

ix1_vlan111: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=400000<TXCSUM_IPV6>
        ether ac:1f:6b:65:a5:a5
        inet6 fe80::ae1f:6bff:fe65:a5a5%ix1_vlan111 prefixlen 64 scopeid 0x11
        inet 10.12.12.1 netmask 0xffffff00 broadcast 10.12.12.255
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
        media: Ethernet autoselect (10Gbase-SR <full-duplex,rxpause,txpause>)
        status: active
        vlan: 111 vlanpcp: 0 parent interface: ix1
        groups: vlan

https://www.thomas-krenn.com/en/products/application/hardware-it-security/opnsense-firewalls/ri1102d.html

ix0@pci0:4:0:0: class=0x020000 card=0x15ac15d9 chip=0x15ac8086 rev=0x00 hdr=0x00
    vendor     = 'Intel Corporation'
    device     = 'Ethernet Connection X552 10 GbE SFP+'
    class      = network
    subclass   = ethernet
ix1@pci0:4:0:1: class=0x020000 card=0x15ac15d9 chip=0x15ac8086 rev=0x00 hdr=0x00
    vendor     = 'Intel Corporation'
    device     = 'Ethernet Connection X552 10 GbE SFP+'
    class      = network
    subclass   = ethernet

ix0: <Intel(R) PRO/10GbE PCI-Express Network Driver, Version - 3.2.12-k> mem 0xfbc00000-0xfbdfffff,0xfbe04000-0xfbe07fff irq 11 at device 0.0 on pci5
ix0: Using MSI-X interrupts with 9 vectors
ix0: Ethernet address: ac:1f:6b:65:a5:a4
ix0: netmap queues/slots: TX 8/2048, RX 8/2048
ix1: <Intel(R) PRO/10GbE PCI-Express Network Driver, Version - 3.2.12-k> mem 0xfba00000-0xfbbfffff,0xfbe00000-0xfbe03fff irq 10 at device 0.1 on pci5
ix1: Using MSI-X interrupts with 9 vectors
ix1: Ethernet address: ac:1f:6b:65:a5:a5
ix1: netmap queues/slots: TX 8/2048, RX 8/2048
ix1: link state changed to UP

[Tsuroerusu]

@fichtner Unfortunately, I only have ix in my production firewalls, so I cannot experiment much at all with my OPNsense setup. However, I do have an identical ix-based card in a server that is not yet fully in production, and I can try running FreeBSD 11.2 live on that when I got back home in a few hours.

[abplfab]

I can use my slave carp firewall if any testing is needed.
@mimugmail looks like you use SFP(+) modules, here I use DAC connected to the switch directly.
Tried to disable TSO / LRO -> no change with 18.7
Tried to disable VLAN hardware filtering -> no change with 18.7
And yes, i use VLAN on ix0

[mimugmail]

I replaced with Twinax ... some result:

ix1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=c500b8<VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,VLAN_HWFILTER,VLAN_HWTSO,TXCSUM_IPV6>
        ether ac:1f:6b:65:a5:a5
        hwaddr ac:1f:6b:65:a5:a5
        inet6 fe80::ae1f:6bff:fe65:a5a5%ix1 prefixlen 64 scopeid 0x2
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
        media: Ethernet autoselect (10Gbase-Twinax <full-duplex,rxpause,txpause>)
        status: active

ix1_vlan111: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=400000<TXCSUM_IPV6>
        ether ac:1f:6b:65:a5:a5
        inet6 fe80::ae1f:6bff:fe65:a5a5%ix1_vlan111 prefixlen 64 scopeid 0x11
        inet 10.12.12.1 netmask 0xffffff00 broadcast 10.12.12.255
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
        media: Ethernet autoselect (10Gbase-Twinax <full-duplex,rxpause,txpause>)
        status: active
        vlan: 111 vlanpcp: 0 parent interface: ix1
        groups: vlan

[ruffy91]

This: https://sourceforge.net/p/e1000/mailman/message/35263903/ points to a HW/FW problem (similar problem with linux kernel 4.7/4.8).
@abplfab Have you tried to loopback the twinax cable to the NIC to make sure it is not a compability problem with the switch?

[abplfab]

@ruffy91 yep. Looks exactly like that. Loop ix0 - ix1 I get link. Switch is a Netgear XS716T :(

[ruffy91]

There is a FW for the switch which describes this problem:
https://kb.netgear.com/000038683/XS708T-XS716T-Firmware-Version-6-6-1-7

[abplfab]

Yep, but I'm running firmware 6.6.3.3

[ruffy91]

There are also problems (maybe unrelated but shows that the NIC FW plays a role with compatibility) with other Mainboards from Supermicro:
https://tinkertry.com/how-to-work-around-intermittent-intel-x557-network-outages-on-12-core-xeon-d
and
https://forums.freebsd.org/threads/driver-for-intel-pci-e-10-gigabit-nic-specifically-x552-x557-at.57536/
I would suggest opening a case with Supermicro to aks if there are known problem or a newer Firmware for the Intel NIC.

We have Huawei R1288H v5 which get regular Intel NIC FW updates (Intel X722), unfortunately SuperMicro does not seem to release the NIC FW to customers as you can see in the linked blogpost.

Maybe you can find and tell the firmware-version in kernel/driver logs as further data points for comparison with other platforms which do not have the problem.

It seems to me that 10GbE is still in it's infancy, despite 40/100GbE now becoming mainstream for the hyperscalers

[abplfab]

Opened a case with supermicro. Lets see.

[abplfab]

Got new firmware for the NICs, but doesn't help :(.
Firmware attached.
"To flash, please boot from a DOS bootable USB stick and run the "7TP8F.BAT" batch file in the attached package. The rest is automatic."
sdvtp2c.zip

[mimugmail]

@abplfab Just to summarize:

OPNsense 18.1.13 works with SFP+ to Switch
OPNsense 18.7 doesnt work with SFP+ to Switch
Downgrade to 18.1.13, power off and it works again to Switch
OPNsense 18.7 from Port1 to Port2 on Dual NIC works
FreeBSD 11.2 live CD doesn't work

What about FreeBSD 12 and 11.1 (to really preclude it's live CD itself)?

@Tsuroerusu Has identical problem, gets fixed with booting old kernel.
Have you already tried the loop? What are your hardware specs? What about live CD?

I'm waiting for my lab to come back and test with ixl instead of ix, but for me everything works. I always test with a second machine (no switch), both Direct Attach and GBic and LWL cable.

[ruffy91]

That was super fast. Unfortunately I am out of ideas.
@fichtner I can only say that we use Intel X722 and 18.7 is the first usable release with the new drivers. We now have working CARP and everything is smooth. Is this revert only for the ix/ixgbe driver or also for ixl?

[fichtner]

@ruffy91 ixl is a separate driver backport we are keeping if you say it works fine now :)

[abplfab]

@mimugmail correct.
FreeBSD 11.1 live CD: works
FreeBSD 12.0-CURRENT live CD: doesn't work

[mimugmail]

So, with a X710 and ixl it works too:

18.7 no VLANs:
ixl0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=6402b8<VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO6,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6>
        ether 3c:fd:fe:9e:e7:48
        hwaddr 3c:fd:fe:9e:e7:48
        inet6 fe80::3efd:feff:fe9e:e748%ixl0 prefixlen 64 scopeid 0x1
        inet 10.55.1.1 netmask 0xffffff00 broadcast 10.55.1.255
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
        media: Ethernet autoselect (10Gbase-Twinax <full-duplex>)
        status: active

18.7. with VLANs:
ixl0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=6002a8<VLAN_MTU,JUMBO_MTU,VLAN_HWCSUM,TSO6,RXCSUM_IPV6,TXCSUM_IPV6>
        ether 3c:fd:fe:9e:e7:48
        hwaddr 3c:fd:fe:9e:e7:48
        inet6 fe80::3efd:feff:fe9e:e748%ixl0 prefixlen 64 scopeid 0x1
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
        media: Ethernet autoselect (10Gbase-Twinax <full-duplex>)
        status: active

ixl0_vlan111: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        ether 3c:fd:fe:9e:e7:48
        inet6 fe80::3efd:feff:fe9e:e748%ixl0_vlan111 prefixlen 64 scopeid 0xa
        inet 10.55.2.1 netmask 0xffffff00 broadcast 10.55.2.255
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
        media: Ethernet autoselect (10Gbase-Twinax <full-duplex>)
        status: active
        vlan: 111 vlanpcp: 0 parent interface: ixl0
        groups: vlan

ixl0@pci0:1:0:0:        class=0x020000 card=0x00088086 chip=0x15728086 rev=0x01 hdr=0x00
    vendor     = 'Intel Corporation'
    device     = 'Ethernet Controller X710 for 10GbE SFP+'
    class      = network
    subclass   = ethernet

ixl0: <Intel(R) Ethernet Connection 700 Series PF Driver, Version - 1.9.9-k> mem 0xdd000000-0xdd7fffff,0xdd808000-0xdd80ffff irq 16 at device 0.0 on pci1
ixl0: using 1024 tx descriptors and 1024 rx descriptors
ixl0: fw 5.0.40043 api 1.5 nvm 5.05 etid 80002892 oem 1.262.0
ixl0: PF-ID[0]: VFs 64, MSIX 129, VF MSIX 5, QPs 768, I2C
ixl0: Using MSIX interrupts with 9 vectors
ixl0: Allocating 8 queues for PF LAN VSI; 8 queues active
ixl0: Ethernet address: 3c:fd:fe:9e:e7:48
ixl0: PCI Express Bus: Speed 8.0GT/s Width x8
ixl0: SR-IOV ready
ixl0: netmap queues/slots: TX 8/1024, RX 8/1024

[Tsuroerusu]

@mimugmail My problem is actually a little different, it is a half-way house of what the rest of you seem to be dealing with.

For me the summary would be:

1. OPNsense 18.1.13 works across the board, no problems.

2. OPNsense 18.7 seems to work when the ix NICs are utilized without VLANs.
   To elaborate a bit: I use a Supermicro AOC-STGN-i2S card (Intel 82599), which provides two ix NICs via SFP+ ports for each of my two firewall nodes. ix0 on each node is the WAN port (and as such has no VLANs or other special stuff), and is connected to a Juniper EX3300 switch via DAC cables. ix1 on each node is connected, also via DAC cables, to a D-Link DGS-3420-52T switch. ix1 is configured only with VLANs, 5 in total, and has no non-VLAN configuration. When I boot into the 18.7 kernel, ix0 works fine and I can access the Internet without any problems, but ix1 are completely dead saying "no carrier". Before upgrading, this setup had been working swimmingly, no incompatibilities at all to speak of for over a year. So my problem seems to be exclusively with VLANs in regard to 18.7 and ix NICs.

3. I have not tried downgrading, because it only took a boot into kernel.old to make things work again, and it was not necessary to power cycle, just a simple reboot was enough.

4. I have another AOC-STGN-i2S card in one of my AMD-based servers (Which admittedly is quite a different beast than my firewalls, which are Atom C2000-based) , and when booting the FreeBSD 11.2 install media in live mode, this is what happens:
   a) The first thing I notice is that, even when unconfigured, the switch shows green lights on those ports.
   b) I can successfully configure both ix0 and ix1 as non-VLAN interfaces.
   c) I can successfully configure both ix0 and ix1 as VLAN interfaces.

5. I have not tried loop-backing the NICs, because ix0 to the Juniper switch works, and in my server both NICs to the D-Link switch works as well. As far as I can tell, I am not suffering from hardware incompatibilities.

From my perspective, vanilla FreeBSD 11.2 seems to work fine for my AOC-STGN-i2S card. So my first thought is that the backport of the driver has a problem for some reason, but I am no kernel hacker at all, so I cannot really tell.

[mimugmail]

Next 18.7 system with ix0, X520 NIC, also working fine. I'm trying to search for a 10G switch to reproduce, but atm it seems to be a very specific problem :(

root@OPNsense:~ # ifconfig ix0
ix0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=c000a8<VLAN_MTU,JUMBO_MTU,VLAN_HWCSUM,TXCSUM_IPV6>
        ether 90:e2:ba:39:1f:10
        hwaddr 90:e2:ba:39:1f:10
        inet6 fe80::92e2:baff:fe39:1f10%ix0 prefixlen 64 scopeid 0x1
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
        media: Ethernet autoselect (10Gbase-Twinax <full-duplex,rxpause,txpause>)
        status: active
root@OPNsense:~ # ifconfig ix0_vlan111
ix0_vlan111: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        ether 90:e2:ba:39:1f:10
        inet6 fe80::92e2:baff:fe39:1f10%ix0_vlan111 prefixlen 64 scopeid 0x9
        inet 10.55.2.2 netmask 0xffffff00 broadcast 10.55.2.255
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
        media: Ethernet autoselect (10Gbase-Twinax <full-duplex,rxpause,txpause>)
        status: active
        vlan: 111 vlanpcp: 0 parent interface: ix0
        groups: vlan
root@OPNsense:~ # dmesg | grep ix0
ix0: <Intel(R) PRO/10GbE PCI-Express Network Driver, Version - 3.2.12-k> port 0xe020-0xe03f mem 0xdde80000-0xddefffff,0xddf04000-0xddf07fff irq 16 at device 0.0 on pci1
ix0: Using MSI-X interrupts with 9 vectors
ix0: Ethernet address: 90:e2:ba:39:1f:10
ix0: PCI Express Bus: Speed 5.0GT/s Width x4
ix0: netmap queues/slots: TX 8/2048, RX 8/2048
vlan0: changing name to 'ix0_vlan111'
ix0: link state changed to UP
ix0_vlan111: link state changed to UP

ix0@pci0:1:0:0: class=0x020000 card=0x00038086 chip=0x10fb8086 rev=0x01 hdr=0x00
    vendor     = 'Intel Corporation'
    device     = '82599ES 10-Gigabit SFI/SFP+ Network Connection'
    class      = network
    subclass   = ethernet

EDIT: This is a Fujitsu RX1300 system ...

[abplfab]

Contacted Netgear, they check with their team...

[fichtner]

This may be related: https://github.com/opnsense/core/commit/4ba0fa679

Try to use Interfaces: Settings: VLAN Hardware Filtering: "Leave default".

[abplfab]

Doesn't help :(

[fichtner]

Ok, bummer. It was a long shot :(

[abplfab]

Tried today with an M5300-52G Netgear Switch (11.0.0.31, B1.0.0.5). Same result: no link.

[Tsuroerusu]

Was this issue resolve or somewhat alleviated in 18.7.1? I ask because I could not find anything about it in the release notes.

[fichtner]

No change in 18.7.1. kernel.old is phased out (now is the 18.7 kernel because there is a new 18.7.1 kernel), but the manual revert to 18.1.11 should still work minus the set verification (i):

# opnsense-update -ikr 18.1.11 -n "18.1\/dummy"

[RehaagJ]

Do I understand correctly that the new ix driver will still be available in 18.7.1? Meaning it should be safe to update on a Denverton system (having worked around the AHCI issue with BIOS settings)?

[fichtner]

Yes.

[fichtner]

This sounds promising: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=221146

I'll provide a test kernel in a bit...

[fichtner]

Relevant src commit: https://github.com/opnsense/src/commit/57b43db5

How to install test kernel from 18.7:

# opnsense-update -kr 18.7.1-ix

[fichtner]

And one more mention via Twitter for affected hardware: https://www.supermicro.com/products/motherboard/Xeon/D/X10SDV-4C_-TLN4F.cfm

[enoch85]

Don't know if it helps but here are some information on my failing ix NICs on 18.7:

```
ix0: mem
ix1: mem
````

I'm @techandme btw.

[fichtner]

No tests with the proposed kernel after all this discussion? ;)

[enoch85]

I'd love to test, but since my system is in production and I don't have a any test system, I'd prefer not to.

Would be nice if someone could confirm that it works. Right now I'm stuck with 18.1 because of this. :/

[RehaagJ]

I can try to test it during the weekend on a Denverton Atom system, but that cannot verify that the problem is solved (I never had that no carrier issue), it can only verify that the driver still works on Denverton. Is such a test useful for you?

[fichtner]

@Tsuroerusu and @abplfab were able to reproduce, their testing would be most useful, but testing that it works either way (release and test kernel) is good to know too

Thanks all. I'll be gone for two weeks so apologies for lack of responses.

Cheers,
Franco

[Tsuroerusu]

@fichtner During the weekend, I will see if I can image one of my firewall nodes and try this out on that one. My situation is the same as @Uica , my systems are production so I have a limited scope to test things, but I can do so on my secondary box after imaging it.

[RehaagJ]

Tested, both release and test kernels are working on Denverton Atom C3558. But I repeat, this does not prove that the problem is fixed, only that I don't see any regression.

[Tsuroerusu]

Okay, this is the situation:
Both my primary and secondary nodes in production have been running kernel.old for 17 days to alleviate the problem of VLAN interfaces on ix1 showing "no carrier", and thus not functioning. As I have stated earlier, my ix0 interfaces both nodes have no VLANs and come up fine, which means that there is WAN access. My ix interfaces are located on a Supermicro AOC-STGN-i2S add-on card, which uses the Intel 82599ES controller.

Today, I performed the following on my secondary node in production:

1. I took a bit-for-bit image of my secondary node, and shut down the machine.

2. After once again powering on the machine, I let it boot directly into the 18.7 kernel, and the problem reappeared.

3. I then did a reboot to make sure that it was not just a fluke, and the problem did indeed persist after rebooting.

4. I then updated the secondary node to 18.7.1 and after rebooting the problem still persisted.

5. I then logged into the system via the IPMI KVM console, and ran:
   opnsense-update -kr 18.7.1-ix

After it finished installing the kernel package, I immediately rebooted the machine.
After rebooting, the problem is still present. :-/ I also tried a second reboot, which had the same result, as well as a power cycle and it also had the same result.

To conclude: As far as my problem of VLAN interfaces on ix1 showing "no carrier" is concerned, the "18.7.1-ix" package had no effect at all.

@fichtner @enoch85

[Tsuroerusu]

Does anybody need me to check anything before I restore my node to its earlier semi-working (ie. kernel.old) state?

[fichtner]

Thanks for testing. No further poking needed. As said on Twitter kernel.old will not work because 18.7.1 installed a newer kernel making 18.7 kernel.old, please use the last hint in this thread to install the 18.1.11 kernel directly.

On 19. Aug 2018, at 12:52, Troels Just [e-mail] wrote:

Does anybody need me to check anything before I restore my node to its earlier semi-working (ie. kernel.old) state?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

[enoch85]

@Tsuroerusu Thanks for confirming!

[abplfab]

Just tried with 18.7.1_3, still no carrier. Supermicro & NetGear still investigating...

[mimugmail]

@enoch85 On my side it's

ix1: <Intel(R) PRO/10GbE PCI-Express Network Driver, Version - 3.2.12-k>

[enoch85]

@mimugmail Newer, and still not working?

[mimugmail]

Sure, it was always working for me with every version.

[mimugmail]

Guys, can you try to add this (or create the file if not exist):

root@OPN:~ # cat /boot/loader.conf.local
hw.ix.allow_unsupported_sfp=1

[fichtner]

PS: can also be added via system: settings: tunables.

[abplfab]

In my case doesn't help :(. Also tried hw.ix.allow_unsupported_sfp=1,1

[Tsuroerusu]

@mimugmail @fichtner The proposed change made no difference in regard to the problem I am seeing.

[mimugmail]

And you driver versions are 3.1.13 or 3.2.12?

[Tsuroerusu]

@mimugmail The one that is in OPNsense 18.7 / 18.7.1

[mimugmail]

And why is mine higher than others?

[enoch85]

And why is mine higher than others?

Maybe because you upgraded with this?

[abplfab]

&%%+"ç"&&/"+%%?!*"/ç DAMN!

I've found the problem. I replaced the Delock SFP+ DAC (Art. 86234) with an Intel XDACBL3M and it works.

• No need for hw.ix.allow_unsupported_sfp=1,1
• Default Kernel / 18.7.2 / 11.1-RELEASE-p13 FreeBSD 11.1-RELEASE-p13 0259f6a4413(stable/18.7) amd64
  Working cable: https://www.brack.ch/intel-direct-attach-kabel-518655
  Damn cable: https://www.brack.ch/delock-direct-attach-kabel-330523

[enoch85]

No need for hw.ix.allow_unsupported_sfp=1,1

So that will make the Intel NIC work?

[abplfab]

In my case it didn't change anything if I have hw.ix.allow_unsupported_sfp=1,1 or not.
Means:
With the Delock cable it doesn't work with hw.ix.allow_unsupported_sfp=1,1 enabled or disabled (always no carrier).
With the Intel cable I didn't try to enable it, as it works without it.

[enoch85]

@abplfab So this could be a cabling issue as well?

@fichtner Any progress on this in the latest releases? I'm stuck on 18.1 because of this. :(

[Tsuroerusu]

@enoch85 My problem is most certainly not cable-related, because everything was working wonderfully for over a year before the 18.7 update. Like you I am also stuck with the 18.1 kernel, which is a big issue as there have been several security updates since then.

As I stated e arlier, when I ran vanilla FreeBSD 11.2 through the LiveUSB, I was able to configure VLANs without an issue, so it seems like its something wrong with the backported driver. So I am crossing my fingers that 19.1 will be re-based to FreeBSD 11.2 and that, that will fix the issue. ( @fichtner )

[abplfab]

@enoch85 / @Tsuroerusu Definitly. With the old driver / Kernel it was working well with Delock cable. After upgrading to 18.7 it stopped working with the delock cable. Intel cable works with 18.1 / 18.7 without any modification.

[enoch85]

@abplfab Do you think something like this work?

Has anyone tried with the 19.1 release?

[abplfab]

@enoch85 could be. But I wouldn't save on cables anymore... Take the original Intel cable (XDACBL3M for 3m) if possible.

[cvbkf]

Hi there,
i have the same problem with an Intel X520-DA2 with direct-attach cables. ix0 is offline with no carrier, ix1 is working. if i boot the system to a ubuntu live cd, both ports are working with 10 gbit/s. the switch on the other end says link is fine with 10 gbit/s, but does not receive any packets

OPNSense ist the latest version with intel driver 3.2.12k. i found this related bug:
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=221146

[Tsuroerusu]

After this whole fiasco, which has left my perimeter firewalls without security updates for several months now, I am rather curious as to whether 19.1 will have any backports of Intel drivers or will it be using the ones in FreeBSD 11.2 ? Hey @fichtner , do you have any information in regard to this?

[fichtner]

has left my perimeter firewalls without security updates for several months now

Sorry, that's a very one-sided approach. People have replaced hardware, cables, tried patches. There's also 19.1-BETA.

If you want to blame OPNsense for any of this, please do while presenting the hard evidence and the intent to leave you without security updates. Looking forward to it? :)

As for 19.1 we are proceeding as planned: January 2019. You can try the BETA from here:

https://forum.opnsense.org/index.php?topic=10135.0

[Tsuroerusu]

has left my perimeter firewalls without security updates for several months now

Sorry, that's a very one-sided approach. People have replaced hardware, cables, tried patches. There's also 19.1-BETA.

If you want to blame OPNsense for any of this, please do while presenting the hard evidence and the intent to leave you without security updates. Looking forward to it? :)

As for 19.1 we are proceeding as planned: January 2019. You can try the BETA from here:

https://forum.opnsense.org/index.php?topic=10135.0

No offense, but WTF!? How in the world did you manage to interpret my remarks as somehow being aggressive and blaming OPNsense on a sort of personal level? I simply said, that this problem has had the consequence of me being unable to update my firewalls for several months, because doing so would cripple my network setup. I never said anything that this was somehow a conspiracy. Frankly, given the amount of time I spent earlier in this discussion trying to help test things, I am slightly ... not offended, but let's say "disappointed" that you would interpret my concern in such a manner.

Also, with all due respect, I use a Supermicro network card (AOC-STGN-i2S) in a system with a Supermicro motherboard (A1SRM-2758F) connected using Supermicro SFP+ DAC cables (CBL-NTWK-0347). The notion that hardware is somehow the problem in my case I think is, frankly, absurd. As I have stated previously, it was all working brilliantly with 18.1 and earlier releases. This all started with 18.7. Frankly, since you mention hard evidence, well there you go, 18.1 worked perfectly, 18.7 does not, and do you honestly expect that I should consider hardware incompatibility as a serious possibility when, 1. it was working fine before 18.7, and 2. I am not mixing brands at all, I am using hardware that is all verified to work together as it is all from Supermicro.

I really do hope that you simply misunderstood what I meant to say, because I really think your remarks were rather unfortunate.

[abplfab]

@Tsuroerusu give the Intel DAC (XDACBL3M) a try. I'm 98.6% sure that this will solve your problems.

[enoch85]

give the Intel DAC (XDACBL3M) a try. I'm 98.6% sure that this will solve your problems.

I just bought one, will be delivered next week. Let's hope it works!

[fichtner]

I really do hope that you simply misunderstood what I meant to say, because I really think your remarks were rather unfortunate.

Likewise.

[Tsuroerusu]

I really do hope that you simply misunderstood what I meant to say, because I really think your remarks were rather unfortunate.

Likewise.

I would be happy to find out that I misunderstood you, however I interpreted a good sense of contempt and arrogance in this remark of yours: "If you want to blame OPNsense for any of this, please do while presenting the hard evidence and the intent to leave you without security updates. Looking forward to it? :)"

I never blamed you or anybody else personally, and I have have tested everything I described and have not just made blind claims. And, please, why did you feel the need to end with a ":)", no offense, but I do not think I can be faulted for interpreting that as contempt directed towards me.

If I am mistaken, and you did not mean it like that, then let's just, virtually, shake hands and and move on. As I said, I really would appreciate to know whether driver backports is going to be a part of 19.1, or whether it will be just the FreeBSD 11.2 drivers, I ask because I tried running the FreeBSD 11.2 live media on my firewalls, and configured VLANs on the ix interfaces and it worked as expected. So my suspicion is that something about the driver backport in 18.7 caused the problem, but I cannot confirm that, and I am crossing my fingers that 19.1 resolves the issue.

[Tsuroerusu]

@Tsuroerusu give the Intel DAC (XDACBL3M) a try. I'm 98.6% sure that this will solve your problems.

No offense, but why should it be necessary for me to spend several hundred euros on cables, when the ones I have (From my system manufacturer) were working perfectly fine for over a year? It was my upgrade to 18.7 that caused the problem, not the hardware.

[mimugmail]

@Tsuroerusu no, it wasn't the update of OPNsense, it was the update of FreeBSD kernel. OPNsense is just the GUI orchestrating all services and configuration. Be sure that many many companys run 10G and DA cables just fine. The problem is that for developers it's hard to replicate since they wont buy such a hardware to figure out the problem.

Be sure you misunderstood Franco .. let's wait for @enoch85 how it works with new cable :)

I have many 10G cards, also 40G, many Gbics and DA cables .. I never had a problem.

[Tsuroerusu]

@Tsuroerusu no, it wasn't the update of OPNsense, it was the update of FreeBSD kernel. OPNsense is just the GUI orchestrating all services and configuration. Be sure that many many companys run 10G and DA cables just fine. The problem is that for developers it's hard to replicate since they wont buy such a hardware to figure out the problem.

Be sure you misunderstood Franco .. let's wait for @enoch85 how it works with new cable :)

I have many 10G cards, also 40G, many Gbics and DA cables .. I never had a problem.

With all due respect, that is obviously what I meant! You can say that one specific component is "OPNsense", fine, but then why is the ISO file named "opnsense" ? Obviously, because it is a, for a lack of a better term, "distribution", so when I say that the "update" for "OPNsense 18.7" caused the problem, I mean that the kernel provided in 18.7 which got installed as part of the update caused the problem, thus the problem was caused by "18.7".

I am a user, not a developer, I will gladly help out to the degree that my skill set allows me to be helpful, but apparently, I cannot simply describe the things I am experiencing from my perspective, apparently, I have to learn whatever jargon/lingo is used between developers before I am even welcomed to say anything. No offense, but do you not feel that, that is just a tiny bit absurd?

And I am very sympathetic to the fact that developers cannot test every single hardware combination! Why do you think I actually dared to risk system uptime to test things a few months ago (i.e. earlier in this discussion) ? Precisely because I know that developers cannot go and buy specific hardware to test things.

Indeed, I expect that thousands of servers and firewalls are using Supermicro's cables, and so was I, that was my entire point, but after upgrading to 18.7, "no carrier" was all I got when using VLANs.

[fichtner]

@Tsuroerusu

You have your own reasons for using the hardware you use. That's fine.

You don't agree with the stance that an open source project has regarding your interests? That's also fine.

But using this platform to justify your stance is suboptimal at best.

You are the sole person in charge of your hardware and software.

So I'm politely asking you to quit this discussion now.

Thank you for your understanding.

[fichtner]

Closing this as there were no negative updates on 19.1-BETA.

[Tsuroerusu]

@fichtner I am using the hardware that I have, because it was working just fine with 18.1, I also said that I fully understand why developers cannot test my particular setup. The only thing I insisted on is that I am not using incompatible hardware as everything I am using is officially validated by Supermicro, WHICH WAS WORKING WITH 18.1, and thus my modest claim was that it was unreasonable for you guys to keep telling me that I just need to change my hardware and spend hundreds of euros on that.

Why do you keep putting words into my mouth? I never said anything about the "stance" of the OPNsense project, all I wanted to ask was about whether there were driver backports planned in 19.1, that was all!

I am not using this platform for anything! I simply asked a question about drivers, and then I ended up responding to the absurd accusations that you made against me, how is that unreasonable?

I fully accept that I am responsible for my hardware.

I am, frankly, a little bit shocked that this is how you choose to treat your users, who, 1. have reported a problem, 2. tried to help, and 3. just asked a question.

But do rest assured, I can promise you that I have no intention of participating in this discussion anymore, and I will not be reporting any problems in the future given that I simply got insulted and shown contempt for simply asking a question.

I wish you and everybody else a pleasant Sunday.
Good bye.

[fichtner]

@Tsuroerusu Okay, listen. Tell me what you in very precise words want us to do and I'll objectively explain why that may or may not be feasible.

[Tsuroerusu]

@Tsuroerusu Okay, listen. Tell me what you in very precise words want us to do and I'll objectively explain why that may or may not be feasible.

This is the heart of the matter, Franco. I did not, and still do not actually want you to do anything, and that is why I have been so amazed (in the negative sense) by your responses today (specifically).

The ONLY thing that I requested was a simple "yes/no" answer to this question:
Will OPNsense 19.1 contain any backported Intel drivers?

[enoch85]

@Tsuroerusu I understand that you're frustrated. Believe me, I'm too! But instead of arguing, why not test the latest 19.1 release to see if it works? That would help all users far more IMHO.

Maybe I can save a few bucks on a new cable. :D

[Tsuroerusu]

@Tsuroerusu I understand that you're frustrated. Believe me, I'm too! But instead of arguing, why not test the latest 19.1 release to see if it works? That would help all users far more IMHO.

Maybe I can save a few bucks on a new cable. :D

@enoch85 I actually agree with you on that, and that is why I was so disappointed that instead of a simple answer to the question I asked, I got absurd accusations thrown at me (not by you), which I then had to respond to.

Your suggestion of testing the 19.1 beta, I have no issue with considering that, it is a perfectly reasonable thing to ask of me. In fact, let me just go further and say, that the reason I asked about the drivers in 19.1 was precisely because I was interested in potentially testing it with my setup, however for that it would be useful to know which drivers I would actually be testing (Because earlier I got the vanilla FreeBSD 11.2 live media to work fine).

My setup is at production-level and because of that, I have to image it before testing things, and before spending an hour or two doing that, I simply needed some information as I have explained.

[cvbkf]

Some update from me, i tried the current 19.1 beta image via live cd, and both ports of the intel x520-da2 are online now.

the driver version is the same as before:

[1] ix0: port 0xc020-0xc03f mem 0xdfe80000-0xdfefffff,0xdff04000-0xdff07fff irq 22 at device 0.0 on pci4

I'll install the beta now, hopefully config from 18.7 can be imported.

edit: performance is really bad, iperf3 measures around 1.2 gbit/s while hitting massive iowait.

[fichtner]

Will OPNsense 19.1 contain any backported Intel drivers?

The answer is no, but not in the sense that the backported drivers of 18.7 are not different from 19.1, because the backport is from FreeBSD 11.2 and 19.1 is based on stock HardenedBSD/FreeBSD 11.2 so the same drivers are included.

I was under the impression that this has been communicated clearly in several of places and I apologise if that wasn't actually the case.

[Tsuroerusu]

Will OPNsense 19.1 contain any backported Intel drivers?

The answer is no, but not in the sense that the backported drivers of 18.7 are not different from 19.1, because the backport is from FreeBSD 11.2 and 19.1 is based on stock HardenedBSD/FreeBSD 11.2 so the same drivers are included.

Thank you for answering my question. :)

I was under the impression that this has been communicated clearly in several of places and I apologise if that wasn't actually the case.

Earlier you linked to a forum post, which stated the following:
"The key difference is the operating system switch from FreeBSD 11.1 to HardenedBSD 11.2."

However, for me, that did not exclude the possibility of any potential driver backports from 11-STABLE or from Intel directly, so that was why I sought clarification from you regarding that point.

The reason why I was interested in this is that shortly after I started experiencing problems with 18.7, I tried booting up the FreeBSD 11.2 live USB media, and configured my ix interfaces with VLANs and that worked in the way I expected them to. So my thinking was that if 19.1 did not contain any modifications to the ixgbe driver compared to the one in FreeBSD 11.2, perhaps it would fix my problem.

I hope this makes sense from my perspective of being a user/sysadmin, and not developer, just trying to use pattern matching and logical inferences to try to find solutions.

[enoch85]

The cable won't show up until 2018-12-13 , so I won't be able to test if it's a cabling issue or not until then.

Can someone else please confirm that 19.1 works?

[cvbkf]

19.1 (installed from ISO) works for me without changing the cables.

interestingly, i switched back to 18.7 by doing a fresh install and found the following: running in live cd mode: ix0 and ix1 are working. the first boot of the fresh install from disk: ix0 & ix1 online. but if i restore a backup, or change the interface settings via GUI ix0 "dies" after the reboot with the ominous "no carrier" status. ix1 stays online.

[mimugmail]

Can you make a diff of config.xml and your backup?

[enoch85]

@cvbkf

Can you make a diff of config.xml and your backup?

Please

I will get the cable tomorrow, so I will test this weekend. :D

[enoch85]

IT WORKS!

What I did:

1. Replaced the cable and did a live boot with 18.7
   a) Got a warning that my WAN interface didn't work
   b) Renewed the IP --> Worked
   c) Noticed that I couldn't reach the servers sites from my WIFI net, but it was reachable from LTE (or outside of my own network)

2. Figured I could live with not reaching the DNS for the sites I host (could just use network outside my own, or thought it might be a firewall rule issue) and went ahead and installed 18.7
   a) After installation I changed back to LibreSSL (which I had originally)
   b) Ran an update to reach 18.7.8

3. Rebooted --> MISSION SUCCESS, everything now works as expected and I can reach my sites from my own network again. Everything regarding ix1 is "up" and so far I have no issues.

Conclusion
Maybe it was a combined error with my SFP+ cable and that something got fixed in the update to 18.7.6. Anyway, I'm happy again since I now can use the latest stable release. :D

[cvbkf]

I am on 19.1-rc1 for a few weeks now, until today both ix0 and ix1 were fine. After a reboot ix0 didn't come online -> the good old "no carrier" problem is back to haunt me. I tried a few things, but for now ix0 stays dead.

But, i made an interesting discovery: If i reset the machine via reset button, the link comes up at 10 Gbit/s - then, the link stays online until "Configuring LAN interface..." at the OPNSense bootup, then it goes down until the next reboot.

Maybe there is some invalid config applied ? Are there any files or something i can provide ?

[Tsuroerusu]

I just upgraded one of my firewalls to 19.1.5 from 18.7 (but using "kernel.old", i.e. the kernel fra 18.1), which I had been running since August because of the issue I had with VLANs on my ix interfaces saying "no carrier", as described earlier in this thread. However, I am sad to say that the issue still persists despite the update to FreeBSD 11.2 in OPNsense, which is really depressing :-(

The strange thing for me is that, as I mentioned before, when 11.2 came out, I tried using the Live boot with my machines, and I could configure VLANs without any issue and run network traffic through them. Which makes this issue even more baffling to me.

And before anybody jumps in with this. Before I went to do the upgrade, the firewalls were working fine with the kernel from 18.1. So this is not a cabling issue, unless the newer Intel drivers have some change that in itself causes compatibility issues with the Supermicro cables that I am using.

At this point, I am millimeters from giving up and buying some different NIC card, and hoping that I will not face the same issue, because at this point I have been without security updates since August. Unfortunately, that will cost me 500 euros before I even know whether it will actually solve the problem.

[cvbkf]

I switched to a Mellanox Connect-X3 EN 2x SFP+ (used), which is working without any flaws.

You could try to compile a newer version of the intel driver (last post in this thread)
https://forum.opnsense.org/index.php?topic=11384.0;topicseen

[Tsuroerusu]

I switched to a Mellanox Connect-X3 EN 2x SFP+ (used), which is working without any flaws.

You could try to compile a newer version of the intel driver (last post in this thread)
https://forum.opnsense.org/index.php?topic=11384.0;topicseen

Thanks for the suggestion, I appreciate that. Unfortunately, the newer driver has the same issue.
Are you using VLANs with that Mellanox card?

[cvbkf]

yes, i do, but just only one (which lead to the "no carrier" problem on the intel x550)

[Tsuroerusu]

yes, i do, but just only one (which lead to the "no carrier" problem on the intel x550)

I must say, the ix driver sure is a wuss, 'ey? A single VLAN, in your case, and it melts down! It would be funny if it wasn't so annoying. I run something like 8 VLANs and I REALLY need them to work, so Mellanox it is for me then. Thanks for the recommendation, as I think I have found a good source for them! Hopefully this will solve my problem. :-)

[DesruX]

I switched to a Mellanox Connect-X3 EN 2x SFP+ (used), which is working without any flaws.

You could try to compile a newer version of the intel driver (last post in this thread)
https://forum.opnsense.org/index.php?topic=11384.0;topicseen

Mention of no carrier bug - recommending the 3.3.6 driver
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=235918

This may also be related
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=221967

Talk of permanent allow unsupported SFP in driver, dated but similar principle.
https://sourceforge.net/p/e1000/mailman/message/28694855/

Another patch for ix driver:
http://www.grosbein.net/freebsd/patches/patch-if_ix.c

Would be great if no artificial restrictions where present in the driver and we only have to worry about actual hw compatibility.

[abplfab]

With 20.1 the problem is back. "no carrier" on the 10Ge I/F. :(

[abplfab]

Booting kernel.old (19.7) doesn't help.

[mimugmail]

Sounds occasional, what happens when plugging of the cable and in again?

[abplfab]

Doesn't help. Exactly the same behavior as in the beginning of this thread. Hardware unchanged, "only" updated to 20.1.

[mimugmail]

ifconfig -vvvvvv please

It worked with 19.7.10?

[abplfab]

Yes with 19.7.10 it worked.

ix0: flags=8802 metric 0 mtu 1500
options=e407bb
ether 0c:c4:7a:97:ed:ce
hwaddr 0c:c4:7a:97:ed:ce
nd6 options=21
media: Ethernet autoselect
status: no carrier
plugged: SFP/SFP+/SFP28 Unknown (Copper pigtail)
vendor: Panduit Corp. PN: PSF1PXA3MBLLN SN: 15219315U-0017B DATE: 2017-01-10
Class: (null)
Length: (null)
Tech: Passive Cable
Media: (null)
Speed: (null)

    SFF8472 DUMP (0xA0 0..127 range):
    03 04 21 00 00 00 00 00 04 00 00 00 67 00 00 00
    00 00 03 00 50 61 6E 64 75 69 74 20 43 6F 72 70
    2E 20 20 20 00 00 0F 9C 50 53 46 31 50 58 41 33
    4D 42 4C 4C 4E 20 20 20 34 20 20 20 01 00 00 F8
    00 00 00 00 31 35 32 31 39 33 31 35 55 2D 30 30
    31 37 42 20 31 37 30 31 31 30 00 00 00 00 00 71
    00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
    00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

[fichtner]

19.7.x OS = 20.1.x OS with no modifications. It does not look like this is a software issue if it works sometimes, but not always.

[abplfab]

19.7.x always stable. After upgrading to 20.1.x no chance to get it working.

[fichtner]

It’s probably a boot timing issue then. On 19.7 the netgraph drivers are loaded, on 20.1 not. See https://forum.opnsense.org/index.php?topic=15653.0

I can’t think of anything else that would make sense software-wise.

On 31. Jan 2020, at 10:04, Fabian Abplanalp [e-mail] wrote:


19.7.x always stable. After upgrading to 20.1.x no chance to get it working.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or unsubscribe.

[AdSchellevis]

other sfp modules (or cables) very often help fix these kind of issues in our experience, often unstable connections point to issues there. The connector contains the transceiver, which is responsible for the connection (some cards even check for specific firmware in the transceiver).