topic Re: ICX 7450-24 unable to use both 40g ports as uplink ports in ICX Switches

ICX 7450-24 unable to use both 40g ports as uplink ports

glydeon — Fri, 29 Dec 2023 23:01:47 GMT

I have ICX 7450-24 updated to 09.0.10hT211 and I am unable to use both 40g ports at the same time to connect to two different TrueNAS servers. It does not matter which cable I use on which 40g port I use. First cable plugged in wins and that server is at 40g. The second server plugged in works for a minute or two, then disconnects. Am I able to use both 40 ports to connect to servers?

As stated, the first server will work fine off of either 40g with any cable I use.

Re: ICX 7450-24 unable to use both 40g ports as uplink ports

Chandini — Sun, 31 Dec 2023 17:27:02 GMT

Hi Glydon

Thank you for reaching us.

From the details you have mentioned, you are using two different TrueNAS servers. I suppose these are for storage purpose.

Does the 40g connection going to 2nd server have same ip address or different ip address ?
Is the 2nd server a backup server or is it used for completely different purpose ?
Which ports in the switch are you connecting to the server ?

Also could you help us with below output from the switch when you connect both links to server.

show media validation
show int e <port number> // 1st port
show int e <port number> // 2nd port

Thanks

Re: ICX 7450-24 unable to use both 40g ports as uplink ports

glydeon — Sun, 31 Dec 2023 20:36:02 GMT

telnet@link2#show media validation

Port Supported Vendor Type
----------------------------------------------------------------------
1/2/1 Yes OTHERS Type : 10GE SR 300m (SFP+)
1/2/2 Yes OTHERS Type : 10GE SR 300m (SFP+)
1/2/3 Yes OTHERS Type : 10GE SR 300m (SFP+)
1/2/4 Yes OTHERS Type : 10GE SR 300m (SFP+)
1/3/1 Yes OTHERSK Type : 40GBASE-Passive Copper 2m (QSFP+)
1/4/1 Yes OTHERSK Type : 40GBASE-Passive Copper 2m (QSFP+)
telnet@link2#

I think my first post was flagged as it has disappeared. So I am going to break this up into 3 posts.

Re: ICX 7450-24 unable to use both 40g ports as uplink ports

glydeon — Sun, 31 Dec 2023 20:36:47 GMT

telnet@link2#show int e 1/3/1
40GigabitEthernet1/3/1 is down, line protocol is down
Port down for 1 hour(s) 59 minute(s) 41 second(s)
Hardware is 40GigabitEthernet, address is cc4e.248a.395d (bia cc4e.248a.395d)
Interface type is 40Gig Copper
Configured speed 40Gbit, actual unknown, configured duplex fdx, actual unknown
Configured mdi mode AUTO, actual unknown
Untagged member of L2 VLAN 1, port state is BLOCKING
BPDU guard is Disabled, ROOT protect is Disabled, Designated protect is Disabled
Link Error Dampening is Disabled
STP configured to ON, priority is level7, mac-learning is enabled
MACsec is Disabled
Flow Control is config enabled, oper enabled, negotiation disabled
Mirror disabled, Monitor disabled
Mac-notification is disabled
VLAN-Mapping is disabled
Not member of any active trunks
Not member of any configured trunks
Port name is architect
IPG MII 96 bits-time, IPG GMII 96 bits-time
MTU 1500 bytes
MMU Mode is Store-and-forward
300 second input rate: 0 bits/sec, 0 packets/sec, 0.00% utilization
300 second output rate: 0 bits/sec, 0 packets/sec, 0.00% utilization
11 packets input, 704 bytes, 0 no buffer
Received 0 broadcasts, 11 multicasts, 0 unicasts
0 input errors, 0 CRC, 0 frame, 0 ignored
0 runts, 0 giants
115783 packets output, 13025215 bytes, 0 underruns
Transmitted 65636 broadcasts, 50126 multicasts, 21 unicasts
0 output errors, 0 collisions
Relay Agent Information option: Disabled
Protected: No
MAC Port Security: Disabled
Flexlink: Disabled

UC Egress queues:
Queue counters Queued packets Dropped Packets
0 0 0
1 0 0
2 0 0
3 0 0
4 0 0
5 0 0
6 0 0
7 7051 0

MC Egress queues:
Queue counters Queued packets Dropped Packets
0 4881 0
1 78616 0
2 720 0
3 24515 0

telnet@link2#

Re: ICX 7450-24 unable to use both 40g ports as uplink ports

glydeon — Sun, 31 Dec 2023 20:39:59 GMT

Both servers have different IPs and are being used for storage. Currently, one is connected @ 40g and the other is connected @ 10g fiber. All on the same switch. I can connected either server to a 40g port with any cable, but just cant connect both at the same time to the two 40g ports on this switch. But I can have one on a 10g fiber SFP+ and the other on 40G on the same switch with no issues.

Re: ICX 7450-24 unable to use both 40g ports as uplink ports

glydeon — Sun, 31 Dec 2023 21:03:37 GMT

I am unable to post the requested info for my second 40g port due to your spam filters. I will try again later.

Re: ICX 7450-24 unable to use both 40g ports as uplink ports

glydeon — Sun, 31 Dec 2023 21:06:24 GMT

UC Egress queues:
Queue counters Queued packets Dropped Packets
0 128293463 0
1 0 0
2 0 0
3 0 0
4 372714 0
5 6049 0
6 0 0
7 183693261 0

MC Egress queues:
Queue counters Queued packets Dropped Packets
0 64952 0
1 400957 0
2 4207 0
3 510675 0

SSH@link2>

Re: ICX 7450-24 unable to use both 40g ports as uplink ports

Chandini — Mon, 01 Jan 2024 16:21:18 GMT

Hi Glydon

Ack and Thanks. Thanks for reaching us. From the details shared the Vendor details of all the transceiver is mentioned as Other meaning its a 3rd party vendor.

1/2/1 Yes OTHERS Type : 10GE SR 300m (SFP+)
1/2/2 Yes OTHERS Type : 10GE SR 300m (SFP+)
1/2/3 Yes OTHERS Type : 10GE SR 300m (SFP+)
1/2/4 Yes OTHERS Type : 10GE SR 300m (SFP+)
1/3/1 Yes OTHERSK Type : 40GBASE-Passive Copper 2m (QSFP+)
1/4/1 Yes OTHERSK Type : 40GBASE-Passive Copper 2m (QSFP+)

Since the transceiver used is 3rd party vendor it would be difficult to understand the behavior of the 3rd party transceiver with the switches as they are manufactured by a different company and we would be unable to validate the behavior of a 3rd part transceiver as it belongs to a different company.

Are you seeing this problem even when you use a Ruckus Transceiver ? If yes could you share a picture of serial details of RUCKUS transceiver you are using ?

Thanks

Re: ICX 7450-24 unable to use both 40g ports as uplink ports

glydeon — Mon, 01 Jan 2024 16:38:30 GMT

I do not have any Rukus transceivers at this time. But I have tested and proved that both cables are good, both 40g mondules are good, and both servers can connect. The only issue is that only one will work at a time.

If I purchase 2 sets of Ruckus transceivers, will this unit support 2 servers connected at the same time through those 40g modules?

Re: ICX 7450-24 unable to use both 40g ports as uplink ports

Chandini — Wed, 03 Jan 2024 16:01:44 GMT

Hi Glydon

Ack and Thanks for reaching us. I understand both 40G ports should be operable and there should not be a problem, But since the transceiver used is 3rd Party the behavior of the unit with the switch would be difficult to understand.

Just to isolate further I would suggest you to log a case with us using the below link. Also there would be a need to share more data to understand the behavior here.

https://support.ruckuswireless.com/contact-us

Thanks

Re: ICX 7450-24 unable to use both 40g ports as uplink ports

glydeon — Fri, 26 Jan 2024 09:37:17 GMT

I purchased some Brocade / Ruckus 40g cables and will be testing with these shortly.

Re: ICX 7450-24 unable to use both 40g ports as uplink ports

glydeon — Fri, 02 Feb 2024 12:38:16 GMT

Well the new cables came in and both are working perfectly. It still does not make sense to me why either one of the other cables would work, but only one at a time. But I am satisfied that the issue was resolved. Thanks for the help. This one was self inflicted on my part.

Issue resolved. This can be closed. Thanks again for all of the help