ICX6610 10G Port Flapping Problems

marin_raskovic — Thu, 22 Feb 2018 16:05:16 GMT

Hi,
We are using two stacked ICX6610 switches and have several servers and storage connected to it. We have observed problems on our 10G ports - on server site (vmware esx hosts) we see constant link down/up events, and on storage ports we have CRC errors and port flapping condition.
No errors are found on 1Gb ports, no flapping, no CRC, no link state changes.
Sometimes we even get to APD event on vmware (we use 10G for NFS connection hosts<->storage).
FastIron version used is 08.0.10g.
Brief description (excluding 1G network connections):
3 ESX hosts named host1, host2, host3 are connected to 10G ports.
host1 - ports 1/3/3 and 2/3/3
host2 - ports 1/3/4 and 2/3/4
host3 - ports 1/3/5 and 2/3/5
No teaming on hosts side - only virtual port based load balancing on virtual switches.
Backup server is connected to ports 1/3/6 and 2/3/6. Here we use NIC teaming at server side (static link aggregateion) and switch side - LAG.
Storage is connected to ports 1/3/1 & 2/3/1 and uses LAG.
From one stack member there is only one active uplink to rest of network - port 1/1/24.

I have copy-pasted our switch config below including port descriptions. Does anybody see some error in configuration?

ICX6610-S-1>show configuration
!
Startup-config data location is flash memory
!
Startup configuration:
!
ver 08.0.10gT7f1
!
stack unit 1
module 1 icx6610-24-port-management-module
module 2 icx6610-qsfp-10-port-160g-module
module 3 icx6610-8-port-10g-dual-mode-module
stack-trunk 1/2/1 to 1/2/2
stack-trunk 1/2/6 to 1/2/7
stack-port 1/2/1 1/2/6
stack unit 2
module 1 icx6610-24-port-management-module
module 2 icx6610-qsfp-10-port-160g-module
module 3 icx6610-8-port-10g-dual-mode-module
stack-trunk 2/2/1 to 2/2/2
stack-trunk 2/2/6 to 2/2/7
stack-port 2/2/1 2/2/6
stack enable
stack mac 609c.9f47.ee38
!
!
!
lag "storage2_10g_ports" static id 6
ports ethernet 1/3/2 ethernet 2/3/2
primary-port 1/3/2
deploy
!
lag "storage1_10g_ports" static id 2
ports ethernet 1/3/1 ethernet 2/3/1
primary-port 1/3/1
deploy
!
lag "BackupServer_10g_ports" static id 1
ports ethernet 1/3/6 ethernet 2/3/6
primary-port 1/3/6
deploy
!
!
vlan 1 name DEFAULT-VLAN by port
!
vlan 6 by port
tagged ethe 1/1/4 to 1/1/9 ethe 1/1/24 ethe 2/1/4 to 2/1/9
!
vlan 7 by port
tagged ethe 1/1/4 to 1/1/9 ethe 1/1/24 ethe 2/1/4 to 2/1/9
untagged ethe 1/1/19 to 1/1/20 ethe 2/1/19 to 2/1/20
!
vlan 110 name ManagementNetwork by port
tagged ethe 1/1/4 to 1/1/9 ethe 1/1/24 ethe 2/1/4 to 2/1/9
untagged ethe 1/1/1 to 1/1/3 ethe 1/1/10 to 1/1/12 ethe 2/1/1 to 2/1/3 ethe 2/1/10
!
vlan 111 name ServersNetwork by port
tagged ethe 1/1/4 to 1/1/9 ethe 1/1/24 ethe 2/1/4 to 2/1/9
untagged ethe 1/1/13 ethe 2/1/11 to 2/1/13
!
vlan 112 name StorageNetwork by port
tagged ethe 1/3/1 to 1/3/5 ethe 2/3/1 to 2/3/5
untagged ethe 1/3/6 ethe 2/3/6
!
vlan 113 name vMotionNetwork by port
tagged ethe 1/3/3 to 1/3/5 ethe 2/3/3 to 2/3/5
!
vlan 155 name Servers2network by port
tagged ethe 1/1/4 to 1/1/9 ethe 1/1/24 ethe 2/1/4 to 2/1/9
!
!
!
!
aaa authentication web-server default local
aaa authentication enable default local
aaa authentication login default local
jumbo
enable super-user-password .....
enable aaa console
hostname ICX6610-S-1
ip address xxx.xxx.xxx.xxx 255.255.255.0
no ip dhcp-client enable
ip default-gateway xxx.xxx.xxx.xxx
!
logging host xxx.xxx.xxx.x
logging persistence
no telnet server
telnet server enable vlan 111
username admin password .....
cdp run
!
!
clock summer-time
clock timezone gmt GMT+01
!
!
ntp
server x.x.x.x minpoll 5 burst
!
!
web-management https
hitless-failover enable
interface ethernet 1/1/4
port-name host01 slot2 port0 vmnic6
!
interface ethernet 1/1/5
port-name host01 slot3 port0 vmnic8
!
interface ethernet 1/1/6
port-name host02 slot2 port0 vmnic6
spanning-tree 802-1w admin-edge-port
!
interface ethernet 1/1/7
port-name host02 slot3 port0 vmnic8
spanning-tree 802-1w admin-edge-port
!
interface ethernet 1/1/8
port-name host03 slot2 port0 vmnic6
spanning-tree 802-1w admin-edge-port
!
interface ethernet 1/1/9
port-name host03 slot3 port0 vmnic8
spanning-tree 802-1w admin-edge-port
!
interface ethernet 1/1/11
spanning-tree 802-1w admin-edge-port
!
interface ethernet 1/1/22
port-name "VLAN 110 uplink to cisco port X - disconnected cable"
!
interface ethernet 1/1/23
port-name "VLAN 111 uplink to cisco port X - disconnected cable"
!
interface ethernet 1/1/24
port-name "Uplink port to central Cisco"
!
interface ethernet 1/3/1
speed-duplex 10G-full
!
interface ethernet 1/3/2
speed-duplex 10G-full
spanning-tree 802-1w admin-edge-port
!
interface ethernet 1/3/3
port-name saphost01 slot0 port0 vmnic2
speed-duplex 10G-full
!
interface ethernet 1/3/4
port-name host02 slot0 port0 vmnic2
speed-duplex 10G-full
spanning-tree 802-1w admin-edge-port
!
interface ethernet 1/3/5
port-name host03 slot0 port0 vmnic2
speed-duplex 10G-full
spanning-tree 802-1w admin-edge-port
!
interface ethernet 1/3/6
speed-duplex 10G-full
spanning-tree 802-1w admin-edge-port
!
interface ethernet 1/3/7
speed-duplex 10G-full
!
interface ethernet 1/3/8
speed-duplex 10G-full
!
interface ethernet 2/1/4
port-name host01 slot2 port1 vmnic7
!
interface ethernet 2/1/5
port-name host01 slot3 port1 vmnic9
!
interface ethernet 2/1/6
port-name host02 slot2 port 1 vmnic7
spanning-tree 802-1w admin-edge-port
!
interface ethernet 2/1/7
port-name host02 slot3 port1 vmnic9
spanning-tree 802-1w admin-edge-port
!
interface ethernet 2/1/8
port-name host03 slot2 port1 vmnic7
spanning-tree 802-1w admin-edge-port
!
interface ethernet 2/1/9
port-name host03 slot3 port1 vmnic9
spanning-tree 802-1w admin-edge-port
!
interface ethernet 2/3/3
port-name host01 slot1 port1 vmnic5
speed-duplex 10G-full
!
interface ethernet 2/3/4
port-name host02 slot1 port1 vmnic5
speed-duplex 10G-full
spanning-tree 802-1w admin-edge-port
!
interface ethernet 2/3/5
port-name host03 slot1 port1 vmnic5
speed-duplex 10G-full
spanning-tree 802-1w admin-edge-port
!
!
!
!
lldp run
!
!
!
!
end

ICX6610-S-1>

Re: ICX6610 10G Port Flapping Problems

BenBeck — Thu, 08 Mar 2018 15:08:59 GMT

What is the media used? Is it showing up as a "state up/down" event in the logs? That would be a physical issue. I would move to 8030 latest since 8010 is a dead code train. Code may have nothing to do with this, however.

topic Re: ICX6610 10G Port Flapping Problems in ICX Switches

ICX6610 10G Port Flapping Problems

Re: ICX6610 10G Port Flapping Problems