Cisco Switches not seeing VLAN from Ruckus/Brocade Switch

[amathias]

Hi all, 

I'm having an interesting issue with the Ruckus/Cisco switches on our campus. 

For context, we had a scheduled power outage at our site back in December, and when power was restored, we started seeing this issue. We have all our core equipment on a generator and did not lose power. No configuration changes happened according to our backups. 

Our core switch is a Ruckus ICX7650-48F. After power was restored, I could not ssh into any Cisco switch directly connected to the core. Any HP/Aruba device directly connected works with SSH and ping. The same is true for any cisco device downstream from an HP/Aruba. Our management vlan is broken between the Ruckus and Ciscos, but I need help finding where. 

Although this is happening, everything around the site is running fine from an operational standpoint. We have wireless and all ethernet working.

We have a Cisco 3650 directly below the core in the rack that I've been troubleshooting. When I checked spanning tree on it, I noticed a VLAN ID mismatch with two different VLANs. Our management VLAN 4 and the Trunk VLAN 666. Nothing has changed in the configuration, and I've double-checked that there isn't a mismatch like it says. I get this error with all Cisco switches.

Below is an attachment with config and spaning-tree results from both the Ruckus and Cisco. If you need more let me know and I'll post it as well. 

Thank you!

Show run

Ruckus ICX7650-48F:

vlan 4 name NetMgmt by port
tagged ethe 1/1/31
spanning-tree

vlan 666 name TRUNK by port
untagged ethe 1/1/31
spanning-tree

Cisco 3650:

interface TenGigabitEthernet1/1/3
 switchport trunk native vlan 666
 switchport mode trunk


show spanning-tree

Ruckus: 

show span vlan 4
PORT 1/1/31 - FORWARDING

show span vlan 666
PORT 1/1/31 - FORWARDING

Cisco: 

show spanning-tree vlan 4

VLAN0004
  Spanning tree enabled protocol ieee
  Root ID    Priority    32772
             Address     006b.f1df.9480
             Cost        8
             Port        10 (GigabitEthernet1/0/10)
             Hello Time   2 sec  Max Age 20 sec  Forward Delay 15 sec

  Bridge ID  Priority    32772  (priority 32768 sys-id-ext 4)
             Address     f44e.0566.f780
             Hello Time   2 sec  Max Age 20 sec  Forward Delay 15 sec
             Aging Time  15  sec

Interface           Role Sts Cost      Prio.Nbr Type
------------------- ---- --- --------- -------- --------------------------------
Te1/1/3             Desg BKN*2         128.51   P2p *PVID_Inc 

show spanning-tree vlan 666

VLAN0666
  Spanning tree enabled protocol ieee
  Root ID    Priority    33434
             Address     006b.f1df.9480
             Cost        8
             Port        10 (GigabitEthernet1/0/10)
             Hello Time   2 sec  Max Age 20 sec  Forward Delay 15 sec

  Bridge ID  Priority    33434  (priority 32768 sys-id-ext 666)
             Address     f44e.0566.f780
             Hello Time   2 sec  Max Age 20 sec  Forward Delay 15 sec
             Aging Time  300 sec

Interface           Role Sts Cost      Prio.Nbr Type
------------------- ---- --- --------- -------- --------------------------------
Te1/1/3             Desg BKN*2         128.51   P2p *PVID_Inc 

[BenBeck]

At a high-level, it looks correct: tagged vlan 4, untagged vlan666 for port 1/1/31

I would check layer 2 mac learning, arp, ping, etc. I would try to decipher if the data path is okay and if something is happening with SSH/Mgmt itself. 

The best bet would be to open a support case (see my signature) and we can help follow the breadcrumbs.