topic Unleashed/ICX7150 C12P problem since 10.0.10F update in ICX Switches

Unleashed/ICX7150 C12P problem since 10.0.10F update

lievensict — Sat, 28 Dec 2024 16:41:26 GMT

I recently upgraded my ICX7150-C12P to 10.0.10f SPR firmware from the 09.0.10j.cd4 SPS and since re-adding the switch to unleashed i have the problem that the switch keeps wanting to join on 5.137.250.224.(see screenshot below) Even tho the switch is already connected to unleashed and otherwise working fine.

When approving this "switch" it isn't able to communicate to this specific interface, it also can't be removed. Removing the existing switch doesn't resolve it either. I can get it working again by specifing the correct IP in unleashed for the discovery, but then i get the annoying 5.137.*.* IP back in the approval list. This isn't a valid IP in my subnet, and also never set in the switch. The switch has also been factory reset since the firmware upgrade and the troubleshooting.

Unleashed logs report: Failed to add ICX [c0:c5:20:99:4f:f1]. The ICX already exists. Please enter a different one. The MAC address from the one i specified (my managemt ip) and the one unleashed tries to connect (5.137.*.*) are the same (?)

I'm running unleashed version 200.16.7.0.402, i have tried to upgrade 200.17 but this gave me alot of issues with my R350 AP's.

I would like to keep on the 10.0 track now, but if there isn't any fix or update expected soon i think it's the best to go back to the 9.0 track.

This is what the unleashed logs tell me:

Dec 28 17:39:03 RuckusAP Beneden sys_wrapper: icx_test_default():ICXD: ICX[id=2]: 5.137.250.224: start to access ICX as user "super".
Dec 28 17:39:03 RuckusAP Beneden syslog: M8d2023S0D0 pid=2769[14546], icxd_state_loop():ICXD: ICX[id=2]: 5.137.250.224: OLD_STATE=[discovery], have_approve=1, icx_username=[]
Dec 28 17:39:02 RuckusAP Beneden ZD-APMgr: Ma33b02S0D0 IPC_thread rcv ping from TACMON
Dec 28 17:38:53 RuckusAP Beneden syslog: M8d2022S0D0 pid=2769[14546], icxd_state_loop():ICXD: ICX[id=2]: 5.137.250.224: NEW_STATE=[discovery], delay 10 seconds.
Dec 28 17:38:53 RuckusAP Beneden syslog: M8d2022S0D0 pid=2769[14546], icxd_discovery():ICXD: ICX[id=2]: 5.137.250.224: icx-test-default done, result is [NO_RESULT]
Dec 28 17:38:53 RuckusAP Beneden sys_wrapper: icx_test_default():ICXD: ICX[id=2]: 5.137.250.224: lastline "==================== LOGIN(test-default): (Sat Dec 28 17:38:43 CET 2024) ====================", expect "assword:".
Dec 28 17:38:51 RuckusAP Beneden syslog: Mad1fd6S0D0 pid=2769[2769], doJobEx():worker: action:[cleanup-response], Done.
Dec 28 17:38:43 RuckusAP Beneden empty: forked dbclient -y -l super 5.137.250.224
Dec 28 17:38:43 RuckusAP Beneden empty: version 0.6.20b started
Dec 28 17:38:43 RuckusAP Beneden sys_wrapper: icx_test_default():ICXD: ICX[id=2]: 5.137.250.224: ping 5.137.250.224 success.
Dec 28 17:38:43 RuckusAP Beneden mping[21230]: status=succ result=34 to_pid=1243 ports=1 ports_signaled=1
Dec 28 17:38:43 RuckusAP Beneden mping[21230]: mping 512: cmd=arping -u -z 195 -q -f -c1 -w1 -S 34 -P 21230 -l/tmp/mping_hm_latency.log -I br0 192.168.168.1 &
Dec 28 17:38:43 RuckusAP Beneden mping[21230]: Target default gateway 192.168.168.1 via br0
Dec 28 17:38:41 RuckusAP Beneden sys_wrapper: icx_test_default():ICXD: ICX[id=2]: 5.137.250.224: start to access ICX as user "super".
Dec 28 17:38:41 RuckusAP Beneden syslog: M8d2022S0D0 pid=2769[14546], icxd_state_loop():ICXD: ICX[id=2]: 5.137.250.224: OLD_STATE=[discovery], have_approve=1, icx_username=[]
Dec 28 17:38:32 RuckusAP Beneden ZD-APMgr: Ma33b01S0D0 IPC_thread rcv ping from TACMON
Dec 28 17:38:31 RuckusAP Beneden syslog: M8d2021S0D0 pid=2769[14546], icxd_state_loop():ICXD: ICX[id=2]: 5.137.250.224: NEW_STATE=[discovery], delay 10 seconds.
Dec 28 17:38:31 RuckusAP Beneden syslog: M8d2021S0D0 pid=2769[14546], icxd_discovery():ICXD: ICX[id=2]: 5.137.250.224: icx-test-default done, result is [NO_RESULT]
Dec 28 17:38:31 RuckusAP Beneden sys_wrapper: icx_test_default():ICXD: ICX[id=2]: 5.137.250.224: lastline "==================== LOGIN(test-default): (Sat Dec 28 17:38:21 CET 2024) ====================", expect "assword:".
Dec 28 17:38:21 RuckusAP Beneden empty: forked dbclient -y -l super 5.137.250.224
Dec 28 17:38:21 RuckusAP Beneden empty: version 0.6.20b started
Dec 28 17:38:21 RuckusAP Beneden sys_wrapper: icx_test_default():ICXD: ICX[id=2]: 5.137.250.224: ping 5.137.250.224 success.
Dec 28 17:38:21 RuckusAP Beneden syslog: Mad1fceS0D0 pid=2769[2769], doJobEx():worker: action:[cleanup-response], Done.
Dec 28 17:38:19 RuckusAP Beneden sys_wrapper: icx_test_default():ICXD: ICX[id=2]: 5.137.250.224: start to access ICX as user "super".
Dec 28 17:38:19 RuckusAP Beneden syslog: M8d2021S0D0 pid=2769[14546], icxd_state_loop():ICXD: ICX[id=2]: 5.137.250.224: OLD_STATE=[discovery], have_approve=1, icx_username=[]
Dec 28 17:38:13 RuckusAP Beneden mping[20980]: status=succ result=34 to_pid=1243 ports=1 ports_signaled=1
Dec 28 17:38:13 RuckusAP Beneden mping[20980]: mping 512: cmd=arping -u -z 195 -q -f -c1 -w1 -S 34 -P 20980 -l/tmp/mping_hm_latency.log -I br0 192.168.168.1 &
Dec 28 17:38:13 RuckusAP Beneden mping[20980]: Target default gateway 192.168.168.1 via br0
Dec 28 17:38:12 RuckusAP Beneden syslog: Mad1fc9S0D0 pid=2769[2769], icxd_av_is_supported():ICXD: AV_Profile[icx-id:2,unit-id=1] The model don't support AV profile
Dec 28 17:38:12 RuckusAP Beneden syslog: upgrade_debug: pid=20976, insert_field_to_upgrade_ap_list(): firmware_file open failure
Dec 28 17:38:12 RuckusAP Beneden syslog: upgrade_debug: pid=20976, get_ap_list(): see [/tmp/unleashed_upgrade/upgrade_ap_list_from_election.xml.ready], ap-list ready!
Dec 28 17:38:12 RuckusAP Beneden syslog: upgrade_debug: pid=20976, get_ap_list(): ap-list loop 0.
Dec 28 17:38:09 RuckusAP Beneden syslog: M8d2020S0D0 pid=2769[14546], icxd_state_loop():ICXD: ICX[id=2]: 5.137.250.224: NEW_STATE=[discovery], delay 10 seconds.
Dec 28 17:38:09 RuckusAP Beneden syslog: M8d2020S0D0 pid=2769[14546], icxd_discovery():ICXD: ICX[id=2]: 5.137.250.224: icx-test-default done, result is [NO_RESULT]
Dec 28 17:38:09 RuckusAP Beneden sys_wrapper: icx_test_default():ICXD: ICX[id=2]: 5.137.250.224: lastline "==================== LOGIN(test-default): (Sat Dec 28 17:37:59 CET 2024) ====================", expect "assword:".
Dec 28 17:38:02 RuckusAP Beneden ZD-APMgr: Ma33b00S0D0 IPC_thread rcv ping from TACMON
Dec 28 17:37:59 RuckusAP Beneden empty: forked dbclient -y -l super 5.137.250.224
Dec 28 17:37:59 RuckusAP Beneden empty: version 0.6.20b started
Dec 28 17:37:59 RuckusAP Beneden sys_wrapper: icx_test_default():ICXD: ICX[id=2]: 5.137.250.224: ping 5.137.250.224 success.
Dec 28 17:37:57 RuckusAP Beneden sys_wrapper: icx_test_default():ICXD: ICX[id=2]: 5.137.250.224: start to access ICX as user "super".
Dec 28 17:37:57 RuckusAP Beneden syslog: M8d2020S0D0 pid=2769[14546], icxd_state_loop():ICXD: ICX[id=2]: 5.137.250.224: OLD_STATE=[discovery], have_approve=1, icx_username=[]
Dec 28 17:37:51 RuckusAP Beneden syslog: Mad1fb7S0D0 pid=2769[2769], doJobEx():worker: action:[cleanup-response], Done.

Re: Unleashed/ICX7150 C12P problem since 10.0.10F update

MariaC862 — Mon, 06 Jan 2025 21:28:08 GMT

Hi!

Thank you for posting you query, just curious, does reverting the ICX to the previous software correct the issue?

Can they see each other as LLDP neighbors? on ICX with #show lldp neighbor and on the AP cli as:

ruckus(ap-mode)# get lldp neighbors

Monitoring Connected ICX Switches

Is automatic approval enabled?

Did the credentials change? could you please verify we have the correct credentials for ICX approval?

Is there any way we can track where that ip is comming from? is there any rogue DHCP server?

Thanks.

Re: Unleashed/ICX7150 C12P problem since 10.0.10F update

lievensict — Sun, 12 Jan 2025 16:14:59 GMT

Hello,

Thank you for your response. Really appreciate the help.

This is a log from the Unleashed master via the CLI. It does show up here but don't know where it gets it from as why the switch would (apparently) send this out.

ruckus(ap-mode)#get lldp neighbors ------------------------------------------------------------------------------- LLDP neighbors: ------------------------------------------------------------------------------- Interface: eth0, via: LLDP, RID: 1, Time: 21 days, 14:42:14 Chassis: ChassisID: mac c0:c5:20:99:4f:f1 SysName: sw-mer-core-1 SysDescr: Ruckus Wireless, Inc. ICX7150-C12-POE, IronWare Version 10.0.10fT213 Compiled on Nov 27 2024 at 06:25:23 labeled as SPR10010f MgmtIP: 5.137.250.224 Capability: Bridge, on Capability: Router, on Port: PortID: mac c0:c5:20:99:4f:f2 PortDescr: GigabitEthernet1/1/2 ------------------------------------------------------------------------------- OK

Output from the switch:

SSH@sw-mer-core-1#show lldp neighbors Lcl Port Chassis ID Port ID Port Description System Name 1/1/2 7047.773c.7b40 7047.773c.7b40 eth0 RuckusAP Beneden 1/2/2 cc98.9136.a4b8 gi10 sw-der-dist-1

I ran a few more command for you to see, that it's not just configured in one of the simple options:

SSH@sw-mer-core-1#show ip Global Settings ttl: 64, arp-age: 10, bootp-relay-max-hops: 4 router-id : 192.168.168.2 SSH@sw-mer-core-1#show vlan Total PORT-VLAN entries: 1 Maximum PORT-VLAN entries: 1024 Legend: [Stk=Stack-Id, S=Slot] PORT-VLAN 1, Name DEFAULT-VLAN, Priority level0, On SSH@sw-mer-core-1#show ip interface sInterface IP-Address OK? Method Status Protocol VRF Eth mgmt1 10.22.3.45 YES NVRAM up up default-vrf Ve 1 192.168.168.2 YES manual up up default-vrf SSH@sw-mer-core-1#show interfaces management 1 GigEthernetmgmt1 is up, line protocol is up Port up for 21 day(s) 15 hour(s) 31 minute(s) 4 second(s) Hardware is GigEthernet, address is c0c5.2099.4ff1 (bia c0c5.2099.4ffd) Configured speed auto, actual 1Gbit, configured duplex fdx, actual fdx Configured mdi mode AUTO, actual AUTO Not a Member of any VLAN , port is untagged, port state is NONE Internet address is 10.22.3.45/24

Yes correct credentials. Turning on automatic approval in Unleashed breaks the connection, i can't add it manually now. So if I turn it off it i can add it manually and it stays.

It is able to ping, but no SSH (also no Telnet). Advanced IP scan shows this:

Pretty sure there is no rogue DHCP server in my network. Haven't seen this IP ever before the update. I'm sure I didn't set it, and it came back even after the reset. Is there any reason this has appeared because of the router firmware?

Even tho this shows up in Unleashed, the connectivity seems fine fortunately. Also, I don't have the time to downgrade now, but will try this hopefully soon if there doens't come another 10.0.10 update. I will update on this post.

Thank you for your time.

Re: Unleashed/ICX7150 C12P problem since 10.0.10F update

dietmar_giessma — Sun, 12 Jan 2025 16:42:54 GMT

Hello, I#ve seen the same situation in my homelab, show lldp local-info shows as management ip only the:

5.137.250.224

related to:

% Information related to '5.137.0.0 - 5.137.255.255'

% Abuse contact for '5.137.0.0 - 5.137.255.255' is 'abuse@rt.ru'

inetnum: 5.137.0.0 - 5.137.255.255
netname: WEBSTREAM
descr: JSC Rostelecom regional branch "Siberia"
remarks: Novosibirsk broadband service
country: RU

A IP bind to the int ve xx, did not anything, I corrected the management IP to want I want :

lldp advertise management-address ipv4 192.168.2.9 ports e 1/1/1 to 1/3/2

Now it works in Unleashed, but this is only a little workarround, not a solution.

Regards.

Dietmar

Re: Unleashed/ICX7150 C12P problem since 10.0.10F update

dietmar_giessma — Sun, 12 Jan 2025 19:23:04 GMT

Hello,

I see also the management IP: 5.137.250.224 with cli:

sh lldp local-info

I set the IP to the same as the ve :

lldp advertise management-address ipv4 ....

now it works. It is a workarround not a solution.

Regards!

Dietmar

Re: Unleashed/ICX7150 C12P problem since 10.0.10F update

MariaC862 — Mon, 13 Jan 2025 17:42:43 GMT

Thank you for your reply, the info shared confirms that the ip 5.137.250.224 is not configured and the rest of data seems normal. Please let me know how the downgrade goes. If the issue stops after the downgrade we will consider escalating it as it seems to be a defect.

Re: Unleashed/ICX7150 C12P problem since 10.0.10F update

MariaC862 — Mon, 13 Jan 2025 18:02:08 GMT

Hi!

Did you see that workaround somewhere or did you discover it your self?

I ask this because I'm trying to find any documentation related to this behavior and so far I haven't found much.

Re: Unleashed/ICX7150 C12P problem since 10.0.10F update

antoine_hannema — Sat, 18 Jan 2025 10:24:33 GMT

I have exactly the same issue, reverting to 09.0.10j.CD6 fixed the situation again. It seems in 10.0.10f there is a defect resulting in a broken ssh/ip link form unleashed to the 7150C12P series switches. Please fix the 10.0.10 code for 7150 series in unleashed environment.

Re: Unleashed/ICX7150 C12P problem since 10.0.10F update

MariaC862 — Mon, 20 Jan 2025 20:01:58 GMT

Hello!

Thanks to everyone who confirmed this issue, to further investigate the problem and escalate it we need you to open a case with TAC support.

Open a Case via online Support Portal

Re: Unleashed/ICX7150 C12P problem since 10.0.10F update

dietmar_giessma — Sat, 25 Jan 2025 18:50:32 GMT

I have opened a case.

Re: Unleashed/ICX7150 C12P problem since 10.0.10F update

antoine_hannema — Sat, 25 Jan 2025 19:48:18 GMT

thx, i've been testing meanwhile, unleashed 200.15.6.212.20 in combination with icx7150C12P running SPS09010j_cd6 work fine. But upgrading to 10.0.10f results in error as the switch advertises 5.137.250.224. Because lievensict (TS) is running 200.16, i suspect the switch firmware 10.0.10f to be broken instead of an issue in unleashed firmware...

Re: Unleashed/ICX7150 C12P problem since 10.0.10F update

dietmar_giessma — Sat, 25 Jan 2025 20:33:54 GMT

Hello,

I have no support for my testdevice:

However, our records indicate that you do not have an active support contract which provides technical assistance.

So I can't get support.

Re: Unleashed/ICX7150 C12P problem since 10.0.10F update

dietmar_giessma — Mon, 27 Jan 2025 07:33:30 GMT

The case was closed, the device has no active support contract.

I have tested in another location with a ICX7150-24P. There I get a different IP, but a whois

says :

inetnum: 5.138.0.0 - 5.138.127.255
netname: Macroregional_South
descr: OJSC Rostelecom Macroregional Branch South
descr: Stavropol, Russia
country: RU

I hope, it is a programming mistake and not a security impact !

Local port: 1/1/1 (not transmitting currently)
+ Chassis ID (MAC address): dcae.eb32.b9a0
+ Port ID (MAC address): dcae.eb32.b9a0
+ Time to live: 120 seconds
+ System name : "SW20-FMGBO-FMG"
+ Port description : "GigabitEthernet1/1/1"
+ System description : "Ruckus Wireless, Inc. ICX7150-24-POE, IronWare Version 10.0.10f_cd1T213"
+ System capabilities : bridge, router
Enabled capabilities: bridge, router
+ 802.3 MAC/PHY : auto-negotiation enabled
Advertised capabilities: 10BaseT-HD, 10BaseT-FD, 100BaseTX-HD,
100BaseTX-FD, fdxSPause, fdxBPause, 1000BaseT-HD,
1000BaseT-FD
Operational MAU type : Other
+ 802.3 Power via MDI: PSE port, power enabled, class 0
Power Pair : A (not controllable)
+ Link aggregation: not capable
+ Maximum frame size: 1522 octets
+ Port VLAN ID: 99
+ Management address (IPv4): 5.138.9.72

Re: Unleashed/ICX7150 C12P problem since 10.0.10F update

Michael-St — Mon, 27 Jan 2025 12:24:10 GMT

RUCKUS Support is already investigating this topic. I will keep you posted.

Re: Unleashed/ICX7150 C12P problem since 10.0.10F update

Michael-St — Fri, 28 Feb 2025 18:10:54 GMT

This issue is not related to Unleashed but rather to ICX 10.0.10f. It is fixed in upcoming ICX 10.0.10f cd releases planned to be published in March.

Re: Unleashed/ICX7150 C12P problem since 10.0.10F update

antoine_hannema — Fri, 28 Feb 2025 19:17:28 GMT

@Michael-St

Thanks for the information!

Meanwhile the workaround by LLDP advertisement works:

lldp advertise management-address ipv4 [IP ADDRESS] ports ethe 1/1/1 to 1/1/12 ethe 1/2/1 to 1/2/2 ethe 1/3/1 to 1/3/2

Rgrds, Antoine

Re: Unleashed/ICX7150 C12P problem since 10.0.10F update

DavyAllinco — Sat, 15 Mar 2025 19:10:38 GMT

Dear Michael,

Thank you for your response. I just tried it with 10.0.10F_CD1, but the problem came back without a manual LLDP advertise management-address ipv4.

I didn't see anything about this issue in the know issues or fixed issues in the release notes.

The switched wanted to join on a different IP as before:

Thank you in advance!