nav[aria-label="Primary Navigation"] { padding: 0; & ul { list-style: none; width: 100%; display: flex; flex-direction: row; justify-content: start; align-items: start; gap: 30px; padding: 0; & li { margin: 0; } & ul li { list-style: none; } } }

Ruckus ICX7150 Stuck at Query state while connecting to smartzone

fauzan_slim

Hi Ruckus Community,

We have 800++ ICX 7150 switches spread across many locations (branch office) connected to SmartZone in Data Center via Cisco SD-WAN. Almost 30% of the switch are indicated offline in the SmartZone, but actually, they are online and working properly. All the switches which disconnected/offline in the SmartZone are stuck at Query state.
Our switches are :
- all using UFI images
- firmware is vary , some using 08090d some using 08095d
- ports ranging from 24 to 48 port
- the WAN latency between DC and branch is under 100ms

Troubleshooting that we try:
- trigger the 'query registrar'
- disable and enable the manager on the switch side
- doing zeroize on crypto key and re-regenerate the SSL certificate on the switch, the doing restart on the switch
- not all the switch is offline, even on the same branch we often found that only several switches is indicated offline on the SmartZone

After we do the step above, it appears some switch is back to an online state again, but after several days the switch is back to offline again

the last step we do is restarting all offline switches but this appears doesn't seem to help that much

is there any WAN link or firewall policy consideration or limitation that causing those issues?

Find more posts tagged with

Accepted answers

All comments

Vineet_nejwala

Hi @fauzan_slim

We would need to focus on a couple of switches first to start with. If this is a virtual controller you need to look if the system capacity is not in issue ? Since you have mentioned that same site few switches are online while some are not this shouldn't be a firewall issue. If you have an active support on the controller I would suggest to open a case with TAC since this will need some dedicated time and logs to debug. Below logs can be attached on TAC case to debug and you can open a case at once using below link : https://support.ruckuswireless.com/contact-us

1)From couple of ICX switches :
show version
show sz status
sh sz client-log detail
show sz server-log detail

2)From the SZ provide the snapshot log and below :
vSZ#show service
vSZ#show version
vSZ#show system-capacity
vSZ#show cpuinfo
vSZ#show diskinfo
vSZ#show meminfo

Best Regards

Vineet

fauzan_slim

Hi @Vineet_nejwala ,

Thanks for the response , here's the output

ON THE ICX SWITCH

1. show version

The output of 'show version'
switch#show version
Copyright (c) Ruckus Networks, Inc. All rights reserved.
UNIT 1: compiled on Sep 23 2019 at 00:00:25 labeled as SPR08090d
(32539748 bytes) from Secondary SPR08090d.bin (UFI)
SW: Version 08.0.90dT213
Compressed Secondary Boot Code size = 786944, Version:10.1.15T225 (mnz10115)
Compiled on Thu Jan 31 07:08:55 2019

HW: Stackable ICX7150-48-POE
==========================================================================
UNIT 1: SL 1: ICX7150-48P-4X1G POE 48-port Management Module
Serial #:FEC3241R015
Software Package: ICX7150_BASE_L3_SOFT_PACKAGE
Current License: 4X1G
P-ASIC 0: type B160, rev 11 Chip BCM56160_B0
==========================================================================
UNIT 1: SL 2: ICX7150-2X1GC 2-port 2G Module
==========================================================================
UNIT 1: SL 3: ICX7150-4X10GF 4-port 40G Module
==========================================================================
1000 MHz ARM processor ARMv7 88 MHz bus
8192 KB boot flash memory
2048 MB code flash memory
1024 MB DRAM
STACKID 1 system uptime is 61 day(s) 12 hour(s) 55 minute(s) 34 second(s)
The system started at 16:23:11 GMT+00 Fri Jun 24 2022

The system : started=warm start reloaded=by "reload"

2. show sz status

SSH@SW-00014-JakartaHarmoni-Ac24-R7150#show sz status

============ SZ Agent State Info ===================
Config Status: None Operation Status: Enabled
State: SZ QUERY Prev State: SZ SSH CONNECTED Event: SZ QUERY RESPONSE

SWR List : None
Active List : 172.15.98.73, 172.15.98.29
DHCP Option 43 : No
DHCP Opt 43 List : None
Passive List : None
Merged List : 172.15.98.73, 172.15.98.29
Merged Idx: 0 IP : 172.15.98.73
Switch registrar host: sw-registrar.ruckuswireless.com
Switch registrar discovery retry count: 88617
Switch registrar host resolve failure count: 88616

SZ IP Used : 172.15.98.29
SZ Query Status :
In Progress. Response Not Received.

SSH Tunnel Status - :
Tunnel Status : Not Initiated
CLI IP/Port : /0
SNMP IP/Port : /0
Syslog IP/Port : /0
HTTP SERVER IP/Port: /0
HTTP CLIENT IP/Port: /0

Timer Status : Running
Timer Details in msec
Start Time Stamp : 962187078
Duration : 64601212
Current Time Stamp: 1022134722
Time Left : 4653568

3. sh sz client-log detail

there is no option to that command instead, I will give the output of 'sh sz logs'

Aug 24 11:00:57:https_connmgr_send_request>Entered.
Aug 24 11:00:57:sz_execute_state_machine>Exit with state/event: SZ QUERY/5, TIMER/2002 RC: 1
Aug 24 11:00:57:sz_execute_state_machine>Entering with state/event: SZ QUERY/5, SZ QUERY RESPONSE/2007
Aug 24 11:00:57:sz_parse_sz_query_response -- Status: 600 <<
Aug 24 11:00:57:sz_execute_state_machine>Exit with state/event: SZ QUERY/5, SZ QUERY RESPONSE/2007 RC: 1
Aug 24 11:00:57:HTTP Request Error:Http remote connection close called.
Aug 25 03:48:38:Sending SZ_SWR_QUERY_RESTART event...
Aug 25 03:48:38:Failed to resolve sw-registrar.ruckuswireless.com
End i/max/iter 165/165/0

4. show sz server-log detail

there is no such command in the switch side

ON THE SZ CONTROLLER

1. show service

2. show version

3.show system-capacity

4. show cpuinfo

5. show diskinfo

6. show meminfo

Best Regards,

Fauzan

fauzan_slim

Hi @Vineet_nejwala

Heres the output below ,

ICX SWITCH

1.show version

Copyright (c) Ruckus Networks, Inc. All rights reserved.
UNIT 1: compiled on Sep 23 2019 at 00:00:25 labeled as SPR08090d
(32539748 bytes) from Secondary SPR08090d.bin (UFI)
SW: Version 08.0.90dT213
Compressed Secondary Boot Code size = 786944, Version:10.1.15T225 (mnz10115)
Compiled on Thu Jan 31 07:08:55 2019

HW: Stackable ICX7150-48-POE
==========================================================================
UNIT 1: SL 1: ICX7150-48P-4X1G POE 48-port Management Module
Serial #:FEC3241R015
Software Package: ICX7150_BASE_L3_SOFT_PACKAGE
Current License: 4X1G
P-ASIC 0: type B160, rev 11 Chip BCM56160_B0
==========================================================================
UNIT 1: SL 2: ICX7150-2X1GC 2-port 2G Module
==========================================================================
UNIT 1: SL 3: ICX7150-4X10GF 4-port 40G Module
==========================================================================
1000 MHz ARM processor ARMv7 88 MHz bus
8192 KB boot flash memory
2048 MB code flash memory
1024 MB DRAM
STACKID 1 system uptime is 61 day(s) 13 hour(s) 18 minute(s) 20 second(s)
The system started at 16:23:12 GMT+00 Fri Jun 24 2022

The system : started=warm start reloaded=by "reload"

2.show sz status

============ SZ Agent State Info ===================
Config Status: None Operation Status: Enabled
State: SZ QUERY Prev State: SZ SSH CONNECTED Event: SZ QUERY RESPONSE

SZ IP Used : 172.15.98.29
SZ Query Status :
In Progress. Response Not Received.

SSH Tunnel Status - :
Tunnel Status : Not Initiated
CLI IP/Port : /0
SNMP IP/Port : /0
Syslog IP/Port : /0
HTTP SERVER IP/Port: /0
HTTP CLIENT IP/Port: /0

Timer Status : Running
Timer Details in msec
Start Time Stamp : 962187078
Duration : 64601212
Current Time Stamp: 1023435120
Time Left : 3353170

3.sh sz client-log detail

there is no such command in the switch , instead I will display you the 'sh sz logs' command

Aug 24 11:00:56:https_connmgr_send_request>Entered.
Aug 24 11:00:56:sz_execute_state_machine>Exit with state/event: SZ QUERY/5, TIMER/2002 RC: 1
Aug 24 11:00:57:sz_execute_state_machine>Entering with state/event: SZ QUERY/5, SZ QUERY RESPONSE/2007
Aug 24 11:00:57:sz_parse_sz_query_response -- Status: 600 <<
Aug 24 11:00:57:sz_execute_state_machine>Exit with state/event: SZ QUERY/5, SZ QUERY RESPONSE/2007 RC: 1
Aug 24 11:00:57:HTTP Request Error:Http remote connection close called.
Aug 25 03:48:38:Sending SZ_SWR_QUERY_RESTART event...
Aug 25 03:48:38:Failed to resolve sw-registrar.ruckuswireless.com
Aug 25 03:49:38:Failed to resolve sw-registrar.ruckuswireless.com
Aug 25 03:50:38:Failed to resolve sw-registrar.ruckuswireless.com
End i/max/iter 167/167/0

4.show sz server-log detail

there is no such command in the switch

SMARTZONE CONTROLLER

1.vSZ#show service

2.vSZ#show version

3.vSZ#show system-capacity

4.vSZ#show cpuinfo

5.vSZ#show diskinfo

6.vSZ#show meminfo

fauzan_slim

Hi @Vineet_nejwala

Here's the output below,

ICX SWITCH

1.show version

HW: Stackable ICX7150-48-POE
==========================================================================
UNIT 1: SL 1: ICX7150-48P-4X1G POE 48-port Management Module
Serial #:FEC3241R015
Software Package: ICX7150_BASE_L3_SOFT_PACKAGE
Current License: 4X1G
P-ASIC 0: type B160, rev 11 Chip BCM56160_B0
==========================================================================
UNIT 1: SL 2: ICX7150-2X1GC 2-port 2G Module
==========================================================================
UNIT 1: SL 3: ICX7150-4X10GF 4-port 40G Module
==========================================================================
1000 MHz ARM processor ARMv7 88 MHz bus
8192 KB boot flash memory
2048 MB code flash memory
1024 MB DRAM
STACKID 1 system uptime is 61 day(s) 13 hour(s) 18 minute(s) 20 second(s)
The system started at 16:23:12 GMT+00 Fri Jun 24 2022

The system : started=warm start reloaded=by "reload"

2.show sz status

============ SZ Agent State Info ===================
Config Status: None Operation Status: Enabled
State: SZ QUERY Prev State: SZ SSH CONNECTED Event: SZ QUERY RESPONSE

SZ IP Used : 172.15.98.29
SZ Query Status :
In Progress. Response Not Received.

SSH Tunnel Status - :
Tunnel Status : Not Initiated
CLI IP/Port : /0
SNMP IP/Port : /0
Syslog IP/Port : /0
HTTP SERVER IP/Port: /0
HTTP CLIENT IP/Port: /0

Timer Status : Running
Timer Details in msec
Start Time Stamp : 962187078
Duration : 64601212
Current Time Stamp: 1023435120
Time Left : 3353170

3.sh sz client-log detail

there is no such command in the switch , instead I will display you the 'sh sz logs' command

there is no such command in the switch

SMARTZONE CONTROLLER

1.vSZ#show service

2.vSZ#show version

3.vSZ#show system-capacity

4.vSZ#show cpuinfo

5.vSZ#show diskinfo

6.vSZ#show meminfo

Regards,

Fauzan

fauzan_slim

Hi @Vineet_nejwala

Heres the output below ,

ICX SWITCH

1.show version

HW: Stackable ICX7150-48-POE
==========================================================================
UNIT 1: SL 1: ICX7150-48P-4X1G POE 48-port Management Module
Serial #:FEC3241R015
Software Package: ICX7150_BASE_L3_SOFT_PACKAGE
Current License: 4X1G
P-ASIC 0: type B160, rev 11 Chip BCM56160_B0
==========================================================================
UNIT 1: SL 2: ICX7150-2X1GC 2-port 2G Module
==========================================================================
UNIT 1: SL 3: ICX7150-4X10GF 4-port 40G Module
==========================================================================
1000 MHz ARM processor ARMv7 88 MHz bus
8192 KB boot flash memory
2048 MB code flash memory
1024 MB DRAM
STACKID 1 system uptime is 61 day(s) 13 hour(s) 18 minute(s) 20 second(s)
The system started at 16:23:12 GMT+00 Fri Jun 24 2022

The system : started=warm start reloaded=by "reload"

2.show sz status

============ SZ Agent State Info ===================
Config Status: None Operation Status: Enabled
State: SZ QUERY Prev State: SZ SSH CONNECTED Event: SZ QUERY RESPONSE

SZ IP Used : 172.15.98.29
SZ Query Status :
In Progress. Response Not Received.

SSH Tunnel Status - :
Tunnel Status : Not Initiated
CLI IP/Port : /0
SNMP IP/Port : /0
Syslog IP/Port : /0
HTTP SERVER IP/Port: /0
HTTP CLIENT IP/Port: /0

Timer Status : Running
Timer Details in msec
Start Time Stamp : 962187078
Duration : 64601212
Current Time Stamp: 1023435120
Time Left : 3353170

3.sh sz client-log detail

there is no such command in the switch , instead I will display you the 'sh sz logs' command

there is no such command in the switch

SMARTZONE CONTROLLER

1.vSZ#show service

2.vSZ#show version

3.vSZ#show system-capacity

4.vSZ#show cpuinfo

5.vSZ#show diskinfo

6.vSZ#show meminfo

Regards,

Fauzan

fauzan_slim

Hi @Vineet_nejwala

Heres the output below ,

ICX SWITCH

1.show version

HW: Stackable ICX7150-48-POE
==========================================================================
UNIT 1: SL 1: ICX7150-48P-4X1G POE 48-port Management Module
Serial #:FEC3241R015
Software Package: ICX7150_BASE_L3_SOFT_PACKAGE
Current License: 4X1G
P-ASIC 0: type B160, rev 11 Chip BCM56160_B0
==========================================================================
UNIT 1: SL 2: ICX7150-2X1GC 2-port 2G Module
==========================================================================
UNIT 1: SL 3: ICX7150-4X10GF 4-port 40G Module
==========================================================================
1000 MHz ARM processor ARMv7 88 MHz bus
8192 KB boot flash memory
2048 MB code flash memory
1024 MB DRAM
STACKID 1 system uptime is 61 day(s) 13 hour(s) 18 minute(s) 20 second(s)
The system started at 16:23:12 GMT+00 Fri Jun 24 2022

The system : started=warm start reloaded=by "reload"

2.show sz status

============ SZ Agent State Info ===================
Config Status: None Operation Status: Enabled
State: SZ QUERY Prev State: SZ SSH CONNECTED Event: SZ QUERY RESPONSE

SZ IP Used : 172.15.98.29
SZ Query Status :
In Progress. Response Not Received.

SSH Tunnel Status - :
Tunnel Status : Not Initiated
CLI IP/Port : /0
SNMP IP/Port : /0
Syslog IP/Port : /0
HTTP SERVER IP/Port: /0
HTTP CLIENT IP/Port: /0

Timer Status : Running
Timer Details in msec
Start Time Stamp : 962187078
Duration : 64601212
Current Time Stamp: 1023435120
Time Left : 3353170

3.sh sz client-log detail

there is no such command in the switch , instead I will display you the 'sh sz logs' command

there is no such command in the switch

SMARTZONE CONTROLLER

1.vSZ#show service

2.vSZ#show version

3.vSZ#show system-capacity

4.vSZ#show cpuinfo

5.vSZ#show diskinfo

6.vSZ#show meminfo

Regards ,

Fauzan

fauzan_slim

Hi @Vineet_nejwala

Heres the output below ,

ICX SWITCH

1.show version

HW: Stackable ICX7150-48-POE
==========================================================================
UNIT 1: SL 1: ICX7150-48P-4X1G POE 48-port Management Module
Serial #:FEC3241R015
Software Package: ICX7150_BASE_L3_SOFT_PACKAGE
Current License: 4X1G
P-ASIC 0: type B160, rev 11 Chip BCM56160_B0
==========================================================================
UNIT 1: SL 2: ICX7150-2X1GC 2-port 2G Module
==========================================================================
UNIT 1: SL 3: ICX7150-4X10GF 4-port 40G Module
==========================================================================
1000 MHz ARM processor ARMv7 88 MHz bus
8192 KB boot flash memory
2048 MB code flash memory
1024 MB DRAM
STACKID 1 system uptime is 61 day(s) 13 hour(s) 18 minute(s) 20 second(s)
The system started at 16:23:12 GMT+00 Fri Jun 24 2022

The system : started=warm start reloaded=by "reload"

2.show sz status

============ SZ Agent State Info ===================
Config Status: None Operation Status: Enabled
State: SZ QUERY Prev State: SZ SSH CONNECTED Event: SZ QUERY RESPONSE

SZ IP Used : 172.15.98.29
SZ Query Status :
In Progress. Response Not Received.

SSH Tunnel Status - :
Tunnel Status : Not Initiated
CLI IP/Port : /0
SNMP IP/Port : /0
Syslog IP/Port : /0
HTTP SERVER IP/Port: /0
HTTP CLIENT IP/Port: /0

Timer Status : Running
Timer Details in msec
Start Time Stamp : 962187078
Duration : 64601212
Current Time Stamp: 1023435120
Time Left : 3353170

3.sh sz client-log detail

there is no such command in the switch , instead I will display you the 'sh sz logs' command

there is no such command in the switch

SMARTZONE CONTROLLER

1.vSZ#show service

2.vSZ#show version

3.vSZ#show system-capacity

4.vSZ#show cpuinfo

5.vSZ#show diskinfo

6.vSZ#show meminfo

Regards ,

Fauzan

fauzan_slim

Hi @Vineet_nejwala

Heres the output below ,

ICX SWITCH

1.show version

HW: Stackable ICX7150-48-POE
==========================================================================
UNIT 1: SL 1: ICX7150-48P-4X1G POE 48-port Management Module
Serial #:FEC3241R015
Software Package: ICX7150_BASE_L3_SOFT_PACKAGE
Current License: 4X1G
P-ASIC 0: type B160, rev 11 Chip BCM56160_B0
==========================================================================
UNIT 1: SL 2: ICX7150-2X1GC 2-port 2G Module
==========================================================================
UNIT 1: SL 3: ICX7150-4X10GF 4-port 40G Module
==========================================================================
1000 MHz ARM processor ARMv7 88 MHz bus
8192 KB boot flash memory
2048 MB code flash memory
1024 MB DRAM
STACKID 1 system uptime is 61 day(s) 13 hour(s) 18 minute(s) 20 second(s)
The system started at 16:23:12 GMT+00 Fri Jun 24 2022

The system : started=warm start reloaded=by "reload"

2.show sz status

============ SZ Agent State Info ===================
Config Status: None Operation Status: Enabled
State: SZ QUERY Prev State: SZ SSH CONNECTED Event: SZ QUERY RESPONSE

SZ IP Used : 172.15.98.29
SZ Query Status :
In Progress. Response Not Received.

SSH Tunnel Status - :
Tunnel Status : Not Initiated
CLI IP/Port : /0
SNMP IP/Port : /0
Syslog IP/Port : /0
HTTP SERVER IP/Port: /0
HTTP CLIENT IP/Port: /0

Timer Status : Running
Timer Details in msec
Start Time Stamp : 962187078
Duration : 64601212
Current Time Stamp: 1023435120
Time Left : 3353170

3.sh sz client-log detail

there is no such command in the switch , instead I will display you the 'sh sz logs' command

there is no such command in the switch

SMARTZONE CONTROLLER

1.vSZ#show service

2.vSZ#show version

3.vSZ#show system-capacity

4.vSZ#show cpuinfo

5.vSZ#show diskinfo

6.vSZ#show meminfo

Regards ,

Fauzan

Raden_Bagas

I have the same problem, is there any solution that can help to solve it?

Ivo

Hello, I would like to ask about a switch Mod. ICX7150-48-4X1G a Sys port lit up in orange and the send in green and no network port works even by turning the device off and on again