Smartzone 100 and R730 Reboots/ Offline Issue

[kevin_7278725]

Is anyone having issues with R730's randomly going Offline and requiring the AP to be rebooted in order to get it back online?  We have been seeing this issue for about 30 days now and support has been unable to tell us why the R730's are doing this.  The AP's are inaccessible most of the time when the issue happens, but sometimes we are able to still ping the AP and actually SSH to the login prompt, but the admin account/password will fail.  Once we power cycle the AP it works fine again.   When the issue is happening the AP is still accepting clients but it has no network access so those clients are broken.  It is very frustrating and Ruckus support has been no help.  The R730's are connected to Ruckus ICX 7650's via 5Gb multigig ports.  The switches report no problems and there are other AP's on the same switch at the time that have no issue, so the problem is just random Access Point specific.  The issue is completely random, no pattern can be found, other than support telling us they are seeing AP kernel panics and that they can't tell us why or how to make stop.   

Smartzone 100 version is 5.1.2.0.302 - which support had us upgrade to as they said that would fix the kernel panics - It has not

R730 version -  5.1.2.0.373

A few of the R730's have not been able to recover from this issue after a reboot and have had to be RMA'd.  Some of them will automatically reboot after 15-30 mins, but if we manually reboot them they typically come back online and work.  Was curious if any else is experiencing this issue with R730's, Smartzone 100's, and ICX 7650's?  

[michael_brado]

ACK your concerns. 


Typically an Engineering test build will turn on additional debugs, which might slow an AP processing a little.
Random / occasional problems are quite difficult to pinpoint sometimes.

[kevin_7278725]

Wanted to provide an update since this have been going on for over a month and Ruckus has only been able to fix 1 of what is currently now 5 issues/problems we are experiencing with their product.  I hope people evaluating Ruckus see this post and are warned of what to expect with Ruckus support/engineering.  They should not have products that kernel panic and have no way of being able to understand why, or take weeks to fix.
 
This is the only item they have fixed 
1) R730 - Kernel Panic - DNS issue >>>>[Resolved] - They fixed this with AP firmware 5.1.2.0.1013

These still have no fix and randomly happen daily
2) R730 - Kernel Panic - Soft Lockup - generates apHeartbeatLost alert and reboots after 1 minute

3) R730 - Kernel Panic - Watchdog Timeout - generates apHeartbeatLost alert and reboots after 1 minute

4) R730 - AP Hangs and no longer works. generates apHeartbeatLost and goes offline. It reboots itself after 30 mins

5) Smartzone Cassandra Database issue - APs show offline in SZ - They believe this is false positive and bug but no fix provided yet.

Ruckus is daily building custom scripts, which they never seem to get correct until multiple attempts, to try and figure out the root of their problems. We feel like we are definitely acting as their QA department and finding bug after bug with the SmartZone 100 and R730's.  Another issue is it seems all their escalation people are on the West Coast so we hear nothing until 1-2PM Eastern time and then it is just to have a remote session to collect more and more logs with no resolution in sight.

[mark_rock_ceoe2]

We have 3 R730 in our area along with R710, R700 and R500.  2 of the R730's in our main meeting area have started to reboot and run for about 5 to 10 minutes and then loose heartbeat and reboot.  We are running 5.1.2.0.203 on our VM Controller and 5.1.2.0.373 on AP Firmware.  Seemed to start after we did the upgrade to 5.1.2.0.203.  Not opened case yet, but will tomorrow.

[kevin_adams_g5a]

over 4 months and Ruckus has still not resolved their problems with the R730's and SmartZone 100.  We have helped them discover at least 7 bugs/issues between the SmartZone 100's and Ruckus R730's but they have only resolved 2 of them.   All seem to point to memory issues with Ruckus code and yet their escalated "tiger" engineering team has still yet to be able to fix the issues so we continually have AP's reboot daily.   Ruckus needs a new tag line as "Simply Better" is no longer accurate.

[kevin_7278725]

Over 4 months and Ruckus has still not resolved their problems with the R730's and SmartZone 100.  We have helped them discover at least 7 bugs/issues between the SmartZone 100's and Ruckus R730's but they have only resolved 2 of those issues.   All seem to point to memory issues with Ruckus code and yet their escalated "tiger" engineering team has still yet to be able to fix the issues so we continually have AP's reboot daily.   Ruckus needs a new tag line as "Simply Better" is no longer accurate.