nav[aria-label="Primary Navigation"] { padding: 0; & ul { list-style: none; width: 100%; display: flex; flex-direction: row; justify-content: start; align-items: start; gap: 30px; padding: 0; & li { margin: 0; } & ul li { list-style: none; } } }

ZD1200 Solid Green lights but dead - 3 different instances this week

unknown

So I am a tech for a system integration company that primarily uses Ruckus in all of our installs. I have had 3 calls this week about wifi down and each of them was the same result. ZD1200 appears to have completely died. All lights are solid green, even on a fresh reboot (never see the red status light at all even while booting). None of the units respond to ping, ssh, console...nothing. I found it very odd the first time I saw it since I have never seen this issue in 15 years of servicing, but to see it 3 times now this week has me very concerned. Not sure what to do next or even what to tell the clients? Do I have 3 randomly bricked ZD1200s in a week?

Find more posts tagged with

Accepted answers

All comments

michael_brado

Very strange, "dead" to you remotely and on the local LAN?
Logs, and open a ticket if not a network issue...

itdept_head_me

ZD1200 "bricking" in the same fashion.. seems quite common,
It's appears not "very strange", take a look at the forums.

Ruckus should publish an analysis of the casues, "upgrade", "component failure" , PSU... etc.
I lost one very recently, exactly the same.

It was powered by some very hefty top quality UPS and there were no surges or power cycles.
also in an air coned room....

Normal practice they recommend is "reset" with a pin, then if that fails " F/D", then RMA.

syamantakomer

Hi Tommy,

It is odd!

Could you try pulling console logs from all three failed devices? What console output says?

I would advise to log a case with console logs from all 3 failed devices and let support check further.

An standard flow of troubleshooting should be:

1- Check power adaptor and power source. Is the device powering up?

Yes = continue to step 2.
No = Hardware fault = Support case >> RMA.

2- Do you see a stable or flashing red LED on status?

Yes = Hardware fault = Support case >> RMA.
No = follow step 3.

3- Check network cable and switch port. Try to use a new cable and connect the ZD to a new switch port and second eth port of the ZD (just in case first Eth port on ZD may be faulty).
4- If above all looks good and still you are not able to access ZD, then follow troubleshooting steps given below in the same sequence (If issue not fixed then continue to next sub step given below):

4.1: Push the 'Reset' button (reset, not FD).
4.2: Reload the power.
4.3: Push the FD button and hold it for 12 seconds and then release it. Detailed procedure can be found here (Article #000010062).
4.4: Connect console cable (for settings click here. Article #000004232) >> Reload the power and check the logs on console window. Collect the logs.

5- Open a support case and provide the logs/output to support. Support may provide RMA but if you have multiple devices showing same issue then support can check and get the failure analysis done (Note: Failure analysis takes a lot of time so findings could take time).

Regards,
Syamantak Omer

unknown

Tommy,

I am having this same exact issue. I went down yesterday and I can not get it to boot up again. I am getting a solid green light on my ZD. I am going to follow Syamantek's steps now.

itdept_head_me

Doing what?
you powerd off?
FW update
power cycle?

I'm getting more interested in the failure modes of these ZD1200

unknown

Haven't had a chance to attempt to console into the units but to reiterate,

The units just sit there with all possible lights on them solid green at all times. Normally, when you reboot a ZD, the status light is flashing red until it is ready...these do not, they boot solid green, never blink, never change. They do not ping, reset button does nothing, I have not found a way to even make a light blink. They are Ice Cubes (frozen).

Like I said, what concerns me is I have never seen this issue at all in 15 years and have now seen it 3 times in 1 week. Hopefully I find an opportunity to attempt a console session soon.

I tried calling support but since the service contract on the units has expired, they hung up on me so I really do not know what to do now. I absolutely hate calling Ruckus for service, especially being a tech and not the owner, they refuse to even answer the simplest question without a service contract.

itdept_head_me

Ruckus are ok for support... but ur not gonna get any freebies.......
every minute of thier time is paid for by service contracts.....

if it's all green solid after following their tech note, it's an RMA , not seen one that works with console in this situation.

unknown

I contacted support yesterday and there was no troubleshooting at all. Told them solid green lights and they are just sending me a new one. So I am going to have a week of no wireless in our buildings. Luckily, secretaries and administrators are all hard wired in.

syamantakomer

Hi Mike,

Could you share the case number of our reference.

unknown

Case ID 01081342

syamantakomer

Hi Mitch,

I can see that case was raised as P1 with critical service impact.

To prevent any delay in resolution, RMA was processed without any delay.

Regards,
Syamantak Omer

unknown

Which version did you apply on zd?

itdept_head_me

No version..... I was told it just stopped, it was a contract job

unknown

Sorry to revive this thread but have 2 more of these now. How long should I wait after filling out the RTF form to hear back from Ruckus?? It's been almost 2 weeks now and I have not even received a conformation of submitting.

syamantakomer

Hi Tommy,

Could you confirm the case number.

In RTF mode of replacement, you have to first send faulty unit to RUCKUS, once received, it will take up-to 15 days to ship the replacement from RUCKUS.

unknown

I assumed I would get a confirmation email after submitting it so I did not write it down. I just filled out a new one today.

Case ID: 01135877

Hopefully I hear back this time.

unknown

Working as a tech lead in a small integration company also.

We're having 5 controllers with this same issue (Green status is ON).

It is definitely DEAD.

Tried removing CF card - nothing!

Tried removing RAM - nothing!

New power supply - nothing!

Ruckus team, please tell us everything is under control.

We're still having 50+ installations with these controllers.

And I cannot do RTF from Russia.

Cause for the replacement unit I will pay import taxes (30% of GPL). No deal((

itdept_head_me

Removing the internal parts will not solve problem.

Already tried this before.

Maybe take a working unit and copy the internal CF card & try that.

syamantakomer

Hi Maxim,

Open a support case and ask for FA (Failure Analysis), if you think it is a hardware defect on multiple devices.

Once faulty units received by FA team, they will check it for any possible hardware defects.

Before you take that path, could you please confirm what logs you see in ZD console?

unknown

Hi, Syamantak!

1. It is definitely the hardware defect:

- Connected through console. Nothing.

- Connected a monitor to the onboard blue VGA port. Nothing.

- Tried a known good CompactFlash from working ZD. Nothing.

2. The worst part is our agreement with distributors.

For ZD and SZ controllers in Russia we have just ONE YEAR of HW warranty from distributor. After one year - this become the problem of me and my customers...

3. Shipping faulty unit from Russia to FA team is almost impossible. I have to spend half a year to fill in EXPORT CONTROL paperwork. I know that with this regulation even the official Ruckus distributors are not shipping faulty units back...

4. With ZoneDirectors 1100 it was an easy part. Just replacing the faulty USB Flash drive inside.

With SZ104 and faulty HDDs it is harder but still possible.

This one is tough in us.

itdept_head_me

I have yet to see a failure analysis for my units that failed for the same reason.....

syamantakomer

Hi,

Could you share the case number.

itdept_head_me

They did not give me a case number.......

but I can give you a serial number for the most recent....

211508000702

within the last 5 months

syamantakomer

Hi,

This was just an RMA case. I don't see any request for FA when this case was raised.

Please note that FA is only done when support/engineering or customer suspects hardware issue with specific set of hardware batch.

If FA was not requested, only replacement will be provided. This is a standard process followed by most of the vendors.

unknown

Same issue i have recived ..Ruckus suggested for RMA

eizens_putnins

For my understanding, Ruckus doesn't repair Zd1200 -- as nobody repairs any similar equipment - repair cost is much higher than replacement cost). I am quit sure faulty units go directly to recycling. Shipping back is just to avoid customers declaring equipment faulty to get additional free part. Of cause, it would be good that Ruckus look on failed equipment to make part reliability reviews.

What was not mentioned in any message -- how old were faulty ZD1200 units? This may be important to know!

We had recently 2x ZD1200 faulty with similar symptoms, but there was serial console output showing bad flash in both cases.

Both units were 5+ years old, and were replaced under RTF without issues. 5+ years is reasonable time to become concerned about flash reliability. It is, of cause, possible, that latest ZD firmware versions write more on flash, so it fails faster, but in this case one of the failed units was on version 9.9 still.

ZD1200 is a relatively cheap device, and if network is important, you should have 2 of them for management (as licenses are additive, second ZD isn't actually much of additional expense as it adds 5 licenses to system), and than RTF is good enough even when support expired.

Russia is of cause specific case, but as it is same for all vendors there, it is just means higher costs using ZD1200 contorllers. I would recommend to use Unleashed or vSZ for all installations in Russia, which would avoid any need for RTF or AR for controllers at all.

Hope it helps.

itdept_head_me

Hi,

Well sorry but that is not how it works in the electronics industry.

Any company not running fault analysis on returned electronics is just asking for trouble

ah..... so bad flash..... its just like telsa

no one should be writing logs to flash memory...... the cycle life can be as low as 500 times...

itdept_head_me

They don't want this discussed in public.

all behind the scenes....

it's the flash.

re-flashing re-inits the storage & marks the bad blocks, but it is a fools game.

Eventually like all flash products it will cascade & lead to full failure.

unknown

Should be easy to prove with one working and one dead?

It should at least boot with a good CF in it? Even if they are locked to the board, which I would doubt.

Have you done any investigation as you have access to units?

Read the flash and checked for bad blocks? Found any issues?

Swapped flash?

I do agree flash would become a problem, my symptoms don't specifically indicate that.

Monitored CPU activity?

Does the VGA out normally show a boot screen like bios or so before loading OS?

There is normally a beep that sounds like a POST beep, perhaps you can see when that normally occurs on a good unit, with a monitor and or SSH or?

Oh, you said you did some CF research, yet not clear on result of replacing flash and why it failed again.

Curious

unknown

I would think the battery is another key failure point?

eizens_putnins

I have seen ZD1200 with died flash, usually it happened after or as a direct result of power loss, but sometimes flash is broken without obvious reason. It should be not a critical issue, as normally ZD1200 should be used in Smart redundancy configuration, and ZD1200 has life-time warranty, so you need just RMA it.

You can make sure taht it is flash issue, connecting by some terminal program to ZD1200 serial port. You'll see obvious errors during boot process.

In some cases, after power loss, flash is OK, but filesystem is damaged, in this case ZD can be recovered, reformatting flash and uploading new image to it. But if you see errors reading flash, don'r waste time and go directly to RMA option...

By the way, when WiFi network is down because of ZD1200 fault, it means that improper design was used -- it must be clear that if system is used with just 1x ZD1200, it becomes the single point of failure, as all WLANs are down, if it fails. As difference in price for system with redundant ZD1200 configuration with at least 10 APs is very small, it can't be anything else as a wrong (or uninformed) decision.

In many cases customers think that price for redundant ZD1200 pair is double, which is not true -- from version 9 licenses on both ZD are summarized, and price for support for redundant ZD1200 is much lower. So for redundant system with 10 APs you need second ZD1200, but as you don't need additional 5 licenses for the first ZD, and support for redundant ZD is really cheap, there is just a really small price difference for redundant solution, and it should be always used.

Anyway, we almost stopped to use ZD1200 in our projects 5 years ago -- small projects can be even better done using Unleashed or cloud, and anything bigger than few APs is best served by vSZ.

eizens_putnins

And I don't think ZD1200 has RTC battery. Never replaced it at least...

eizens_putnins

Beginning of this thread was about problems with warranty support in Russia, which are the same for all vendors because of limiting import regulations. So in Russia you probably should use only Unleashed, no ZS1200 -- as it seems clear enough that ZD1200 will be EOL in some near future, and why you want you additional device, which can make problems to you, and which you actually don't need?

APs itself are extra reliable, and it doesn't matter that you can't use life-time warranty in Russia - I had just a few replacement in 15 years, which makes it much lower than 0.1% failures.

unknown

ZD1200 has a battery in it.

unknown

Thanks for the responses!

Many small project don’t have redundant zone directors. Not every project has a support contract. This one I inherited doesn’t have either. This site is over 7 years old so unleashed was probably not an option, yet is the best now.

I agree, I didn’t like the single point of failure, unlike other brands where the wifi won’t go down without the controller. Or the other solutions here that auto heal. I did find that while Unleashed will switch to another if the master fails, the wifi goes down while switching, a small problem. Doesn’t seem to happen on another brand I’ve worked with.

I didn’t know the zone directors have a lifetime warranty lol. Guess I will call them today….. That will probably take weeks.

I was curious about the symptoms of different failures as this one has no signs of life except green LEDs. No beep, no-cli response. It did start up a few times before complete failure, so may be a flash issue.

I found out switching to Unleashed gains some features and loses some features. Most notably Unleashed has cloud access and an app, yet WPA3 is not in Unleashed unlike recent ZD firmware. Any other comments on ZD vs Unleashed? Other than the security issue with cloud access.

I haven’t explored the other options as the others require purchases, not available in a pinch and this is a small low budget site.

Thanks

unknown

@chris_borderit Mine died last Tuesday, opened a support case and received a new unit this morning. Excellent service for an eight year old piece of kit without a support contract on it. The whole support process has been refreshingly excellent, wish other vendors were half as helpful.

unknown

Nope, these guys are horrible, among the worst for RMA. Ubiquiti and netgear, no problem. These guys are a joke. They hassle me while I wait for another department for a response.

Unless you pay them for an 8 year piece of kit, they won't move anything forward. They also require return before shipping a replacement, unless you have been paying more than the unit is worth for 8 years.

Then proof of purchase for an 8 year old device with lifetime warranty, that had support on it for years, so it's registered.

Pulling it all out and putting in Ubiquiti after this fiasco, don't buy Duckus Commspoop.

itdept_head_me

@chris_borderit

yep but those two are garbage for security.....I know this because i did a "programmer /hacker" evaluation.

both sets of kit are banned from our IT procurement processes..

but you only need to do a CVE search....

Rucus have good code development & are on the ball, it's not like speaking to primary school students.

However they dropped the ball on the standalone logging and painted themselves into a corner.

A bit like apple did on their power saving methodology that screwed over their networking.........