R500 APs port flapping on Cisco Switch

[scott_bailey_gv]

Hello,

I have 4x R500 APs with a 1200 Zone Director on version 9.13.3.0 build 22. The APs are connected to a Cisco WS-C2960X-48LPS-L POE switch on version 15.2(2)E6.

The issue is that the 4 ports that these APs are connected to randomly go down throughout the day as shown here:

Mar  1 13:03:43: %LINK-3-UPDOWN: Interface GigabitEthernet1/0/3, changed state to down
Mar  1 13:03:47: %LINK-3-UPDOWN: Interface GigabitEthernet1/0/3, changed state to up
Mar  1 13:03:49: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet1/0/3, changed state to up
Mar  1 13:04:14: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet1/0/3, changed state to down
Mar  1 13:04:15: %LINK-3-UPDOWN: Interface GigabitEthernet1/0/3, changed state to down
Mar  1 13:04:18: %LINK-3-UPDOWN: Interface GigabitEthernet1/0/3, changed state to up
Mar  1 13:04:20: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet1/0/3, changed state to up

The AP themselves do not show as rebooting but I have verified that users do lose connection when this happens. I cannot reproduce the issue and I cannot find any sort of pattern. Some days one of the ports will not drop at all but the other 3 will. Or some days it might be only one or 2 that drop randomly.

I have verified the cabling multiple times, I even pulled out brand new patch cables and used them and still get the same result. I tried moving one of the APs to an older cisco switch and left it there for a day and it did not drop at all. So that would leave me to believe it's something on that switch itself but I can't seem to track it down. There are no POE events on the switch when a port goes down. I also have other POE devices (mostly phones) that are plugged into the same switch and do not have an issue.

The only thing that stands out to me is that there are a lot STP topology changes on the AP VLAN that originate from the switch ports that the APs are connected to and they line up with when the switch ports drop but I'm not sure if that's a cause or effect of the ports dropping. I originally had spanning-tree bpdu guard and loop guard configured on the ports but I removed them from half of them but there was no change in behavior.

I did open a case with Ruckus support and they looked at the logs and said they see nothing on their side that would cause this. I'm hoping someone else has run into this issue.

Any help would be appreciated as I'm starting to run out of ideas of where to look.

Thanks!

 

[michael_brado]

Hi Scott,

   Do you use PoE devices on all the other switch ports too?  Can you connect the APs
on different "blocks" of switch ports, or alternating top/bottom (kitty-corner) instead of
right on top and next to each other in a block of 4?  It might be that the switch is having
a PoE drain on the one chip that powers sets of 4 ports per PCB.  I've seen it before.
I hope this separation helps.

[scott_bailey_gv]

Hi Michael,

Thanks for the reply. These are 2 switches in a stack configuration. I currently have 2 APs on the switch and 2 APs on the other. I have moved an AP to another block on ports on the switches but unfortunately still get the same result. 

I don't have many other POE devices on these switches. I only have 5 other devices and they are phones and they are pretty spread out over the switch. 

[hyosang_choi]

I have experienced similary symptom too..

I do not yet find reason.

Our ap is R500 connected to Cisco 2960 too, and it happened frequently up/down at interface connected the AP.

Anyone experienced similary symptom?

[scott_bailey_gv]

Just want to update this in case anyone else runs into it. It took me awhile to track it down but my issue was that I was hitting this bug:

https://bst.cloudapps.cisco.com/bugsearch/bug/CSCus35889/?referring_site=bugquickviewclick

The "energy-efficient ethernet" feature was causing the issue. Applying the command "no power efficient-ethernet" to the ports having the issue resolved it.