topic Re: R500 APs port flapping on Cisco Switch in To Be Moved

R500 APs port flapping on Cisco Switch

scott_bailey_gv — Wed, 01 Mar 2017 20:59:24 GMT

Hello,

I have 4x R500 APs with a 1200 Zone Director on version 9.13.3.0 build 22. The APs are connected to a Cisco WS-C2960X-48LPS-L POE switch on version 15.2(2)E6.

The issue is that the 4 ports that these APs are connected to randomly go down throughout the day as shown here:

Mar 1 13:03:43: %LINK-3-UPDOWN: Interface GigabitEthernet1/0/3, changed state to down
Mar 1 13:03:47: %LINK-3-UPDOWN: Interface GigabitEthernet1/0/3, changed state to up
Mar 1 13:03:49: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet1/0/3, changed state to up
Mar 1 13:04:14: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet1/0/3, changed state to down
Mar 1 13:04:15: %LINK-3-UPDOWN: Interface GigabitEthernet1/0/3, changed state to down
Mar 1 13:04:18: %LINK-3-UPDOWN: Interface GigabitEthernet1/0/3, changed state to up
Mar 1 13:04:20: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet1/0/3, changed state to up

The AP themselves do not show as rebooting but I have verified that users do lose connection when this happens. I cannot reproduce the issue and I cannot find any sort of pattern. Some days one of the ports will not drop at all but the other 3 will. Or some days it might be only one or 2 that drop randomly.

I have verified the cabling multiple times, I even pulled out brand new patch cables and used them and still get the same result. I tried moving one of the APs to an older cisco switch and left it there for a day and it did not drop at all. So that would leave me to believe it's something on that switch itself but I can't seem to track it down. There are no POE events on the switch when a port goes down. I also have other POE devices (mostly phones) that are plugged into the same switch and do not have an issue.

The only thing that stands out to me is that there are a lot STP topology changes on the AP VLAN that originate from the switch ports that the APs are connected to and they line up with when the switch ports drop but I'm not sure if that's a cause or effect of the ports dropping. I originally had spanning-tree bpdu guard and loop guard configured on the ports but I removed them from half of them but there was no change in behavior.

I did open a case with Ruckus support and they looked at the logs and said they see nothing on their side that would cause this. I'm hoping someone else has run into this issue.

Any help would be appreciated as I'm starting to run out of ideas of where to look.

Thanks!

Re: R500 APs port flapping on Cisco Switch

michael_brado — Wed, 01 Mar 2017 22:47:54 GMT

Hi Scott,

Do you use PoE devices on all the other switch ports too? Can you connect the APs
on different "blocks" of switch ports, or alternating top/bottom (kitty-corner) instead of
right on top and next to each other in a block of 4? It might be that the switch is having
a PoE drain on the one chip that powers sets of 4 ports per PCB. I've seen it before.
I hope this separation helps.

Re: R500 APs port flapping on Cisco Switch

scott_bailey_gv — Thu, 02 Mar 2017 14:40:45 GMT

Hi Michael,

Thanks for the reply. These are 2 switches in a stack configuration. I currently have 2 APs on the switch and 2 APs on the other. I have moved an AP to another block on ports on the switches but unfortunately still get the same result.

I don't have many other POE devices on these switches. I only have 5 other devices and they are phones and they are pretty spread out over the switch.

Re: R500 APs port flapping on Cisco Switch

hyosang_choi — Fri, 03 Mar 2017 12:59:33 GMT

I have experienced similary symptom too..

I do not yet find reason.

Our ap is R500 connected to Cisco 2960 too, and it happened frequently up/down at interface connected the AP.

Anyone experienced similary symptom?

Re: R500 APs port flapping on Cisco Switch

scott_bailey_gv — Mon, 06 Mar 2017 21:36:50 GMT

Just want to update this in case anyone else runs into it. It took me awhile to track it down but my issue was that I was hitting this bug:

https://bst.cloudapps.cisco.com/bugsearch/bug/CSCus35889/?referring_site=bugquickviewclick

The "energy-efficient ethernet" feature was causing the issue. Applying the command "no power efficient-ethernet" to the ports having the issue resolved it.

Re: R500 APs port flapping on Cisco Switch

michael_brado — Wed, 08 Mar 2017 18:01:21 GMT

Scott, I don't have a SmartNet contract to view the bug info. Can you please copy/paste it for us here?
I'd like to capture it in a knowledge article. Thanks!

Re: R500 APs port flapping on Cisco Switch

scott_bailey_gv — Wed, 08 Mar 2017 18:58:21 GMT

Hi Michael,

Here you go:

"EEE" on 2960X and PD device causing intermittent link flapCSCus35889

Symptom:
Dec 30 03:34:54: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet1/0/26, changed state to down
Dec 30 03:35:04: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet1/0/26, changed state to up
Dec 30 03:49:50: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet1/0/26, changed state to down
Dec 30 03:50:00: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet1/0/26, changed state to up

Line protocol flapping, some times with link up/down also, which would cause PDs like AP disconnected/reboot.

Conditions:
When "Energy-Efficient Ethernet" feature enabled on both Catalyst 2960X switchport and on the PDs.

Affected images:
c2960x-universalk9-mz.152-2.E.bin
c2960x-universalk9-mz.150-2.EX5

Workaround:
Disable Energy-Efficient Ethernet (EEE) by below CLI command:
2960X(config-if)#no power efficient-ethernet

Re: R500 APs port flapping on Cisco Switch

hyosang_choi — Thu, 09 Mar 2017 00:44:54 GMT

Thank you very much, Scott.

To. Michael.

You can find details w/o login on above URL.

https://quickview.cloudapps.cisco.com/quickview/bug/CSCus35889

And plz upload artical for this issue on KB.

I have experienced this issue recently too.

Regards.

Re: R500 APs port flapping on Cisco Switch

edgars_muceniek — Mon, 20 Mar 2017 07:05:47 GMT

Thanks Scott, we faced same problem but our poe switch is Cisco small business SG300-28PP. "no eee enabled" on switch port solved ports flapping.

Re: R500 APs port flapping on Cisco Switch

michael_brado — Mon, 20 Mar 2017 17:00:37 GMT

Thanks all, KBA-6303 will appear tomorrow.