R500 APs keep losing heardbeat, disconnecting and then reconnecting each day at the same time, why?

[christopher_hay]

Hi.  My R500s keep disconnecting and then reconnecting each day at the same time (6:01pm) according to the monitor log.  It's really odd and they just started doing this recently.  Perhaps I made a change to a configuration or something that caused this?  I don't use mesh, have 3 R500s and a ZD 1200 connected to Verizon FIOS Gigabit internet and have setup separate 2.4gb and 5gb networks.  Thanks

[christopher_hay]

Hi.  So I decided to reboot or power cycle my ZD 1200 to see if it might help.  Everything came back up online as it should including the 3 wired APs.  But now my 3 R500 APs lose heartbeat and reconnect at the time I power cycled the ZD!  So strange.  I did the original ZD power cycle at 9:27am and now for the last couple days the APs go offline and come right back online at that time.  I can't understand this behavior - so odd.  Any ideas?  Thanks.

[christopher_hay]

Hi.  The daily disconnecting and reconnecting of my 3 APs is still happening.  Each day precisely at the same time.  This is a home setup so nothing odd going on on my network. 
Example from my monitor logs:

2019/02/08  19:13:33Medium AP[Family Room] joins with uptime [2851541] s and last disconnected reason [Heartbeat Loss]
2019/02/08  19:13:31Medium AP[Susan Office] joins with uptime [2851512] s and last disconnected reason [Heartbeat Loss]
2019/02/08  19:13:22Medium AP[Harrison Room] joins with uptime [2851486] s and last disconnected reason [Heartbeat Loss]
2019/02/08  19:13:13Medium AP[Family Room] heartbeats lost
2019/02/08  19:13:11Medium AP[Susan Office] heartbeats lost
2019/02/08  19:13:02Medium AP[Harrison Room] heartbeats lost
2019/02/07  19:13:16Medium AP[Susan Office] joins with uptime [2765099] s and last disconnected reason [Heartbeat Loss]
2019/02/07  19:13:15Medium AP[Family Room] joins with uptime [2765124] s and last disconnected reason [Heartbeat Loss]
2019/02/07  19:13:03Medium AP[Harrison Room] joins with uptime [2765069] s and last disconnected reason [Heartbeat Loss]
2019/02/07  19:12:56Medium AP[Susan Office] heartbeats lost
2019/02/07  19:12:53Medium AP[Family Room] heartbeats lost
2019/02/07  19:12:43Medium AP[Harrison Room] heartbeats lost


Any ideas?
Many thanks.

[tony_heung]

Hi Christopher,

From reading the above logs, checking the uptime, eg AP (Susan Office) on Feb 7th, is 2765099 .  This equals to 32 days uptime, meaning the last time you rebooted the ZD and AP was 32 days ago (from Feb 7th) at 19:13 (assuming you had NTP and timezone setup correctly).

And the next event of the same AP has uptime 2765099 which is 33 days, aka ~86400 seconds after the first event.

It would interpret the AP is up and running serving client with no service interruption (unless you do tunnel mode otherwise local breakout traffic would not be impacted by the AP mgmt ip issue), and it is just the ping to the AP mgmt IP address is somehow lost.

As it is almost a perfect number of 86400, I would make one guess which is the DHCP leasing time.  Are you running DHCP Server on the ZD to provide IP address to the AP?  If so, please check the DHCP parameter particular the lease time.  You can find it from Configure -> System -> DHCP Server -> Lease Time.  The options available is 6-hr, 12-hr, 1-day, 2-day, 1-week or 2-week.  If it is currently set to 1 day, it is highly likely the lost of heartbeats is due to the expiry of the dhcp lease to the AP IP address.

If it is the case, I suppose you have a flat network, and all your home devices are using ZD as the dhcp server, then check the pool size and if it is big enough even though at the end of the lease expiration, it should still allow the AP to maintain the original IP address rather than force it to renew.

Let's see if it is moving to the right direction of the root cause, before we discuss to come up the best option moving forward including options to use external dhcp server (eg: the verizon router).

--tony

[matthewlampe]

I have a very similar issue on business network. Any ideas?

[dan_kern]

Your problem sounds similar to this old problem some ran across years ago.  Not sure if this will help, but check it out and see if it relates to your issue.   https://www.kahuna-fi.com/post/all-ruckus-unleashed-ap-s-lose-heartbeat-simultaneously