topic Re: Users unable to get IP address after several months running normally! in ZoneDirector

Users unable to get IP address after several months running normally!

flavio_borup — Tue, 16 Apr 2019 13:02:54 GMT

My ZD is 9.9.0.0 build 216 and i can ́t do any change right now
I have several old APs which doesn ́t allow me to upgrade

I ́m running this very same solution in the last 3 years with no problems at all

Yesterday, some users complain about the were unable to get IP addresses

My DHCP Server is a Windows 2012
My Ruckus environment have 12 APs and several SSIDs and Serveral VLANs
My network uses HP/3COM Switchs models 2600, 5550, etc

After doing a investigation:

The problem only affected a single SSID and is related to a single VLAN
At DHCP Server logs, i can see SOME clients are still able to get IPs, but randomly, some users can ́t. After some network tracing, i can see DHCP requests arriving at DHCP Server and all requests that come to DHCP are served with no problem.

But the clients with problems, the DHCP request does not arrive at the interface, for some reason, the traffic was blocked at some point and the DHCP packets were not arriving at the DHCP Server.. Again, for some clients, other can connect with no problem.

Yesterday, afet several months of uptime i rebooted my ZD and all things got normal again. Ok, maybe it was a "glitch", anyway, after the reboot.. everyting got back normal

Today, the problem is back! After some tests i did the following: I changed the VLAN ID of the SSID and clients were able to get IP address again (each VLAN has its own IP range), so, in the new range, things were nromal. And, after re-configuring back as before, the problem was gone. First a reboot, now a little change in configuration to solve the problem. Weird!

Some monutes ago, the problem is back again! and i didnpt reboot yet, but now, no matter what change i do, the problem is here, some clients can leas IPs and some not (randomly) and if the pacjets arrive at DHCP Server they ́re leased with no problem, but some packets simply didn ́t arrive at the DHCP Server interface to be processed

So, what ́s next? What should I do?

Re: Users unable to get IP address after several months running normally!

nickzourdos — Tue, 16 Apr 2019 18:29:44 GMT

Are your DHCP scopes filling up? How many clients are connected to the network, and what is the subnet mask of each of these VLANs?

Another possibility is that you have a rogue DHCP server running on your network. Try running a tool like this one to see how many DHCP servers are being detected, or alternatively you could run a packet capture using Wireshark to see how many servers are responding to your client's DHCP request.

Re: Users unable to get IP address after several months running normally!

flavio_borup — Tue, 16 Apr 2019 18:35:17 GMT

No, plenty of addresses, more than 80% of a full Class-C range available
I ́m sure there are no rogue DHCPs at the network, but it ́s a good idea, to re-check

Re: Users unable to get IP address after several months running normally!

andrew_giancol1 — Tue, 16 Apr 2019 18:49:18 GMT

I'd run some Wireshark, and look for the replies. could be your IP helper statement or something else.
`set capture eth0 stream`
where eth0 is your active ethernet on the AP ( 0 or 1 )
Then go to Wireshark, (windows version only) and set up a remote interface by typing the IP of your AP, then the port 2002. Many interfaces will start to appear, only select eth0. Once there, use the following filter
`bootp`
Hope that helps.

Re: Users unable to get IP address after several months running normally!

flavio_borup — Tue, 16 Apr 2019 19:26:46 GMT

Great, i like it, but as a first-tme suer for Ap ́s cmd-line, see my commands below:

rkscli: get eth

Port Interface 802.1X Logical Link Physical Link Label
-----------------------------------------------------------------
2 eth2 None Up Up 100Mbps full 10/100/1000 PoE
1 eth1 None Down Down 10/100
0 eth0 None Down Down 10/100
OK

rkscli: set capture eth2 stream
eth2: Invalid wlan interface name

rkscli: get ipmode wan
IP Mode: dual
OK

rkscli: get ipmode lan
IP Mode: ipv4
OK

rkscli: get ipaddr wan
IP Address: (static, vlan 1), IP: 10.121.60.59 Netmask 255.255.255.0 Gateway 10.121.60.1
OK

Re: Users unable to get IP address after several months running normally!

andrew_giancol1 — Tue, 16 Apr 2019 19:28:09 GMT

set capture eth2 stream

Re: Users unable to get IP address after several months running normally!

flavio_borup — Tue, 16 Apr 2019 19:29:58 GMT

rkscli: set capture eth2 stream
eth2: Invalid wlan interface name

Re: Users unable to get IP address after several months running normally!

flavio_borup — Tue, 16 Apr 2019 19:31:52 GMT

i found the get wlanlist and there is a mapping between wlanX and SSIDs
but...

rkscli: set capture wlan0 (or any wlan in the list) stream
parameter error

Re: Users unable to get IP address after several months running normally!

andrew_giancol1 — Tue, 16 Apr 2019 19:32:05 GMT

in wireshark(WS);
capture>options
in the WS - Capture interfaces box hit the "manage interfaces" button
in the Manage Interfaces Box, hit "Remote Interfaces" tab
on the Manage Interfaces Tab, hit the + on the lower left
put in your IP 10.121.60.59 and use port 2002

Re: Users unable to get IP address after several months running normally!

andrew_giancol1 — Tue, 16 Apr 2019 19:33:24 GMT

Ugh. i mis-typed the command.

rkscli: set capture eth2 stream

Re: Users unable to get IP address after several months running normally!

andrew_giancol1 — Tue, 16 Apr 2019 19:34:26 GMT

Try hitting `set cap` then use tab to complete.

Re: Users unable to get IP address after several months running normally!

flavio_borup — Tue, 16 Apr 2019 19:39:23 GMT

ok, now i got, wlan100 for 2.4 GHz radio, i ́m testing WS remote capture, and i can see a lot of "sub interfaces" available

Re: Users unable to get IP address after several months running normally!

flavio_borup — Tue, 16 Apr 2019 19:41:26 GMT

now, after selecting the wlan40 (my case)

---------------------------

---------------------------
The capture session could not be initiated on interface 'rpcap://[10.121.60.59]:2002/wlan40' (Unknown error (pcap bug; actual error cause not reported)).
---------------------------

Please check that "rpcap://[10.121.60.59]:2002/wlan40" is the proper interface.

Help can be found at:

https://wiki.wireshark.org/WinPcap
https://wiki.wireshark.org/CaptureSetup

---------------------------
OK
---------------------------

Re: Users unable to get IP address after several months running normally!

flavio_borup — Tue, 16 Apr 2019 19:43:01 GMT

---------------------------

---------------------------
The capture session could not be initiated on interface 'rpcap://[10.121.60.59]:2002/eth2' (Unknown error (pcap bug; actual error cause not reported)).
---------------------------

Please check that "rpcap://[10.121.60.59]:2002/eth2" is the proper interface.

Help can be found at:

https://wiki.wireshark.org/WinPcap
https://wiki.wireshark.org/CaptureSetup

---------------------------
OK
---------------------------

Re: Users unable to get IP address after several months running normally!

andrew_giancol1 — Tue, 16 Apr 2019 19:45:36 GMT

can you attach a screen shot? of your remote session failing?

Re: Users unable to get IP address after several months running normally!

andrew_giancol1 — Tue, 16 Apr 2019 19:48:15 GMT

also, perhaps we'll need to use br0 or the wlan your client is failing on.

lacking the pcap from the AP you can run wireshark directly on the server. Are you forwarding DHCP requests via IP helper, or is your dhcp on the same subnet?

Re: Users unable to get IP address after several months running normally!

flavio_borup — Tue, 16 Apr 2019 19:48:30 GMT

After trying the "ANY" interface, i ́m able to see packets and now i ́m filtering by "Bootp" and i think, for now, that i ́m able to start diagnosing something

Re: Users unable to get IP address after several months running normally!

andrew_giancol1 — Tue, 16 Apr 2019 19:49:52 GMT

Excellent! Just remember, to make sure your eth interfaces on the computer your using aren't selected.

Best of Luck Flavio!

Re: Users unable to get IP address after several months running normally!

flavio_borup — Tue, 16 Apr 2019 20:03:18 GMT

Now, going on by the main/original subject:

I ́ve applied a filter based on DHCP Transaction ID : (bootp.id == 0xd585f0f8) && ((bootp.option.dhcp == 1) || (bootp.option.dhcp == 5))

I can see 5 DHCP Discovers, clearly showing wifi client trying to get an Ip address, but for now, i have no idea on where the hell is going on

To where are these packets going? If there was a rogue DHCP Server, should I see the (rogue) dhcp server packets responses? SO, i ́m 99% sure that there is no rogue DHCP operating in the segment

Is consistent, client can get an IP and packets are no arriving at the DHCP Server (i can see that ncapturing packets at the real DHCP Server)

Packets are getting lose on their way...

Re: Users unable to get IP address after several months running normally!

andrew_giancol1 — Tue, 16 Apr 2019 20:05:09 GMT

ok so you're looking for the server to ACK or NAK these replies. do you see that?