topic Re: kernel panics R700 after upgrade to 9.10.0.0 build 218 in Access Points - Indoor and Outdoor

kernel panics R700 after upgrade to 9.10.0.0 build 218

arthur_hulsman — Fri, 12 Jun 2015 09:54:20 GMT

may 2015
We upgrade to software version 9.10.0.0 build 218 on several access points R700 we experience kernel panics.
Memory leak ER-2252 not yet fixed in the this version (:

Can you explain what caused the kernel oops ?

Herewith the output
### Kernel Oops ###

0xcff00000[1784]: Image: type=2 index=2 total_boot=6

@ j=4c9095d2

Oops: Kernel access of bad area, sig: 11 [#1]

P1024 RDB

Modules linked in: br_dhcp(P) wispr_http(P) crypto_aes_cbc_talitos av(P) br_dnat(P) afmod(P) crypto_aes_ccm(P) crypto_aes_cbc crypto(P) v54smartmedia(P) br_pif(P) br_ci ath_pci wlan_u(P) wlan_acl wlan_xauth wlan_ccmp wlan_wep wlan_tkip wlan_scan_sta wlan_scan_rks ath_spectral(P) ath_dfs(P) ath_rate_v54beamflex(P) ath_hal(P) wlan rflow(P) rks_dns(P) rudb(P) rks_ioctl

NIP: d141d168 LR: d141d094 CTR: c003cd44

REGS: cc68f7f0 TRAP: 0300 Tainted: P (2.6.32.24)

MSR: 00029000 CR: 80004088 XER: 00000000

DEAR: 00100104, ESR: 00800000

TASK = cc4e2560[1365] 'avpd' THREAD: cc68c000

GPR00: cdd82468 cc68f8a0 cc4e2560 ffffffea 00000002 cc68f924 00000100 d1424c88

GPR08: cdd82460 00100100 d1468910 00200200 00000000 10025ca8 10009f9c 10009f74

GPR16: 10002ee4 00000000 bf811fb0 00000000 00000000 cc68f8e8 00000000 00000100

GPR24: d1424c98 00043c88 00000007 cb3b6ec0 cc68f924 00000100 00000002 000018a6

NIP [d141d168] __dns_cache_group_dump_dn_list+0x128/0x218 [rks_dns]

LR [d141d094] __dns_cache_group_dump_dn_list+0x54/0x218 [rks_dns]

Call Trace:

[cc68f8a0] [d141d1a8] __dns_cache_group_dump_dn_list+0x168/0x218 [rks_dns] (unreliable)

[cc68f8e0] [d141e13c] get_dn_by_ip+0x4c/0x100 [rks_dns]

[cc68f910] [d1422fec] dns_ioctl+0x258/0xb3c [rks_dns]

[cc68fc40] [d13fa194] rks_ioctl+0xd4/0x1d0 [rks_ioctl]

[cc68fe70] [c01c9320] sock_ioctl+0x10c/0x3b0

[cc68fe90] [c0098ee8] vfs_ioctl+0x34/0x90

[cc68fea0] [c0099108] do_vfs_ioctl+0x84/0x724

[cc68ff10] [c00997e8] sys_ioctl+0x40/0x88

[cc68ff40] [c000f7c8] ret_from_syscall+0x0/0x3c

Instruction dump:

3d20d142 1f3f002c 38e94c88 7d27ca14 81090038 2f880000 419e00bc 8168000c

7d47ca14 81280008 3b070010 38080008 <91690004> 912b0000 7d38ca14 3929001c

Re: kernel panics R700 after upgrade to 9.10.0.0 build 218

michael_brado — Fri, 12 Jun 2015 20:52:45 GMT

Arthur, the developers believe they have a fix for ER-2285 that more matches your call trace.

The 9.10 MR1 will include the change, but if you're willing to try it (*with patch code caveats)

please open a Tech Support ticket and ask for Michael Brado. The engineer who says he has

the fix will provide the test build, if you let us know what platform of ZoneDirector you have.

Re: kernel panics R700 after upgrade to 9.10.0.0 build 218

arthur_hulsman — Mon, 15 Jun 2015 13:54:32 GMT

Michael herewith the casenumber 295627 we request the support engineer to contact you.
Do you know if the memory leak ER-2252 has been resolved in this software version.

Re: kernel panics R700 after upgrade to 9.10.0.0 build 218

michael_brado — Mon, 15 Jun 2015 23:12:37 GMT

Engineering has resolved both ER issues, fixes are being rolled into the firmware builds,
and will be available in the next official Maintenance Releases (9.9.2, 9.10.1, 9.12.1) which
we expect to be posted this summer. 9.10.1 is currently scheduled for mid-July.

Re: kernel panics R700 after upgrade to 9.10.0.0 build 218

michael_brado — Tue, 16 Jun 2015 16:04:21 GMT

Yes Arthur, it looks like ER-2252 fix is integrated in 9.10.0.0.308+, so will be in 9.10 MR1.

Re: kernel panics R700 after upgrade to 9.10.0.0 build 218

arthur_hulsman — Fri, 14 Aug 2015 12:33:57 GMT

We upgrade the sofware version [9.10.0.0.218] to [9.12.0.0.336] on production
We will monitor if the memory leak and kernel panics has been resolved.

The upgrade take some extra time because we first have to upgrade our ZD1100 to ZD1200 in the acceptance environment to run the 9.12.0.0.336 version.

Re: kernel panics R700 after upgrade to 9.10.0.0 build 218

michael_brado — Fri, 14 Aug 2015 15:57:36 GMT

ZoneFlex 9.10.1.0.59 (MR1) has been published to Support downloads pages yesterday, for ZD1100/1200/3000/5000 model controllers.

Re: kernel panics R700 after upgrade to 9.10.0.0 build 218

arthur_hulsman — Mon, 14 Sep 2015 09:01:39 GMT

Michael, 13 augustus 2015 software image has been upgraded from [9.10.0.0.218] to [9.12.0.0.336] we still have the same kernel crashes.

### Kernel Oops ###0xcff00000[1784]: Image: type=1 index=1 total_boot=3

@ j=93955451
Oops: Kernel access of bad area, sig: 11 [#1]
P1024 RDB
Modules linked in: br_dhcp(P) wispr_http(P) crypto_aes_cbc_talitos av(P) br_dnat(P) afmod(P) crypto_aes_ccm(P) crypto_aes_cbc crypto(P) v54smartmedia(P) br_pif(P) br_ci ath_pci wlan_u(P) wlan_acl wlan_xauth wlan_ccmp wlan_wep wlan_tkip wlan_scan_sta wlan_scan_rks ath_spectral(P) ath_dfs(P) ath_rate_v54beamflex(P) ath_hal(P) wlan rflow(P) rks_dns(P) rudb(P) rks_ioctl
NIP: d153d168 LR: d153d094 CTR: c003cd44
REGS: cb4777f0 TRAP: 0300 Tainted: P (2.6.32.24)
MSR: 00029000 CR: 80004088 XER: 00000000
DEAR: 00100104, ESR: 00800000
TASK = cc3ff540[1381] 'avpd' THREAD: cb474000
GPR00: cada5428 cb4778a0 cc3ff540 ffffffea 00000002 cb477924 00000100 d1544c88
GPR08: cada5420 00100100 d1550f44 00200200 00000000 10025ca4 10009f98 10009f70
GPR16: 10002ee4 00000000 bf8faf70 00000000 00000000 cb4778e8 00000000 00000100
GPR24: d1544c98 0000c2bc 00000007 cb4d8960 cb477924 00000100 00000002 0000046d
NIP [d153d168] __dns_cache_group_dump_dn_list+0x128/0x218 [rks_dns]
LR [d153d094] __dns_cache_group_dump_dn_list+0x54/0x218 [rks_dns]
Call Trace:
[cb4778a0] [d153d1a8] __dns_cache_group_dump_dn_list+0x168/0x218 [rks_dns] (unreliable)
[cb4778e0] [d153e13c] get_dn_by_ip+0x4c/0x100 [rks_dns]
[cb477910] [d1542fec] dns_ioctl+0x258/0xb3c [rks_dns]
[cb477c40] [d151a194] rks_ioctl+0xd4/0x1d0 [rks_ioctl]
[cb477e70] [c01c9320] sock_ioctl+0x10c/0x3b0
[cb477e90] [c0098ee8] vfs_ioctl+0x34/0x90
[cb477ea0] [c0099108] do_vfs_ioctl+0x84/0x724
[cb477f10] [c00997e8] sys_ioctl+0x40/0x88
[cb477f40] [c000f7c8] ret_from_syscall+0x0/0x3c
Instruction dump:
3d20d154 1f3f002c 38e94c88 7d27ca14 81090038 2f880000 419e00bc 8168000c
7d47ca14 81280008 3b070010 38080008 <91690004> 912b0000 7d38ca14 3929001c

Create a new case 00329193

Re: kernel panics R700 after upgrade to 9.10.0.0 build 218

john_d — Mon, 14 Sep 2015 15:19:52 GMT

As Michael said, for the 9.12 series it will be available in 9.12.1: "next official Maintenance Releases (9.9.2, 9.10.1, 9.12.1) "

Major releases have their maintenance releases run in parallel, so 9.10.1 is currently the only one that contains the fix to this issue, not 9.12.0.0.336.

Re: kernel panics R700 after upgrade to 9.10.0.0 build 218

michael_brado — Mon, 14 Sep 2015 17:13:20 GMT

And confirming, a search of the bug database on 'dns_cache_group_dump_dn_list' does indictate ER-2285 as the culprit.

Re: kernel panics R700 after upgrade to 9.10.0.0 build 218

monnat_systems — Mon, 14 Sep 2015 17:47:25 GMT

michael.

what is the reason for this bug?

Re: kernel panics R700 after upgrade to 9.10.0.0 build 218

michael_brado — Mon, 14 Sep 2015 18:00:36 GMT

Not obvious, but Engineering calls it kernel panic, application reboot.

Re: kernel panics R700 after upgrade to 9.10.0.0 build 218

arthur_hulsman — Mon, 14 Sep 2015 19:05:36 GMT

Michael, do you know the release date for software version 9.12.1 ?

Re: kernel panics R700 after upgrade to 9.10.0.0 build 218

michael_brado — Tue, 15 Sep 2015 16:18:00 GMT

Offical word from QA is final test currently in progress, expecting release tonight, and posting this week.
I'll send an official Announcement to Forum ZD/AP groups.

Re: kernel panics R700 after upgrade to 9.10.0.0 build 218

robin_cain_7457 — Thu, 17 Sep 2015 21:56:10 GMT

Hi, any news on 9.12.1 Michael?

Re: kernel panics R700 after upgrade to 9.10.0.0 build 218

michael_brado — Thu, 17 Sep 2015 22:19:13 GMT

An important issue, Win10 being reported as Win8 by application recognition, has pushed expected release to Mon 9/21.