Radius authentication stopped working on ICX 7250

[stephan_schuste]

Dear Colleagues,

I have a couple of ICX 7250, all configured quite similar. On one of them radius authentication stopped working. I just can't login by using my AD credentials anymore. Only local auth is working. I checked on the Firewall and saw that the switch is not even sending packets to the Radius Server (windows nps).

I started debugging on that switch and to compare it on a working switch as well.
The broken one shows me a debug log like this:

Debug: Jul 24 15:36:10 Function aaa_authenticate_start_internal, session:17, cloud_enable:0, login:username, SSH/Con/Tel(get_ui_session_idx):1, cloudUI(get_ui_session_idx):0, AAA method:2
Debug: Jul 24 15:36:10 RADIUS authentication for context 17
Debug: Jul 24 15:36:10 RADIUS ERROR: Client index 255 out of range.
Debug: Jul 24 15:36:10 Failed to get a RADIUS clientCB; Returning AAA_BUSY for context 17
Debug: Jul 24 15:36:10 Function aaa_authenticate_callback, session:17, cloud_enable:0, login:username, SSH/Con/Tel(get_ui_session_idx):1, cloudUI(get_ui_session_idx):0, AAA method:2, rc:5
Debug: Jul 24 15:36:10 Authentication Timeout or busy, rollover to next server or method.
Debug: Jul 24 15:36:10 Authentication: Next method is Local.
Debug: Jul 24 15:36:10 Function aaa_authenticate_callback, session:17, cloud_enable:0, login:username, SSH/Con/Tel(get_ui_session_idx):1, cloudUI(get_ui_session_idx):0, AAA method:3, rc:4
Debug: Jul 24 15:36:10 Authentication Timeout or busy, rollover to next server or method.
Debug: Jul 24 15:36:10 Authentication: No more method left.
Debug: Jul 24 15:36:10 Function aaa_authenticate_callback, session:17, cloud_enable:0, login:username, SSH/Con/Tel(get_ui_session_idx):1, cloudUI(get_ui_session_idx):0, AAA method:0, rc:3
Debug: Jul 24 15:36:10 Authentication failed.
Debug: Jul 24 15:36:10 aaa_send_aaa_response()..session 17, err_code=3 deferred_response_id=f81a89
Debug: Jul 24 15:36:29 Authentication Stopped, AAA Session context is 17.
Debug: Jul 24 15:36:29 ====AAA: Cleanup session 17 information.

on a switch with working radius auth it looks like this:

Debug: Jul 24 15:39:57 ====AAA: Cleanup session 17 information.
Debug: Jul 24 15:39:57 Function aaa_authenticate_start_internal, session:17, cloud_enable:0, login:username, SSH/Con/Tel(get_ui_session_idx):1, cloudUI(get_ui_session_idx):0, AAA method:2
Debug: Jul 24 15:39:57 RADIUS authentication for context 17
Debug: Jul 24 15:39:57 RADIUS ERROR: Client index 255 out of range.
Debug: Jul 24 15:39:57 Reseting RADIUS Client structure
Debug: Jul 24 15:39:57 RADIUS: Reset client 0, Session type 1, Total number of active clients=1
Debug: Jul 24 15:39:57 AAA: Open RADIUS UDP port
Debug: Jul 24 15:39:57 RADIUS message received from server of len 127.
Debug: Jul 24 15:39:57 Radius secret len ?25, total len ?127
Debug: Jul 24 15:39:57 RADIUS Timer cancelled for client 0.
Debug: Jul 24 15:39:57 RADIUS server ACCEPTed request
Debug: Jul 24 15:39:57 Function aaa_authenticate_callback, session:17, cloud_enable:0, login:username, SSH/Con/Tel(get_ui_session_idx):1, cloudUI(get_ui_session_idx):0, AAA method:2, rc:1
Debug: Jul 24 15:39:57 Authentication successful.
Debug: Jul 24 15:39:57 aaa_send_aaa_response()..session 17, err_code=1 deferred_response_id=a86a8
Debug: Jul 24 15:39:57 Closing RADIUS UDP port
Debug: Jul 24 15:39:57 RADIUS: radius_authenticate_stop for client Idx 0. Actv Clients left 0
Debug: Jul 24 15:39:57 Reseting RADIUS Client structure
Debug: Jul 24 15:39:57 Authorization status - accept.
Debug: Jul 24 15:39:57 aaa_send_aaa_response()..session 17, err_code=1 deferred_response_id=a86aa

 So both are showing the Error "RADIUS ERROR: Client index 255 out of range." but the working one is performing a "Reseting RADIUS Client structure" after that and the broken one don't.

Both switches are running SW: Version 08.0.90kT211

best
stephan

[Chandini]

Hi Stephan

Noted and Thanks for reaching out to us on portal.

[point_of_presen]

Similar behaviour here after an update from 8.0.80e to 8.0.95s. It worked for a few days then all logins started failing. Logs look like this:

Debug: Jan 7 17:55:11 ====AAA: Cleanup session 17 information.
Debug: Jan 7 17:55:11 Function aaa_authenticate_start_internal, session:17, cloud_enable:0, login:mike, SSH/Con/Tel(get_ui_session_idx):1, cloudUI(get_ui_session_idx):0, AAA method:2
Debug: Jan 7 17:55:11 RADIUS authentication for context 17
Debug: Jan 7 17:55:11 RADIUS ERROR: Client index 255 out of range.
Debug: Jan 7 17:55:11 Reseting RADIUS Client structure
Debug: Jan 7 17:55:11 RADIUS: Reset client 0, Session type 1, Total number of active clients=1
Debug: Jan 7 17:55:11 AAA: Open RADIUS UDP port
Debug: Jan 7 17:55:11 Error: Radius Authentication - Cannot create local UDP port..returning AAA_BUSY.
Debug: Jan 7 17:55:11 Closing RADIUS UDP port
Debug: Jan 7 17:55:11 RADIUS: radius_authenticate_stop for client Idx 0. Actv Clients left 0
Debug: Jan 7 17:55:11 Reseting RADIUS Client structure
Debug: Jan 7 17:55:11 Function aaa_authenticate_callback, session:17, cloud_enable:0, login:mike, SSH/Con/Tel(get_ui_session_idx):1, cloudUI(get_ui_session_idx):0, AAA method:2, rc:5
Debug: Jan 7 17:55:11 Authentication Timeout or busy, rollover to next server or method.
Debug: Jan 7 17:55:11 Authentication: Next method is RADIUS.
Debug: Jan 7 17:55:11 RADIUS authentication for context 17
Debug: Jan 7 17:55:11 RADIUS ERROR: Client index 255 out of range.
Debug: Jan 7 17:55:11 Reseting RADIUS Client structure
Debug: Jan 7 17:55:11 RADIUS: Reset client 0, Session type 1, Total number of active clients=1
Debug: Jan 7 17:55:11 AAA: Open RADIUS UDP port
Debug: Jan 7 17:55:11 Error: Radius Authentication - Cannot create local UDP port..returning AAA_BUSY.
Debug: Jan 7 17:55:11 Closing RADIUS UDP port
Debug: Jan 7 17:55:11 RADIUS: radius_authenticate_stop for client Idx 0. Actv Clients left 0
Debug: Jan 7 17:55:11 Reseting RADIUS Client structure
Debug: Jan 7 17:55:11 Function aaa_authenticate_callback, session:17, cloud_enable:0, login:mike, SSH/Con/Tel(get_ui_session_idx):1, cloudUI(get_ui_session_idx):0, AAA method:2, rc:5
Debug: Jan 7 17:55:11 Authentication Timeout or busy, rollover to next server or method.
Debug: Jan 7 17:55:11 Authentication: Next method is Local.
Debug: Jan 7 17:55:11 Function aaa_authenticate_callback, session:17, cloud_enable:0, login:mike, SSH/Con/Tel(get_ui_session_idx):1, cloudUI(get_ui_session_idx):0, AAA method:3, rc:4
Debug: Jan 7 17:55:11 Authentication Timeout or busy, rollover to next server or method.
Debug: Jan 7 17:55:11 Authentication: No more method left.
Debug: Jan 7 17:55:11 Function aaa_authenticate_callback, session:17, cloud_enable:0, login:mike, SSH/Con/Tel(get_ui_session_idx):1, cloudUI(get_ui_session_idx):0, AAA method:0, rc:3
Debug: Jan 7 17:55:11 Authentication failed.
Debug: Jan 7 17:55:11 aaa_send_aaa_response()..session 17, err_code=3 deferred_response_id=d50d70