topic Re: ICX7750 Egress Queue Drops in ICX Switches

ICX7750 Egress Queue Drops

kvon12 — Mon, 14 Oct 2024 18:10:07 GMT

Good morning,

Can someone point me in the right direction to resolve egress queue drops? This is happening on a pair of stacked ICX7750s running SWR08095g. All drops are happening in queue 0. I was hoping to increase the buffers on that queue for the ports that are experiencing the issue. I have pasted below a sample output of the drops, but it doesn't format well. The bolded and underlined number is the drops.

UC Egress queues:
Queue counters Queued packets Dropped Packets
0 18144016316 768740
1 0 0
2 0 0
3 0 0
4 3137333 0
5 228089 0
6 0 0
7 5348571 0

MC Egress queues:
Queue counters Queued packets Dropped Packets
0 994025 0
1 4730751 0
2 110670 0
3 0 0
4 2344920 0

Re: ICX7750 Egress Queue Drops

kvon12 — Tue, 15 Oct 2024 13:28:38 GMT

I found how to adjust the buffers in the command reference. I tried queue-share-level first and that did not resolve the issue. I am trying port-share-level now and will report back.

Re: ICX7750 Egress Queue Drops

kvon12 — Wed, 16 Oct 2024 21:45:42 GMT

Port-share-level also failed to resolve the issue. We are still seeing UC egress queue 0 dopped packets.

Re: ICX7750 Egress Queue Drops

BenBeck — Thu, 17 Oct 2024 14:33:14 GMT

Hey Kvon12,

Apologies for the delayed response. I typed out a whole response on your initial submission, but seems I did not hit reply! Egress drops are due to buffer overflowing. This can be due to multiple reasons. I see you have tried to adjust buffers already and that has not helped. We may want to go back and look at the reason why. Things I would check:

Do we have traffic going from higher speed to lower speed (ex. 10g to 1g)? This inherently causes buffers to fill

Do you see flow control/discards on some or all ports? (show stat eth x/y/z)

Do you see egress buffer drops on all units/ports?

Are above statistics happening in bursts or are they steadily incrementing?

This is a tough one to do over a forum, so I would advise opening a support case to get the best help.

Re: ICX7750 Egress Queue Drops

kvon12 — Thu, 17 Oct 2024 17:47:40 GMT

Good morning BenBeck,

Thanks for the response. I would love to open a support case but sadly we do not have support on these devices.

Do we have traffic going from higher speed to lower speed (ex. 10g to 1g)? This inherently causes buffers to fill. - We have drops on a some ports that are 10g to 1g and some that are 10g to 10g.

Do you see flow control/discards on some or all ports? (show stat eth x/y/z)

Port 2/1/45 Counters:
InOctets 52727597869 OutOctets 334847473648
InPkts 168742111 OutPkts 303047076
InBroadcastPkts 737311 OutBroadcastPkts 3905535
InMulticastPkts 3076227 OutMulticastPkts 6584819
InUnicastPkts 164928573 OutUnicastPkts 292556722
InBadPkts 1
InFragments 0
InDiscards 0 OutErrors 0
CRC 1 Collisions 0
InErrors 1 LateCollisions 0
InGiantPkts 0
InShortPkts 0
InJabber 0 OutDiscards 31518
InFlowCtrlPkts 0 OutFlowCtrlPkts 0
InBitsPerSec 4568464 OutBitsPerSec 37707440
InPktsPerSec 1404 OutPktsPerSec 4214
InUtilization 0.47% OutUtilization 3.83%

Do you see egress buffer drops on all units/ports? - not all but most ports are showing egress drops.

Are above statistics happening in bursts or are they steadily incrementing?- seems to be bursts

Thanks again for the help.