topic Re: load balancing mode of a LAG in ICX Switches

load balancing mode of a LAG

harri_waltari — Mon, 16 Sep 2019 11:46:10 GMT

I need to check the load balancing mode of the lags on the physical switches (ICX6610, 08.0.30t) facing our esxi cluster. On the cluster side, at the virtual switches, the balancing mode is currently "Source and destination IP address". The manual is a bit vague of the modes and I can't seem to find a command to modify the mode in the physical switch.

The manual says (a snippet...):

Layer 2 Bridged TCP/UDP: Source and destination MAC addresses, source and destination IP addresses, and source and destination TCP/UDP ports.
...
Layer 2 Bridged IPv4 TCP/UDP Source and destination IP addresses, and source and destination TCP/UDP ports.

So what is the difference here with "bridged tcp/udp" and "bridged ipv4 tcp/udp"? And are those different, selectable modes?

Eg "src & dst ip address and tcp/udp port" is one of the possible selections on the cluster side. Would that be a better combination for icx?

Re: load balancing mode of a LAG

simon_pollard — Mon, 16 Sep 2019 12:18:28 GMT

Hi

LAG load balancing is fixed and cannot be changed, the method used depends on the traffic type (this is described in the L2 config guide). Symmetric load balancing can be enabled if required (some firewalls need this) but it does not alter the traffic distribution across the links.

Re: load balancing mode of a LAG

harri_waltari — Mon, 16 Sep 2019 12:23:40 GMT

Simon, thanks for your prompt response.

What you would see what is the correct or most proper setting then on the cluster side?

And would you please open the difference btw "bridged tcp/udp" and "bridged ipv4 tcp/udp"?

Re: load balancing mode of a LAG

simon_pollard — Tue, 17 Sep 2019 06:02:16 GMT

Hi, I think this could be a documentation error, I will need to check and will get back to you ASAP.

Re: load balancing mode of a LAG

simon_pollard — Wed, 18 Sep 2019 06:05:59 GMT

I can confirm that this is an error in the documentation and the entry for "Layer 2 Bridged TCP/UDP" will be removed from the next version of the L2 config guide.

The entries for "bridged ipv4 tcp/udp" and "bridged ipvd tcp/udp" describe the hashing for TCP/UDP traffic.

Re: load balancing mode of a LAG

harri_waltari — Wed, 18 Sep 2019 08:12:03 GMT

Simon, thanks for this. It really clarifies a bit.

So, if the mode in the switch is fixed to "Source and destination IP addresses, and source and destination TCP/UDP ports" and the modes available on the vsphere are (amogst others) "src and dst ip address", "src and dst ip address and tcp/udp port" or "src and dst tcp/udp port" would you think that "src and dst ip address and tcp/udp port" is the closest match? It may be obvious but I'm baffled with nuances here. Also how crucial are the actual effects of (slightly) wrong selection?

Re: load balancing mode of a LAG

simon_pollard — Wed, 18 Sep 2019 09:45:44 GMT

Hi

I'm afraid I can't comment on the vsphere config. with any confidence as it's not something that I have any experience with, but from what you have described "src and dst ip address and tcp/udp port" looks like the best match to the ICX behavior. The only way to be certain with be to perform some testing which is not something that we would be able to do.

Hopefully someone else on here has some relevant experience and can share some real-world insights.

Re: load balancing mode of a LAG

harri_waltari — Wed, 18 Sep 2019 10:06:44 GMT

Simon,

I understand. Thanks for your input.

Re: load balancing mode of a LAG

vu_pham_ghtztqm — Wed, 18 Sep 2019 18:37:13 GMT

Hi Hwa - LAG hashing on ICX is locally significant on the ICX, its result will decide which port to map the egress traffic toward the ESX. Similarly, LAG hashing on ESX is locally significant on the ESX, its result will decide which port to map egress traffic toward the ICX. Since it's two independent process, so I believe the hashing algorithm doesn't have to match between ICX and ESX.

Thanks,

Vu

Re: load balancing mode of a LAG

harri_waltari — Fri, 20 Sep 2019 06:07:00 GMT

Hi Vu,
I'm thinking the returning traffic. Say, from the host the traffic is sent out via one interface and the return traffic is sent from the switch to the host via another interface because of different load balancing mechanism. And of course vice versa. Would this affect the performance of either devices?
If there is no effect, what is the point of having several load balancing algorithms at all?

Re: load balancing mode of a LAG

vu_pham_ghtztqm — Fri, 20 Sep 2019 19:22:11 GMT

Hi Hwa - I see your points. I think there is 'symmetric lag hashing' option that maybe be able to do what you're thinking, but it only work on IP packet and not Layer 2 packets. ICX6k don't support it, but ICX7K do. I don't know if VMware or Linux even support it. So it's not really a good option for load-balancing either.

Since lag hashing result is locally significant, I think even the hashing algorithm is the same on both LACP peer, it doesn't guarantee that the same pair of traffic will always get mapped to the same port on both end. What can be expected is that the ICX would always map SrcA/DstB to the same port as long as that port is up, this could be Port1 depend on the hash result of ICX. And similarly ESX would probably always map SrcB/DstA to the same port as long as that port is up, this could be Port 2 depend on the hash result of ESX.

At the high level on the ICX6K, when the traffic is Layer3 (being routed between VLAN/VE), it uses the IP headers and protocol fields in hashing. When the traffic is bridged/Layer2 (being switched in the same VLAN), it depends on the traffic type, IP and/or Ethernet headers along with Layer 4 ports info are used in hashing. I believe this is a standard method among vendors, and should be very similar between them.

Thanks,

Vu

Re: load balancing mode of a LAG

harri_waltari — Mon, 23 Sep 2019 10:14:06 GMT

Vu, Thanks for you informative input. Really appreciate it. I think i get it now.