Let's try to manually set your channels first. This will be dependant on other WiFi channels in your surrounding area, but this is a start.
In your Unleashed dashboard go to Access Points>Select AP1>Edit>Radio B/G/N (2.4G)>Override Channel: 1 / Radio A/N/AC (5G)>Override Channel: 157. Ok/Save.
AP2, follow the same procedure but for 2.4G choose channel 6 and for 5G choose channel 161.
AP3, follow the same procedure but for 2.4G choose channel 11 and for 5G choose channel 165.
Since you have no QoS in place and most of your users are doing simple web browsing, I would rate limit them to 1Mbps up/down. You can do this at the router level or WLAN level, and since we are on the topic of Unleashed: WLAN>Advanced Options>Access Control>Rate Limit>Uplink: 1Mbps Downlink: 1Mbps.
Each AP is on its own VLAN correct? What is the subnet size? When the network becomes unusable, are clients spread out across the 3 AP's or is one being overloaded?
If you have 400 clients connected to a /24 IP subnet you are running out of addresses after 256 client connections. Maybe you can elaborate more on what happens when the network becomes "unusable"?
I may also recommend turning load balancing on, which will help spread clients between your access points: Go to Wi-Fi Networks > Edit > Show advanced options > Radio Control, and select the check box next to Load Balancing. Experiment with this option enabled and adjust accordingly.
At least one more AP to share client load would only be beneficial. I didn't read what Auth type on your WLAN? Simple WPA2/PSK may offer best performance over a Guest Access/HotSpot or 802.1x network, when roaming is involved. No benefit for R600 two Eth port to same switch (Link Aggregation), only 802.11ac Wave2 APs (R710, R610).
But can you see how much of your 100mb WAN pipe is in use? It may be the bottleneck for Internet browsing, as mentioned above.