Is it possible to have 2 controllers with different software versions? The plan is for one to stay on the recommended version 22.214.171.124.222 and the other on the latest 5.x.x How we organized AP groups, it is most convenient for us to have 2 controllers. We have a data center and we can also have separate clusters if needed.
You can have two controllers on separate versions, but they cannot be clustered (connected) together. They must be two separate, independent SZ clusters. It really depends on your requirements now and in the future for scaling-up/expanding your network.
However, you could run 3.6.2 and 5.x APs on a single cluster. To do so you'd start with a 3.6.2 cluster and upgrade it to 5.x.x. You can then optionally leave the required zones on 3.6.2 AP firmware (i.e. not upgrade the zones to 5.x). So you can use this option to run some zones in 3.6.2, and others on 5.x.x. But note that any cluster upgraded to 5.x, can no longer support 3.5 (or earlier) APs. So before making any decisions, you will first need to check the 5.x release notes to see if any of your APs are affected.
Ultimately, the path you take will be determined by your existing infrastructure, the number of APs required to be support by each cluster, licensing, support budget etc.