The two are not mutually exclusive.
Smart-roam without the BSS-minrate will keep track of client RSSI and if it hits the threshold it will kick the client off, ideals the client it in a an area that another AP will provide a better signal, so the client then is looking for a new and better AP and that should be the AP with the stronger signal, but if the signal level is not all that different and the client driver has a preference to connect to the last known BSSID it will most likely connect back to the same Access point. The APs keeps a lot of metric on the clients and uses internal response timer to try to encourage the client to make the right choice, but it's the client's decision.
BSS-minrate will effectively reduce the coverage area, in that the beacon as well as other management frames at a higher data rate will not go as far as a lower data rate frame.
You can have over laps. For example of you set SmartRoam to a low value 1 (SNR/RSSI threshold of 5) and you set the BSS minrate to 12 Mbps, you will get out of range of the AP's 12 Mbps Beacon range way before you would hit the 5 dB RSSI for a SmartRoam of 1, so your client will disconnect from the old AP (out of range) and connect to a new one assuming there is another AP that the client can hear at 12 Mbps, before SmartRoam is triggered. You can have it the other way around as well, If you set SmartRoam to 10 which requires that the client SNR be 60 dB above noise and you the BSS minrate set to 6 Mbps, you will hit the SmartRoam threshold before the client get out of the 6 Mbps range.
As for the threshold recommendation, it will really depend on the coverage. If the deployment were designed for (lower band) 5 GHz coverage of -65 dBm or better everywhere, then you can start with either the SmartRoam of 5 (SNR/RSSI=20 dB) or if you wan to use bss minrate you can start with 12 Mbps. From there you can adjust either to optimize for the environment. That is assuming the the coverage is uniform, if not use the value appropriate to the area with the sparse coverage area.