I mostly agree with Robert's assessment here. I've got an R500, R600, R700, and R710, and I would say the first 3 have almost the same coverage despite having widely different beamflex and spatial stream configurations, and being separated by a few hundred bucks price-wise too....
The R710 truly does seem to provide better coverage, but I would only say it's maybe a 10-20% improvement in range vs speed, and I mostly attribute that to its far superior receive sensitivity compared to the other APs.
Bottom line, especially given the great price point of the R300 vs the R500, I think it's unlikely that it's more cost effective to replace with a R500 vs deploying more R300's. I also agree with Robert on mounting position, but I will additionally add: Wall mounting is almost never something you want to do with the Ruckus APs unless your goal is vertical coverage (e.g. if it's a townhome with a lot of stories but not very wide rooms). For a more traditional house, you probably want the AP in the flat orientation. And in my experience, flat on a book case vs dome-down from the ceiling doesn't make a huge difference, UNLESS something is obstructing the book case mounting position that makes the ceiling position provide a better line of sight.
However, one point that hasn't been made here is that you've got an 802.11n AP, and the R310/R500 and above are all 802.11ac. With 80MHz channel bonding and 256QAM rates, I've observed a substantial performance improvement at ALL ranges when switching a residential AP from 802.11n to 802.11ac, as long as the client devices support it. So that might effectively improve video streaming range and give you the equivalent effect of improved coverage.