It is probably a bug. We have NOT encountered this though years ago there was a bug where we would not be able to connect to something that hadn't been connected to in a long time. When you would open your SSH session it would hang for a while then fail... but a subsequent attempt worked fine because the process started up on the switches.
I do not remember what version that was presumably some ancient 08.0.30 code from years ago. Honestly, everything has been rock solid for us on all our 64xx, 6610, 7150's and 7450's.
We are running 08.0.80ca on everything that supports it, and it has been bug-free for us as is 08.0.30sa on everything that doesn't support the 08.0.80ca.
Here is how we do our SSH configuration (I am not saying you are doing yours wrong on that collectively this is the configuration options we use that have something to do with SSH...)...
crypto key zeroize rsa
crypto key zeroize dsa
crypto key generate rsa mod 2048
Then we make a list where device management can come from... edit to suit your taste:
ip access-list standard 99
permit host 10.1.2.3
permit 10.1.0.0 0.0.255.255
Without Radius we use this block:
aaa authentication web-server default local
aaa authentication enable default local
aaa authentication login default local
aaa authentication login privilege-mode
enable aaa console
console timeout 30
no telnet server
no web-management http
ssh access-group 99
web access-group 99
ip ssh authentication-retries 2
ip ssh timeout 30
ip ssh idle-time 30
ip ssh scp disable
ip ssh encryption disable-aes-cbc
I would probably try a code update and consider adding or changing some of your configuration options to include some of the above arguments... certainly there is quite a bit more than SSH going on here, but I included related material.