If the ip address had changed, you would get a time-out.
"Connection refused" means it is not accepting ssh connections anymore.
Possible reasons for this are plenty, you know best what you changed when you logged in that first time.
If you can still access via the console you might be able to fix it.
Otherwise, the fastest option is to re-flash your card.
/emgi