Everyone seems to be missing the fact that the OP said he is connecting the RasPi DIRECTLY to the PC.
No. I didn't miss that. Whilst the address doesn't matter if there are only two computers in the network - the Ubuntu computer is also tethered to an Android device. I'm guessing this is using the Android for an Internet connection - so it should therefore be using private network addresses.
Ubuntu was assigning an IP 192.xx.xx.xxx
to the connected tethered android device.
Whilst using a non-private address range is unlikely to be the problem on it's own (unless attempting to contact another computer in that address) it is a good idea to instead use a private network address range. I did also say this is unlikely to be the problem, but best practice to set this up using the network range designed for such networks. Using the same subnet as the next network hop / DNS entries etc. can cause significant network problems for the Internet connection.
Normally a connection refused from ssh does not necessarily indicate an underlying network problem, but in this case there is also a problem with pinging between the computers, but only in one direction.
I could successfully ping the IP from PC side but not from the RPi side.
I think it is worth therefore trying to fix what appears to be network problem first. One issue you can get where you can ping in one direction, but not the other is either a duplicate IP address, in this case it's physically connected so unless there is another interface one of the computers that is unlikely. Another is that routing is not set-up on one side of the route, although this is more likely to result in pings failing in the other direction (ie the computer with multiple interfaces sends it request in the wrong direction, but when responding from a request on a known network learns the arp entry of that computer). It may be the case that the implied route hasn't been added to the routing table for some reason. It may therefore also be worth looking at the arp table "arp -an" (duplicate arp entries / outdated arp entries can cause these sort of symptoms as well)
Looking at the routing tables and the ifconfig outputs can at least identify some potential problems / eliminate these (also show us packet loss etc which could be due to a faulty cable / network interface) .
Another reason for ping not working could be that the PC is configured to not respond to ICMP echo requests (perhaps a firewall). I am trying to keep the information we are investigating as simple as possible at this stage - so sticking to basic network commands.
It could be that there is nothing wrong with the network other than the PC refusing the ICMP ping, and that the problem with ssh is that the Pi is refusing the ssh connection, but with the potential network problems indicated by the ping it's worth checking network levels 1 to 3 prior to investigating the application layer.