About SSH brute force attacks.

[jbudd]

One of my Pi's is exposed to the internet by port forwarding on my router (Not port 22!)

As an experiment I opened port 22 for a while and I quickly got lines in /var/log/auth.log like this

Code: Select all

sshd[12399]: Unable to negotiate with 103.89.88.209 port 51370: no matching key exchange method fou
sshd[12402]: Unable to negotiate with 103.89.88.209 port 53368: no matching key exchange method fou
sshd[12419]: Unable to negotiate with 103.89.88.209 port 53440: no matching key exchange method fou
sshd[12444]: Unable to negotiate with 103.89.88.209 port 55135: no matching key exchange method fou
sshd[12447]: Unable to negotiate with 103.89.88.209 port 55705: no matching key exchange method fou
sshd[12449]: Unable to negotiate with 103.89.88.209 port 57382: no matching key exchange method fou
sshd[12452]: Unable to negotiate with 103.89.88.209 port 57532: no matching key exchange method fou
sshd[12471]: Unable to negotiate with 103.89.88.209 port 60401: no matching key exchange method fou
sshd[12473]: Unable to negotiate with 103.89.88.209 port 61876: no matching key exchange method fou
sshd[12476]: Unable to negotiate with 103.89.88.209 port 63279: no matching key exchange method fou

I am puzzled - what are these port numbers doing in the log? Only port 22 and one other are open?

[B.Goode]

Port 22 is the target (listening) port on your sshd-serving RPi.

Probably what you are seeing is the source/originating port on the machine 103.89.88.209 from which the suspected 'attack' is being launched?

For enlightenment, I suspect that if you make a loopback ssh connection to pi@localhost and check relevant logs you will see that your RPi chose a fairly 'random' portnumber for the outgoing connection attempt.

Edit: seems to be a well known source for suspect activity, see https://www.abuseipdb.com/check/103.89.88.209

[droleary]

Services that accept continuous connections usually move sessions to a new port so that they can continue to accept new connections on their main port. It's covered in the third paragraph on the Wikipedia Port article:

the ports that network clients connect to for service initiation provide a multiplexing service, so that multiple simultaneous communication sessions may be initiated from these ports. After an initial service request connects to the well-known port number, the port is freed by switching the servicing of the request to a dedicated, connection-specific port number.

[tpyo kingg]

Those ports in the log are the port numbers that the remote system is trying to connect from. The SSH daemon will always use the same port to receive connections. Wikipedia is not correct in that context.

The utility lsof is easy to install and will show you the connections on your machine.
Then you can see for yourself both ends of the connection, if you have active SSH sessions. Otherwise, it will just show the daemon listening. Here awk filters out the irrelevant parts:

Code: Select all

sudo lsof -n | awk '$1=="sshd" && $8=="TCP" || NR==1'

If you install sshguard, you can automatically block repeated attempts from the same address. However, most of the attacks are distributed and not be able to trigger anything. Also, the scans will find whatever port you are using, because that's what automated scans do:

• The Hail Mary Cloud And The Lessons Learned (2013)
• There's No Protection In High Ports Anymore. If Indeed There Ever Was (2013)

The Raspberry Pi is often easy prey because of the default account and password. So changing the password prior to putting it on the net is essential. But in general, it is even better to turn of password authentication for SSH and just allow keys.

If you like jokes about moving port numbers, then you could try this: https://blog.benjojo.co.uk/post/ssh-por ... -with-totp
But it would be hard to implement behind a router.

[allfox]

There is something called "fail2ban", it would read the /var/log/auth.log, and block any IP that brute forcing your SSH daemon, for 10 minutes.

You need to know some iptables to manually configure it. But if you never played with your iptables, then by default it would work.

Install it with:
sudo apt update
sudo apt install fail2ban

Then everything is set.

[jbudd]

Yes, I do use fail2ban. You apt-get install it and it starts working straight away (for ssh).
Which is lucky because it's complexity defies my attempts to modify it beyond changing the ban time.
I have looked for nice easy to understand tutorials but they inevitably refer to files which don't exist, don't work, or prevent the fail2ban server from starting.
I guess that the structure of fail2ban changes with each version so the tutorials are out of date.

I opened port 22 to get live data to work with on in my attempts to tweek fail2ban.
Normally I use a high port number and have not seen a login attempt in more than a year.
Of course log files are cycled so I might have missed them. So I wanted to implement a repeat offender jail and send email when it is triggered. I think my mobile phone may be banned for a week, but no luck with email so far!

The appearence of high port numbers (originating port) in my log was alarming, but it seems not to be an issue. Yes, I have read the links from Tpyo Kingg - thanks. Useful info, I wish they were from 2017 instead of 2012!

Key based connection - no doubt it's a good thing, I think it essentially gives you a long and unguessable password - even longer than mine.
Well I've looked for simple tutorials. The page Securing Your Raspberry Pi https://www.raspberrypi.org/documentati ... ecurity.md would be great but it talks about client and server without saying which is the Pi and which is the PC. Nor does it say how you set up Putty or other ssh package to use them. So I'm too scared to try it.

Thanks for your replies.

[tpyo kingg]

The info there from 2013 is still valid. Peter Hansteen stays on top of those topics and updates the blog or writes a new blog post when there is any change one direction or another.

But, if you are looking for an easier tool than fail2ban but with more functionality, you might take a look at sshguard instead.

About the key-based authentication, no private key leaves your system, so there is no "password" to guess. Instead the public / private halves of the key are used to make and verify and respond to a message between the server and the client. If I understand correctly (which may be a leap), the process goes like this:

• The private key (a secret) is kept on the client
• The public key is copied to where the SSH server can use it
• The SSH client contacts the server
• The SSH server uses the requested account's designated public key to encrypt a random number and send that back to the client
• The SSH client decrypts that random number using the private key and bundles it up in a message along with the session identification number and returns the MD5 hash of that message to the server.
• The SSH server compares the received hash to the one it calculated itself. If there is a match, the login is allowed. If not, the next public key is tried, until authentication happens or the server runs out of public keys to try.

See: An Illustrated Guide to SSH Agent Forwarding : How Key Challenges Work
and RFC 4252 : The Secure Shell (SSH) Authentication Protocol : 7. Public Key Authentication Method: "publickey"
and, penultimately, OpenSSH Source (portable edition)
and ultimately OpenSSH source itself

(In this context, the SSH server is the machine you are connecting to and the SSH client is the machine you are connecting from. )

[DougieLawson]

One of my Pi's is exposed to the internet by port forwarding on my router (Not port 22!)

As an experiment I opened port 22 for a while and I quickly got lines in /var/log/auth.log like this

Code: Select all

sshd[12399]: Unable to negotiate with 103.89.88.209 port 51370: no matching key exchange method fou
sshd[12402]: Unable to negotiate with 103.89.88.209 port 53368: no matching key exchange method fou

I am puzzled - what are these port numbers doing in the log? Only port 22 and one other are open?

That's perfectly normal. There are 4 billion systems out there in the public internet, a small percentage are scanning the other 3.999 billion trying to find an open port.

Ignore them or install fail2ban to block them.

The main rule with opening ssh to the four billion is to ensure that you have a strong password for pi or better disallow logins with passwords - force the use of a public/private key pair.

https://www.raspberrypi.org/documentati ... ecurity.md has more details.

[tpyo kingg]

Well I've looked for simple tutorials. The page Securing Your Raspberry Pi https://www.raspberrypi.org/documentati ... ecurity.md would be great but it talks about client and server without saying which is the Pi and which is the PC. Nor does it say how you set up Putty or other ssh package to use them. So I'm too scared to try it.

Regarding that concern, password authentication and key-based authentication can co-exist. So just postpone the last step, that of turning off password authentication, until you are sure your keys work. PuTTY has a weird key format so I avoid it myself and just stick with the standard OpenSSH client that came as part of the distro you have on your PC. Which distro is your PC running?

Again, the SSH server is the machine you are connecting to and the SSH client is the one you are connecting from. It all depends on where you have sshd running. If you are planning to connect to your Pi from another machine, the Pi is the server and the other machine is the client.

Option 1) If you want to stick with PuTTY or have to on a legacy OS then How To Create SSH Keys With PuTTY to Connect to a VPS might be relevant if you scroll down to "Generating OpenSSH-compatible Keys for Use with PuTTY" Where they talk about VPS or "droplet" substitute your Pi.

Option 2) Or as another option, since all distros these days have a full set of SSH client utilities, it is easier to use them to set things up to connect from any distro on your PC to your Pi. In that case use these steps from your PC instead:

• mkdir -m 700 ~/.ssh/
• cd ~/.ssh/
• ssh-keygen -t ed25519 -f my-rpi.ed25519 -C "jbudd at laptop computer"
• ssh-copy-id -i ~/.ssh/my-rpi.ed25519 jbudd@xx.yy.zz.aa

Where xx.yy.zz.aa is the address of your RPi. See "man ssh-keygen" and "ssh-copy-id" for reference information.

[allfox]

I agree that key pair authentication won't send the private key out.

But I doubt about "there is no key to guess".

Think about these steps:
1 Server send a challenge of a public key encrypted random number.
2 I make up a private key, and use it to decrypt the challenge.
3 If the server don't let in, I would try next key. Repeat until I'm in.

It is able to brute force. However in practice it would take almost forever to try every possible private key. And fail2ban would make it even longer.

[tpyo kingg]

Think about these steps:
1 Server send a challenge of a public key encrypted random number.
2 I make up a private key, and use it to decrypt the challenge.
3 If the server don't let in, I would try next key. Repeat until I'm in.

Think instead about how key-based authentication actually works: You can't make up a matching private key to decrypt the challenge on the first guess. That's rather the point of today's algorithms. Even if you get multiple guesses, it's computationally infeasible with non-quantum computing, not counting the time delays, and even then you have the number of tries limited by the server.

About add-ons, I accept that fail2ban has the brand recognition but I would urge an evaluation of sshguard so that an informed decision can be made about which to use, if any.

[jbudd]

Umm Dougie, I think you must have speed read my post and missed some of the detail, thanks for your reply though.

Tpyo kingg: My PC runs Windows 10 and I generally use MobaXTerm to access the command line, rarely to access the GUI. Occasionally Putty for the command line too.

Again, the SSH server is the machine you are connecting to and the SSH client is the one you are connecting from. It all depends on where you have sshd running. If you are planning to connect to your Pi from another machine, the Pi is the server and the other machine is the client.

[From https://www.raspberrypi.org/documentati ... ecurity.md] Generating a key pair in Linux is done using the ssh-keygen command on the client;

So I have to generate a key pair on my Windows PC, and on my Android phone, and on my tablet, and on each machine I could be using at work? Do I need to do it once per machine or once per program/app that will make the connection?

And it seems that the raspberrypi.org documentation is assuming that people will want to ssh from a Linux PC. Given the market penetration of Windows, that's a little bit unhelpful.

I will take a look at sshguard, and try and absorb the info from your links. It's always good to learn new stuff.

I'm not too concerned about getting hacked. Even the chinese government would take a long time to guess my password at random, even if they used distributed attacks to minimse countermeasures such as fail2ban.
But as you can tell, I'm pretty ignorant about security.

Once ssh is nicely secure, next I have to look at the other password protected services I use. More key pairs I suppose.

[tpyo kingg]

So I have to generate a key pair on my Windows PC, and on my Android phone, and on my tablet, and on each machine I could be using at work? Do I need to do it once per machine or once per program/app that will make the connection?

It doesn't matter where the keys are generated but it is a Very Good Idea to have each device use separate keys to log in. The target machine (server) can accept multiple keys for any given account. I'm not sure what the limit might be but it is not a small number. The essential aspect is that the keys work, whether you are using PuTTY or a normal client.

And it seems that the raspberrypi.org documentation is assuming that people will want to ssh from a Linux PC. Given the market penetration of Windows, that's a little bit unhelpful.

Given the way Windows got and maintains that market penetration and the mess it makes on the net, it's a little bit unhelpful to do anything other than erase it. I myself neither use nor condone use of it. But let's put that aside and focus on helping get you connected to the RPi with keys from what sounds like an interesting array of devices.

So I have to generate a key pair on my Windows PC, and on my Android phone, and on my tablet, and on each machine I could be using at work? Do I need to do it once per machine or once per program/app that will make the connection?

Once per machine ought to do it, if the program or app works with regular SSH keys. That would be what I read as the generally accepted best practice. The idea is that if a device gets compromised or lost (happens with mobile and tablets too often) then you just remove the bad keys and keep rolling. Whereas if you used one key for everything everywhere, you would be greatly inconvenienced when canceling a device and be expanding the window for attack at the same time.

For most threat models, it is analogous to not having to outrun the bear, just be able to outrun the slower guy. The automated attacks go for the easy wins and if you raise the bar even a little, they will ignore you and move on.

When you use MobaXTerm, do you type something like "ssh jbudd@xx.yy.zz.aa" to connect to the RPi? If so does it have ssh-keygen available?

Is the tablet Android or iOS?

[B.Goode]

So I have to generate a key pair on my Windows PC, and on my Android phone, and on my tablet, and on each machine I could be using at work? Do I need to do it once per machine or once per program/app that will make the connection?

And it seems that the raspberrypi.org documentation is assuming that people will want to ssh from a Linux PC. Given the market penetration of Windows, that's a little bit unhelpful.

Public/Private key pairs are a very powerful flexible toolkit.

You don't have to deploy them in any prescribed way, you can adapt them to your needs. You can, for example, just generate a key-pair once (literally once) and then reuse those keys everywhere.

For Windows, there are key generation and key agent tools shipped as part of the standard download for PuTTY.

But I sympathize: it does indeed seem a little mind-bending at first exposure. But it does all work together, across multiple OS platforms, remarkably well.

[jbudd]

When you use MobaXTerm, do you type something like "ssh jbudd@xx.yy.zz.aa" to connect to the RPi? If so does it have ssh-keygen available?

Is the tablet Android or iOS?

Yes it's ssh <id>@<IP or hostname>.
It looks as though MobaXTerm does include the ability to generate / use keys. It asks for a "key passphrase" which I'll have to look up. A password to protect the key from other people on my PC?

My tablet is Android and mostly redundant (the battery is always flat when I want to use it), but I wanted to insinuate that Apple devices could be involved too...

I see the sense in having a different key pair for each device.

[tpyo kingg]

It looks as though MobaXTerm does include the ability to generate / use keys. It asks for a "key passphrase" which I'll have to look up. A password to protect the key from other people on my PC?

The passphrase is more or less a password to protect the keys if they get lifted or copied somewhere public either by accident or by malevolent people.

(As an aside, the MobaXTerm 8.2 release notes say that it includes an SSH agent. So if you're constantly logging in and out of the same machine, you might load the keys into the agent on your PC after logging into your PC and thereby reduce the number of times you need to enter the passphrase when connecting out the RPi.)

OS X is almost as easy and capable as GNU/Linux though they lag far behind in versions of OpenSSH it's still up to the task. I haven't used iOS though nor much Android. Digging around, it looks like Termius is one option, though proprietary. JuiceSSH might be another. However, you'll have to look around. The basic concept is the same: generate the keys, transfer or paste the public key into ~/.ssh/authorized_keys on the RPi -- one key per line / one line per key. In some you may have to stay with RSA keys but in others you might be able to use ECDSA or Ed25119 instead. Avoid the deprecated DSA algorithm for keys.

[jbudd]

Edit: the subject has slipped from "About SSH brute force attacks." - probably should be a new thread.

Attempting connecting with keys, I get "Server refused our key" and it falls back to asking for the password.
I tried to follow the tutorial at https://dvpizone.wordpress.com/2014/03/ ... key-pairs/


Here's what I did:
I experimented with Putty since it's more common than MobaXTerm
On my PC
Generate a key pair with Puttykeygen
Save the public key as MyPi.pub and the private key as MyPi.ppk.
Open MyPi.pub in Notepad++.

On the Pi, via ssh
mkdir ~/.ssh
chmod 700 .ssh
edit ~/.ssh/authorized_keys
Copy/Paste everything from MyPi.pub into the file
Save
chmod 644 ~/.ssh/authorized_keys

edit /etc/ssh/sshd_config
Append this

UsePAM no
PermitRootLogin no
AllowUsers <obfuscated>
RSAAuthentication yes
PubkeyAuthentication yes
AuthorizedKeysFile .ssh/authorized_keys
PasswordAuthentication yes

Save
sudo systemctl restart ssh

On the PC, in Putty
Click data on the left and insert auto-login username
Click SSH on the left then Auth
Browse for MyPi.ppk
Click Session on the left. Insert the ip address.
Give the session a name and Save
Click on the saved session and Open.

This is the contents of ~/.ssh/authorized_keys:

Code: Select all

---- BEGIN SSH2 PUBLIC KEY ----
Comment: "rsa-key-20180507"
AAAAB3NzaC1yc2EAAAABJQAAAQEAl2nVL0rsH35Kgpwtr8E1/yZU/VfWGuoc1Gyl
VDftDEYbLTrYl0keiT5p0l5OOkXHCE23NOLNy3aS2C2/3Pw/NDUqsJwgQcC/s7jf
d61Th2zNdB4CTZbkD2qtTenJmAkJ+MLwOQ8VH4bSJ/crQy46aeWRcsWlb7Zc+vj/
ggp30xs4Qxc0/dN+cwvgJB1GGVUXgxIN3g820BpbsTzh4uXPR5v3pkBgMLnhCJMR
dql8SbMrubcwsu72BaZbBJwJDADN6kbU+uHrhBczoqa5b32HoHtZ4aL1bWt5yU1R
9u9mBknMgF+4gMR8TvRMG8pL+ncLVY2qs6FIBUs4Ga3QgLK1BQ==
---- END SSH2 PUBLIC KEY ----

Note that this isn't "one key per line, one line per key". It has new lines \n not carriage returns \r

[tpyo kingg]

I tried to follow the tutorial at https://dvpizone.wordpress.com/2014/03/ ... key-pairs/

Ok. Perhaps a step was missed on the PC when using PuTTYgen. You seem to have a key pair but the format of the public key needs conversion.

That looks like a RFC 4716 formatted key. While on the RPi, try converting the format.

Code: Select all

cd ~/.ssh/
mv authorized_keys authorized_keys.orig
ssh-keygen -i -f authorized_keys.orig > authorized_keys
cat authorized_keys

The -i option imports a RFC 4716 formatted key and the -f gets it from a particular file.

Then while the window is still open, try logging in again using the key pair.

[jbudd]

That fixed it!

Thanks very much for your help and advice.

[droleary]

I'm not too concerned about getting hacked. Even the chinese government would take a long time to guess my password at random, even if they used distributed attacks to minimse countermeasures such as fail2ban.
But as you can tell, I'm pretty ignorant about security.

I'd suggest looking directly at iptables to set up firewall rules that minimize your attack surface. While fail2ban is nice, there's no reason to wait for it to kick in for networks that never should be accessing your servers in the first place. Either blacklist IP ranges that are persistent troublemakers, or whitelist all the IP ranges you know you're going to be coming in from.