calculon
Posts: 36
Joined: Tue Jun 26, 2012 9:07 am

Reverse ssh - ddns question

Tue Mar 14, 2017 1:22 pm

I have a remote Pi acting as a camera behind a 3G dongle. I'm using reverse ssh to access its pictures. I'm using another Pi at home as the server. Home Pi is has a static address from a ddns provider.

I'm running this script in cron to create the tunnel, periodically check it is open and create again if it is not.

Code: Select all

#!/bin/bash
createTunnel() {
  /usr/bin/ssh -o "ServerAliveInterval 30" -o "ServerAliveCountMax 3" -fNR \*:8080:localhost:8080 homepi.ddns.name -p 443
  if [[ $? -eq 0 ]]; then
    echo Tunnel created successfully
  else
    echo An error occurred creating a tunnel. RC was $?
  fi
}
/bin/pidof ssh
if [[ $? -ne 0 ]]; then
  echo Creating new tunnel connection
  createTunnel
fi
Everything runs smoothly until homepi's IP address changes. Then, even after the ddns service has picked up on the change, the remote Pi will not. I need to reboot it or enter

Code: Select all

/usr/bin/ssh -o "ServerAliveInterval 30" -o "ServerAliveCountMax 3" -fNR \*:8080:localhost:8080 homepi.ddns.name -p 443
manually to get the tunnel back up.

I think (in simple newbie terms, that's what I am when it comes to ssh tunnels) the remote Pi still thinks the tunnel is up as the ssh process hasn't stopped, so isn't trying to make a new connection. As the IP address of the server has changed, remote Pi can't be contacted. Rebooting or manually running the command cures this.

Please can you suggest a solution? Or have I failed to spot the real problem?

calculon
Posts: 36
Joined: Tue Jun 26, 2012 9:07 am

Re: Reverse ssh - ddns question

Tue Mar 14, 2017 4:35 pm

Here's what the remote Pi tunnel.log shows for successive runs of the cron (every five minutes) if I change the IP of the server...

First, the ssh PID number and an error message

Code: Select all

[email protected]:~ $ cat tunnel.log
1616
Timeout, server homepi.ddns.name not responding.
Then when it finds the server

Code: Select all

[email protected]:~ $ cat tunnel.log
Creating new tunnel connection
Warning: Permanently added the ECDSA host key for IP address '[xx.xx.xx.xx]:443' to the list of known hosts.
Tunnel created successfully
Warning: remote port forwarding failed for listen port 8080
(is this warning a clue? Does it mean port 8080 is still listening to the old ddns IP?)
Then the new PID is shown

Code: Select all

[email protected]:~ $ cat tunnel.log
1761
But there is no connection! However, if I kill that process

Code: Select all

[email protected]:~ $ kill -9 1761
A new tunnel opens, there is no error message about port 8080 and we have success!

Code: Select all

[email protected]:~ $ cat tunnel.log
Creating new tunnel connection
Tunnel created successfully
[email protected]:~ $ cat tunnel.log
1823
Is there something I need to add to the script (see original post) to kill the old process and close the port when the connection is lost?

calculon
Posts: 36
Joined: Tue Jun 26, 2012 9:07 am

Re: Reverse ssh - ddns question

Tue Mar 14, 2017 5:37 pm

Now I've tried adding

Code: Select all

-o ExitOnForwardFailure=yes
to the ssh command but all that does is turn the 'remote port forwarding failed' message from a warning into an error! I'm giving up for tonight - it seems to me that the old ssh process needs to be killed when contact is lost with the server, and that this isn't happening at the moment. As I said earlier, I'm a newbie with ssh tunnels - is there anyone who can help me?

calculon
Posts: 36
Joined: Tue Jun 26, 2012 9:07 am

Re: Reverse ssh - ddns question

Tue Mar 14, 2017 7:38 pm

I've only just realized I really should have posted this in the 'troubleshooting' section. If a mod sees this please can you move it for me?

User avatar
emgi
Posts: 357
Joined: Thu Nov 07, 2013 4:08 pm
Location: NL

Re: Reverse ssh - ddns question

Tue Mar 14, 2017 7:52 pm

It seems the ssh process does not terminate but keeps on running in a sort of stalled state.
That would explain why the crontab check fails to detect a problem.
Perhaps you can try a slightly different approach:
Regularly check if the ip address has changed. (nslookup to a file, check if equal to previous)
If changed, kill the ssh process and recreate the tunnel.

/emgi

calculon
Posts: 36
Joined: Tue Jun 26, 2012 9:07 am

Re: Reverse ssh - ddns question

Wed Mar 15, 2017 1:31 pm

That looks elegant and I'm sure it would work - but beyond my capabilities at the moment. I haven't done any programming for 30 years, so I'm taking baby steps... I understand the concepts, it's the syntax of Linux that I am trying to learn.

I was wondering about putting in a line to kill the ssh process just before the ssh command - but I need to work out how to extract the process ID from the /bin/pidof ssh line and use it in a kill -9 $whatever command. That's my next task.

User avatar
emgi
Posts: 357
Joined: Thu Nov 07, 2013 4:08 pm
Location: NL

Re: Reverse ssh - ddns question

Wed Mar 15, 2017 3:26 pm

That could be something like:

Code: Select all

echo $(/bin/pidof ssh)

calculon
Posts: 36
Joined: Tue Jun 26, 2012 9:07 am

Re: Reverse ssh - ddns question

Thu Mar 16, 2017 1:16 pm

I added a couple of lines to the script (lines 3+4)

Code: Select all

#!/bin/bash
createTunnel() {
  old_ssh=`echo $ /bin/pidof ssh`
  kill -9 $old_ssh
  /usr/bin/ssh -o "ServerAliveInterval 30" -o "ServerAliveCountMax 3" -fNR \*:8080:localhost:8080 homepi.ddns.name -p 443
  if [[ $? -eq 0 ]]; then
    echo Tunnel created successfully
  else
    echo An error occurred creating a tunnel. RC was $?
  fi
}
/bin/pidof ssh
if [[ $? -ne 0 ]]; then
  echo Creating new tunnel connection
  createTunnel
fi
They must have some syntax errors because I get some error messages

Code: Select all

Creating new tunnel connection
tunnel_script.sh: line 4: kill: $: arguments must be process or job IDs
tunnel_script.sh: line 4: kill: /bin/pidof: arguments must be process or job IDs
tunnel_script.sh: line 4: kill: ssh: arguments must be process or job IDs
Warning: Permanently added the ECDSA host key for IP address '[xx.xx.xx.xx]:443' to the list of known hosts.
Tunnel created successfully
But for some reason it now works - no warning message about port 8080. The remote Pi tracks the ddns change of the server and rebuilds the tunnel automatically, so I'm happy. But if you can tell me what the syntax errors are, I'd be grateful - I'd like it to be error-free, even if it makes no difference!

Edit - @emgi - I now see you have put brackets around /bin/pidof ssh, which I missed. I'll try that.

calculon
Posts: 36
Joined: Tue Jun 26, 2012 9:07 am

Re: Reverse ssh - ddns question

Thu Mar 16, 2017 3:43 pm

OK - I changed

Code: Select all

old_ssh=`echo $ /bin/pidof ssh`
to

Code: Select all

old_ssh=`echo $(/bin/pidof ssh)`
it made a difference - the log file now said

Code: Select all

Creating new tunnel connection
kill: usage: kill [-s sigspec | -n signum | -sigspec] pid | jobspec ... or kill -l [sigspec]
Warning: Permanently added the ECDSA host key for IP address '[xx.xx.xx.xx]:443' to the list of known hosts.
Tunnel created successfully
Warning: remote port forwarding failed for listen port 8080
But the port forwarding warning was back, and it didn't work. So I went back to the old version -

Code: Select all

Creating new tunnel connection
tunnel_script.sh: line 4: kill: $: arguments must be process or job IDs
tunnel_script.sh: line 4: kill: /bin/pidof: arguments must be process or job IDs
tunnel_script: line 4: kill: ssh: arguments must be process or job IDs
Warning: Permanently added the ECDSA host key for IP address '[xx.xx.xx.xx]:443' to the list of known hosts.
Tunnel created successfully
The errors were back, but it worked again! So I think I'll leave it alone now. But I'm curious as to why one incorrect version worked but not the other, especially if the kill command wasn't being called on either occasion.

User avatar
emgi
Posts: 357
Joined: Thu Nov 07, 2013 4:08 pm
Location: NL

Re: Reverse ssh - ddns question

Thu Mar 16, 2017 6:32 pm

Sorry for putting you in the wrong direction with the use of the echo command.
That was merely intended as an example.
The expression $(between quotes) is a replacement for the awkward single quotes.
A sample shell script perhaps illustrates this best:

Code: Select all

#!/bin/bash

old_ssh=$(/bin/pidof ssh)
echo $old_ssh
Otherwise I think you are moving in the right direction.
Tweaking this script will be a real learning experience!

/emgi

Return to “General discussion”