UFW is blocking my ftp traffic.

[eddie3000]

Hello friends.

I have a raspberry with raspbian buster, which I believe to be based on debian 10. I have it with openssh-server and openvpn running.

I have a script that downloads and uploads to different ftp servers that are not mine. The script is triggered using cron. I am using wget and curl. None of them work with ufw enabled, not even from the command line.

Here are the UFW rules:

Code: Select all
To Action From
-- ------ ----
22 ALLOW IN Anywhere
1194 ALLOW IN Anywhere
80 ALLOW IN Anywhere
443 ALLOW IN Anywhere
21 ALLOW IN Anywhere
20,21/tcp ALLOW IN Anywhere
22 (v6) ALLOW IN Anywhere (v6)
1194 (v6) ALLOW IN Anywhere (v6)
80 (v6) ALLOW IN Anywhere (v6)
443 (v6) ALLOW IN Anywhere (v6)
21 (v6) ALLOW IN Anywhere (v6)
20,21/tcp (v6) ALLOW IN Anywhere (v6)

21/tcp ALLOW OUT Anywhere
21 ALLOW OUT Anywhere
21/tcp (v6) ALLOW OUT Anywhere (v6)
21 (v6) ALLOW OUT Anywhere (v6)



When I disable UFW the script works fine. I have reset ufw various times and re-entered all the rules, one at a time, but without success.

I have another computer with debian 10 recently installed, same setup as the raspberry. Openssh-server and openvpn, and the exact same script triggered from cron. With only the ssh ports and vpn ports allowed in ufw, it works flawlessly.

Code: Select all
To Action From
-- ------ ----
22 ALLOW IN Anywhere
1194 ALLOW IN Anywhere
22 (v6) ALLOW IN Anywhere (v6)
1194 (v6) ALLOW IN Anywhere (v6)



I somehow believe that ufw on my raspberry is not setting up iptables correctly, and ufw reset is not working. What can I do? The easiest solution for me would be to reinstall from scratch as it would only take be about half an hour. But I know nothing about iptables and it might be educational to fix it instead of reinstalling. Can this be all fixed done via ssh without getting locked out as well?

[B.Goode]

Welcome to the Raspberry Pi forums.

Hello friends.

I have a raspberry with raspbian buster, which I believe to be based on debian 10. I have it with openssh-server and openvpn running.

I have a script that downloads and uploads to different ftp servers that are not mine. The script is triggered using cron. I am using wget and curl. None of them work with ufw enabled, not even from the command line.

Here are the UFW rules:

Code: Select all
To Action From
-- ------ ----
22 ALLOW IN Anywhere
1194 ALLOW IN Anywhere
80 ALLOW IN Anywhere
443 ALLOW IN Anywhere
21 ALLOW IN Anywhere
20,21/tcp ALLOW IN Anywhere
22 (v6) ALLOW IN Anywhere (v6)
1194 (v6) ALLOW IN Anywhere (v6)
80 (v6) ALLOW IN Anywhere (v6)
443 (v6) ALLOW IN Anywhere (v6)
21 (v6) ALLOW IN Anywhere (v6)
20,21/tcp (v6) ALLOW IN Anywhere (v6)

21/tcp ALLOW OUT Anywhere
21 ALLOW OUT Anywhere
21/tcp (v6) ALLOW OUT Anywhere (v6)
21 (v6) ALLOW OUT Anywhere (v6)



When I disable UFW the script works fine. I have reset ufw various times and re-entered all the rules, one at a time, but without success.

I have another computer with debian 10 recently installed, same setup as the raspberry. Openssh-server and openvpn, and the exact same script triggered from cron. With only the ssh ports and vpn ports allowed in ufw, it works flawlessly.

Code: Select all
To Action From
-- ------ ----
22 ALLOW IN Anywhere
1194 ALLOW IN Anywhere
22 (v6) ALLOW IN Anywhere (v6)
1194 (v6) ALLOW IN Anywhere (v6)



I somehow believe that ufw on my raspberry is not setting up iptables correctly, and ufw reset is not working. What can I do? The easiest solution for me would be to reinstall from scratch as it would only take be about half an hour. But I know nothing about iptables and it might be educational to fix it instead of reinstalling. Can this be all fixed done via ssh without getting locked out as well?

I can't tell you exactly how to configure ufw, I don't use it. but I can give you a hint.


It's a long time (years... ) since I had to do battle with internet security for the insecure-by-design ftp protocol. I think there might be 2 gotchas to deal with. You need multiple outbound ports for ftp? Maybe you have done that. The other is that the server at the other (server) end needs to open a random port back to your client, not the 20/21 that was used outbound.

Finding an alternative to ftp seems like the best way forward?

[eddie3000]

Finding an alternative to ftp seems like the best way forward?

No. I do not own the ftp servers I fetch files from or send files to. It's their administrator's decision, and I need to work with them as is. I already mentioned some time ago that they weren't encrypting their user or password, but they don't seem to care too much about it. Nothing bad has happened either in the past few years anyway. But, as I said, it's not something I can do anything about.

Why does it work on debian with the rules I showed before, but does not work on raspbian?

[knute]

Well I have a couple of questions:

Are there more rules in UFW than you posted?

Are your problems with outgoing FTP or incoming?

And a couple of observations:

Your rules seem too complex for a simple installation.

There is a fault in UFW that prevents inbound passive FTP from working correctly. If that is your problem, I'll post the fix.

If there are more rules, I can see why your outgoing wget and curl don't work.

[eddie3000]

Thank you very much for your answers.

The rules I have are the ones I have posted. Just copied and pasted the output of "sudo ufw numbered verbose".

I started off just entering the rules I entered on my debian pc, and while that works on debian it does not on the raspberry. I further added more rules to see if that helped, those are the extra rules you see in the first post. On my debian pc I have no need to do "ufw allow ftp" because it just works.

The script is the same on both the raspberry pi and the computer with debian. It uses wget to fetch files from one ftp server. It uses curl to upload files to another ftp server. Now I come to think of it, I use ubuntu on my laptop which also uses a similar script, I have ufw enabled, and it works as well.

There is a fault in UFW that prevents inbound passive FTP from working correctly. If that is your problem, I'll post the fix.

Yes, I think that's my issue. It stops when at the PASV... point.

[eddie3000]

Yes, I've just read about passive and active ftp connections. I had no idea about that. I'm now trying to figure out what port range is open on the ftp server. I might as well just ring the server administrators. But I still don't understand why ufw doesn't block the random port connections on debian. The rules should be just the same right?:

Code: Select all

To Action From
-- ------ ----
22 ALLOW IN Anywhere
1194 ALLOW IN Anywhere
22 (v6) ALLOW IN Anywhere (v6)
1194 (v6) ALLOW IN Anywhere (v6) 

[knute]

You don't want to open a port range. You want the firewall to take care of it for you. What you need to do is add the following to the end of /etc/ufw/before.rules:

Code: Select all

*raw
:PREROUTING ACCEPT [0:0]
-A PREROUTING -p tcp --dport 21 -j CT --helper ftp
COMMIT

That will make inbound passive FTP work. You only need to open port 21.

I would really like to see the command line you use to enter one of your ufw rules, please?

[eddie3000]

In debian I entered:

Code: Select all

sudo ufw allow 22

for ssh and

Code: Select all

sudo ufw allow 1194

for openvpn.

I also set it to

Code: Select all

sudo ufw default deny

.

And ftp uploading and downloading just works.

On the raspberry it doesn't work. That's why, as an ignorant user I am, I entered all types of stuff like:

Code: Select all

sudo ufw allow 21

.

Code: Select all

sudo ufw allow out 21

.

Code: Select all

sudo ufw allow in to any port 21

.

Code: Select all

sudo ufw allow 1:65000/tcp

Code: Select all

sudo ....

Code: Select all

sudo ufw reset

.

And repeating the silly stuff all over again. Maybe I'd be best just starting from scratch and reinstalling. But before I will try your solution, Knute, as soon as I can. Thank you for your help.

[DougieLawson]

OpenVPN port 1194 is UDP/IP not TCP/IP.

FTP uses two ports 20 & 21 (unless you run passive (aka PASV)).

[knute]

In debian I entered:

Code: Select all

sudo ufw default deny

.

I don't know if that blocks outgoing or not but it is not required and if it blocks outgoing you are going to have all sorts of issues.

And ftp uploading and downloading just works.

As you now know there are two types of FTP, passive and active. I have found there are differences in the default mode for FTP clients. If you are not sure what mode it is using it may have slipped by. You only need port 20 open if passive is off.

[eddie3000]

Thank you guys.

The ftp I'm using I believe is passive.

If I'm not mistaken, ufw default deny I think only affects the inwards direction. I think it's ufw default deny outgoing for outwards traffic.

Openvpn by default uses udp, but I have set it to tcp. It works. That's not the problem.

I tried your suggestion adding the following to the end of /etc/ufw/before.rules:

*raw
:PREROUTING ACCEPT [0:0]
-A PREROUTING -p tcp --dport 21 -j CT --helper ftp
COMMIT

I don't know what happened. My raspberry wouldn't start after that. Data corruption? Bad sd? Dunnow.

I have reinstalled raspbian lite on the sdcard. It seems to be working at the moment, maybe the sd card is ok? It's quite old (three years now) so it might be time for a change. I will try and enable ufw again tomorow, but I think I might swap the sd card just in case and discard that.

Once I get it all up and running again, I will try and turn on the firewall

Thanks.

[eddie3000]

Ok, I don't understand a thing.

I have had to re-install raspbian because it wouldn't boot. After setting it all up again, and making sure it functioned correctly, I enabled ufw.

Code: Select all

sudo ufw allow 22
sudo ufw allow 1194
sudo ufw default dent
sudo ufw enable

The ftp uploads and downloads in passive mode work. Just like in my debian computer. But I hadn't realized that ufw was blocking web browsing as well, on both debian and raspbian. At least it's good that they are now behaving the same way. Hopefully the fix for one will fix the other.

I have tried to allow ports 80, 443 and 943, but no luck.

I am using openvpn to connect to the machines to access the machine's local resources and browse the web. Not that I need to use a vpn for browsing the web, I just happen to browse the web at the same time. It's inconvenient to keep connecting and disconnecting from the vpn server if I'm doing various things at a time. So disabling ufw is the way to go for the moment. Everything works without ufw.

I don't understand, it's too complex for me.

[dickon]

Why are you running a firewall? What makes you think you need one?

[knute]

I don't think I actually understand your configuration. Are you running an FTP server on the Pi? Are you running OpenVPN server on the Pi? Are you running apache on the Pi? If not you don't need to open any of those ports. Please explain in detail your configuration.

Please show me the result of sudo ufw status verbose.

[eddie3000]

Sorry for not being specific. I will start all over and tell you the story of my lazy life. But things have changed since I started this thread, because ftp passive connection do work now with ufw enabled.

I have a raspberry pi, at home, with raspbian 10 buster. It has openvpn (using the pivpn install script), and openssh-server (which already comes installed I think). I connect to it from my laptop (ubuntu) through the vpn mainly to access my local NAS, (another different raspberry pi on my local network).

I do not have any ftp servers running. I have to work with somebody's windows ftp server to download some files, do some stuff on those files, and send the result to somebody else's windows ftp server. Both servers seem to be working in that so called passive mode. I use wget and curl to download and upload to these ftp servers. This job is automated with a script and crontab on the raspberry vpn server.

At work I have another computer, not a raspberry, running debian 10 buster. Similar automated scripts to ones on the raspberry. I set this one up some time ago, and because it seemed to be doing so well I thought I'd automate what I manually did on my raspberry pi vpn server. Just laziness, I guess, but it saves me and my co-workers a few minutes everyday.

I had to reínstall raspbian a few posts ago for some strange reason after doing something Knute recommended... It wouldn't boot. Not even the rainbow screen appeared. But now, after setting the raspberry pi again, both the raspbian and debian machines are behaving similarly.

I'm not accusing Knute of anything. It's just a coincidence that it happened just then. Right Knute?

Because I am normally connected to my home machine via vpn to access my NAS, I also do web browsing, and that all goes through the vpn server. Because I'm lazy, it's a pain having to switch the vpn connection on and off every now and then. Before the reinstall, the ftp wouldn't work with ufw enabled. But now my ftp scripts do work perfectly, but web browsing doesn't.

I hadn't noticed that the debian computer didn't allow web browsing because I only connected to it when necessary, and didn't do any web browsing.

It's my fault for not writing things down in the past, but I do recall doing something paranoid to the firewall on the raspberry pi to deny all input and output, set some rules, and edited an openvpn config file to allow dns forwarding to get things going. All went well until I started doing the ftp stuff from the raspberry pi. It was then that I attempted to reset the firewall doing sudo ufw reset, but even then I still couldn't upload nor download from the ftp servers.

So after reinstalling raspbian I I started off with only these rules:

Code: Select all

Status: active

     To                         Action      From
     --                         ------      ----
[ 1] 22                       ALLOW IN    Anywhere                  
[ 2] 1194                      ALLOW IN    Anywhere                  
[ 3] 22 (v6)                  ALLOW IN    Anywhere (v6)             
[ 4] 1194 (v6)                 ALLOW IN    Anywhere (v6)             

Ftp works, web browsing does not. I now tried this to allow web browsing:

Code: Select all

Status: active

     To                         Action      From
     --                         ------      ----
[ 1] 22                       ALLOW IN    Anywhere                  
[ 2] 1194                      ALLOW IN    Anywhere                  
[ 3] 943                        ALLOW IN    Anywhere                  
[ 4] 80                         ALLOW IN    Anywhere                  
[ 5] 443                        ALLOW IN    Anywhere                  
[ 6] Anywhere                   ALLOW IN    192.168.1.66             
[ 7] Anywhere                   ALLOW IN    192.168.1.0/24           
[ 8] 22 (v6)                  ALLOW IN    Anywhere (v6)             
[ 9] 1194 (v6)                 ALLOW IN    Anywhere (v6)             
[10] 943 (v6)                   ALLOW IN    Anywhere (v6)             
[11] 80 (v6)                    ALLOW IN    Anywhere (v6)             
[12] 443 (v6)                   ALLOW IN    Anywhere (v6)             

At least I have ftp connections working with ufw, which means my initial problem is solved.

I've run into a different problem, nothing to do with the initial one. The only thing in common is that ufw is in the way. Somebody at the debian forum recommended just turning ufw off. Is that a safe way to go?

[knute]

If your computer or network are open to the internet then I think you need a firewall. UFW is a simple way to configure that firewall. If you are using wget and curl from your Pi through the VPN or not to remote servers, opening an inbound port on your Pi is not going to do anything for you.

The codes I gave you would have no effect if you are not running an FTP server on your Pi.

On your firewall rules, line 6 is redundant with line 7.

I don't know much about VPNs but the ones I am familiar with go to a specific place and make you part of that network. An inbound VPN connection would need an open port(s). An outbound might need something. I know that my PPTP outbound connection from my Pi needs access through the firewall but it doesn't open a specific port.

So please correct me if I'm misunderstanding you. You are connecting remotely to your Pi via a VPN and then trying to browse the internet. Are you forwarding the VPN connection through a router to your Pi or is the VPN connected to your network via the router?

[eddie3000]

So please correct me if I'm misunderstanding you. You are connecting remotely to your Pi via a VPN and then trying to browse the internet. Are you forwarding the VPN connection through a router to your Pi or is the VPN connected to your network via the router?

Good question. I'm not sure. I have a router, the one the isp installed. I then have two unmanaged switches, one upstairs and another downstairs. My raspberries are connected to one of the switches.

The router has two ports (ssh and vpn) set pointing at one raspberry pi only. My raspberry pi sends me the public ip every time it changes, my laptop automatically updates the ovpn file so I don't have to do anything. I connect to my vpn whenever I'm not at home usually to access files on my my nas, which is another raspberry pi.

The nas has ufw blocking all in and out ports except for the port for sftp/ssh. Every user at home and myself have a keypair to access files over the local network. Here, ufw works perfectly. I disable ufw every now and then to perform updates, and enable it again afterwards.

Whenever I connect to my vpn server, I always connect via ssh to it's local address over vpn. The ssh port open to the internet from the router is there only in case the vpn server isn't working for whatever the reason, so I can ssh in and reboot in the hope that will remotely solve the problem.

I haven't had time to do anything lately, but I think my network browsing incapability through my vpn server is not ufw's fault but the vpn missing some type of configuration. I can't remember what I did when I got it first up and running quite some time ago.

Thanks knute. The initial thread problem I think is solved, even though we will never really know what the cause was as I had to reinstall.

I think the lesson I have learned is "Don't be lazy, don't count on your memory. Write everything you do down, keep a log and backup all your important config files." Not doing so is asking for trouble, especially as one gets older and older and older.... and older

[knute]

I think the lesson I have learned is "Don't be lazy, don't count on your memory. Write everything you do down, keep a log and backup all your important config files." Not doing so is asking for trouble, especially as one gets older and older and older.... and older

Just so you know, that only gets worse