jfirestorm44
Posts: 18
Joined: Tue Apr 21, 2020 5:45 pm

Help needed: Fail2ban setup

Sat May 23, 2020 12:19 pm

Hello I am hoping someone here has used this program and can help me figure out what I'm doing wrong. I've been able to figure out most things on this RPi but I'm throwing my hands up at this one. It's been kicking my butt for days now.

I am trying to set up a jail for jellyfin. I believe I have created and updated all of the correct files but I still can't seem to get it to work. I have tried many many varitions of the below configs but this is what I have right now.

Here is what was added to my jail.local. I think this should be pretty straight forward.

Code: Select all

[jellyfin]
port     = http,https
logpath  = /var/lib/docker/containers/d5964d76ca147b7ef74cbd48a0c696c311edb6696c49622aa474efbc73333358/d5964d76ca147b7ef74cbd48a0c696c311edb6696c49622aa474efbc73333358-json.log
maxretry = 3
bantime  = 30
banaction = iptables-multiport
filter   = jellyfin
enabled  = true

Here's my /filter.d/jellyfin.conf. I kept getting date errors so I added a datepattern. I don't even know if it's correct but I'm not getting errors so I'm guessing it is. I have tried many variation of the failregex (some much longer) and am currently using the one below because I would think it should catch everything up to the IP address and the following word. I learned regex about 3 days ago so it could be completely wrong.

Code: Select all

# Fail2Ban for jellyfin
#
#
[Definition]
datepattern = ^.*%%Y-%%b-%%dT%%H:%%M:%%S:%%f$
failregex = ^.*\[<HOST>\]\sInvalid$

ignoreregex =

What a login error looks like in my jellyfin log. What concerns me about this log line is the DTG comes at the end of the line. Every example I have found online it seems everyones log files have the DTG at the beginning. Would this be the cause of the problem? Can I make fail2ban look towards the end of the line for the DTG?

Code: Select all

{"log":"[11:45:54] [ERR] [4] Emby.Server.Implementations.HttpServer.HttpListenerHost: Error processing request: [12.123.12.123] Invalid username or password entered. URL: http://jellyfin.mydomain.duckdns.org/Users/authenticatebyname\n","stream":"stdout","time":"2020-05-22T11:45:54.938405196Z"}

And here is what my test looks like:

Code: Select all

Running tests
=============

Use   failregex filter file : jellyfin, basedir: /etc/fail2ban
Use      datepattern : ^.*Year-MON-DayT24hour:Minute:Second:Microseconds$
Use         log file : /var/lib/docker/containers/d5964d76ca147b7ef74cbd48a0c696c311edb6696c49622aa474efbc73333358/d5964d76ca147b7ef74cbd48a0c696c311edb6696c49622aa474efbc73333358-json.log
Use         encoding : UTF-8


Results
=======

Failregex: 0 total

Ignoreregex: 0 total

Date template hits:

Lines: 3278 lines, 0 ignored, 0 matched, 3278 missed
[processed in 0.29 sec]

Missed line(s): too many to print.  Use --print-all-missed to print all 3278 lines

No matter how many times I've tried this with many different failregex and datepatterns I can't seem to get a match. I have also ran it against a test.log which only had failed log-in attempts pasted into it. What am I doing wrong? Any help would be greatly appreciated I'm on day 4 of trying to get this working and it's driving my crazy now!

Thank you in advance.

bls
Posts: 735
Joined: Mon Oct 22, 2018 11:25 pm
Location: Seattle, WA
Contact: Twitter

Re: Help needed: Fail2ban setup

Sat May 23, 2020 2:38 pm

I think your failregex is incorrect. It describes a string that ends in "Invalid". But, that's not what the log entry shows. I'd expect your failregex to be something like "^.*\[<HOST>\]\.*Invalid.*$" which would catch any error with the string "Invalid" in it.

Also, if you're not using the pattern prefix_line stuff, i'm not sure that the date pattern matters, since it's grabbed with ".*". I'm not sure if fail2ban really cares about the date time (as in uses it for other purposes), or if it's just part of the pattern match.

One thing that I've found extremely useful for saving my patience in debugging these things is to extract one single log entry to a file and use that for testing with fail2ban so that there's a lot less "stuff" to deal with.

HTH
Pi tools:
Free your network from your router's DHCP/DNS and run it on a Pi:https://github.com/gitbls/ndm
Quickly and easily build customized-just-for-you SD Cards: https://github.com/gitbls/sdm
Easy strongSwan VPN installer/manager: https://github.com/gitbls/pistrong
Lightweight Virtual VNC Config: https://github.com/gitbls/RPiVNCHowTo

jfirestorm44
Posts: 18
Joined: Tue Apr 21, 2020 5:45 pm

Re: Help needed: Fail2ban setup

Sat May 23, 2020 4:15 pm

Thanks for the reply. I added the changes to the filter. I also put the log with just one line into a test file and ran against that. Still no match though.

Code: Select all

jfirestorm44@raspberrypi:/etc/fail2ban $ sudo fail2ban-regex /etc/fail2ban/test/test.log /etc/fail2ban/filter.d/jellyfin.conf

Running tests
=============

Use   failregex filter file : jellyfin, basedir: /etc/fail2ban
Use      datepattern : ^.*Year-MON-DayT24hour:Minute:Second:Microseconds$
Use         log file : /etc/fail2ban/test/test.log
Use         encoding : UTF-8


Results
=======

Failregex: 0 total

Ignoreregex: 0 total

Date template hits:

Lines: 1 lines, 0 ignored, 0 matched, 1 missed
[processed in 0.00 sec]

|- Missed line(s):
|  {"log":"[11:45:54] [ERR] [4] Emby.Server.Implementations.HttpServer.HttpListenerHost: Error processing request: [12.123.12.123] Invalid username or password entered. URL: http://jellyfin.mydomain.duckdns.org/Users/authenticatebyname\n","stream":"stdout","time":"2020-05-22T11:45:54.938405196Z"}
Here my new filter:

Code: Select all

# Fail2Ban for jellyfin
#
#
[Definition]
datepattern = ^.*%%Y-%%b-%%dT%%H:%%M:%%S:%%f$
failregex = ^.*\[<HOST>\].*Invalid.*$

ignoreregex =
Anything else I could be missing?

bls
Posts: 735
Joined: Mon Oct 22, 2018 11:25 pm
Location: Seattle, WA
Contact: Twitter

Re: Help needed: Fail2ban setup

Sat May 23, 2020 6:40 pm

I found a solution that works on your log. First, the datepattern string should be

Code: Select all

%%Y-%%m-%%dT%%H:%%M:%%S
Not sure what the %%b is that you used for the month, but seems that you need to use %%m instead. There doesn't appear to be a string to match against the nanoseconds. See https://docs.python.org/3/library/time.html, for example.

Secondly, I stripped the failregex down to

Code: Select all

 ^.*: Error processing request: \[<HOST>\] Invalid .*$
This works against the one logline you've provided. Depending on what other log lines look like, this may be sufficient, or you may need to add additional regex or expand this one.

One last thing. I ran into a page that seems to be VERY useful for regex debugging: https://regex101.com/

Good luck. Always handy with fail2ban :lol:
Pi tools:
Free your network from your router's DHCP/DNS and run it on a Pi:https://github.com/gitbls/ndm
Quickly and easily build customized-just-for-you SD Cards: https://github.com/gitbls/sdm
Easy strongSwan VPN installer/manager: https://github.com/gitbls/pistrong
Lightweight Virtual VNC Config: https://github.com/gitbls/RPiVNCHowTo

jfirestorm44
Posts: 18
Joined: Tue Apr 21, 2020 5:45 pm

Re: Help needed: Fail2ban setup

Sat May 23, 2020 9:48 pm

Thanks for the resource page. It didn't solve my problem but was most definitely informative and allowed me to see some mistakes. I was using %%b which is used when the month is in format MMM. I changed it to %%m now as you suggested. I got the %%f from GitHub where another guy was having DTG issues and was told to use %%f for microseconds.

I tried your failregex and it didn't work. I'm dang near convinced this is some kind of DTG issue. From the .8 manual it says
In order for a log line to match your failregex, it actually has to match in two parts: the beginning of the line has to match a timestamp pattern or regex, and the remainder of the line has to match your failregex. If the failregex is anchored with a leading ^, then the anchor refers to the start of the remainder of the line, after the timestamp and intervening whitespace.
I don't know if it's been changed or improved in the newest version to support looking further into the log lines or if there is a work around code to make it do that.

I was hoping this would be easier to do but it's kicking my butt. Oh and I have found those regex testing pages very useful.

bls
Posts: 735
Joined: Mon Oct 22, 2018 11:25 pm
Location: Seattle, WA
Contact: Twitter

Re: Help needed: Fail2ban setup

Sat May 23, 2020 11:34 pm

jfirestorm44 wrote:
Sat May 23, 2020 9:48 pm

I tried your failregex and it didn't work. I'm dang near convinced this is some kind of DTG issue.
That's fascinating :? . Here's what I tested. I thought I copied the stuff correctly from your post. You'll know for sure.

Code: Select all

pisrv1~/tmp# cat f2bin.txt 
{"log":"[11:45:54] [ERR] [4] Emby.Server.Implementations.HttpServer.HttpListenerHost: Error processing request: [12.123.12.123] Invalid username or password entered. URL: http://jellyfin.mydomain.duckdns.org/Users/authenticatebyname\n","stream":"stdout","time":"2020-05-22T11:45:54.938405196Z"}
pisrv1~/tmp# cat /etc/fail2ban/filter.d/jellyfin.conf 
# Fail2Ban for jellyfin
#
#
[Definition]
datepattern = %%Y-%%m-%%dT%%H:%%M:%%S
failregex = ^.*: Error processing request: \[<HOST>\] Invalid .*$

ignoreregex =
pisrv1~/tmp# fail2ban-regex --print-all-matched --print-all-missed --verbosity=4 --VD f2bin.txt jellyfin.conf

Running tests
=============

Use   failregex filter file : jellyfin, basedir: /etc/fail2ban
Use      datepattern : Year-Month-DayT24hour:Minute:Second
Use         log file : f2bin.txt
Use         encoding : UTF-8


Results
=======

Failregex: 2 total
|-  #) [# of hits] regular expression
|   1) [1] ^.*: Error processing request: \[<HOST>\] Invalid .*$
|      12.123.12.123  Fri May 22 11:45:54 2020
`-

Ignoreregex: 0 total

Date template hits:
|- [# of hits] date format
|  [1] Year-Month-DayT24hour:Minute:Second
|      # weight: 0.004 (1.000), pattern: %Y-%m-%dT%H:%M:%S
|      # regex:   (?:^|\b|\W)((?P<Y>\d\d\d\d)-(?P<m>1[0-2]|0[1-9]|[1-9])-(?P<d>3[0-1]|[1-2]\d|0[1-9]|[1-9]| [1-9])T(?P<H>2[0-3]|[0-1]\d|\d):(?P<M>[0-5]\d|\d):(?P<S>6[0-1]|[0-5]\d|\d))(?=\b|\W|$)
`-

Lines: 1 lines, 0 ignored, 1 matched, 0 missed
[processed in 0.00 sec]

|- Matched line(s):
|  {"log":"[11:45:54] [ERR] [4] Emby.Server.Implementations.HttpServer.HttpListenerHost: Error processing request: [12.123.12.123] Invalid username or password entered. URL: http://jellyfin.mydomain.duckdns.org/Users/authenticatebyname\n","stream":"stdout","time":"2020-05-22T11:45:54.938405196Z"}

Pi tools:
Free your network from your router's DHCP/DNS and run it on a Pi:https://github.com/gitbls/ndm
Quickly and easily build customized-just-for-you SD Cards: https://github.com/gitbls/sdm
Easy strongSwan VPN installer/manager: https://github.com/gitbls/pistrong
Lightweight Virtual VNC Config: https://github.com/gitbls/RPiVNCHowTo

jfirestorm44
Posts: 18
Joined: Tue Apr 21, 2020 5:45 pm

Re: Help needed: Fail2ban setup

Sun May 24, 2020 12:41 pm

Holy cow man it worked! Good lord what a relief that is. OMV was easy to get working but jellyfin has been such a pain. I really appreciate the help. Programs like this without a dedicated forum make it hard to get help sometimes. You're awesome!

bls
Posts: 735
Joined: Mon Oct 22, 2018 11:25 pm
Location: Seattle, WA
Contact: Twitter

Re: Help needed: Fail2ban setup

Sun May 24, 2020 2:55 pm

jfirestorm44 wrote:
Sun May 24, 2020 12:41 pm
Holy cow man it worked! Good lord what a relief that is. OMV was easy to get working but jellyfin has been such a pain. I really appreciate the help. Programs like this without a dedicated forum make it hard to get help sometimes. You're awesome!
Glad to hear that! Configuring fail2ban is one of those make-you-older-faster apps until you wrestle it into submission.
Pi tools:
Free your network from your router's DHCP/DNS and run it on a Pi:https://github.com/gitbls/ndm
Quickly and easily build customized-just-for-you SD Cards: https://github.com/gitbls/sdm
Easy strongSwan VPN installer/manager: https://github.com/gitbls/pistrong
Lightweight Virtual VNC Config: https://github.com/gitbls/RPiVNCHowTo

Return to “Advanced users”