Report after one month live RaspbianWheezy web server use

[kaspencer]

Greetings all.

Some will have seen my posting of my webserver setup for the Raspbian Wheezy OS with Apache2, MySQL and PHP.
You can find the link to the PDF file at (http://www.raspberrypi.org/phpBB3/viewt ... 30#p238830).
As I have now been running that server live on the Internet for pretty nearly a full month, I thought that people might be interested in a brief report. The details of the setup are all in the PDF given in the original posting so I won't repeat them here, but here is the news:

1. Hits: between 29th November and 27th December, the server received well over 22,000 hits to the main virtual host. These are user hits, not maintenance or other retrievals. They were from more than 65 countries worldwide, and consequently came in at all times of day and night.

2. Data: Between the same period, well over 1.7GByte of data was transferred from the server. These data include lots of music files (MP3) of original recordings (not copies of commercial recordings), video files (MP4, WMV and MPG) and images, mainly to do with music (guitars and pipe organs).

3. Downtime: the server has been down early most Sunday mornings (GMT) each week for a couple of hours - for the reason for this, please read the "Issues" section below. As well as this there were more-or-less two days when it was not noticed that Apache had stopped - the stoppage was related to the Sunday morning stoppages mentioned earlier and explained below.

4. Security: reading the logs reveals several attempts each week to either bring the server down or to obtain illegal access. Some of these were from a hacker using the Morpheus tool on several different IP addresses. Each address has been blocked as soon as it has been identified. There have also been several other attempted access that appear suspicious: people looking for various scripts and so on. Whereas when I used a Belkin ADSL2+ router, SYN Flood attacks used to get through and crash my network, I find that my cable router is much more resilient at protecting my LAN from these types of attack.
Personally, I'd like to see more debate on the security of Raspbian Wheezy in general.

5. Performance: Since 1995, I have had several domains, including .COM, .CO.UK and .ORG.UK which are run on commercial servers. However, I have also run many other domains and sub-domains from servers in my office: these platforms have included Red Hat and Fedora Core Linux, SCO Unix, Windows NT Professional, Windows 2000 Advanced Server, and currently Windows Server 2003 R2 on hardware ranging from Pentium II, III, 4, and Core2Duo and Core2Quad, with RAM varying from 64Mbyte to 16Gbyte. It is interesting to note that it does seem that the Raspberry Pi compares quite well with the web server function of the other platforms, although it is (not unexpectedly) quite a bit slower to start delivery of a large video file. My current Internet connection is fast (Fibre-to-the-cabinet @ 40Mbps nominal, 35Mbps most of the time).

Generally the Raspberry Pi as a web server has been hugely successful, and when I complete my setting up of my third Raspberry Pi as a Domain Controller and File server, I shall switch off my Windows 2003 R2 Server, saving a lot on my annual electricity bill. But I do wish to mention a couple of
issues.

1. Segmentation Faults in Child Processes: I have noted from the error logs that there are on average, about 24 Segmentation Faults recorded each day (approximately one per hour). These are all Fault Code 11, but I am not familiar with their significance. They do not seem to damage the server as it remains functional.
Can anyone issue any advice?

2. Weekly Apache2 Shutdown: As mentioned in section 3 above, the server shuts itself down each Sunday at about 6:00am (GMT). The error log report states, in two separate lines-
Msg Line 1 Graceful Restart requested, doing restart.
Msg Line 2 Seg fault or similar nasty error detected in the parent process.
Apache2 fails to restart. However restarting it manually can be accomplished without a problem, although the following error is reported-
Msg Line 1 Warning PID file [address] overwritten -- unclean shutdown of previous Apache run?
Msg Line 2 Apache 2.2.22 (Debian) configured - resuming normal operations.
My suspicion is that this is something to do with the Log File Rotation?
Can anyone issue any advice?

I hope that those considering the embarkation of a similar project to my own will find the report helpful, and in turn would appreciate advice on the issues raised from anyone with a greater (or lesser!) expertise than myself.

Happy New Year to all,

Kenneth Spencer

[benthepoet]

Number 2 is most definitely the log rotation going off, that's when the cron job is usually set for that (Sunday morning).

The segmentation faults might not be Apache but actually caused by PHP (if you're using this). In that case you'd want to see if there are any corresponding errors in your PHP logs.

[kaspencer]

Thanks Ben ...

I am quite sure that you are right - the Sunday 6:25am halt does seem to be log rotation related. However, there are no errors in the Log Rotator logs - just a final statement that Apache2 will be restarted. The restart attempt takes place but fails apparently silently until Apache reports the SEG fault.

Anyway, for the time being, I have set up a Cron job to restart Apache2 after a five minute wait and I'll see how that gets on.

On another point - progress is made on the SMB/LDAP Domain Controller setup - thanks for your help - I'll be in touch on that one shortly!

Happy New Year

Ken

[technion]

Regarding security..

Any webserver is generally subject to garbage in this respect, and it doesn't always mean there's a "security" issue. I remember two years after the Microsoft IIS "Code Red" worm, my Apache box was being scanned for said worm several times a second.
People on forums everywhere came up with "security" policies to blacklists associated IP addresses, but the reality is:

* If you are vulnerable to said issue, you won't have time to act and blacklist. Your server has already been taken over. You're only searching for vulnerabilities you know about, and if you know about it, just patch it already.
* Usually you are not vulnerable to said issue (see above, searching for Windows vulnerabilities on a Linux machine), at which point, said attacker has dissapeared to scan other servers.

On a wider scale however, Linux in general has had several security improvement projects, including:
SELinux
Exec Shield (kernel based)
fortify_source (compiler based)
iptables (it's installed, but not setup and no guide anywhere recommends doing so)

I'd expect some, such as exec shield, wouldn't actually work on an ARM processor due to their requirement for certain registers. However, where all the above seems missing in Raspbian, the general attitude of "just run xyz as root" seems prevalent on this forum, which does bring up some concerns.

[PRKid]

Hi Ken,

Thank you so much for pulling this together and the pdf write-up, it's appreciated! I am relatively new to RPi, yet, I amg etting really excited about all it's potential. I really appreciate all the efforts that you and others have made to this endeavor. I wanted to report back on my setup. I am using RPi w/Wheezy and followed your pdf setup and all as gone well so far. I went all the way through backing up my SD card per you recommendations then proceeded to the FTP section. I was able to create my new web folder (to place my website files) successfully using your proftpd instructions. At that point I stopped and rebooted my system to then continue my work today. This is when I ran into a problem. When I boot up I see 3 errors, as follows: (1) [FAIL] startpar: service(s) returned failure: hostname.sh ... failed!, then, (2) Starting ftp server: proftpd_none_proftpd[2233] mod_tls_memcache/0.1: port not enabled _none_proftpd[2233]: Fatal: ServerName: missing arguments on line 15 of '/etc/....failed!, and, (3) [FAIL] startpar: service(s) returned failure: proftpd...failed!

In addition, it could not start Xtightvnc as well.

So now, when I log into either Pi or root, it comes up with: root@(none):~#

Any thoughts/feedback from you and/or others would be appreciated!

Thank you!
PRKid

[kaspencer]

Hello PR ...

Sorry to hear of your errors on startup after attempting configuration of your Rasperry Pi. I think that some of the errors are closely related.

1. The hostname.sh error
Check that you have put a valid hostname in a valid form into /etc/hostname. I have heard of a bug in Debian which can cause mis-allocation of a hostname, possibly related to DHCP problems. Are you using a dynamic IP address? If so you could try setting a static address - you will no doubt find advice on how to do that on this forum if you need it.

2. and 3. Proftd Not Starting
This is related to (1.) above. The clue is in the reference to the servername.

Finally, the root@(none) is also occurring owing to the absence of a configured machine name. I understand that Debian may create a hostname of "none" when there is a failure to allocate a proper hostname. I cannot be sure, and others may know more about this kind of issue than I do, but I'd say look carefully at all the settings where your machine name is involved.

Good luck with it!

Update on the Webserver Behaviour
For those that are interested: I have now been running one of my websites on the Raspberry Pi since the end of November 2012. It really has been exceedingly successful, with minimum downtime. The server has had 150,000 hits in 5 months, and has served 15Gbyte of data in such forms as MP3s, Videos, and Virtual Organ Sample Set files in the same period.

All the best

Ken

[PRKid]

Ken,

Thanks for the prompt response! I also figured that they must all be interrelated. I created the hostname file per your instructions and what I have is the following:

raspbianwheezy_xxxxxxxxxxx

It's just one line, the x's are a particular name. I have been able to access the websites and all of that just fine per the outlined instructions. I do have a static IP (via wireless).

I will check in the forum for other clues into the hostname issue. Again, thank you!

p.s. I am pretty excited about this webserver. This is the best outlined process I've seen for such.

[PRKid]

Ken,

Good news to report! I changed my hostname and all is working fine! I basically shorten the name and did not use the underscore (_). So, wanted to post jic someone else runs into this issue. Thus, either my hostname was too long, or, it does not allow the use of the underscore.

Thanks for triggering me to think a little longer about my hostname.

PRKid

[PRKid]

Ken,

During my setup of the virtual host and using your file "oursites.conf" the system gives me an error indicating that line 28 (CustomLog) requires two or three arguments. In your pdf, it only has one argument! The first argument, the file name which you have. Then it requires a custom log format string or format name, and, an optional "env=" clause. Is this a missing in your write-up? I did see that a couple of folks had similar issues in your other post, yet, no one mentioned a fix. I'll keep checking to see what I need and how I can fix such. Thanks!

==> Is it missing the nickname "common"? (per Apache2: http://httpd.apache.org/docs/2.2/logs.html)

[kaspencer]

Thanks Pr ...

There are indeed restrictions on the characters that can be included in a hostname, and I am not surprised that the underline character might have been part of the problem, as well as the length. All that is made more obscure when the error trapping and reporting aren't always perfect!

Regarding the "oursites.conf" file issue which you mention, I'll have a look at that early next week as I am about to go our for the day!

All the best

Ken

[PRKid]

Thanks Ken!

I added the "common" argument and it appears to be working fine. Is there a way to validate such?

On another note, a warning that comes up (just in case others ask) is that about the ports. Apache2 default on ports.conf is port 80 (also on the virtualhost). So, when we use it on the oursites it will give you a warning. Those 2 lines can only be on one of the files.

In addition, in my instance, pcmanfm process was using port 80, thus, I changed the Apache2 to another port above 8000.

[kaspencer]

Hello again, PR ...

Regarding the "oursites.conf" file and the missing "common" keyword ...

I have checked my working version of the equivalent of the "oursites.conf" file listed in my Web Server paper, and, as I expected, I don't have the parameter keyword "common" in the line mentioned. So I have checked in the file "apache2.conf", in which the log file formats are defined, and I find the following:

#
# The following directives define some format nicknames for use with a CustomLog directive
# (see below).
#
LogFormat "%v:%p %h %l %u %t \"%r\" %>s %O \"%{Referer}i\" \"%{User-Agent}i\"" vhost_combined
LogFormat "%h %l %u %t \"%r\" %>s %O \"%{Referer}i\" \"%{User-Agent}i\"" combined
LogFormat "%h %l %u %t \"%r\" %>s %O" common
LogFormat "%{Referer}i -> %U" referer
LogFormat "%{User-agent}i" agent

You might like to look at the equivalent lines in your own "apacahe2.conf" file, in particular to check which, if any, keywords are supplied for the second parameter of the Log Format definitions.

That may help anyone else with the same issues.

All the best

Ken

[PRKid]

Thank you very much Ken! I'll check out my apache2.conf file. This is provided some great learnings for me and hope to pass such on to others. Again, thanks!

[PRKid]

Ken,

My apache2.conf file has identical custom log directives as the ones you posted. Yet, what's interesting is that if I exclude the argument "common" from the oursites.conf file I get an error from Apache2. When I include the argument "common" it runs just fine. Now the key for me will be to select the appropriate argument. Since I am still a newbie, going through the learning curve I'll keep reading and digging around. Again, thanks!

[philhudson91]

Any issues with SD card burnouts?

[PRKid]

Hi Ken,

Just a quick update...my web server is working really well thanks to all your previous hard work on this, a great process write-up, as well as, the support via the forum. I had everything in my setup the same with the exception of adding the argument "common" (as I mentioned before) to the oursites.conf file (on line "CustomLog").

I setup the forwarding of my DNS and my web statistics are working as well. The only issue I have is that of the last step...protecting the web stats. I tried several variations of what you recommended yet for whatever reason webalizer never asks for my id/pw. Any additional thoughts would be appreciated. Again, thank you!

PRKid