Monitor linux resources the easy way

[capslock118]

Hi folks,

I had a problem. I have what I thought to be an overloaded pi. When I was doing work remotely I would sometimes lose the connection because too much was running all at the same time. Cutting out a handful of services, like apache web server and mysql, really helped solve my problem.

However I was craving some data on how my system is actually doing during any given time.

So I wrote some python to monitor my CPU, temp, memory, disk I/O, and network traffic. I know there are already a ton of different projects out there that already does this, but I wasn't really happy with the potential implementation and hey, this is a raspberry pi, I should just be doing this myself for the educational factor

I'm using /proc/ kernel files, taking the data from there and then shipping the data out to a google spreadsheet through a google form. It's simple for sure and that's what so pleasing to me. Google provides really nice timeline graphs and effectively free data storage and hosting, so how could I resist that?

The code is nothing glamorous but the end-result is something quite nice for my needs.

If you're looking for something like this, my github project is here:
https://github.com/JoelSlowik/LoadMonitor

further explanation and is here (and results of my efforts with pretty graphs are also here):
http://joelslowik.blogspot.com/2013/01/ ... y-way.html

If you have any suggestions I'd like to hear them.

Cheers.

[leona-pi]

Having a look around the forum for a similar kind of thing, I think this is what I'm looking for.

I do not wish to run a webserver, php and mysql on my PI and I don't want connect to it remotely (or rather, let others!).

So I think what I would like to do is collect these stats and then send them to my external web server, which runs php, mysql and all that.

So rather than send the results to Google, I would like to create a JSON (or xml) file to send to my server, then use some php (or CGI) charting software to render the graphs.

So I think your project will work for me, I will now need to learn Python to produce the JSON output then find a way to get it 'securely' to my webserver, since I read FTP is not secure, I my only other option is webDav (no SSH or TFTP on our servers ).

[mtron]

Hello! I gave your code a try and ran into a problem with the memory module. I use a Model B with 512 MB Ram.

free reports:

Code: Select all

             total       used       free     shared    buffers     cached
Mem:        497716     201764     295952          0      16396     104960
Swap:       524284          0     524284

cat /proc/meminfo

Code: Select all

MemTotal:         497716 kB
MemFree:          293984 kB
Buffers:           16492 kB
Cached:           106440 kB
SwapCached:            0 kB
Active:            94364 kB
Inactive:          92840 kB
Active(anon):      65780 kB
Inactive(anon):      328 kB
Active(file):      28584 kB
Inactive(file):    92512 kB
Unevictable:        1668 kB
Mlocked:            1668 kB
SwapTotal:        524284 kB
SwapFree:         524284 kB
Dirty:                28 kB
Writeback:             0 kB
AnonPages:         65952 kB
Mapped:            19228 kB
Shmem:               492 kB
Slab:               7960 kB
SReclaimable:       3440 kB
SUnreclaim:         4520 kB
KernelStack:        1504 kB
PageTables:         1976 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:      773140 kB
Committed_AS:     685848 kB
VmallocTotal:     139264 kB
VmallocUsed:         996 kB
VmallocChunk:     136308 kB

when launching "python /opt/LoadMonitor/src/Main.py" i get:

Code: Select all

Traceback (most recent call last):
  File "/opt/LoadMonitor/src/Main.py", line 14, in <module>
    MEMTOTAL, MEMFREE = MemoryLoad.getMemory()
  File "/opt/LoadMonitor/src/MemoryLoad.py", line 12, in getMemory
    return int(MemTotalColumns[1]), int(MemFreeColumns[3])
ValueError: invalid literal for int() with base 10: 'kB\n'

Do you know how i might fix this?

[capslock118]

Are you on raspbmc? and what kernel? I was expecting that these files should be the same across os's as long as the same kernel is being used.

[mtron]

no i run the default raspbian wheezy with
Linux raspberrypi 3.2.27+ #285 PREEMPT Tue Nov 20 17:49:40 GMT 2012 armv6l GNU/Linux

[mtron]

and another bugreport: i replaced sda1 with sdb1 in Main.py

Code: Select all

python /opt/LoadMonitor/src/Main.py
Traceback (most recent call last):
  File "/opt/LoadMonitor/src/Main.py", line 16, in <module>
    DISKLOAD = DiskLoad.getDiskLoad("sdb1")
  File "/opt/LoadMonitor/src/DiskLoad.py", line 42, in getDiskLoad
    read, write = deltaTime(diskAssignment, INTERVAL)
  File "/opt/LoadMonitor/src/DiskLoad.py", line 32, in deltaTime
    time1 = getDiskIO(diskAssignment)
  File "/opt/LoadMonitor/src/DiskLoad.py", line 23, in getDiskIO
    if ioData:
UnboundLocalError: local variable 'ioData' referenced before assignment

/proc/diskstats reports sdb1 as:

Code: Select all

   8       0 sda 1058 30 8614 2890 318901 79205 69844040 25632960 0 1391060 25631680
   8       1 sda1 926 8 7431 2670 318901 79205 69844040 25632960 0 1390830 25631310
   8      16 sdb 22453 507 2557595 157840 65 181 2032 7760 0 100530 165210
   8      17 sdb1 22338 504 2556700 157620 65 181 2032 7760 0 100310 164990

is there anywhere else i need to make changes to monitor sdb1 ?

[capslock118]

@mtron - I was having some trouble getting consistent data locations when using .split(" "). It turns out .split was smarter about white-space when I just issue .split(). I think this should be enough to fix your problems.

I've also dropped using a google form and implemented the gspread library as a middle-man for using googles api. Ultimately the spreadsheet is cleaned up too (charts are in their own sheet and columns are organized better) so you will need to re-copy it; I don't see making any more changes to the spreadsheet now.

Thanks for looking at this.