Bruce Crowthorne
Posts: 100
Joined: Wed Jan 11, 2012 7:19 pm
Location: Sussex, UK

Am I asking too much of a Pi Zero W?

Fri Oct 20, 2017 5:41 pm

My PiZeroW keeps crashing. It will run for periods of an hour and sometimes to up to 12 hours, the duration seems to be random.

It is running Jessie and has a PiCamera and a USB stick attached to it.
It runs two programmes..
First is a bash shell which logs data every minute to a couple of files in the main file system.
The other is much more demanding. It is a Python3 programme that runs the camera, taking a picture every 5 seconds and writes these to the USB drive. Every fourth picture is then split up and sent via MQTT to another machine via Wifi.

From the logging programme I see that it runs with load averages of well over 1, typically : 2.35, 0.86, 0.31.
It seems to have plenty of memory available.

So the question is - I am asking too much of a PiZero?
I am thinking of using my Pi3 that I use as a development system to run this - I am hoping that the 4 cores will help spread the load and maybe stop crashing!

What are your thoughts?
Thanks
Bruce

piras77
Posts: 147
Joined: Mon Jun 13, 2016 11:39 am

Re: Am I asking too much of a Pi Zero W?

Fri Oct 20, 2017 7:48 pm

I don't see why your application might crash your Pi. There are many things that can crash a Pi. If you're lucky you'll see a relevant diagnostic message in /var/log/messages, /var/log/syslog or /var/log/kern.log. However, events like a failed power supply, an electromagnetic event, a failing wire, SD card failure, etc. won't show up. You may check for those / try to exchange components to single out the problem.

User avatar
davidcoton
Posts: 5505
Joined: Mon Sep 01, 2014 2:37 pm
Location: Cambridge, UK
Contact: Website

Re: Am I asking too much of a Pi Zero W?

Fri Oct 20, 2017 8:48 pm

Does it work with just one program at a time?
If not, fix the program that fails. A Pi3B won't help.
If it does, then you might be in weird interaction territory. A Pi3B may or may not help.
Keep an eye on memory use via "top" in a console window. First guess would be a memory leak in one of the programs.
Location: 345th cell on the right of the 210th row of L2 cache

User avatar
Gavinmc42
Posts: 4855
Joined: Wed Aug 28, 2013 3:31 am

Re: Am I asking too much of a Pi Zero W?

Sat Oct 21, 2017 2:23 am

Is that python programming running all the time?
I tried that years ago, it died eventually, I am guessing memory leaks or OS conflicts.

What I would do instead is use Linux tools cron and watch.
But even 5 secs is a bit close, 10 secs may work better.

Cron will run stuff on minute intervals.
Watch can do it on second intervals.
This has worked for years in my apps.
Have not yet got around to replacing this with an Ultibo version.

MQTT, not sure if that will work/port to Ultibo, but then I did not expect camera or VC4 to work this year.
You can do multicore in Python, but one reason I use Ultibo is to understand multithread/multicore stuff at a lower level.
Since i2c seems to be rock solid on Ultibo (cannot say that of Linux) some of your stuff should work just as well if not better.
But re Zero W, the W bit is not working yet in Ultibo.

You might want to try PiCore instead of Raspbian, it is much smaller, well supported and about the most reliable Linux OS I have used on Pi's.
Raspbian has a tendency to shoot it's own SDcard, well it did years ago, not sure if it is better now.
PiCore only writes to SD when you tell it too ;)

Using the Pi3 for Zero development is the way I am heading, waiting for nice Aarch64 Linux OS without bloat.
USB boot on Zero is great, no need for sdcard while developing.
Works well for Ultibo, nearly for PiCore, no idea how to for Raspbian.

I think just changing OS and make a smaller python app that is called by cron/watch will fix your issues.
Learn shell script too, nearly everything can be done with script, just it runs a bit (a lot ) slower.
Not a problem for stuff that only runs every few minutes.
I'm dancing on Rainbows.
Raspberries are not Apples or Oranges

piras77
Posts: 147
Joined: Mon Jun 13, 2016 11:39 am

Re: Am I asking too much of a Pi Zero W?

Sat Oct 21, 2017 7:17 am

Why should a memory leak in a user application crash a Pi?

Bruce Crowthorne
Posts: 100
Joined: Wed Jan 11, 2012 7:19 pm
Location: Sussex, UK

Re: Am I asking too much of a Pi Zero W?

Sat Oct 21, 2017 8:38 am

Thanks for the replies, let me try and address the suggestions in turn.

1) There is nothing showing up in any of the logs.
The only error message shows up on the console, this is the first lines of the report (the italics are mine) .....

Message from syslogd@RPi0W at Oct 21 09:00:17 ...
kernel:[52852.488232] Internal error: Oops: 17 [#1] ARM

Message from syslogd@RPi0W at Oct 21 09:00:17 ...
kernel:[52852.534926] Process swapper (pid: 0, stack limit = 0xc08a4188)

Message from syslogd@RPi0W at Oct 21 09:00:17 ...
kernel:[52852.537565] Stack: (0xc08a5ed8 to 0xc08a6000)



2) I have reduced what is running to only run the offending programme
The listing is included at the end of this post just in case anyone spots anything.

Normally I run a bash shell that logs various bits of data on cpu load, memory space and what programmes are running. Unfortunately the only thing that shows up that is out of the "normal" is the CPU load.

3) I don't think it is a memory leak. Nothing shows up in my systems management data (my bash shell script) and the frequency of the crashes is random - it can be 10 minutes or 15 hours - so I would have expected something of a more consistently long period.

4) I don't actually use a scheduler like cron or watch. I use a "non blocking timer" (actually two in this case) to time the picture capture and the writing to MQTT. See the "Main" part of the listing.

Have a look at the code and see if you can spot anything .....

Code: Select all

#!/usr/bin/env python
# -*- coding: utf-8 -*-
#
#  File camcappubXX.py, merges camcapture11.py and PyMQtest_pubpic34.py. Was picamdoc3_1_to fileand_loopV1_X.py
#  V1_0 - Just a merge of the two files.
#  Note 5 second delay between pictures, transfers to MQ every 20 sec
#    12000 pictures each 5sec apart gives 5x12000 seconds of pictures =60000sec, 1000min, 16.6 hours
#  V11 Initial try at real version 
#  V12 Add Keyboard breakout, comment out print statements
#

from time import sleep
import picamera                     # PiCamera driver
import datetime as dt               # Used to watermark each picture
import shutil                       # Used to copy files (shell utilities)
import paho.mqtt.client as mqtt     # MQ client
import time                         # Used for date & time plus delays

camera = picamera.PiCamera()
camera.resolution = (1280, 720)
camera.start_preview()

broker_address="192.168.1.203"                                             # Set up MQTT server on 203 
client = mqtt.Client("P1")                                                 # Start MQTT Client
client.connect("192.168.1.203", 1883, 60)                                  # Connect to server

i = 1                                                                      # Initialise picture counter
takepic = 5                                                                # How often to take a picture
transferMQ = 20                                                            # How often to send to MQ
timepic = time.time()                                                      # picture initial timer
timeMQ = time.time()                                                       # MQTT initial timer

client.loop_start()                                                        # initial start before loop

#  Functions -------------------------------------------------------------------------------------------
def transfer_to_MQTT():                                                     # Function to write to MQ
#    print ("Transfer to MQTT")
    file = open("/home/pi/usbdrive/snapshot/imagenow.jpg", "rb")            # open the file, note r=read, b=binary
    imagestring = file.read()                                               # read the file
    byteArray = bytes(imagestring)                                          # convert to byte string
    client.publish(topic="highmount/camera", payload= byteArray ,qos=0)     # publish it to the MQ queue

def take_picture():
#    print ("Take Pic")
    global i
    camera.annotate_text = dt.datetime.now().strftime("%d-%m-%Y-%H-%M")     # Print time on each picture
    camera.capture('/home/pi/usbdrive/images/image%d.jpg'%(i,))             # The %i gives the incementing file name
    shutil.copyfile ('/home/pi/usbdrive/images/image%d.jpg'%(i,), '/home/pi/usbdrive/snapshot/imagenow.jpg')    
    i = i+1                                                                 # Incement picture counter
    if i > 12000:                                                           # Reset counter after 12000 pics
        i = 1

#Main loop  ------------------------------------------------------------------------------------------
try:
    while True:                                                            # Loop forever
        t1 = time.time()
        if t1 - timepic >= takepic:
            take_picture()                                             # *****   Call function to take a picture
            timepic = time.time()                                          # re read start time to take a picture

        if t1 - timeMQ >= transferMQ:
            transfer_to_MQTT()                                         # *****   Call function to do MQTT transfer
            timeMQ = time.time()                                           # re read start time to transfer to MQTT
            
except KeyboardInterrupt:
    print (" Control C breakout ....")

Thanks in advance
Bruce

User avatar
Gavinmc42
Posts: 4855
Joined: Wed Aug 28, 2013 3:31 am

Re: Am I asking too much of a Pi Zero W?

Sat Oct 21, 2017 12:25 pm

Why should a memory leak in a user application crash a Pi?
No idea, but I could watch memory get gobbled up over time when the Python code was running.
This was years ago, 2012/13, was not then or now a Linux guru to figure it out, so I tried alternative method -shell/cron/watch that just works.
I was also doing i2c sensor reads that only worked sometimes? Turns out there was a bug in i2c-tools.
Not really interested in going back to figure it out, too many revision changes in everything since then.

The bit of code that looks familiar is

Code: Select all

while True:   #Loops forever
Make two apps, no loops, run once and exit , use watch and cron to run these two apps.
One of them might have leaks or whatever it is called in Linux land.
It is Linux so many, many ways to do things.
All I know is cron and watch just have to run better than any code I will ever write :lol:
I'm dancing on Rainbows.
Raspberries are not Apples or Oranges

Bruce Crowthorne
Posts: 100
Joined: Wed Jan 11, 2012 7:19 pm
Location: Sussex, UK

Re: Am I asking too much of a Pi Zero W?

Mon Oct 30, 2017 3:56 pm

Not sure if this is an answer or just a work around!
I moved the same code from my PiZeroW to a Pi3 and it has been running fine for 3 days so far.

So I am happy that it is working, but it would be nice to know why the PiZero acted up.
Regards
Bruce

User avatar
Gavinmc42
Posts: 4855
Joined: Wed Aug 28, 2013 3:31 am

Re: Am I asking too much of a Pi Zero W?

Tue Oct 31, 2017 2:50 am

Interesting, I first had issues with Python running 24/7 on an old model A- 256MB ram.
Pi3 has 1GB, running 3 days, hmm wonder how long it will run :D
I suspect it is memory problems but I would not bet on it, Linux + Python is complex.
Do you have an old Pi with 256MB, check it it locks up sooner.
I'm dancing on Rainbows.
Raspberries are not Apples or Oranges

mikerr
Posts: 2826
Joined: Thu Jan 12, 2012 12:46 pm
Location: UK
Contact: Website

Re: Am I asking too much of a Pi Zero W?

Tue Oct 31, 2017 8:22 am

What happens if the camera fails to take a picture one time ? ( I don't see error handling code)
Android app - Raspi Card Imager - download and image SD cards - No PC required !

User avatar
jahboater
Posts: 6300
Joined: Wed Feb 04, 2015 6:38 pm
Location: Wonderful West Dorset

Re: Am I asking too much of a Pi Zero W?

Tue Oct 31, 2017 11:32 am

Gavinmc42 wrote:
Tue Oct 31, 2017 2:50 am
Do you have an old Pi with 256MB, check it it locks up sooner.
Another way perhaps is to set the GPU mem to a high value.
https://github.com/raspberrypi/document ... /memory.md

Bruce Crowthorne
Posts: 100
Joined: Wed Jan 11, 2012 7:19 pm
Location: Sussex, UK

Re: Am I asking too much of a Pi Zero W?

Tue Oct 31, 2017 6:55 pm

Hi Mike, the PiZero just falls over. Nothing that I can find in any of the logs. The error code is in my first post, an amusing Oops 17.
I have been logging memory and storage usage every minute or so, but I didn't detect anything unusual. I was expecting memory or storage leaks but didn't find anything.
TBH I am happy that the Pi3 is running OK. Touch wood! So far it has been up for 4days and counting ......

Return to “Troubleshooting”