morguslethe
Posts: 11
Joined: Thu Jun 11, 2015 8:58 pm

Re: Very strange behavior (time.strftime() kills main thread

Mon Jun 15, 2015 6:46 pm

It might help to flesh out how I THOUGHT this could be done:

Set events for 20 pins. Trigger alarm on rising edge for every pin. (ideally, all 20 alarms can be triggered at the same time)
When in "alarm", print something, then wait for another rising edge using wait_for_edge.
when wait for edge comes, go into a while loop and check every second if the signal is stable on 0. when it is, exit and go from the beginning.

this did not work because wait_for_edge does not work in a sub-thread. so I tried eliminating "waiting while in callback" by using a global flag. for every pin I have a log which state has been triggered. If "alarm" was already triggered for one pin, the callback goes into the "stage two". In stage two I still wait in a while loop to see if signal is stable... I don't know how else to do it. wait_for_edge doesn't even work, removing the event doesn't work, if I do it outside the callback it's just going to trigger again half a second later...and when I fix that, everything works, right up until the end when a GHOST rising edge somehow wrongly triggers a new event, even though I quadruple checked nothing of the sort happens. It's like some rising edges gets queued while the thread is busy. So, a callback thread kind of blocks a new thread spawning, but it kind of doesn't... And three alarms going into stage two at the same time still don't work.

Ugly hack code, works well with one or two things happening simultaneously...:

Code: Select all

import RPi.GPIO as GPIO
GPIO.setmode(GPIO.BCM)
import time
import sys

GPIO.setup(17, GPIO.IN, pull_up_down=GPIO.PUD_DOWN)
GPIO.setup(27, GPIO.IN, pull_up_down=GPIO.PUD_DOWN)
GPIO.setup(22, GPIO.IN, pull_up_down=GPIO.PUD_DOWN)


stages = {17: 0, 27: 0, 22: 0}

#threaded callback function
def alarm(channel):
	
	try:
		if(stages[channel] == 0):

			print("Alarm  on pin "+str(channel)+" at "+time.strftime("%H:%M:%S %d/%m/%Y"))
			
			stages[channel] = 1
			
		elif stages[channel] == 1: 
			stages[channel]=2
			print("Stage two on pin "+str(channel)+" at "+time.strftime(" ob %H:%M:%S %d/%m/%Y"))
			
			
			while 1:
				v1 = GPIO.input(channel)
				time.sleep(0.333)
				v2 = GPIO.input(channel)
				time.sleep(0.333)
				v3 = GPIO.input(channel)

				if(v1 == 0 and v2 == 0 and v3 == 0):

					print("Stable signal on pin "+str(channel)+" at "+time.strftime("%H:%M:%S %d/%m/%Y"))
					
					stages[channel] = 0
					
					break		
		else:
			return
			
	except:
		print("Alarm thread exception")
		e = sys.exc_info()[0]
		print(str(e))
		 
GPIO.add_event_detect(17, GPIO.RISING, callback=alarm, bouncetime=2000)
GPIO.add_event_detect(27, GPIO.RISING, callback=alarm, bouncetime=2000)
GPIO.add_event_detect(22, GPIO.RISING, callback=alarm, bouncetime=2000)


try:
	print("This is the main thread. It will print a dot every second, forever.")
   
	while 1:
		time.sleep(1)
		print(".")
	print("If this prints something is strange")
   
except KeyboardInterrupt:
   GPIO.cleanup()
GPIO.cleanup()           # clean up GPIO on normal exit

Edit:
That looks impressive... Alright, I'll try. will it work like this:

-set 20 pins to in and pulled down
-set 20 event detections on rising edge, with callback function alarm (or is it alarm1, alarm2, alarm3 ... for each pin its own callback ??)
-when an event for a specific pin gets triggered, update value in "stages" array and print
-when a new rising edge gets triggered for that pin, check array, and go into stage two
-in stage two, print, then check for stable signal with a while loop
-when signal is stable, change "stages" array value to 0.
---this can happen at different pins at the same time

EDIT2:

Both libraries work well now, IF I comment the while loop. Joan, you said I can use many threads, but it looks like this while is blocking others from doing their job... What do you think is happening?

I truly think there's only this hurdle left, no matter which library I use.

Code: Select all

import pigpio
import time
import sys

pi = pigpio.pi()

pi.set_mode(17, pigpio.INPUT)
pi.set_mode(27, pigpio.INPUT)
pi.set_mode(22, pigpio.INPUT)
pi.set_pull_up_down(17, pigpio.PUD_DOWN)
pi.set_pull_up_down(27, pigpio.PUD_DOWN)
pi.set_pull_up_down(22, pigpio.PUD_DOWN)

stages = {17: 0, 27: 0, 22: 0}

def alarm(gpio, level, tick):
	try:
		if(stages[gpio] == 0):
			print("Alarm  on pin "+str(gpio)+" at "+time.strftime("%H:%M:%S %d/%m/%Y"))
			stages[gpio] = 1
		else:
			print("Stage two on pin "+str(gpio)+" at "+time.strftime("%H:%M:%S %d/%m/%Y"))
			
			while 1:
				v1 = pi.read(gpio)
				time.sleep(0.333)
				v2 = pi.read(gpio)
				time.sleep(0.333)
				v3 = pi.read(gpio)

				if(v1 == 0 and v2 == 0 and v3 == 0):

					print("Stable signal on pin "+str(gpio)+" at "+time.strftime("%H:%M:%S %d/%m/%Y"))
			
					stages[gpio] = 0
					
					break
	except:
		print("Alarm thread exception")
		e = sys.exc_info()[0]
		print(str(e))
		
pi.callback(17, pigpio.RISING_EDGE, alarm)
pi.callback(27, pigpio.RISING_EDGE, alarm)
pi.callback(22, pigpio.RISING_EDGE, alarm)


print("This is the main thread. It will print a dot every second, forever.")
   
while 1:
	time.sleep(1)
	print(".")

User avatar
joan
Posts: 14755
Joined: Thu Jul 05, 2012 5:09 pm
Location: UK

Re: Very strange behavior (time.strftime() kills main thread

Mon Jun 15, 2015 7:32 pm

This is the original code updated for multiple gpios.

Code: Select all

#!/usr/bin/env python

# morguslethe_2.py
# 2015-06-15
# Public Domain

import time

import pigpio

GPIOS=[2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21]

cbf_state=[0]*32
cb=[None]*32

def cbf(gpio, level, tick):
   global cbf_state
   if cbf_state[gpio] == 0:
      if level == 1: # rising edge
         ts = time.strftime("%H:%M:%S %d/%m/%Y")
         print("Rising edge on "+str(gpio) + " at " + ts)
         cbf_state[gpio] = 1 # wait for falling edge
   elif cbf_state[gpio] == 1:
      if level == 0: # falling edge
         ts = time.strftime("%H:%M:%S %d/%m/%Y")
         print("Falling edge on "+str(gpio) + " at " + ts)
         cbf_state[gpio] = 2 # wait 3 seconds, set watchdog timeout
         pi.set_watchdog(gpio, 3000) # 3000ms
   else: # cbf_state == 2
      if level == 2: # watchdog expired
         ts = time.strftime("%H:%M:%S %d/%m/%Y")
         pi.set_watchdog(gpio, 0) # cancel watchdog
         print("Exit from stage two at "+ ts)
         cbf_state[gpio] = 0 # back to original state

pi = pigpio.pi() # connect to local Pi

for gpio in GPIOS:
   pi.set_mode(gpio, pigpio.INPUT)
   pi.set_pull_up_down(gpio, pigpio.PUD_DOWN)
   cb[gpio] = pi.callback(gpio, pigpio.EITHER_EDGE, cbf)

try:
   while True:
      print("dot")
      time.sleep(1)
except:
   pass

print("cancelling")

for gpio in GPIOS:
   pi.set_watchdog(gpio, 0) # make sure watchdog is cancelled
   cb[gpio].cancel() # cancel callback
pi.stop() # disconnect from local Pi

ghp
Posts: 1466
Joined: Wed Jun 12, 2013 12:41 pm
Location: Stuttgart Germany
Contact: Website

Re: Very strange behavior (time.strftime() kills main thread

Mon Jun 15, 2015 8:21 pm

Good news, it is a slow problem. And obviously each of the pins is handled the same way independent of the other pins.
I usually try to handle problems like these with state machines. For your problem, the inputs are [0] or [1] and possibly you need some time events. For the pure academic implementation, see [gamma et al, design patterns], state machine.
Sounds complicated, but is incredible powerful and fast.

A simple example is here. Assume you poll your GPIO each 0.05 sec, and provide the inputs to a state machine, one instance for each pin. The sample just waits for 5 '1'-inputs in sequence (in your example, you will have higher value), and prints a line when found.
pattern.gif
pattern.gif (12.83 KiB) Viewed 627 times
Each state has a 'entry'-method, 'exit' method and of course a handle which processes signals.
First state is START. When a '1' is found, there is an itermediate 'WFL' (wait for low).
In state 'I1', there is a counter which simulates a timeout. When 5 times a '1' is found, then timeout strikes.
state.jpg
state.jpg (19.31 KiB) Viewed 627 times
(Wanted to be the first one posting a state chart here).
The implementation is

Code: Select all

#
# state machine sample
debug = True

class STATE:
    def entry(self):
        pass
    def exit(self):
        pass
    def handle(self, sig):
        return self
    def name(self):
        return self.__class__.__name__
    
class START ( STATE):
    def entry(self):
        """will never be called"""
        pass
    def exit(self):
        pass
    def handle(self, sig):
        if sig == 0:
            return I0()
        if sig == 1:
            return WFL()
        return self

class WFL ( STATE):
    
    def entry(self):
        pass
    def exit(self):
        pass
    def handle(self, sig):
        if sig == 0:
            return I0()
        return self
    

class I0( STATE):
    def handle(self, sig):
        if sig == 0:
            self
        if sig == 1:
            return I1()
        return self

class I1( STATE):
    cnt = None
    
    def entry(self):
        self.cnt = 0
    
    def handle(self, sig):
        #
        # simulate timeout
        #
        self.cnt += 1
        if self.cnt == 5:
            return I1T()
        if sig == 0:
            return I0()
        if sig == 1:
            return self
    

class I1T( STATE):
    def handle(self, sig):
        if sig == 0:
            return I0()
        if sig == 1:
            return self
    def entry(self):
        print("found an event !!!!")
    

class StateMachine:
    state = None
    def __init__(self):
        self.state = START()
        
    def handle(self, signal):
        newState = self.state.handle(signal)
        if debug:
            print( '{f:s} --[{s:s}]--> {t:s}'.format(f=self.state.name(),s=str(signal),t=newState.name() ) )
        if newState != self.state:
            self.state.exit()
            self.state = newState
            self.state.entry()
            
        
stateMachine = StateMachine()
#
# simulate inputs
#
events = [ 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0]

for evt in events:
    stateMachine.handle( evt)         
If you follow this approach, you will draw your own state diagram, and design the state classes. Test it with some input pattern and then wrap it in a polling-gpio-reader.

Hope this helps
Gerhard

morguslethe
Posts: 11
Joined: Thu Jun 11, 2015 8:58 pm

Re: Very strange behavior (time.strftime() kills main thread

Mon Jun 15, 2015 10:37 pm

ghp wrote: Good news, it is a slow problem. And obviously each of the pins is handled the same way independent of the other pins.
I usually try to handle problems like these with state machines. For your problem, the inputs are [0] or [1] and possibly you need some time events. For the pure academic implementation, see [gamma et al, design patterns], state machine.
Sounds complicated, but is incredible powerful and fast.
...
Thanks for your input, I read through your post wholly. I think I always had a pretty good idea of what I wanted to do, it was a simple problem, resulting in a simple state machine I guess :P The bad part was the implementation. I also specifically wanted to avoid polling, since it's inferior to interrupts... Good suggestion though :)
joan wrote: This is the original code updated for multiple gpios.
...
Reading your code made me wonder what a watchdog was again, I checked the docs, and BOOM, it ended up being the tool I needed, and lacked before.

Doing anything slow or new with events while the program was inside a threaded callback proved to be a complete failure. Spawning a different thread with different limitations is exactly what I needed. I needed a different thread (because that's the only alternative to cumbersome polling) that would somehow check if a pin has a stable signal, call a function, and pass it some parameter. A watchdog is exactly that: it's a thread that runs somewhere else, you can have many of them (one on each pin), detecting their trigger is as simple as comparing a value, and from what I saw, they are quite robust, since they keep running even after you kill the program! Since I have no experience of spawning new threads by hand, much less telling them what to do, the watchdog REALLY is my best friend. There could not have been a more perfect function for my problem right now. I'm very relieved :D

What I learned over the last few grueling days:
-callback threads should be really fast, only setting a few flags and watchdogs, or spawning other threads. The designated callback thread has to be freed up fast so other interrupts get their chance to do work.
-my assumption that a callback spawns a new thread for every new event (at least for a different pin) was dead wrong. if it did, I would have finished this three days ago
-from my experience you can't really do ANYTHING with threads once you're inside a callback thread. there's always some sneaky error, or it just stops your program! BIG PAIN to solve
-good documentation and help are priceless

From what I've tested until now I think I should be in the clear. I want to thank every one of you for contributing, if you hadn't I probably would have given up. Doubly thanks to you joan for including such a great watchdog and creating the library! The program works smoothly and it looks clean too!

See you soon

Code: Select all

import pigpio
import time
import sys

pi = pigpio.pi()

sobe = {17: 1, 27: 2, 22: 3}
stages = {17: 0, 27: 0, 22: 0}

for pin in sobe:
	pi.set_mode(pin, pigpio.INPUT)
	pi.set_pull_up_down(pin, pigpio.PUD_DOWN)

def alarm(gpio, level, tick):

	try:
		if level == 2:
			print("Watchdog expired! Signal stable. Exit on pin"+str(gpio))
			pi.set_watchdog(gpio,0)
			stages[gpio] = 0
			return
			
		if(stages[gpio] == 0):
			print("Alarm  on pin "+str(gpio)+" at "+time.strftime("%H:%M:%S %d/%m/%Y"))
			stages[gpio] = 1
		elif(stages[gpio] == 1):
			print("Stage two on pin "+str(gpio)+" at "+time.strftime("%H:%M:%S %d/%m/%Y"))
			
			print("Setting watchdog")
			pi.set_watchdog(gpio, 1500)
			stages[gpio] = 2
		else:
			return
			
	except:
		print("Alarm thread exception")
		e = sys.exc_info()[0]
		print(str(e))

for pin in sobe:
	pi.callback(pin, pigpio.RISING_EDGE, alarm)

print("Start")

while 1:
	time.sleep(1)
	print(".")

User avatar
joan
Posts: 14755
Joined: Thu Jul 05, 2012 5:09 pm
Location: UK

Re: Very strange behavior (time.strftime() kills main thread

Tue Jun 16, 2015 7:57 am

morguslethe wrote: ...
A watchdog is exactly that: it's a thread that runs somewhere else, you can have many of them (one on each pin), detecting their trigger is as simple as comparing a value, and from what I saw, they are quite robust, since they keep running even after you kill the program!
...
They will do, they are running on the daemon. In my example code I made sure each one was stopped by a watchdog 0 call in the exception code.
...
What I learned over the last few grueling days:
-callback threads should be really fast, only setting a few flags and watchdogs, or spawning other threads. The designated callback thread has to be freed up fast so other interrupts get their chance to do work.
...
That's a good lesson to learn. At best a callback should just set a flag and let the main thread handle the "heavy lifting".

Return to “Python”