Python recursion error

[dkat]

The continuous loop code finally works great in my updated RPi-based plant watering system but errors out when the maximum recursion depth is exceeded.

The logic requires taking different paths through the code, depending on tests along the way. Not having GOTOs in Python, I used functions to perform the tests and direct the program flow to other functions depending on the test results.

As written, the functions never return. They just point to other functions, which eventually result in an action (do nothing, turn a pump on, or turn a pump off). Before too many loops, the maximum recursion depth is exceeded.

If I could just clear the recursion cache at the end of each loop (if that's a thing), I'd have the issue resolved. Otherwise, I'm looking at a re-write using functions that all return as intended. I suppose I could do that but the little I've read about iterative vs recursive programming so far seems to me like an awfully complex work-around.

As-is, the loop simply starts at the first function. It calls one of two functions, depending. Those call other functions, etc.

Here's a link to a pdf of the logic flow diagram. The overall area is the main loop. The inner shaded area loops six times (once for each water pump) for each main loop.
https://www.dropbox.com/s/5vqyqnlap4dv8 ... w.pdf?dl=0

Here's a link to a pdf of the python code.
https://www.dropbox.com/s/ly0i46u1e6h3n ... y.pdf?dl=0

Any ideas on resolving the recursion depth error would be greatly appreciated.
Especially if they're easy.

Thanks.

[Paeryn]

The continuous loop code finally works great in my updated RPi-based plant watering system but errors out when the maximum recursion depth is exceeded.

The logic requires taking different paths through the code, depending on tests along the way. Not having GOTOs in Python, I used functions to perform the tests and direct the program flow to other functions depending on the test results.

As written, the functions never return. They just point to other functions, which eventually result in an action (do nothing, turn a pump on, or turn a pump off). Before too many loops, the maximum recursion depth is exceeded.
...
Any ideas on resolving the recursion depth error would be greatly appreciated.
Especially if they're easy.

Thanks.

Every time you call a function the location of where to return to is pushed onto a stack, when a function ends it takes that location off the stack and execution continues from there. If you never return from functions and have them endlessly calling each other then Python eventually objects because the stack becomes full. There's also overheads of variables and such local to each invocation of the functions.

Recursive is ok so long as you do eventually return within some predetermined depth. Some languages can detect and optimise tail recursion but Python doesn't.

The way around it is to write your code so that functions correctly return rather endlessly calling each other.

[Heater]

dkat,

Not having GOTOs in Python, I used functions to perform the tests and direct the program flow to other functions depending on the test results.

Calling functions to transfer program control from on place to another is not a replacement for GOTO. As pointed out above. So don't do that.

As it turns out any algorithm can be coded without the use of GOTO. Just use conditionals ("if"), loops ("for", "while", etc) and multiway conditionals.

What you need to know about is "state machines". I'm afraid that if that means nothing to you then you will have to google it.

Also I'm afraid that creating state machines in Python is ridiculously more confusing and complex than any other language I know.

See for example:
https://www.zeolearn.com/magazine/writi ... -in-python

https://python-3-patterns-idioms-test.r ... chine.html

https://dev.to/karn/building-a-simple-s ... -in-python

[jahboater]

If I could just clear the recursion cache at the end of each loop (if that's a thing), I'd have the issue resolved.

Is there such a thing as longjmp() in Python? I doubt it.
Assuming by "recursion cache" you mean the "stack".

Otherwise, I'm looking at a re-write using functions that all return as intended.

As noted above by the last two posts - that's the answer,

[Paeryn]

Is there such a thing as longjmp() in Python?

Not as far as I'm aware, the nearest would involve using exceptions to unwind the call stack but that would be a mess to manage especially if there's mutual recursion involved.

[Heater]

Using exceptions for normal program flow control is a no no.

I'm pretty sure what our OP needs is a state machine. As he says "The logic requires taking different paths through the code, depending on tests along the way". This is a very normal way to organize things.

Is it just me or is creating state machines in Python just ridiculously complex? I did a quick search around for examples/tutorials on how to make Python state machines and they all seem to be completely nuts. For example:

https://pysm.readthedocs.io/en/latest/e ... te-machine

In C and other languages state machines can be made with simple a simple "switch" statement in and endless loop. But Python has no such thing.

I guess one could do it long hand with a long string of "if", "elif", "else". Like so:

Code: Select all

state = 0

while True:
    if state == 0:
        print("Entering state 0")
        if something:
            doSomething()
            state = 1
    elif state == 1:
        print("Entering state 1")
        if somethingElse:
            doSomethingElse()
            state = 2
    elif state == 2:
        print("Entering state 2")
        if somethingOther:
            doSomethingOther()
            state = 3
    elif state == 3:
        print("Entering state 0")
        if somethingMore:
            doSomethingMore()
            state = 0
    else:
        print("Perhaps an error...")

In each "if" above we test some thing (read inputs, messages, whatever) and call a function to do whatever is required.

When it's done we change the "state" variable to move us on to the next junk of logic. These states need not go in order, 1, 2, 3 but can be set to steer the code in whatever direction is required. Basically a complicated, long winded way to get GOTO back again. Although here we are not jumping from line number to line number we use the "state" variable to keep track of where we are.

I guess when you code gets a bit larger and more complex then using a state machine library like pysm begins to make sense.

Any other ideas?

[ghp]

Perhaps you want to have a look to "The State Machine Compiler" on http://smc.sourceforge.net/.
There are a lot of examples and howto on this site.
Even if you do not want to add generated code to your app, the examples are a good entry point to understand state machines.

[ghp]

Had a look to your code. What I understand is that you have six temp/moisture sensors, six pumps switched by a relais board.
The waterlevel sensor is not used ?

Writing the code object oriented would be a good approach.
Something like:

Code: Select all

class Waterer:
    START = 0
    PUMP = 1
    SEEP = 2
    
    def __init__(self, pump_io):
        self.pump_io = pump_io
        self.pump_state = False
        self._pump_off()
        self.state = START
        self.pump_time = 20
        self.seep_time = 20
        
    def _pump_off (self):
        GPIO.output(self.pump_io, True)
        
    def _pump_on (self):
        GPIO.output(self.pump_io, False)
        
    def set_pump_time(self, value):
        """ just an example how to modify the presets in the class"""
        self.pump_time = value
        
    def sensor(self, temp_value, moisture_value):
        self.temp_value = # remember different values 
        ....

    def tick(self):
        if self.state == START:
            if self.moisture_value < self_moisture_threshold
                self._pump_on()
                self.state = self.PUMP
                self.tick_count = self.pump_time
            else:
                self.state = self.SEEP
                self.tick_count = seep_time
                
        elif self.state == self.PUMP:
            self.tick_count -= 1
            if self.tick_count < 0:
                self._pump_off()
                self.state = self.SEEP
                self.tick_count = self.seep_time
                
        elif self.state == self.SEEP:
            self.tick_count -= 1
            if self.tick_count < 0:
                self.state = self.START
                            

waterers = [
    Waterer( 2 ),
    Waterer( 3 ),
    Waterer( 4 ),
    Waterer( 17),
    Waterer( 27),
    Waterer( 22),
    Waterer( 23),
    Waterer( 24),
]

while True:
    sleep(1)
    for z in range(0,InstalledSensors):
        
        sensor = chirp_modbus.SoilMoistureSensor(address=z+i, serialport='/dev/ttyUSB0')
        temp = C2F(sensor.getTemperature())
        moist = sensor.getMoisture()
        waterers[z].sensor( temp, moist) # send sensor values before tick
        waterers[z].tick()
        

This code is not fully developed. The ideas are: pumptime and seeptime are integers which are just counted down till zero. The tick() event is sent to the classes in a sec time frame.
Sensor values are (in the sample) once each second. Could be slower.

The state machine is using switch statements and no entry-exit actions. Therefor it is needed to set some values on the transition, possibly repeated times.

The application logic what I understood is: pump 20 secs, then allow to seep for 20 secs. Then when too dry restart the cycle.
The boilerplate code you have with handling the loop counter is almost removed from the class logic and only once in the outside.

[Heater]

ghp,

Thanks for the link to "The State Machine Compiler".

But good grief, what a throw back to the 1990's.

What an overly complicated way to get a simple thing done.

[dkat]

Thank you all for your kind and informed replies. You've put me on the right track!

[Marck1]

Rewriting the algorithm alliteratively, if possible, is generally a better idea. It's to avoid a stack overflow. Return the current value of the recursion limit, the maximum depth of the Python interpreter stack. This limit prevents infinite recursion from causing an overflow of the C stack and crashing Python.