danjperron
Posts: 3691
Joined: Thu Dec 27, 2012 4:05 am
Location: Québec, Canada

Two cores problem (solved)

Sat Jan 30, 2021 12:57 am

I just test the _thread module using Pi calculation.

Seems to work fines until I tried to increase the loop. For some unknown reason the core 1 stop.

This is the result if the loop is 1000.
>>> %Run -c $EDITOR_CONTENT
PI calculation
One core cpu :
3.141593
time 0.061 seconds to calculate Pi using a loop of 1000.

Two cores cpu:
3.141593
time 0.041 seconds to calculate Pi using a loop of 1000.
>>>
But if you try a loop of 3000 it hangs. but only on core1.

This is the source code. It is just a split method to calculate PI on a loop.

Code: Select all

import utime
import _thread
MaxCount=1000


threadDone = False
threadResult = 0.0

def showTimeLapse(start,MaxCount):
    lapse= (utime.ticks_ms() - start) / 1000.0
    print(" time {:.3f} seconds".format(lapse),end="")
    print(" to calculate Pi",end="")
    print(" using a loop of {}.".format(MaxCount))



def PiCalcThread(Start, End, theLock):
    global threadResult
    global threadDone
    r = PiCalc(Start,End)
    #lock to transfer data
    with theLock:
        threadDone = True
        threadResult = r

# using  the Nilakantha series.
def PiCalc(Start,End):
    r = 0.0
    d1 = Start * 2.0
    d2 = Start * 2.0 + 1.0
    d3 = Start * 2.0 + 2.0
    for i in range(Start,End):
        d1 = d1 + 2.0
        d2 = d2 + 2.0
        d3 = d3 + 2.0 
        if  (i & 1) == 0:
           r = r + ( 4.0 / ( d1 * d2 * d3))
        else:
           r = r - ( 4.0 / ( d1 * d2 * d3))
    return r

print("PI calculation")
print("One core cpu : ")
start = utime.ticks_ms()

result = 3.0 + PiCalc(0,MaxCount)
print(result)
showTimeLapse(start,MaxCount)
print("\nTwo cores cpu:")
# creation d'une a lock to sync thread
threadLock = _thread.allocate_lock()

# start timer
start = utime.ticks_ms()

#start thread
_thread.start_new_thread(PiCalcThread,(0,MaxCount // 2,threadLock))

#calculation premiere partie en utilisant core 0 (main core)
result = 3.0 + PiCalc(MaxCount // 2, MaxCount)

# wait for the thread to end

while True:
    with threadLock:
        if threadDone:
            break

result = result + threadResult
print(result)
showTimeLapse(start,MaxCount)
Then what I'm doing wrong ?
Last edited by danjperron on Tue Mar 30, 2021 4:29 pm, edited 1 time in total.

arj
Posts: 23
Joined: Sun Jun 10, 2012 2:18 pm

Re: Two cores problem

Mon Feb 08, 2021 2:30 pm

I'm also getting all sorts of strange issues when trying to use the second core with MicroPython (I'm using MicroPython v1.14 on 2021-02-05; Raspberry Pi Pico with RP2040).

Have you worked out how to stop something running once started with _thread.start_new_thread?

I've also found that a def "Function" must have at least 2 parameters as you don't seem to be able to pass a single parameter with start_new_thread.

DWiskow
Posts: 56
Joined: Sun Apr 09, 2017 1:56 pm

Re: Two cores problem

Mon Feb 08, 2021 3:52 pm

arj wrote:
Mon Feb 08, 2021 2:30 pm

Have you worked out how to stop something running once started with _thread.start_new_thread? . . .

You need the second thread to complete (rather than simply running in an endless while loop).

See example here https://www.raspberrypi.org/forums/vie ... 6&t=302889 where I set a flag in the primary thread to tell the secondary thread to terminate.

You will also find that the normal Ctrl-C only terminates the MicroPython code running on core 0 . . . so, you also need to trap keyboard interrupts and instruct the code running in core 1 to terminate . . . otherwise will have to unplug/reset the Pico every time !

DWiskow
Posts: 56
Joined: Sun Apr 09, 2017 1:56 pm

Re: Two cores problem

Mon Feb 08, 2021 4:33 pm

danjperron wrote:
Sat Jan 30, 2021 12:57 am
Then what I'm doing wrong ? . . .

When I run your code with 'maxcount = 3000', I sometimes get a hang and occaisonaly I get a 'TypeError' as shown below.
Traceback (most recent call last):
File "<stdin>", line 60, in <module>
File "<stdin>", line 37, in PiCalc
TypeError: unsupported types for __mul__: '', ''

I had at theory . . . the RD2040 has no hardware floating point capability [but optimised floating point multiply and divide functions have been created and built into the RS2040 ROM alongside the UF2 bootloader] . . . when these floating point routines are simultaneously used by both threads a timing conflict/error appears to occasionally happen [and the longer the parallel execution time extends the more likely the problem to happen].

If you try 'maxcount' values from 1700 upwards, you will find a variety of different errors happen . . . all of which appear to show some evidence of memory corruption.

but, If you comment out the code, to only run core 0 OR core 1 . . . I know, that removes the whole point of the example . . . then you will find the code runs successfully with all values of maxcount

To prove/disprove this theory, you could rewrite the 'PiCalc' function to only use integer math to see if that resolves the issue.
Last edited by DWiskow on Mon Feb 08, 2021 6:22 pm, edited 3 times in total.

DWiskow
Posts: 56
Joined: Sun Apr 09, 2017 1:56 pm

Re: Two cores problem

Mon Feb 08, 2021 5:50 pm

DWiskow wrote:
Mon Feb 08, 2021 4:33 pm
I have at theory . . .

Well that was a rubbish theory . . . integer math generates the same outcome

Code: Select all

import utime
import _thread
MaxCount=1000

threadDone = False
threadResult = 0

def showTimeLapse(start,MaxCount):
    lapse= (utime.ticks_ms() - start) / 1000
    print(" time {:.3f} seconds".format(lapse),end="")
    print(" to calculate Pi",end="")
    print(" using a loop of {}.".format(MaxCount))

def PiCalcThread(Start, End, theLock):
    global threadResult
    global threadDone
    r = PiCalc(Start,End)
    #lock to transfer data
    with theLock:
        threadDone = True
        threadResult = r

def intDiv (a, b):
    c = a % b

# using the Nilakantha series.
def PiCalc(Start,End):
    r = 0
    d1 = Start * 2
    d2 = Start * 2 + 1
    d3 = Start * 2 + 2
    for i in range(Start,End):
        d1 = d1 + 2
        d2 = d2 + 2
        d3 = d3 + 2 
        if  (i & 1) == 0:
           r = r + ( (400000000) // ( d1 * d2 * d3))
        else:
           r = r - ( (400000000) // ( d1 * d2 * d3))
    return r

print("PI calculation")
print("One core cpu : ")
start = utime.ticks_ms()

# single core execution
result = 300000000 + PiCalc(0,MaxCount)
print("{:1.7f}".format(result/100000000))

showTimeLapse(start,MaxCount)
print("\nTwo cores cpu:")

# duel core execution
threadLock = _thread.allocate_lock()                                # create a lock to sync thread
start = utime.ticks_ms()                                            # start timer
_thread.start_new_thread(PiCalcThread,(0,MaxCount // 2,threadLock)) # start thread to calculate 2nd part on core 1
result = PiCalc(MaxCount // 2, MaxCount)                            # calculate 1st part on core 0 (main thread)
while True:                                                         # wait for the thread to end
    with threadLock:
        if threadDone:
            break
result = 300000000+(result + threadResult)
print("{:1.7f}".format(result/100000000))
showTimeLapse(start,MaxCount)

But, no surprise, it runs faster than floating point !
PI calculation
One core cpu :
3.1415927
time 0.048 seconds to calculate Pi using a loop of 1000.

Two cores cpu:
3.1415927
time 0.036 seconds to calculate Pi using a loop of 1000.
Last edited by DWiskow on Mon Feb 08, 2021 6:20 pm, edited 4 times in total.

arj
Posts: 23
Joined: Sun Jun 10, 2012 2:18 pm

Re: Two cores problem

Mon Feb 08, 2021 5:53 pm

DWiskow wrote:
Mon Feb 08, 2021 3:52 pm
arj wrote:
Mon Feb 08, 2021 2:30 pm

Have you worked out how to stop something running once started with _thread.start_new_thread? . . .

You need the second thread to complete (rather than simply running in an endless while loop).

See example here https://www.raspberrypi.org/forums/vie ... 6&t=302889 where I set a flag in the primary thread to tell the secondary thread to terminate.

You will also find that the normal Ctrl-C only terminates the MicroPython code running on core 0 . . . so, you also need to trap keyboard interrupts and instruct the code running in core 1 to terminate . . . otherwise will have to unplug/reset the Pico every time !
Yes I saw your code where you set the flag to break out of the loop. My code was actually based on yours. I'm, like you, finding that the loop in the background function seems to crash which in turn prevents it exiting cleanly when the terminateThread flag is set. I notice your comments about the potential floating point issue. It seems something strange is happening. What version of MicroPython are you using? I have MicroPython v1.14 on 2021-02-05; Raspberry Pi Pico with RP2040

DWiskow
Posts: 56
Joined: Sun Apr 09, 2017 1:56 pm

Re: Two cores problem

Mon Feb 08, 2021 6:03 pm

arj wrote:
Mon Feb 08, 2021 5:53 pm
I'm, like you, finding that the loop in the background function seems to crash . . .

I am not having any problems with the 'background ' code running on core 1 crashing.

If you run your main code within a try/except construct and ensure you set the 'terminateThread' flag on both a normal clean exit AND ALSO on a Ctrl-C KeyboardInterrupt, as shown below, you should have no issues

Code: Select all

# main code  ...
try:
    while True:
	do something
	...
	do something
    terminateThread = True
    
except KeyboardInterrupt:
    terminateThread = True
    exit()

MicroPython v1.13-290-g556ae7914 on 2021-01-21; Raspberry Pi Pico with RP2040

DWiskow
Posts: 56
Joined: Sun Apr 09, 2017 1:56 pm

Re: Two cores problem

Mon Feb 08, 2021 6:11 pm

something to bear in mind [from the MicroPython documentation https://docs.micropython.org/en/latest/ ... hread.html] . . .
_thread – multithreading support

This module implements a subset of the corresponding CPython module, as described below. For more information, refer to the original CPython documentation: _thread.

This module is highly experimental and its API is not yet fully settled and not yet described in this documentation.
Last edited by DWiskow on Wed Feb 10, 2021 2:49 pm, edited 1 time in total.

hippy
Posts: 9540
Joined: Fri Sep 09, 2011 10:34 pm
Location: UK

Re: Two cores problem

Mon Feb 08, 2021 6:28 pm

DWiskow wrote:
Mon Feb 08, 2021 4:33 pm
. . . all of which appear to show some evidence of memory corruption.
I am also seeing what looks like evidence of memory corruption, even when running on a single core. 'n = ord(sys.stdin.read(1))' with an error that 4 characters were passed to ord() when .read(1) should only ever return a single character - viewtopic.php?f=146&t=302944

Don't know if related. Things mostly seem to work but it seems bizarre things do happen.

arj
Posts: 23
Joined: Sun Jun 10, 2012 2:18 pm

Re: Two cores problem

Mon Feb 08, 2021 6:32 pm

DWiskow wrote:
Mon Feb 08, 2021 6:11 pm
something to bear in mind [from the MicroPython documentation https://docs.micropython.org/en/latest/ ... hread.html] . . .
_thread – multithreading support

This module implements a subset of the corresponding CPython module, as described below. For more information, refer to the original CPython documentation: _thread.

This module is highly experimental and its API is not yet fully settled and not yet described in this documentation.
Yes, I saw that and asked a question on the MicroPython forums https://forum.micropython.org/viewforum.php?f=21. I'm new to MicroPython (although old at Z80/6502/68000/C/C++/C#/JavaScript...and who knows what others). It feels like something odd is happening. My code is trying to update a WS2812 LED strip so it's using the PIO/State machine to send data to the LED's. I don't know if that's an issue. I might try the code without sending anything to the LED's

arj
Posts: 23
Joined: Sun Jun 10, 2012 2:18 pm

Re: Two cores problem

Mon Feb 08, 2021 6:38 pm

hippy wrote:
Mon Feb 08, 2021 6:28 pm
DWiskow wrote:
Mon Feb 08, 2021 4:33 pm
. . . all of which appear to show some evidence of memory corruption.
I am also seeing what looks like evidence of memory corruption, even when running on a single core. 'n = ord(sys.stdin.read(1))' with an error that 4 characters were passed to ord() when .read(1) should only ever return a single character - viewtopic.php?f=146&t=302944

Don't know if related. Things mostly seem to work but it seems bizarre things do happen.
We all seem to have come across the same odd behaviour. I wonder if it's worth ensuring we are all using the same version of MicroPython to narrow down the issue? I'm using MicroPython v1.14 on 2021-02-05; Raspberry Pi Pico with RP2040. I'm not sure if it is the latest or most stable.

DWiskow
Posts: 56
Joined: Sun Apr 09, 2017 1:56 pm

Re: Two cores problem

Mon Feb 08, 2021 6:53 pm

There is also a problem if printing from both cores . . .

Code: Select all


import time
import _thread
import math
from sys import stdout

def cpu2task():
    with lock:
        # time.sleep(1)
        print("THE_QUICK_BROWN_FOX_JUMPS_OVER_THE_LAZY_DOGS_BACK")
        # now do some stuff
        time.sleep(1)
        
print("Interspersed print test")
#wait to print first message
time.sleep(0.1)

#get the lock
lock = _thread.allocate_lock()
loops = 0
_thread.start_new_thread(cpu2task, ())

# this print arrives in pieces
print(math.pi)

while lock.locked() == True:
    loops += 1

print("Waited for",loops, "loops")

, , , the output gets jumbled !

But, I would classify that as a ‘user problem’, rather than a MicroPython issue. :lol:

hippy
Posts: 9540
Joined: Fri Sep 09, 2011 10:34 pm
Location: UK

Re: Two cores problem

Mon Feb 08, 2021 7:00 pm

I should have read the documentation before my first attempt to use threading, but this amused me ...

Code: Select all

>>> import twothreads
dnThraancdelbeadc ke x(cmeopstti orne cienn tt hcraelald  lsatsatr)t:e
  b yF i<lfeu n"ct<isotnd iTnh>r"e,a dlTiwnoe  a1t,  0ixn2 <0m0o0d8u27l0e>>

eTy FpielEer r"otrwo:t hfruneacdtsi.opny "t, alkiense  0 15p,o siinti o<nmaold uarlgeu>m
tTtypse Ebrurto r1 : wfeurnec tgiiovnen 
akes 1 positional arguments but 0 were given

arj
Posts: 23
Joined: Sun Jun 10, 2012 2:18 pm

Re: Two cores problem

Tue Feb 09, 2021 9:17 am

Here is my code that demonstrates the core 1 crashing problem. It's connected to a WS2812 LED strip. Enter R,G,B for a static colour or S to animate the colours running from core 1. With line 109 (pixels_set(i, rgb)) commented out it works without any problem. When line 109 is not commented out it crashes core 1 after 28 times around the loop. I'm assuming the problem is because the code running in core 1 is accessing the state machine used for the LED control?

Code: Select all

import array
import time
import rp2
import random
import select
import sys
import utime
import _thread
from machine import Pin

from sys import stdin, exit
from _thread import start_new_thread
from utime import sleep



################################################################################
# Configure the number of WS2812 LEDs.
NUM_LEDS   = 60
PIN_NUM    = 6
brightness = 0.2   # <<< Any value between 0.0 and 1.0
version    = "PicoLED V1.00 enter R,G,B or S to run sparkle on core 1"


################################################################################
# global variables to share between both threads/processors
# 
terminateThread = False



##########################################################################
@rp2.asm_pio(sideset_init=rp2.PIO.OUT_LOW, out_shiftdir=rp2.PIO.SHIFT_LEFT, autopull=True, pull_thresh=24)
def ws2812():
    T1 = 2
    T2 = 5
    T3 = 3
    wrap_target()
    label("bitloop")
    out(x, 1)               .side(0)    [T3 - 1]
    jmp(not_x, "do_zero")   .side(1)    [T1 - 1]
    jmp("bitloop")          .side(1)    [T2 - 1]
    label("do_zero")
    nop()                   .side(0)    [T2 - 1]
    wrap()
 

# Create the StateMachine with the ws2812 program, outputting on pin
sm = rp2.StateMachine(0, ws2812, freq=8_000_000, sideset_base=Pin(PIN_NUM))
 
# Start the StateMachine, it will wait for data on its FIFO.
sm.active(1)

# Display a pattern on the LEDs via an array of LED RGB values.
ar = array.array("I", [0 for _ in range(NUM_LEDS)])



##########################################################################
def pixels_show():
    dimmer_ar = array.array("I", [0 for _ in range(NUM_LEDS)])
    for i,c in enumerate(ar):
        r = int(((c >> 8) & 0xFF) * brightness)
        g = int(((c >> 16) & 0xFF) * brightness)
        b = int((c & 0xFF) * brightness)
        dimmer_ar[i] = (g<<16) + (r<<8) + b
    sm.put(dimmer_ar, 8)
    time.sleep_ms(10)

def pixels_set(i, color):
    ar[i] = (color[1]<<16) + (color[0]<<8) + color[2]

def pixels_fill(color):
    for i in range(len(ar)):
        pixels_set(i, color)
    pixels_show()
    

##########################################################################
BLACK  = (0, 0, 0)
RED    = (255, 0, 0)
ORANGE = (255, 150, 0)
YELLOW = (255, 240, 0)
GREEN  = (0, 255, 0)
CYAN   = (0, 255, 255)
BLUE   = (0, 0, 255)
PURPLE = (180, 0, 255)
WHITE  = (255, 255, 255)
COL1   = (255, 124, 23)

COLORS = (BLACK, RED, ORANGE, YELLOW, GREEN, CYAN, BLUE, PURPLE, WHITE)



##########################################################################
def ModeSparkle(wait_ms):
    global terminateThread
    count=1
    
    while True:
        for i in range(NUM_LEDS):
            if terminateThread == True:
                print("Ending ModeSparkle")
                return
            r = random.randint(0,255)
            g = random.randint(0,255)
            b = random.randint(0,255)
            rgb = [r,g,b]
            pixels_set(i, rgb)
        pixels_show()
        print ("Sparkle ", count)
        count=count+1
        time.sleep_ms(wait_ms)
                   

try:
    print(version)
    while True:
        buffLine = input(":").lower()

        # Static colours    
        if (buffLine == "?"):
            print(version)

        if (buffLine.startswith("r")):
            terminateThread = True 
            pixels_fill(RED)

        if (buffLine.startswith("g")):
            terminateThread = True 
            pixels_fill(GREEN)

        if (buffLine.startswith("b")):
            terminateThread = True 
            pixels_fill(BLUE)
            
        if (buffLine.startswith("k")):
            terminateThread = True 
            pixels_fill(BLACK)
            
        if (buffLine.startswith("w")):
            terminateThread = True 
            pixels_fill(WHITE)

        if (buffLine.startswith("s")):
            terminateThread = True
            time.sleep_ms(100)
            terminateThread = False
            
            try:
                lastthread = _thread.start_new_thread(ModeSparkle, (500,))
            except:
                print("Error starting thread")


except KeyboardInterrupt:
    print("Ctrl+C")
    terminateThread = True   #terminate thread


terminateThread = True
print("Program END")

DWiskow
Posts: 56
Joined: Sun Apr 09, 2017 1:56 pm

Re: Two cores problem

Tue Feb 09, 2021 11:26 pm

arj wrote:
Tue Feb 09, 2021 9:17 am
With line 109 (pixels_set(i, rgb)) commented out it works without any problem ...
You are simultaneously using 'pixel_set' in both core 0 and core 1 . . . which means you are potentially trying to set/update the content of array 'ar' in two places at the same time . . . this in most probably the root cause of your crash.

You need ensure that variables are only set by one core [but you can still read them in the other core] . . . and/or use 'thread locking' to allow one core to have exclusive access whilst it updates any variable used in both cores.

The locking mechanism is shown in the first post (Pi Calculation example by danjperron).

arj
Posts: 23
Joined: Sun Jun 10, 2012 2:18 pm

Re: Two cores problem

Wed Feb 10, 2021 9:50 am

DWiskow wrote:
Tue Feb 09, 2021 11:26 pm
You are simultaneously using 'pixel_set' in both core 0 and core 1
No, pixel_set is NOT being used simultaneously in both cores. The static colours are set from the main code on core 0 whilst the animation is happening on core 1. They are not happening at the same time, it's one or the other.

You can prove that by commenting out the pixel_set lines on core 0 and it still crashes after 28 loops of the animation code.

DWiskow
Posts: 56
Joined: Sun Apr 09, 2017 1:56 pm

Re: Two cores problem

Wed Feb 10, 2021 10:20 am

arj wrote:
Wed Feb 10, 2021 9:50 am
No, pixel_set is NOT being used simultaneously in both cores ...
You call pixels_set from with the ModeSparkle function [which is running in background on core 1] . . . you also call pixel_set from within pixels_fill which is called whenever you input a command [which is running in foreground on core 0] . . . the ar array can therefore be updated from two places in parallel (which may clash at some point ) !

The fact that you set the terminateThread flag immediately before calling pixels_fill [which then calls pixel_set ] doesn’t mean that the background task has terminated [only that you have set a flag for it to terminate when it gets round to it], it may well call pixel_set before it terminates !

Similarly, you are also calling pixels_show from both cores, which is updating data in the PIO state machine FIO from two places in parallel.

Have a look at this https://youtu.be/aIFElaK14V4 which explains about communicating between the two cores.

This may or may not be the cause of the crash, but it is definitely an issue.

arj
Posts: 23
Joined: Sun Jun 10, 2012 2:18 pm

Re: Two cores problem

Wed Feb 10, 2021 2:01 pm

Thanks. I see what you mean. I'll try adding some delays and see if that helps.

DWiskow
Posts: 56
Joined: Sun Apr 09, 2017 1:56 pm

Re: Two cores problem

Wed Feb 10, 2021 2:42 pm

Have a look at my latest serialUSB code (below).

Code: Select all

from sys import stdin, stdout, exit
from utime import sleep
import _thread

bufferSize = 1024                 # size of buffer
buffer = [' '] * bufferSize       # buffer
bufferEcho = False                # echo bytes 
bufferNextIn, bufferNextOut = 0,0 # next in/out pointers
terminateThread = False           # terminate thread flag

def bufferSTDIN(threadLock):
    global buffer, bufferSize, bufferEcho, bufferNextIn, terminateThread
    while True:
        try:
            stdinByte = stdin.read(1)
        except:
            stdinByte = '?'
        if bufferEcho:
            stdout.write(stdinByte)
        with threadLock:
            buffer[bufferNextIn] = stdinByte        
            bufferNextIn += 1
            if bufferNextIn == bufferSize:
                bufferNextIn = 0
            if terminateThread:
                _thread.exit()

def bufferGetByte(threadLock):
    global buffer, bufferSize, bufferNextOut, bufferNextIn
    with threadLock:
        if bufferNextOut == bufferNextIn:
            return ''
    n = bufferNextOut
    bufferNextOut += 1
    if bufferNextOut == bufferSize:
        bufferNextOut = 0
    return (buffer[n])

def bufferGetLine(threadLock):
    global buffer, bufferSize, bufferNextOut, bufferNextIn
    with threadLock:
        if bufferNextOut == bufferNextIn:
            return ''
        bufferNextInSaved = bufferNextIn
    n = bufferNextOut
    while n != bufferNextInSaved:
        if buffer[n] == '\x0d':
            break
        if buffer[n] == '\x0a':
            break        
        n += 1
        if n == bufferSize:
            n = 0
    if (n == bufferNextInSaved):
            return ''
    line = ''
    n += 1
    if n == bufferSize:
        n = 0
    while bufferNextOut != n:
        if buffer[bufferNextOut] == '\x0d':
            bufferNextOut += 1
            if bufferNextOut == bufferSize:
                bufferNextOut = 0
            continue
        if buffer[bufferNextOut] == '\x0a':
            bufferNextOut += 1
            if bufferNextOut == bufferSize:
                bufferNextOut = 0
            break
        line = line + buffer[bufferNextOut]
        bufferNextOut += 1
        if bufferNextOut == bufferSize:
            bufferNextOut = 0
    return line
#
# main ...
#
try:
    threadLock = _thread.allocate_lock()
    bufferSTDINthreadID = _thread.start_new_thread(bufferSTDIN,(threadLock,))
    while True:
        buffLine = bufferGetLine(threadLock)
        if buffLine:
            stdout.write(buffLine+'\n')
            if buffLine == '\x1b\x1b':  # ESC+ESC
                break
        sleep(0.01)
    terminateThread = True
    
except KeyboardInterrupt:
    terminateThread = True
    exit()


As python is a high level language [and each python instruction results in many machine level instructions] it is quite possible that a variable updated in a secondary core/thread could be only partially updated [i.e. in the process of being updated] at the point it is referenced/used in a primary thread . . . this may result in unpredictable behaviors . . . hence the need for thread locking, even when a protocol is in place that no two threads will update/change a variable (i.e. a protocol where one core/thread always owns the variable and is allowed to update it, the other core/thread only ever reads that variable and never updates it).

You will see the use of 'threadLock = _thread.allocate_lock()', and then, in the bufferGetLine(threadLock) function and bufferSTDIN(threadLock) function, the use of with threadLock: to bound mutually exclusive code and only allow one or other function to actually run code that could rely upon variables that are in the process of being changed by the other function.

This works because with threadLock: checks to see if the referenced lock is currently in a locked state, and then waits until it is unlocked before allowing the bounded code to run.

Combining thread locking with the use of a circular buffer (effectively a FIFO buffer of a definable size) for passing significant amounts of data between the cores/threads, together with a principle that only one core/thread owns/updates a given variable (effectively a tiny FIFO buffer of a fixed size), means that multiple cores/threads should never 'trip' over each other.

arj
Posts: 23
Joined: Sun Jun 10, 2012 2:18 pm

Re: Two cores problem

Wed Feb 10, 2021 6:05 pm

Ok, all of the LED pixel stuff removed and it still crashes after 99 times around the outer loop when running on core 1.

Code: Select all

import array
import time
import rp2
import random
import select
import sys
import utime
import _thread
from machine import Pin

from sys import stdin, exit
from _thread import start_new_thread
from utime import sleep

terminateThread = False

##########################################################################
def ModeSparkle(wait_ms):
    global terminateThread
    count=1
    
    while True:
        for i in range(1,60):
            if terminateThread == True:
                print("Ending ModeSparkle")
                _thread.exit()
                return
            r = random.randint(0,255)
            g = random.randint(0,255)
            b = random.randint(0,255)
            rgb = [r,g,b]
        print ("Sparkle ", count)
        count=count+1
        time.sleep_ms(wait_ms)
                   

try:
    _thread.start_new_thread(ModeSparkle, (20,))    # <<< Line 38 - crashes here
    
    while True:
        #ModeSparkle(200)                           # <<< Line 41 - works fine here
        pass
    
except KeyboardInterrupt:
    print("Ctrl+C")
    terminateThread = True   #terminate thread
    time.sleep_ms(1000)

terminateThread = True
print("Program END")

danjperron
Posts: 3691
Joined: Thu Dec 27, 2012 4:05 am
Location: Québec, Canada

Re: Two cores problem

Wed Feb 10, 2021 7:29 pm

ok I think it is a memory problem! looks like a memory alloc dealloc.

I just add gc.collect and everything works.

Kind of work with my pi.py but still crash.

On this code the RGB= [r, g , b] is the culprid!

Code: Select all

import array
import time
import rp2
import random
import select
import sys
import utime
import _thread
from machine import Pin
import gc

from sys import stdin, exit
from _thread import start_new_thread, allocate_lock
from utime import sleep

terminateThread = False
count=1

##########################################################################
def ModeSparkle(wait_ms,threadLock):
    global terminateThread
    global count
    i = 0
    r = 0
    g = 0
    b = 0
    
    while True:
        for i in range(1,60):
            with threadLock:
                if terminateThread == True:
                    print("Ending ModeSparkle")
                    _thread.exit()
                    return
            r = random.randint(0,255)
            g = random.randint(0,255)
            b = random.randint(0,255)
            rgb = [r,g,b]
        with threadLock:
            gc.collect()
            
            print ("Sparkle ", count)
        count=count+1
        time.sleep_ms(wait_ms)

def ModeSparkle2(wait_ms, threadLock):
    global terminateThread
    global count
    while True:
        with threadLock:
            if terminateThread == True:
                print("Ending ModeSparkle")
                _thread.exit()
                return
        count=count+1
        time.sleep_ms(wait_ms)



try:
    
    # create a lock
    threadLock = allocate_lock()
    i = 1
    _thread.start_new_thread(ModeSparkle, (20, threadLock))    # <<< Line 38 - crashes here. Not anymore
    
    while True:
        #ModeSparkle(200)                           # <<< Line 41 - works fine here
        pass
    
except KeyboardInterrupt:
    print("Ctrl+C")
    with threadLock:
        terminateThread = True   #terminate thread
    time.sleep_ms(1000)

terminateThread = True
print("Program END")

danjperron
Posts: 3691
Joined: Thu Dec 27, 2012 4:05 am
Location: Québec, Canada

Re: Two cores problem

Wed Feb 10, 2021 7:39 pm

ok I removed the gc.collect and modified the code to not create an array inside the loop. Now it is working.

Looks like a memory problem. We need to be sure that we don't create object inside a loop. The garbage collection is not very effective on the second core.

Code: Select all

import array
import time
import rp2
import random
import select
import sys
import utime
import _thread
from machine import Pin
import gc

from sys import stdin, exit
from _thread import start_new_thread, allocate_lock
from utime import sleep

terminateThread = False
count=1

##########################################################################
def ModeSparkle(wait_ms,threadLock):
    global terminateThread
    global count

    rgb = [0, 0, 0]
    while True:
        for i in range(1,60):
            with threadLock:
                if terminateThread == True:
                    print("Ending ModeSparkle")
                    _thread.exit()
                    return
            rgb[0] = random.randint(0,255)
            rgb[1] = random.randint(0,255)
            rgb[2] = random.randint(0,255)
        with threadLock:
            print ("Sparkle ", count)
            count=count+1
        time.sleep_ms(wait_ms)


try:
    
    # create a lock
    threadLock = allocate_lock()
    i = 1
    _thread.start_new_thread(ModeSparkle, (20, threadLock))    # <<< Line 38 - crashes here. Not anymore
    
    while True:
        #ModeSparkle(200)                           # <<< Line 41 - works fine here
        pass
    
except KeyboardInterrupt:
    print("Ctrl+C")
    with threadLock:
        terminateThread = True   #terminate thread
    time.sleep_ms(1000)

terminateThread = True
print("Program END")

arj
Posts: 23
Joined: Sun Jun 10, 2012 2:18 pm

Re: Two cores problem

Wed Feb 10, 2021 8:15 pm

Cheers for that. I'm brand new with MicroPython and appreciate your help to find a solution. Should we report it to the MicroPython development guys so they can make it more stable in future versions?

DWiskow
Posts: 56
Joined: Sun Apr 09, 2017 1:56 pm

Re: Two cores problem

Wed Feb 10, 2021 10:03 pm

arj wrote:
Wed Feb 10, 2021 8:15 pm
Should we report it to the MicroPython development guys? . . .

Yes . . . but we need a simple and repeatable example . . . and the one I created bellow doesn't seem to fail , so I am not sure we have fully isolated/identified the issue yet

Code: Select all

from utime import sleep
import _thread, gc

def core1(threadLock):
    while True:
        with threadLock:
            buffer = [' '] * 3
            buffer [0] = '1'
            buffer [1] = '2'
            buffer [2] = '3'
            sleep(0.5)

def free(full=False):
    gc.collect()
    F = gc.mem_free()
    A = gc.mem_alloc()
    T = F+A
    P = '{0:.2f}%'.format(F/T*100)
    if not full: return P
    else : return ('Total:{0} Free:{1} ({2})'.format(T,F,P))
#
# main ...
#
threadLock = _thread.allocate_lock()
_thread.start_new_thread(core1,(threadLock,))
while True:
    print(free(True))
    sleep(1)
Last edited by DWiskow on Wed Feb 10, 2021 10:19 pm, edited 1 time in total.

DWiskow
Posts: 56
Joined: Sun Apr 09, 2017 1:56 pm

Re: Two cores problem

Wed Feb 10, 2021 10:17 pm

the one I created [in my previous post ] doesn't seem to fail . . .

but LINE 19 of my original code below does seem to create a memory leak !

The code runs perfectly, with no change in the memory usage/allocation, until some data is keyed into the host computer serial terminal and LINE 19 of the background/core 1 code is executed.

Code: Select all

from sys import stdin, stdout, exit
from utime import sleep
import _thread, gc

bufferSize = 1024                 # size of buffer
buffer = [' '] * bufferSize       # buffer
bufferEcho = False                # echo bytes 
bufferNextIn, bufferNextOut = 0,0 # next in/out pointers
terminateThread = False           # terminate thread flag

def bufferSTDIN(threadLock):
    global buffer, bufferSize, bufferEcho, bufferNextIn, terminateThread
    while True:
        try:
            stdinByte = stdin.read(1)
        except:
            stdinByte = '?'
        if bufferEcho:
            stdout.write(stdinByte)
        with threadLock:
            buffer[bufferNextIn] = stdinByte #*** LINE 19, which causes memory leak ***#
            bufferNextIn += 1
            if bufferNextIn == bufferSize:
                bufferNextIn = 0
            if terminateThread:
                _thread.exit()

def bufferGetByte(threadLock):
    global buffer, bufferSize, bufferNextOut, bufferNextIn
    with threadLock:
        if bufferNextOut == bufferNextIn:
            return ''
    n = bufferNextOut
    bufferNextOut += 1
    if bufferNextOut == bufferSize:
        bufferNextOut = 0
    return (buffer[n])

def bufferGetLine(threadLock):
    global buffer, bufferSize, bufferNextOut, bufferNextIn
    with threadLock:
        if bufferNextOut == bufferNextIn:
            return ''
        bufferNextInSaved = bufferNextIn
    n = bufferNextOut
    while n != bufferNextInSaved:
        if buffer[n] == '\x0d':
            break
        if buffer[n] == '\x0a':
            break        
        n += 1
        if n == bufferSize:
            n = 0
    if (n == bufferNextInSaved):
            return ''
    line = ''
    n += 1
    if n == bufferSize:
        n = 0
    while bufferNextOut != n:
        if buffer[bufferNextOut] == '\x0d':
            bufferNextOut += 1
            if bufferNextOut == bufferSize:
                bufferNextOut = 0
            continue
        if buffer[bufferNextOut] == '\x0a':
            bufferNextOut += 1
            if bufferNextOut == bufferSize:
                bufferNextOut = 0
            break
        line = line + buffer[bufferNextOut]
        bufferNextOut += 1
        if bufferNextOut == bufferSize:
            bufferNextOut = 0
    return line

def free(full=False):
    gc.collect()
    F = gc.mem_free()
    A = gc.mem_alloc()
    T = F+A
    P = '{0:.2f}%'.format(F/T*100)
    if not full: return P
    else : return ('Total:{0} Free:{1} ({2})'.format(T,F,P))

#
# main ...
#
try:
    threadLock = _thread.allocate_lock()
    bufferSTDINthreadID = _thread.start_new_thread(bufferSTDIN,(threadLock,))
    buffline = ''
    count = 0
    while True:
        buffLine = bufferGetLine(threadLock)
        if buffLine:
            stdout.write(buffLine+'\n')
            if buffLine == '\x1b\x1b':  # ESC+ESC
                break
        sleep(0.05)
        count = count + 1
        if count == 5 * 20:
            print(free(True))
            count = 0
    terminateThread = True
    
except KeyboardInterrupt:
    terminateThread = True
    exit()

Return to “MicroPython”