DWiskow
Posts: 56
Joined: Sun Apr 09, 2017 1:56 pm

USB serial communication using second processor/thread of the RD2040

Sat Feb 06, 2021 2:15 pm

The code detailed below demonstrates how to use the second thread (and processor) of the RD2040 Dual Core microprocessor,, to run a function ‘bufferSTDIN()’ in background.

The ‘bufferSTDIN()’ function waits for a byte from ‘stdin’ (the serial USB port) and then stores it in a circular buffer. Because ‘bufferSTDIN()’ is running as a second thread, it is effectively running in background or parallel to the main code, which is running in foreground. This means it can wait for input from the serial USB port (stdin.read) without blocking code in the rest of the program from continuing to run.

This approach exploits the dual processors of the RD2040 to achieve very efficient non-blocking serial communication written entirely with standard MicroPython.

Code: Select all

#
# USB serial communication for the Raspberry Pi Pico (RD2040) using the second RD2040
# thread/processor (written by Dorian Wiskow - Janaury 2021) 
#
from sys import stdin, exit
from _thread import start_new_thread
from utime import sleep
# 
# global variables to share between both threads/processors
# 
bufferSize = 1024                 # size of circular buffer to allocate
buffer = [' '] * bufferSize       # circuolar incomming USB serial data buffer (pre fill)
bufferEcho = True                 # USB serial port echo incooming characters (True/False) 
bufferNextIn, bufferNextOut = 0,0 # pointers to next in/out character in circualr buffer
terminateThread = False           # tell 'bufferSTDIN' function to terminate (True/False)
#
# bufferSTDIN() function to execute in parallel on second Pico RD2040 thread/processor
#
def bufferSTDIN():
    global buffer, bufferSize, bufferEcho, bufferNextIn, terminateThread
    
    while True:                                 # endless loop
        if terminateThread:                     # if requested by main thread ...
            break                               #    ... exit loop
        buffer[bufferNextIn] = stdin.read(1)    # wait for/store next byte from USB serial
        if bufferEcho:                          # if echo is True ...
            print(buffer[bufferNextIn], end='') #    ... output byte to USB serial
        bufferNextIn += 1                       # bump pointer
        if bufferNextIn == bufferSize:          # ... and wrap, if necessary
            bufferNextIn = 0
#
# instantiate second 'background' thread on RD2040 dual processor to monitor and buffer
# incomming data from 'stdin' over USB serial port using ‘bufferSTDIN‘ function (above)
#
bufferSTDINthread = start_new_thread(bufferSTDIN, ())

#
# function to check if a byte is available in the buffer and if so, return it
#
def getByteBuffer():
    global buffer, bufferSize, bufferNextOut, bufferNextIn
    
    if bufferNextOut == bufferNextIn:           # if no unclaimed byte in buffer ...
        return ''                               #    ... return a null string
    n = bufferNextOut                           # save current pointer
    bufferNextOut += 1                          # bump pointer
    if bufferNextOut == bufferSize:             #    ... wrap, if necessary
        bufferNextOut = 0
    return (buffer[n])                          # return byte from buffer

#
# function to check if a line is available in the buffer and if so return it
# otherwise return a null string
#
# NOTE 1: a line is one or more bytes with the last byte being LF (\x0a)
#      2: a line containing only a single LF byte will also return a null string
#
def getLineBuffer():
    global buffer, bufferSize, bufferNextOut, bufferNextIn

    if bufferNextOut == bufferNextIn:           # if no unclaimed byte in buffer ...
        return ''                               #    ... RETURN a null string

    n = bufferNextOut                           # search for a LF in unclaimed bytes
    while n != bufferNextIn:
        if buffer[n] == '\x0a':                 # if a LF found ... 
            break                               #    ... exit loop ('n' pointing to LF)
        n += 1                                  # bump pointer
        if n == bufferSize:                     #    ... wrap, if necessary
            n = 0
    if (n == bufferNextIn):                     # if no LF found ...
            return ''                           #    ... RETURN a null string

    line = ''                                   # LF found in unclaimed bytes at pointer 'n'
    n += 1                                      # bump pointer past LF
    if n == bufferSize:                         #    ... wrap, if necessary
        n = 0

    while bufferNextOut != n:                   # BUILD line to RETURN until LF pointer 'n' hit
        
        if buffer[bufferNextOut] == '\x0d':     # if byte is CR
            bufferNextOut += 1                  #    bump pointer
            if bufferNextOut == bufferSize:     #    ... wrap, if necessary
                bufferNextOut = 0
            continue                            #    ignore (strip) any CR (\x0d) bytes
        
        if buffer[bufferNextOut] == '\x0a':     # if current byte is LF ...
            bufferNextOut += 1                  #    bump pointer
            if bufferNextOut == bufferSize:     #    ... wrap, if necessary
                bufferNextOut = 0
            break                               #    and exit loop, ignoring (i.e. strip) LF byte
        line = line + buffer[bufferNextOut]     # add byte to line
        bufferNextOut += 1                      # bump pointer
        if bufferNextOut == bufferSize:         #    wrap, if necessary
            bufferNextOut = 0
    return line                                 # RETURN unclaimed line of input

#
# main program begins here ...
#
# set 'inputOption' to either  one byte ‘BYTE’  OR one line ‘LINE’ at a time. Remember, ‘bufferEcho’
# determines if the background buffering function ‘bufferSTDIN’ should automatically echo each
# byte it receives from the USB serial port or not (useful when operating in line mode when the
# host computer is running a serial terminal program)
#
# start this MicroPython code running (exit Thonny with code still running) and then start a
# serial terminal program (e.g. putty, minicom or screen) on the host computer and connect
# to the Raspberry Pi Pico ...
#
#    ... start typing text and hit return.
#
#    NOTE: use Ctrl-C, Ctrl-C, Ctrl-D then Ctrl-B on in the host computer terminal program 
#           to terminate the MicroPython code running on the Pico 
#
try:
    inputOption = 'LINE'                    # get input from buffer one BYTE or LINE at a time
    while True:

        if inputOption == 'BYTE':           # NON-BLOCKING input one byte at a time
            buffCh = getByteBuffer()        # get a byte if it is available?
            if buffCh:                      # if there is...
                print (buffCh, end='')      # ...print it out to the USB serial port

        elif inputOption == 'LINE':         # NON-BLOCKING input one line at a time (ending LF)
            buffLine = getLineBuffer()      # get a line if it is available?
            if buffLine:                    # if there is...
                print (buffLine)            # ...print it out to the USB serial port

        sleep(0.1)

except KeyboardInterrupt:                   # trap Ctrl-C input
    terminateThread = True                  # signal second 'background' thread to terminate 
    exit()

Two further functions getByteBuffer() and getLineBuffer() are provided, that run as part of the main program in foreground. These functions check if there is a ‘byte’ or complete ‘line’ available in the buffer and return a ‘byte’ or ‘line’, respectively..

Using this technique, and these functions, MicroPython code that engages and interacts with software running on another computer (connected to the Pico over USB) can be very easily developed.

Load this code into main.py on your Raspberry Pi Pico and then run a serial terminal program (e.g. putty, minicom or screen) on the host computer (connected to the appropriate serial USB port) to see it working.

NOTE: you may find it necessary to remove power from the Pico to reset it, as Ctrl-C in the REPL only appears to terminate MicroPython running in foreground (on the first processor of the RD2040). I found that using Ctrl-C, Ctrl-C, Ctrl-D then Ctrl-B in the host computer terminal program will properly terminate the MicroPython code running on both processors on the Pico (as it effectively resets the RD2040)
Last edited by DWiskow on Sun Feb 07, 2021 3:10 am, edited 1 time in total.

arj
Posts: 24
Joined: Sun Jun 10, 2012 2:18 pm

Re: USB serial communication using second processor/thread of the RD2040

Sat Feb 06, 2021 6:33 pm

Thanks, I was just about to start something similar so I can control a LED strip from a PC app (via the USB port). Although I've been a programmer since the good old ZX80 days I'm new to MicroPython and a little confused by the "Ctrl-C, Ctrl-C, Ctrl-D then Ctrl-B" bit. Can you explain a little more on that please.

DWiskow
Posts: 56
Joined: Sun Apr 09, 2017 1:56 pm

Re: USB serial communication using second processor/thread of the RD2040

Sun Feb 07, 2021 3:08 am

There are a number of useful shortcuts for interacting with the MicroPython REPL. See below for the key combinations;

  • Ctrl-A on a blank line will enter raw REPL mode. This is similar to permanent paste mode, except that characters are not echoed back.
  • Ctrl-B on a blank like goes to normal REPL mode.
  • Ctrl-C cancels any input, or interrupts the currently running code.
  • Ctrl-D on a blank line will do a soft reset (and will auto run anything defined in main.py).
  • Ctrl-E enters ‘paste mode’ that allows you to copy and paste chunks of text. Exit this mode using Ctrl-D.
  • Ctrl-F performs a “safe-boot” of the device that prevents boot.py and main.py from executing

So, the “ Ctrl-C, Ctrl-C, Ctrl-D then Ctrl-B” sequence

  • interrupts the currently running code [twice to make sure]
  • Executes a soft reset
  • and goes to normal REPL mode

arj
Posts: 24
Joined: Sun Jun 10, 2012 2:18 pm

Re: USB serial communication using second processor/thread of the RD2040

Sun Feb 07, 2021 12:25 pm

That's very helpful, thanks.

Have you come across a source of documentation for the Raspberry Pico specific MicroPython libraries? I guess these are evolving quite quickly at the moment. The MicroPython site has details on specific hardware but nothing for the Pico yet.

mlewus
Posts: 2
Joined: Tue Feb 23, 2021 12:05 am

Re: USB serial communication using second processor/thread of the RD2040

Wed Feb 24, 2021 9:35 am

@DWiskow, are you aware of any documentation as to how modem control signals are implemented in the (micropython) pico USB serial interface? I'm working with some software that was written for the ESP 8266 and it assumes certain reset functionality associated with DTR and CTS. I can't find any Pico documentation about how those control signals are implemented on the Pico.

There is some level of implementation because, for instance, DTR needs to be asserted from the PC side when opening the port, in order to communicate with the Pico. Any other documentation you may have on this and other control signals would be appreciated.

Thanks

blimpyway
Posts: 618
Joined: Mon Mar 19, 2018 1:18 pm

Re: USB serial communication using second processor/thread of the RD2040

Wed Feb 24, 2021 11:20 pm

Thanks, I was about to ask how can I stop Pico from reading application traffic over serial and getting back into repl.

So any application talking to Pico over serial should avoid sending CTRL-A trough CTRL-F byte codes as parts of its normal traffic?

hippy
Posts: 9906
Joined: Fri Sep 09, 2011 10:34 pm
Location: UK

Re: USB serial communication using second processor/thread of the RD2040

Thu Feb 25, 2021 12:23 am

blimpyway wrote:
Wed Feb 24, 2021 11:20 pm
So any application talking to Pico over serial should avoid sending CTRL-A trough CTRL-F byte codes as parts of its normal traffic?
I didn't have any issues with anything but Ctrl-C when I was reading the stdin of the USB virtual serial port. You can prevent Ctrl-C entering the REPL using -

Code: Select all

import micropython
micropython.kbd.intr(-1) # Disable Ctrl-C
micropython.kbd.intr(3)  # Re-enable Ctrl-C
After that I could read all raw 8-bit character bytes, 0x00 through 0xFF.

The only show-stopper for me is I haven't been able to figure out how to detect if a 'break' has been received or have it put 0x00 in the receive buffer as physical UART's do, as PIO UART's can be made to do..

Take care if disabling Ctrl-C in 'main.py' because it will be hard to get back to a REPL if you don't have a means of re-enabling Ctrl-C


pmulvey
Posts: 14
Joined: Tue Feb 16, 2021 9:25 am
Location: Athlone, Ireland

Re: USB serial communication using second processor/thread of the RD2040

Thu May 13, 2021 8:39 am

I have used this background code to send commands to the Pico from my laptop Python program. I plan on using this to connect an ESP8266 to the Pico so that I can use the Pico in slave mode (next best thing to I2C slave).
However... after sending about 1000 bytes to the Pico it stops responding. This is a consistent problem and never fails to occur. Any ideas anyone? The following is a piece of code that I found to implement a select case type structure to handle the various commands that I send to the Pico.

Code: Select all

def one():
    return "January"
 
def two():
    return "February"
 
def three(x):
    print (x)
    return "March"
 
def four():
    return "April"
 
def five():
    return "May"
 
def six():
    return "June"
 
def seven():
    return "July"
 
def eight():
    return "August"
 
def nine():
    return "September"
 
def ten():
    return "October"
 
def eleven():
    return "November"
 
def twelve():
    return "December"
 
 
def numbers_to_months(argument,a):
    switcher = {
        1: one,
        2: two,
        3: three,
        4: four,
        5: five,
        6: six,
        7: seven,
        8: eight,
        9: nine,
        10: ten,
        11: eleven,
        12: twelve
    }
    # Get the function from switcher dictionary
    func = switcher.get(argument, lambda: "Invalid month")
    # Execute the function
    # print (func())
    func(a)


numbers_to_months(3, "Hello")

hippy
Posts: 9906
Joined: Fri Sep 09, 2011 10:34 pm
Location: UK

Re: USB serial communication using second processor/thread of the RD2040

Thu May 13, 2021 9:54 am

There have been all sorts of issues using '_thread' and I don't think they have all been resolved. Until they are; the most logical explanation for anything using '_thread' which doesn't work is because '_thread' is broken.

pmulvey
Posts: 14
Joined: Tue Feb 16, 2021 9:25 am
Location: Athlone, Ireland

Re: USB serial communication using second processor/thread of the RD2040

Sat May 15, 2021 4:39 pm

It seems that we might be putting in for unnecessary frustration by trying to get the Pico to do stuff at this immature stage of its Micropython implementation.

Return to “MicroPython”