DrRocksalt
Posts: 5
Joined: Mon Oct 15, 2018 5:03 am

Python Serial jumping ahead / While loop problem on boot only

Mon Oct 15, 2018 3:00 pm

I'm having a very strange problem with a While loop in Python I hope you all could help with. I didn't have time before running to work this morning to pull the code, so I'll be working with pseudo code or from memory (or an old version I have on my work computer).

A little background: a few friends and I do the 24 Hours of Lemons races, where the car has to cost less than $500. We drive an engine swapped Volvo wagon. I use an Arduino with custom circuit boards to take in readings from the car like RPM, oil pressure, temp, etc., control a dash with lots of LEDs and stuff, and drive the solenoids for the trans. I also have a Raspberry Pi in the box as well. After this Pi boots, it automatically launches a Python script, communicating with the Arduino over GPIO serial to take in the car's data and log it, as well as a GPS receiver. The Pi has an extension wifi antenna, so every time it passes our router on the side of the track, it pushes data to a laptop running a Matlab GUI to look at data. On our last race it worked somewhat well. Some pictures of our idiocy are on our instagram: https://www.instagram.com/gunbarrelcobras/

Here's the recent problem: I didn't think I'd changed anything since the last race as far as the Python script goes. But now, the handshaking between Arduino and Pi is broken. Here's the general steps:

1) Arduino and Pi are powered on. Arduino starts listening for the Pi to send "@"
2) Once Pi boots, sends "@" to the Arduino. If Arduino sees that, send "@" back.
3) If Pi hears "@" after sending one, send "=" to confirm
4) Once Arduino hears "=", it knows connection is successful, everybody goes about their day.

Here's a code snippet:

Code: Select all

ser = serial.Serial('/dev/ttyS0', 250000, timeout=.01)
sleep(.2)
dashcoms = False
dashLEDstate = False
try:
    contime1 = time()
    while dashcoms == False:
        ser.write("@")
        onresponse = ser.readline()
        if onresponse == '@\r\n':
            dashcoms = True
            ser.write("=")
            GPIO.output(COM_LED, True)
            print("Dash Connected!")
        else:
            if time()-contime1 > 0.25:
                print("No dash connection")
                dashLEDstate = not dashLEDstate
                GPIO.output(COM_LED, dashLEDstate)
                contime1 = time()
except KeyboardInterrupt:
        print('\n')
        
And then on the Arduino side (again, just a snippet):

Code: Select all

  if (!rpiconn ) {    // If the RPi has yet to connect, only listen for @
    while (Serial1.available() > 0) {
      int readin = Serial1.read();
      if (readin == 64) {            // If Serial1 reads in a "@" (ASCII 64), the Pi is checking dash coms.  Send back a "@"
        Serial1.println("@");
        Serial.println("Handshaking with RPi");
      }
      else if (readin == 61) {
         rpiconn = true;
        Serial.println("RPi connected!");
    }
  }
  else {
    while (Serial1.available() > 0) {
      char received = Serial1.read();
      if (received != '\n') {
        inData += received;
      }
      if (received == '\n') {
        // parsing RPi data, etc.
      }
    }
  }
On power-up, the loop sends "@", receives"@", and sends "=" to the Arduino (I monitor the regular serial port on the Ardiuno, Pi coms is on Serial1 on a Mega). It does NOT, however, print anything, and then immediately goes to saying there's no dash connected, even though it has sent "=".

Where I'm confused, is if I then stop that script and run it again (after resetting the Arduino's hand-shaking), it works perfectly, and the Arduino reports seeing exactly what it did on the first time.

What on earth would make a While loop fail like this only once after boot, but then work perfectly after? I feel like I'm losing my mind!
Last edited by DrRocksalt on Tue Oct 16, 2018 3:28 pm, edited 1 time in total.

DrRocksalt
Posts: 5
Joined: Mon Oct 15, 2018 5:03 am

Re: While loop problem on boot only

Tue Oct 16, 2018 6:29 am

Okay, a bit more debugging, and I'm getting somewhere, but still very confused.

I've now moved the last response from the Pi to the Arduino to outside of the while loop, after the communication has been confirmed the first time. This still makes it fail. What I've found, though, is that I can put in a long delay before sending that final handshake symbol, and a print line of "I'm sending it now" just before. The thing I don't understand is that the Arduino gets the final handshake symbol 10 seconds before the code finally says it's sending it. I have no idea how this can happen. I must be drastically misunderstanding something.

User avatar
Paeryn
Posts: 2171
Joined: Wed Nov 23, 2011 1:10 am
Location: Sheffield, England

Re: While loop problem on boot only

Tue Oct 16, 2018 8:51 am

That sounds like you are maybe seeing the effect of output being buffered, especially if you have stdout redirected to a file. If stdout is a terminal then it is flushed on every newline, if it is a pipe or redirected to a file then it isn't.

Try explicitly flushing the print() lines, e.g.

Code: Select all

    print("Dash Connected!", flush = True)
She who travels light — forgot something.

DrRocksalt
Posts: 5
Joined: Mon Oct 15, 2018 5:03 am

Re: While loop problem on boot only

Tue Oct 16, 2018 3:15 pm

Interesting. These print lines are just to the the terminal, so they should be flushing, right? I can certainly try adding flushes though, and seeing what happens.

I guess my befuddlement comes from the fact that the serial port seems to be acting ahead of the actual script. Here's a snippet of my current code (note I've changed the handshaking characters to not be identical so as to prevent some weird echo issue):

Code: Select all

ser = serial.Serial('/dev/ttyS0', 250000, timeout=.01)  # chagned from 115200 12/14/17
sleep(.2)
dashcoms = False
dashLEDstate = False
try:
    contime1 = time()
    while dashcoms == False:
        ser.write("[")
        onresponse = ser.readline()
        onresponse = onresponse.strip()
        sleep(1)
        if onresponse == ']':
            dashcoms = True
            GPIO.output(COM_LED, True)
        else:
            if time()-contime1 > 0.25:
                print("No dash connection")
                dashLEDstate = not dashLEDstate
                GPIO.output(COM_LED, dashLEDstate)
                contime1 = time()
except KeyboardInterrupt:
        print('\n')
print("Dash Connected!")
sleep(12)
print("Sending _")
ser.write("_")
If I take out the sleep(12), the Arduino receives the "_" before the while loop is broken. If I change the sleep(12) to something longer like sleep(30), I can time the difference with a stopwatch. The Arduino receives the final "_" 10 seconds before the RPi terminal prints "Sending _"

Apologies for reiterating, but I want to make sure I'm not missing something obvious.

As for running successfully the second time after ctrl+C quitting the first time failing, that MAY be because ctrl+C doesn't quit the script very well (my fault), and it's still running in the background. I just learned that I should ps and then kill the process. When I'm not at work I'll check that.

DrRocksalt
Posts: 5
Joined: Mon Oct 15, 2018 5:03 am

Re: Python Serial jumping ahead / While loop problem on boot only

Thu Oct 18, 2018 3:03 pm

More weirdness observed.

Further down in the code, in a loop that runs as long as the car is on, the Raspberry Pi takes some of its parameters and pushes it to the Arduino so that the dashboard can display things like disk space, IP address, latitude and longitude, etc. I do this by starting the line with "*", separate the info with "#" and then end the string with "&", and then a new line. In the Arduino code this makes it easy to throw out the communication unless it starts with "*" and ends with "&", and then if that's all good, chop up everything between the "#" symbols in the order I know the Pi puts them together.

I know this string is being assembled correctly. It prints to terminal on the Pi correctly. But when I monitor what the Arduino is receiving, the received string is a very bizarre. What it should look like (no "#" here, I've just separated with commas for readability):

Code: Select all

wifi, ip, file count, disk space, lat, lon, sat num, alt, foldername, gpsspeed, signal strength, loggingStatus, havemessage
But the Arduino thinks it is getting that, plus another one or more of it shoved in the middle:

Code: Select all

wifi, ip, file count, disk space, lat, lon, wifi, ip, file count, disk space, lat, lon, wifi, ip, file count, disk space, lat, lon, sat num, alt, foldername, gpsspeed, signal strength, loggingStatus, havemessage ,sat num, alt, foldername, gpsspeed, signal strength, loggingStatus, havemessage, sat num, alt, foldername, gpsspeed, signal strength, loggingStatus, havemessage
Every once in a while a correct one shows up. It's like the serial is writing over itself or something. My gut feeling is that it's a similar problem to the one I mentioned above. Serial flushes (both Pi and Arduino, both in and output) don't seem to do anything.

I work close to Sparkfun, so I can go pick up a new Pi and memory card, with the hope being that it is some Pi side hardware issue.

Any help is appreciated.

User avatar
Paeryn
Posts: 2171
Joined: Wed Nov 23, 2011 1:10 am
Location: Sheffield, England

Re: Python Serial jumping ahead / While loop problem on boot only

Thu Oct 18, 2018 3:35 pm

Is this only happening when the code is being run automatically as start up? Just wondering if you are somehow managing to have multiple copies of the program running at the same time.
She who travels light — forgot something.

DrRocksalt
Posts: 5
Joined: Mon Oct 15, 2018 5:03 am

Re: Python Serial jumping ahead / While loop problem on boot only

Tue Oct 23, 2018 3:02 am

Alright, the race is over. We got 2nd place in our class, didn't blow up the car, and got an award for our intake setup being weird. Good times.

I bought a new Pi and cloned the disk image to a new card, but that failed. The new card wouldn't even boot the old Pi. I would have tried that a bunch more, but each write took eternity on my Mac. Only later did I find some other way to write it that would have sped up the process.

The weird problem of sending a serial command 10 seconds before Python runs the command is still there. The other weird serial issue I was getting may be some weird interference problem.

I mentioned we had a GPS receiver on the car. I start this using threading, and admittedly I got that code chunk from a tutorial, so I'm not super familiar with it. I know it worked last race, but not now. What I noticed is that when the code initially started, the Arduino was receiving the correct string, but the GPS receiver hadn't initialized and was just sending NaN (actual line received in the car, in the format I mentioned previously):

Code: Select all

*None##0#7996 MBs#nan#nan#12#nan ft#preOctHPR#nan mph#0%#0#!&
but then as soon as the GPS receiver initialized, things got bad (bad lines are random, this is just a sample):

Code: Select all

*None##0#7996 MBs#39.735361182#-103.893295299#12#5039.4 ft#preOctHPR#0.0*None##0#7996 MBs#39.735361182#-103.893295299#12#5039.4 ft#preO*None##0#7996 MBs#39.735361182#-103.893295299#12#5039.4 ft#preOctHPR#0.0 mph#0%#0#!&
which is obviously no good with repeated data and such. So somehow the GPS stuff is messing things up THIS time, though it worked flawlessly last race.

You might think I should have caught this, but when I have time to debug, I'm at my own house, away from the car, and the GPS receiver is attached to the car at a team mates house, so I never saw this issue. They're not too much money, so I may just buy a spare to have at my place to debug.

Return to “Python”