cyberblitz
Posts: 9
Joined: Sun Aug 19, 2012 12:17 pm

LCD gibberish woes :(

Wed Jun 26, 2019 9:57 pm

Hi,

I am attempting to pull data from a websites API that changes frequently and display the data on an LCD (a I2C, 20 Column, 4 line LCD to be precise). I have managed to access the data and break it up to how i would like to see it on the LCD. I am also able to get the standby screen to work, showing a clock and the words, "No Active Rescues" when there's no data to show.

The problem i am having is when the display switches between the standby screen to the data screen and back again. The LCD starts to show gibberish and the only way to get to work again is to turn the Raspberry Pi off and restart it. But as soon as data comes in, reverting from the standby screen, gibberish shows again.

After scouring the internet it appears that the LCD might be receiving two or more message packets at the same time (because of the multiprocessing) resulting with the gibberish display, but the code seems fine to me (using print to show the results seems to suggest this). I have tried threading and multiprocessing, and Pipe() following some suggestions but can't get it to work. The problem with Pipe is that i need to send two different messages to one end (not at the same time) using the other end to receive the message. I might not have fully comprehended how to use this bit of code (i'm kinda new to Python), so any help would be appreciated.

PLEASE, PLEASE any help would be appreciated. I've been stuck on this for days :cry:

ADDN: I had tried a bit of code from another source where they too had problems like mine, so they created a class, thus:

Code: Select all

Class mem:
  lcd_connection = Pipe()
  
But no matter what i try, the rest of code only sees the 'lcd_connection' variable as a tuple, becasue the Pipe() method requires two variables (i have no idea how they got it to work).

Code: Select all

import requests
from time import sleep
import time
import datetime
import multiprocessing as mp


# import I2C_LCD_Driver
# mylcd = I2C_LCD_Driver.lcd()

# api URL
rescue_url = "https://api.somesite.com/rescues"
headers = {"Authorization": "Bearer Key"}
json_data = []
dta_bool = False


def lcd_enter(comm):
    while True:

        ln1, ln2, ln3, ln4 = comm.recv()

        #  mylcd.lcd_clear()
        #  mylcd.lcd_display_string(ln1, 1)
        # mylcd.lcd_display_string(ln2, 2)
        # mylcd.lcd_display_string(ln3, 3)
        # mylcd.lcd_display_string(ln4, 4)

        print(ln1)
        print(ln2)
        print(ln3)
        print(ln4)


def get_data(url):
    try:
        data = requests.get(url, headers=headers).json()
        data = data['data']
        return [dta['attributes'] for dta in data if dta['attributes']['status'] == 'closed']
    except requests.exceptions.RequestException:
        sleep(5)
        data = requests.get(url, headers=headers).json()
        data = data['data']
        return [dta['attributes'] for dta in data if dta['attributes']['status'] == 'open']


# function to convert string datetime value when rescue commenced (usually entered by Mecha)
def dte_convert(dte):
    d1 = datetime.datetime.strptime(dte, "%Y-%m-%dT%H:%M:%S.%fZ") + datetime.timedelta(hours=10)
    d2 = datetime.datetime.now()  # set variable to current dateTime
    dur_s = (d2 - d1).total_seconds()  # similar to datediff and calculates to seconds
    dy = divmod(dur_s, 86400)
    hrs = divmod(dy[1], 3600)
    mins = divmod(hrs[1], 60)
    ret = '%d d %d h %d m' % (dy[0], hrs[0], mins[0])
    return ret


def clock(comm):
    while True:
        if not json_data:
            try:
                comm.send(("Date: %s" % time.strftime("%d/%m/%Y"),
                           "Time: %s" % time.strftime("%H:%M:%S"),
                           'No Active Rescues',
                           ''))

                # sleep(1)


def main(comm):
    while True:
        global json_data
        json_data = get_data(rescue_url)
        if json_data:
            for each in json_data:
                comm.send(("C: %s" % each['client'],
                           "P: %s" % each['platform'] + " CR: %s" % each['codeRed'],
                           "S: %s" % each['system'],
                           "A: %s" % dte_convert(each['createdAt'])))

        else:
            comm.close()
        sleep(5)


if __name__ == '__main__':
    j_data, c_data = mp.Pipe(False)
    t_main = mp.Process(target=main, args=(c_data,))
    t_clock = mp.Process(target=clock, args=(c_data,))
    lcd_proc = mp.Process(target=lcd_enter, args=(j_data,))
    t_main.start()
    t_clock.start()
    lcd_proc.start()
   # t_main.join()
   # t_clock.join()
   # lcd_proc.join()


cyberblitz
Posts: 9
Joined: Sun Aug 19, 2012 12:17 pm

Re: LCD gibberish woes :(

Thu Jun 27, 2019 11:26 am

Thanks.

I'm just wondering if anyone has any advice to the problem at hand? This issue is driving me to distractions!

pcmanbob
Posts: 9611
Joined: Fri May 31, 2013 9:28 pm
Location: Mansfield UK

Re: LCD gibberish woes :(

Thu Jun 27, 2019 4:55 pm

May be if you showed use what the lines on the display should contain and post a picture of what it actually displays , we may get an idea of what the problem is .

As no one can test run your program debugging it remotely is going to be a problem.
We want information… information… information........................no information no help
The use of crystal balls & mind reading are not supported

Andyroo

Re: LCD gibberish woes :(

Thu Jun 27, 2019 8:06 pm

I had a similar problem on my eInk screen where it would lock due to two threads calling it at once.

I ended up implementing one thread to write to the screen when it gets JSON data on a queue
The queue was fed from other threads in the program, each doing one job - build the display up and format the info into JSON

cyberblitz
Posts: 9
Joined: Sun Aug 19, 2012 12:17 pm

Re: LCD gibberish woes :(

Thu Jun 27, 2019 10:16 pm

Trying to actively hit the website to check for active data whilst in clock mode, then switch when there is data to display is the problem. The LCD screen displays utter gibberish when trying to do this.

The output when there is no data should look like this:
12/05/2019
10:00:02 <-- this should show the seconds counting without interruption
No Active Rescues
When there is data, it should look like this:
C: somebody
P: PS CR: YES
S: some system
A: 00:00:10 <--- this will count up as time passes
This is the JSON dump from the site. I have made changes to it to keep the website hidden.

Code: Select all

{
    "meta": {
        "count": 2,
        "limit": 25,
        "offset": 0,
        "total": 2
    },
    "data": [
        {
            "id": "f4b53c08",
            "type": "rescues",
            "attributes": {
                "client": "Nort",
                "codeRed": true,
                "data": {
                    "langID": "en",
                    "status": {},
                    "IRCNick": "Nort",
                    "boardIndex": 3,
                    "markedForDeletion": {
                        "marked": false,
                        "reason": "None.",
                        "reporter": "Noone."
                    }
                },
                "notes": "",
                "platform": "pc",
                "quotes": [
                    {
                        "author": "nothing",
                        "message": "nothing",
                        "createdAt": "2019-06-27T21:48:55.368294Z",
                        "updatedAt": "2019-06-27T21:48:55.368272Z",
                        "lastAuthor": "Mecha"
                    },
                    {
                        "author": "nothing",
                        "message": "nothing",
                        "createdAt": "2019-06-27T21:48:55.368312Z",
                        "updatedAt": "2019-06-27T21:48:55.368305Z",
                        "lastAuthor": "Mecha"
                    },
                    {
                        "author": "nothing",
                        "message": "nothing",
                        "createdAt": "2019-06-27T21:53:44.696563Z",
                        "updatedAt": "2019-06-27T21:53:44.696536Z",
                        "lastAuthor": "nothing"
                    },
                    {
                        "author": "nothing",
                        "message": "nothing",
                        "createdAt": "2019-06-27T21:54:18.176691Z",
                        "updatedAt": "2019-06-27T21:54:18.176678Z",
                        "lastAuthor": "nothing"
                    }
                ],
                "status": "inactive",
                "system": "Aornum",
                "title": null,
                "outcome": null,
                "unidentified": [],
                "createdAt": "2019-06-27T21:48:55.450Z",
                "updatedAt": "2019-06-27T21:54:18.246Z",
                "firstLId": null
            },
            "relationships": {
                "users": {
                    "data": []
                },
                "firstL": {
                    "data": null
                },
                "epics": {
                    "data": []
                }
            },
            "links": {
                "self": "/rescues"
            }
        },
        {
            "id": "ea2ecf46",
            "type": "rescues",
            "attributes": {
                "client": "nothing",
                "codeRed": false,
                "data": {
                    "langID": "en",
                    "status": {},
                    "IRCNick": "nothing",
                    "boardIndex": 2,
                    "markedForDeletion": {
                        "marked": false,
                        "reason": "None.",
                        "reporter": "Noone."
                    }
                },
                "notes": "",
                "platform": "ps",
                "quotes": [
                    {
                        "author": "nothing",
                        "message": "Col 359",
                        "createdAt": "2019-06-27T21:30:26.217976Z",
                        "updatedAt": "2019-06-27T21:30:26.217957Z",
                        "lastAuthor": "nothing"
                    },
                    {
                        "author": "nothing",
                        "message": "nothing",
                        "createdAt": "2019-06-27T21:47:16.317612Z",
                        "updatedAt": "2019-06-27T21:47:16.317592Z",
                        "lastAuthor": "nothing"
                    }
                ],
                "status": "inactive",
                "system": "Col 359",
                "title": null,
                "outcome": null,
                "unidentified": [],
                "createdAt": "2019-06-27T21:30:26.304Z",
                "updatedAt": "2019-06-27T21:47:19.901Z",
                "firstLId": null
            },
            "relationships": {
                "user": {
                    "data": [
                        {
                            "id": "48178386",
                            "type": "nothing"
                        }
                    ]
                },
                "firstL": {
                    "data": null
                },
                "epics": {
                    "data": []
                }
            },
            "links": {
                "self": "/rescues/"
            }
        }
    ],
    "included": [
        {
            "id": "48178386",
            "type": "nothing",
            "attributes": {
                "name": "nothing",
                "data": null,
                "joined": "2018-12-20T07:03:30.000Z",
                "platform": "ps",
                "createdAt": "2018-12-20T07:03:30.079Z",
                "updatedAt": "2018-12-20T07:03:30.079Z",
                "deletedAt": null,
                "userId": "083feb7b"
            },
            "relationships": {
                "cars": {
                    "data": []
                }
            }
        }
    ]
}

pcmanbob
Posts: 9611
Joined: Fri May 31, 2013 9:28 pm
Location: Mansfield UK

Re: LCD gibberish woes :(

Fri Jun 28, 2019 10:19 am

But what does the display actually show when you say it shows gibberish ?

just trying to get an idea if its because of some formatting error or line wrap, or is its because you are writing 2 lots of data to the display from different parts of you program at the same time.
We want information… information… information........................no information no help
The use of crystal balls & mind reading are not supported

cyberblitz
Posts: 9
Joined: Sun Aug 19, 2012 12:17 pm

Re: LCD gibberish woes :(

Fri Jun 28, 2019 9:17 pm

It's just gibberish - a collection of randomly placed symbols, characters, cursors ect, filling all spaces, nothing you can make any sense from. Its no different than opening a compiled exe file in notepad text file.

pcmanbob
Posts: 9611
Joined: Fri May 31, 2013 9:28 pm
Location: Mansfield UK

Re: LCD gibberish woes :(

Sat Jun 29, 2019 9:36 am

Sounds very much like you either have 2 programs or 2 parts of your program writing to the same LCD at the same time , or the data being written is corrupt.

when you are writing data to the LCD have you tried also printing the same data on the screen assuming you have a screen attached to the pi so you can see what is actually being written.
We want information… information… information........................no information no help
The use of crystal balls & mind reading are not supported

cyberblitz
Posts: 9
Joined: Sun Aug 19, 2012 12:17 pm

Re: LCD gibberish woes :(

Sat Jun 29, 2019 10:01 am

Yea, I figured that was the case, but I have no idea how to rectify it.

This is the code without any multiprocessing/threading. It works fine, except the seconds on the clock update only as quick as the data is downloaded. I was hoping to have the seconds on the clock tick away and check the API every 5 seconds in the background. Once the background process receives data, it displays it on the screen, replacing the clock, and vice versa when no data is received.

Any ideas??

I also changed the code a little after figuring how to filter the API.

Code: Select all

import requests
from time import sleep
import time
import datetime
# import I2C_LCD_Driver
# mylcd = I2C_LCD_Driver.lcd()

json_data = []

def get_data():
    fin = []
    global json_data
    rescue_url = "https://api.someapisite.com/rescues?[status]=open"   # ?$not[status]=closed"
    headers = {"Authorization": "Bearer key"}
    data = requests.get(rescue_url, headers=headers).json()
    data = data['data']
    json_data = [dta['attributes'] for dta in data]
    if json_data:
        for each in json_data:
            fin_val = ["C: %s" % each['client'],
                       "P: %s" % each['platform'] + " CR: %s" % each['codeRed'],
                       "S: %s" % each['system'],
                       "A: %s" % dte_convert(each['createdAt'])]
            fin.append(fin_val)
        return fin


# function to convert string datetime value when rescue commenced (usually entered by Mecha)
def dte_convert(dte):
    d1 = datetime.datetime.strptime(dte, "%Y-%m-%dT%H:%M:%S.%fZ") + datetime.timedelta(hours=10)
    d2 = datetime.datetime.now()  # set variable to current dateTime
    dur_s = (d2 - d1).total_seconds()  # similar to datediff and calculates to seconds
    dy = divmod(dur_s, 86400)
    hrs = divmod(dy[1], 3600)
    mins = divmod(hrs[1], 60)
    ret = '%d d %d h %d m' % (dy[0], hrs[0], mins[0])
    return ret


def clock():
    if not json_data:
        fin_val = ("Date: %s" % time.strftime("%d/%m/%Y"),
                   "Time: %s" % time.strftime("%H:%M:%S"),
                   'No Active Rescues',
                   '')

        return fin_val


def lcd_data(in_data):

    ln1, ln2, ln3, ln4 = in_data

        #  mylcd.lcd_clear()
        #  mylcd.lcd_display_string(ln1, 1)
        # mylcd.lcd_display_string(ln2, 2)
        # mylcd.lcd_display_string(ln3, 3)
        # mylcd.lcd_display_string(ln4, 4)

    print(ln1)
    print(ln2)
    print(ln3)
    print(ln4)


while True:
    json_data = get_data()
    t_clock = clock()

    if json_data:
        for data in json_data:
            lcd_data(data)
            sleep(3)
    else:
        lcd_data(t_clock)
    sleep(5)

pcmanbob
Posts: 9611
Joined: Fri May 31, 2013 9:28 pm
Location: Mansfield UK

Re: LCD gibberish woes :(

Sat Jun 29, 2019 10:20 am

Sorry no idea never written any programs that included threading....
We want information… information… information........................no information no help
The use of crystal balls & mind reading are not supported

cyberblitz
Posts: 9
Joined: Sun Aug 19, 2012 12:17 pm

Re: LCD gibberish woes :(

Sun Jun 30, 2019 12:04 am

This is the closest I can get this to work.
It doesn't work as I would like but close enough.

I'll leave this thread open in case anybody else has any better ideas..

Code: Select all

import requests
from time import sleep
import time
import datetime
import multiprocessing as mp
# import I2C_LCD_Driver
# mylcd = I2C_LCD_Driver.lcd()

json_data = []

def get_data(pi):
    while True:
        fin = []
        global json_data
        rescue_url = "https://api.somesite.com/rescues?[status]=open"   # ?$not[status]=closed"
        headers = {"Authorization": "Bearer key"}
        data = requests.get(rescue_url, headers=headers).json()
        data = data['data']
        json_data = [dta['attributes'] for dta in data]
        if json_data:
            for each in json_data:
                fin_val = ["C: %s" % each['client'],
                           "P: %s" % each['platform'] + " CR: %s" % each['codeRed'],
                           "S: %s" % each['system'],
                           "A: %s" % dte_convert(each['createdAt'])]
                fin.append(fin_val)
            pi.send(fin)
        else:
            pi.send(None)
        sleep(5)


# function to convert string datetime value when rescue commenced (usually entered by Mecha)
def dte_convert(dte):
    d1 = datetime.datetime.strptime(dte, "%Y-%m-%dT%H:%M:%S.%fZ") + datetime.timedelta(hours=10)
    d2 = datetime.datetime.now()  # set variable to current dateTime
    dur_s = (d2 - d1).total_seconds()  # similar to datediff and calculates to seconds
    dy = divmod(dur_s, 86400)
    hrs = divmod(dy[1], 3600)
    mins = divmod(hrs[1], 60)
    ret = '%d d %d h %d m' % (dy[0], hrs[0], mins[0])
    return ret


def clock():
    if not json_data:
        fin_val = ("Date: %s" % time.strftime("%d/%m/%Y"),
                   "Time: %s" % time.strftime("%H:%M:%S"),
                   'No Active Rescues',
                   '')

        return fin_val


def lcd_data(in_data, clr_lcd):

   ln1, ln2, ln3, ln4 = in_data
   # if clr_lcd:
        #mylcd.lcd_clear()
          #mylcd.lcd_display_string(ln1, 1)
        # mylcd.lcd_display_string(ln2, 2)
        # mylcd.lcd_display_string(ln3, 3)
        # mylcd.lcd_display_string(ln4, 4)

   print(ln1)
   print(ln2)
   print(ln3)
   print(ln4)


if __name__ == '__main__':

    pi_out, pi_in = mp.Pipe()
    p = mp.Process(target=get_data, args=(pi_in,))
    p.start()

    while True:

        json_data = pi_out.recv()

        t_clock = clock()

        if json_data is not None:
            for data in json_data:
                lcd_data(data, True)
                sleep(3)
        else:
            lcd_data(t_clock, False)







cyberblitz
Posts: 9
Joined: Sun Aug 19, 2012 12:17 pm

Re: LCD gibberish woes :(

Mon Jul 01, 2019 5:45 am

SOLVED

Would love to hear if anyone else has any better ideas..

This is the code I came up with, using threading:

Code: Select all

import requests
import datetime
from time import sleep
import time
from threading import Thread
# import I2C_LCD_Driver
# mylcd = I2C_LCD_Driver.lcd()

json_dataG = []

def get_data():

    global json_dataG
    while True:
        try:
            fin = []
            rescue_url = "https://api.somesite.com/rescues?[status]=open"  # ?$not[status]=closed"
            headers = {"Authorization": "Bearer key"}
            data = requests.get(rescue_url, headers=headers).json()
            data = data['data']
            json_data = [dta['attributes'] for dta in data]
            if json_data:
                for each in json_data:
                    fin_val = ["C: %s" % each['client'],
                               "P: %s" % each['platform'] + " CR: %s" % each['codeRed'],
                               "S: %s" % each['system'],
                               "A: %s" % dte_convert(each['createdAt'])]
                    fin.append(fin_val)
            else:
                fin = None

            json_dataG = fin
            sleep(5)

        except requests.exceptions.Timeout:
            continue



# function to convert string datetime value when rescue commenced (usually entered by Mecha)
def dte_convert(dte):
    d1 = datetime.datetime.strptime(dte, "%Y-%m-%dT%H:%M:%S.%fZ") + datetime.timedelta(hours=10)
    d2 = datetime.datetime.now()  # set variable to current dateTime
    dur_s = (d2 - d1).total_seconds()  # similar to datediff and calculates to seconds
    dy = divmod(dur_s, 86400)
    hrs = divmod(dy[1], 3600)
    mins = divmod(hrs[1], 60)
    ret = '%d d %d h %d m' % (dy[0], hrs[0], mins[0])
    return ret

def clock():
        fin_val = ("Date: %s" % time.strftime("%d/%m/%Y"),
                   "Time: %s" % time.strftime("%H:%M:%S"),
                   'No Active Rescues',
                   '')

        return fin_val

def lcd_data(in_data):

   ln1, ln2, ln3, ln4 = in_data
    #if clr_lcd:
       # mylcd.lcd_clear()
          #mylcd.lcd_display_string(ln1, 1)
        # mylcd.lcd_display_string(ln2, 2)
        # mylcd.lcd_display_string(ln3, 3)
        # mylcd.lcd_display_string(ln4, 4)

   print(ln1)
   print(ln2)
   print(ln3)
   print(ln4)

def main():
    while True:
        try:

            global json_dataG
            t_clock = clock()
            if json_dataG is not None:
                for data in json_dataG:
                    lcd_data(data)
                    sleep(3)
            else:
                lcd_data(t_clock)
        except ValueError:
            continue

t1 = Thread(target=get_data)
t2 = Thread(target=main)
t1.start()
t2.start()


Return to “Interfacing (DSI, CSI, I2C, etc.)”