OCR scanning image with Raspberry

[P-solver]

Hi, this question is a part of my "project" that can be found here: https://www.raspberrypi.org/forums/view ... 2#p1634022

Background:
I want to extract two numeric values from an LCD display.
The values changes depending on outside temperature etc.
I have gone from the idea of extracting the info directly from the voltage signals sent to the LCD display, to trying to extract the info from a image of the display instead.
I will mount an Raspberry Pi that takes an photo of the display every 30 second, and saves it in a folder on the Pi.
On this Pi I then would like to extract the two values, by running some sort of code.

Question(s):
a. Is it possible to extract the values "X" and "Y" by running the example image below in a sort of "OCR-scanning" code? (Result from the code should be "X=28", Y=29")
b. If "a" is possible, what should I look up when it comes to the coding, to be able to extract the two values into variables? Is there any function in python for this that I can look up?

Example photo of display ("X" and "Y" is what I want to extract as two numeric variables):

display.PNG (192.83 KiB) Viewed 528 times

[jamesh]

Rest of project is here https://www.raspberrypi.org/forums/view ... 1&t=268597

But will let this thread run separately as it's quite interesting, and only a small part of the entire project, but technically challenging.

[boyoh]

I see you have started another thread on your project , Did you complete the first thread
You are taking a more difficult rout , by trying to use the display image to get your signals
in to the Pi by using code. Why have you dropped the idea of inputting your Sensor signals
in to the Pi inputs, You must understand, the Pi is not " Plug & Play" , I suggest you read
up on how different sensors work ,and how to inter face them to the GPIOs

Regards BoyOh

[neilgl]

Layout is not too bad, but the question is - can we read numbers from an lcd screen or image thereof?, forgetting sensors and GPIO...

[nigelbartlett1]

Try Tesseract. I tried it to read low resolution scans of a paper form, looking for a particular 13 digit number. It was accurate about 50% of the time. If you can accept single digit errors then that bumped up the success rate to just below 80%. Never got a chance to try it with higher resolution scans; certainly worth a go.

Code: Select all

sudo apt -y install tesseract-ocr
# For help:
man tesseract

[boyoh]

Light sensor (4).jpg (120.56 KiB) Viewed 397 times

This might help you to get the flashing led signal in the Pi I/P It will receive a Active o low

regards BoyOh

[neilgl]

Excellent layout but we are now trying to get OCR working for the LCD display on his heater ?

[P-solver]

I see you have started another thread on your project , Did you complete the first thread
You are taking a more difficult rout , by trying to use the display image to get your signals
in to the Pi by using code. Why have you dropped the idea of inputting your Sensor signals
in to the Pi inputs, You must understand, the Pi is not " Plug & Play" , I suggest you read
up on how different sensors work ,and how to inter face them to the GPIOs

Regards BoyOh

As i wrote in the beggining of this new thread, this is a part of my entire project, and to make it easier and more searchable for other people that might look for a similar solution to this specific problem I created a new post for this.

I don't know if I'm taking a more difficult route, since I've explained that I have 2 choices (as far as I know) to get the value I would like from the heater:
1: Read the signals sent to the display on the heater
2. Reading the display with some sort of OCR scanning

I have not dropped the idea of sending the signals into the Pi. The thing about this specific part of the project is that I can not use a ordinary sensor to get the correct value(s) from the heater display.

I have full understanding that this is not "plug n play".

Just to be clear:
Getting the signals from the flashing led is totaly separat from this specific part of the project.
The flashing light info is to be taken from a voltage meter in the hallway. The value(s) from the display are to be taken from the heater in the basement.

[P-solver]

Try Tesseract. I tried it to read low resolution scans of a paper form, looking for a particular 13 digit number. It was accurate about 50% of the time. If you can accept single digit errors then that bumped up the success rate to just below 80%. Never got a chance to try it with higher resolution scans; certainly worth a go.

Code: Select all

sudo apt -y install tesseract-ocr
# For help:
man tesseract

Hi and thanks! This is exactly what I'm looking for.
I have now installed and setup my Pi and will do some tests with Tesseract this weekend.
I'll be back with more info on how the end result is turning out.

[boyoh]

Might I make suggestion for reading your Temperature .this can be carried out with out interfering with you system as it is
Use sing Thermistor 's as your sensors , They can be used in conjunction with different meters Ohm meter / Volt meter / Current meter
You could the use the reading off your existing meter as a calibration point All this can be at low voltage level to suit the Pi
All items are common and easy to buy

This is only suggestion Regards Boyoh

[neilgl]

Yes, good layout (mods be aware) and a thermistor will measure temperature (resistance is dependent on temperature) but we want to read the screen display of the heater using some images and some software and a pi camera (OCR). I need to install/test that later.

[P-solver]

Try Tesseract. I tried it to read low resolution scans of a paper form, looking for a particular 13 digit number. It was accurate about 50% of the time. If you can accept single digit errors then that bumped up the success rate to just below 80%. Never got a chance to try it with higher resolution scans; certainly worth a go.

Code: Select all

sudo apt -y install tesseract-ocr
# For help:
man tesseract

Hi and thanks! This is exactly what I'm looking for.
I have now installed and setup my Pi and will do some tests with Tesseract this weekend.
I'll be back with more info on how the end result is turning out.

Ok, first report regarding test with tesseract.

Equipment at the moment used for testing:
- A "manual" photo of the display from my phone (since I haven't got the pi-camera delivered yet)
- Raspberry Pi 4
- Installed/Downloaded/updated software: Rasbian, Tesseract, Pillow
- Some Python code

How I did:
1. I put the image "IMG_0126" on the Pi desktop in Rasbian
2. Created the code "img_conv.py" with Pyton and saved it on the Pi desktop in Raspbian
3. Ran the code and got a output in form of the image "bw_display", and also a string of text showing "2829)" (Yes missing one character)

Important notes:
- Regarding the pytesseract, I got some real strange values of its output until I captured the "displays numbers only".
So I guess the importance in carrying on this part of the project is:

1. Have a really good image to start with. Luckily the display has background light, so I'm thinking of making a "screen or protection" from the Pi camera that covers the display from any reflections of existing lights.
2. Trying to capture only the text in the image that you want, when processing the image will help for a good end result.

More updates will come as I get my Pi camera.


img_conv.py

Code: Select all

from PIL import Image
from PIL import ImageFilter
import pytesseract

# open the "original" image 
orig_image = Image.open('IMG_0126.jpg')

# capture the area of the text to "read" by setting "top left" and "right bottom" values in the image
left   = 1660
top    = 1200
right  = left + 620
bottom = top + 180

# copy the captured area
display = orig_image.crop((left, top, right, bottom))

# Modify the image of your "text" to b/w etc to get an image as smooth and "easy readable" as possible, and save it as "code_bw"
gray = display.convert('L')
blackwhite = gray.point(lambda x: 0 if x < 66 else 255, '1')
blackwhite.save("code_bw.jpg")

# open the just saved file and smoothen it some more....
im = Image.open('code_bw.jpg')
smooth_im = im.filter(ImageFilter.SMOOTH_MORE)

# "read" the image with pytesseract and only allow numbers etc.
text = pytesseract.image_to_string(smooth_im, config='--psm 13 --oem 3 -c tessedit_char_whitelist=()0123456789')

# print result from your OCR-reading
print(text)

IMG_0126:

display.PNG (192.83 KiB) Viewed 268 times

bw_display:

bw_display.PNG (16.76 KiB) Viewed 268 times

[boyoh]

Yes, good layout (mods be aware) and a thermistor will measure temperature (resistance is dependent on temperature) but we want to read the screen display of the heater using some images and some software and a pi camera (OCR). I need to install/test that later.

From BoyoH
Is there a possibility that the Camera frame rate and the Temperature display are seeing some
noise and some slight mains ripple on the display. There is no synchronization between camera
and display. There might be some poor filtering on the display power supply. I might be well out
on this idea But nothing ventured nothing gained

Regards BoyOh