Lelouch Dragneel
Posts: 2
Joined: Thu Sep 18, 2014 7:32 am

Need help for project for blind people

Thu Feb 05, 2015 4:21 am

Hi All,
I want to work on a project which can help blind people to read books and documents not in the braille format . The rough idea is to make wearable gloves fitted with cameras , which the person needs to hover over the page and with the help of sensors adjust the glove at the right position, after that the camera will take the picture and convert it into some doc form or say pdf. The next step is that with the help of a software read line by line from the doc and convert it into audio, which can be heard by the person through earphones.
This is just a rough idea of the project and I need help from the people of this forum, if they can guide me in any small way to start this project as I am a total beginner in pi.
Any kind of help will be highly appreciated.

sdjf
Posts: 1388
Joined: Fri Mar 16, 2012 5:20 am
Location: California
Contact: Website

Re: Need help for project for blind people

Thu Feb 05, 2015 4:31 pm

You might get more help in a subforum devoted to special projects, I am not sure which one.

But the idea will be that after the person makes an image, it will need to be processed by OCR (optical character recognition software) and then that output sent to text to speech software such as flite.

THe problem I see is that OCR software can be pretty finnicky, and not do a very good job unless the quality of the image is very, very good, and perfectly level. I am not sure how good the OCR software is that is available for the Pi, but I think that is a very important factor.

For help with the camera aspects, let's see if anyone else answers you, or you could look at other forum posts about using the camera module.
FORUM TIP: To view just one person's posting history, sign in, click on their user name, then click on "Search User's Posts." || My 512MB Model B only runs Arch.

shappat
Posts: 1
Joined: Wed Apr 15, 2015 6:11 am

Re: Need help for project for blind people

Wed Apr 15, 2015 6:29 am

I want to work on a project which can help blind people to read books and documents not in the braille format . The rough idea is to make wearable gloves fitted with cameras , which the person needs to hover over the page and with the help of sensors adjust the glove at the right position, after that the camera will take the picture and convert it into some doc form or say pdf. The next step is that with the help of a software read line by line from the doc and convert it into audio, which can be heard by the person through earphones.
malik66

tsiegel
Posts: 1
Joined: Wed Jul 15, 2015 12:07 am

Re: Need help for project for blind people

Wed Jul 15, 2015 12:47 am

I'm not sure you'd want to use cameras on gloves, it would be much easier to just use a camera that is already on a stand (ocr works much better from a steady image) then feed the image to the pi, and allow it to process the image, then output via festival of flite, or if you wanted a better speech synthesizer, the emic2 from Parallax is an excellent choice, and since it's based on Dectalk, many blind users will already recognize it, plus it's human sounding speech can't be beat for quality.
https://www.parallax.com/product/30016
I don't know of any cameras that work on the pi (I just got mine a couple months ago), and I'm still working on getting it talking properly, but that's another story. But regardless, desktop cameras would be best, it's really hard to get a decent picture for ocr with anything handheld, most of the ocr programs for the iphone work much better when a stand is used for the phone to hold it still while taking an image of the text to be read.
hth.

ame
Posts: 3172
Joined: Sat Aug 18, 2012 1:21 am
Location: Korea

Re: Need help for project for blind people

Wed Jul 15, 2015 3:17 am

Or you could buy an OCR pen, like this one:
http://www.irislink.com/c2-2842-189/IRI ... Table.aspx

It's not Open Source however, but someone might have hacked on it a bit.

The original is probably the C-Pen, but that's not open source either.

Is this project for blind people, or partially-sighted? They key with Braille is that you can feel the line, so you know which direction to read. With gloves, or a pen, you have to be able to see the lines (even just a blurry outline) to guide the camera. If you have a scanner or full-page camera then the software can follow the text.

johnf
Posts: 28
Joined: Fri Dec 23, 2011 10:35 pm

Re: Need help for project for blind people

Sat Aug 01, 2015 10:19 pm

There is a well established process called "The vOICe" (add Dr. Peter Meijer to the websearch!). This aims to sonify a low res image from a brow or specframe cam , with very simple audio code. The system has OCR capabilities, colour analysis, etc.
It is barely functional on a Pi-B, presumably will work much better on a B-2.
Basically written for PC, now available for Android 4.2.
Well worth checking out. The performance is far ahead of any other surgical or cyborg devices on the market.
The reading of paper printing with OCR certainly implies fairly good alignment of camera , perhaps a bitmap image would lend itself to auto alignment software, to detect inter line gaps before OCR begins.
The vOICe could benefit from a detachable head-camera, say with a magnetic fixing to specs frame or headband... To check a document , timetable, salad platter (!) would require a neat wrist movement to snap the camera off, quizz the object, and neatly pop it back again.
There would need to be a corresponding switching of picture pixel rates to permit OCR to interrupt the normal sonifications.
The suggestion offered is that The vOICe is the foundation system , giving a degree of sight replacement, and the OCR device applied as a hand-mobilised adjunct to give precision colour analysis and OCR, and all manner of clever tricks .
As things stand, webcams and equivalent spyglasses are best suited to the wearable setup, which requires a USB cable connection from controls to headpiece. Is it feasible to somehow cable the Pi Cam to the motherboard in the same way?

FRED36
Posts: 1
Joined: Sun May 07, 2017 9:12 am

Re: Need help for project for blind people

Sun May 07, 2017 9:41 am

Hello, I don´t thiink your project is well defined : why camera's for blind ?? It is to my opinion much cheaper to take a (human voice) audio file (e.g. see LIBRIVOX) and program something like my WDS / EDOS system. ( please have a look at youtube, search for fw3664 ) Of course you will need an army of volunteers to build a collection! Regards, FW.

johnf
Posts: 28
Joined: Fri Dec 23, 2011 10:35 pm

Re: Need help for project for blind people

Sun May 07, 2017 9:59 pm

Things have come an awful long way since 2015.
Ref. your remark of "why cameras for blind?".. well, have a look at seeingwithsound.com which is the fountain of knowledge for Sight Replacement. This is as good as a cheap autonomous system can get.
The user has to learn the meaning of "soundscapes" corresponding to low-res images.
Also, go to After-Sight.com which is an effort, based upon Raspberry-Pi , giving the user not only the system known as " The vOICe " but also the Terradeep object recognition app. All for $200..
The vOICe is also available on Android smart glasses such as the ORA-800 which of course offer internet connection and useful apps.
Thanking you for your interest.
john

chad546
Posts: 1
Joined: Tue Aug 22, 2017 2:53 am

Re: Need help for project for blind people

Tue Aug 22, 2017 3:01 am

the problem in OCR api's is that it still read the words on the paper even though the blind person did not place the paper properly.

User avatar
startrek.steve
Posts: 372
Joined: Thu Aug 30, 2012 7:19 pm

Re: Need help for project for blind people

Tue Aug 22, 2017 7:35 am

I work with Visually and Hearing Impaired children, and there are various pieces of hardware that do this, Giraffe Reader on the Ipad, and on the Android Braillenote Touch there is a built in OCR reader in the Android tablet part of the Braillenote. Looks like youre trying to re-invent the wheel. (Probably cheaper than a £3000 braillenote though!)
Pi 2 running LibreElec Krypton, Pi 2 running Wheezy Desktop, Pi 1 headless Wheezy, downloading Radio, Pi 1 running Picore headless Media Server, Pi Zero to be an Old Time Radio Project. Pi 3 testing. Pi Zero W downloading Radio shows headless.

PiGraham
Posts: 3261
Joined: Fri Jun 07, 2013 12:37 pm
Location: Waterlooville

Re: Need help for project for blind people

Tue Aug 22, 2017 8:15 am

sdjf wrote:
Thu Feb 05, 2015 4:31 pm

THe problem I see is that OCR software can be pretty finnicky, and not do a very good job unless the quality of the image is very, very good, and perfectly level. I am not sure how good the OCR software is that is available for the Pi, but I think that is a very important factor.
chad546 wrote:
Tue Aug 22, 2017 3:01 am
the problem in OCR api's is that it still read the words on the paper even though the blind person did not place the paper properly.
This might be the issue that can be usefully addressed - guiding the blind user to optimal positioning of the camera relative to the text using haptic (feel) or other non-visual feedback to enable effective OCR.

A Hough filter can find straight lines in images. OPenCV includes such filter functions. If the text is square-on and level the lines of text will be parallel and aligned with the pixel grid. If the view is off centre or rotated that can be measured.
A bounding box around the text should not overlap the edge of the image.

Isolating single words and attempting OCR on them as a fast rate may provide additional guidance before attempting the whole text.

A mechanism that moved or vibrated could guide the hand of the user to correct alignment. An inertial measurement unit (accelerometer) can measure shake and help to take a picture when the camera is steady.

The haptic devices might be vibration motors (as used in phones) or a moving weight (to feel the balance) or motors that apply force to the fingers touching the edge of the device or many other possibilities. Sound might also be useful.

Gloves may not be the best idea because they might interfere with touch and the device will probably not be in intensive use in the hand. For long reading sessions a table stand device is probably better. For occasional hand-held use a pendant on a lanyard might be easier.

Return to “Assistive technology and accessibility”

Who is online

Users browsing this forum: No registered users and 1 guest