Sight for the Blind for <100$

[seeingwithsound]

Hi Jacob,

I haven't tried the After-Sight image on Raspberry Pi 3 yet, but I remember that time in between Teradeep object recognitions was some 5 seconds on Raspberry Pi 2. What is it on Raspberry Pi 3?

Thanks,

Peter


The vOICe for Android
http://www.seeingwithsound.com/android.htm
https://play.google.com/store/apps/deta ... OICe.vOICe

[mikey11]

Peter, the time on the rpi 3 was about as fast as it would finish speaking the last one for teradeep.

performance of everything on the rpi 3 seemed lagless in comparison to the rpi 2 even with multiple functions operating.

I stopped in on the forum today with a small update:

Although I have not been working on things, other people are.

A gent who I will refer to as GO has been working on making performance improvements to the code. When I jumped from the rpi 2 to the rpi 3 the battery life tanked.

This is due to me being a fabricobbler, and not a real programmer or electronics engineer.

I was using a method of constantly polling to look for keystrokes which was maxing out the CPU, and apparently was not doing multi CPU coding effectively.

This gentleman has been working on these issues, and although I haven't seen anything yet, I will let him go through his debugging, and then I will review it, and try to understand the fundamental changes to the code.

I will then work on having it possible to push it as an update.

With those advances, performance should improve even on the rpi 2, but my bigger hope is that power consumption on the rpi 3 is reduced to a reasonable level.

A different gentleman has been working on things that struck his interest. Namely investigating other OS distributions for the rpi 3. I used minibian which is a small size implementation of raspbian. This was done to be able to keep using the cheapest SD cards on the market. It turns out technology keeps jumping faster than my cost concerns do. It probably makes some sense to not care too much about the size of the underlying OS. I appreciate this point of view, and am keen to hear his opinion as he explores.

This gent is also investigating the bluetooth function for the rpi 3. which will eliminate wires. Of course we have to come up with a scripted menu driven config tool to make it happen. Last I heard from him was that he was getting devices to connect, but was not getting the sound system to push through bluetooth. Something to do with pulseaudio or something like that. I don't know the details exactly.

In the future, I will try to steer these fellows to this forum, as the more eyes that see problems, often the solutions come from someone's arcane knowledge of different things that we don't ourselves understand.

I hope everyone is doing well, I just returned from a trip to Germany with my children who are half German. It was my first time in the old world, and I found it enchanting. As a lover of the countryside and hater of cities, Germany just reinforced that for me. I spent some time west of Poland and east of Dresden in the countryside, then traveled to Stralsund. The further away I am from the larger parts of civilization the better I feel regardless of the country.

I had a very kind offer from John Peacocke (who has contributed in this forum on many occasions) to come visit in Ireland when he corresponded with me about some interesting observations regarding ultrasound and it's potential applications to the device as an alternate form of input for different people who may be mobility impaired and can't use the rotary knob or keyboard.

Unfortunately, my visit was coming to a close, and Ireland was not in the cards this time. However, as my children age, I have more heritage to explore including Finn, Swede, Irish, and Welsh. So I hope I can meet some forum members on my future trips!

My best to all this summer; keep cool, and be well.

[Gavinmc42]

I only just noticed this post.
Time of Flight sensors are getting cheaper.
A search for ToF will find the post and links.

Basically it is an optical sensor that detects how long a laser beam takes to be reflected back.
Off the shelf from Adafruit etc
https://www.adafruit.com/product/3317
Not much better than a stick but the next gen versions are multi region, not just one spot.
This one goes to 4meters and 16 spots
http://www.st.com/content/st_com/en/pro ... l53l1.html

You can get up to 320x240 ToF cameras, smaller than Kinect.
But doing everything on a smart phone make for a pure software solution.

[scottpete]

Hi, all. Sorry for nosing in on this conversation. I started on pages 1 and 2 then skipped to the end and have a lot of catching up to do. I am very excited to read what has been going on over the course of time. Here is my background:

I am an ophthalmologist in the US who treats retinal diseases and have a significant proportion of my patient base with visual impairment.

Discussing technology solutions for low vision was often frustrating due to the high cost of, in my opinion, primitive CC TV-type devices being >$1000-$2500 US. For elderly patients with fixed incomes this is a non-starter. Researching online has led me here and I have now a few pi3 with camera modules to begin prototyping, but progress has been slow. I have also just started to learn coding (I am admittedly a dilettante and novice). My goal is to create devices with a price point of $50-100. much in line with what is going on here. As a newbie my abilities are severely limited, but growing. I am following this thread to try to keep up with this very robust discussion for the benefit of my patients and friends. Thank you all for your expertise and work - hopefully I can be a contributing member to the discussion in the future as my experience allows. Thanks!

Scott

[seeingwithsound]

Hi Scott,

Welcome to this long-running thread! Posts have been infrequent lately, but the quest for effective and affordable solutions continues. I'm very pleased to find an ophthalmologist here, which demonstrates that you are open to considering alternative and multi-disciplinary approaches. BTW, I'm not an expert on Raspberry Pi usage, and others here know much more about that part. My focus is more on Android and Windows options.

Regards,

Peter


The vOICe for Android on Smart Glasses
http://www.seeingwithsound.com/android-glasses.htm

The vOICe Web App
http://www.seeingwithsound.com/webvoice.htm

[mikey11]

Hi Scott,

I don't mind answering questions that can help you, but I can't devote a lot of time right now.

I have not been contributing much lately, as I have two young kids that take up my spare time right now. I expect this to change eventually as they get old enough for school.

The device we have made is not necessarily for those with low vision so much as for the blind or severely vision impaired.

The impression I get it that you would like to take a camera, zoom in, and then redisplay the zoomed portion.

In terms of keeping the costs down, the raspberry pi is capable of helping you reach your goal without a doubt. The camera module for the pi is OK, but it has a narrow field of view. something like 65-80 degrees whereas a wide angle lens will get 120+ degrees.

You can buy wide angle lenses for the raspberry pi module, or you could use another webcam that has a wide angle lens built in. That is what we did for this project (though the smaller field of view cameras also work). You may not require a wide field of view for your task. If so, the camera module is a good choice.

It sounds like you would require a display as well though. This is where things would get expensive in my opinion. Not only because displays are expensive, but because they tend to eat battery life like crazy.

The device we made does not have a display, and relies on an audio interface only.

In any case, feel free to post to the forum, and I will help where I can.

[johnf]

Hi, all. Sorry for nosing in on this conversation. I started on pages 1 and 2 then skipped to the end and have a lot of catching up to do. I am very excited to read what has been going on over the course of time. Here is my background:

I am an ophthalmologist in the US who treats retinal diseases and have a significant proportion of my patient base with visual impairment.

Discussing technology solutions for low vision was often frustrating due to the high cost of, in my opinion, primitive CC TV-type devices being >$1000-$2500 US. For elderly patients with fixed incomes this is a non-starter. Researching online has led me here and I have now a few pi3 with camera modules to begin prototyping, but progress has been slow. I have also just started to learn coding (I am admittedly a dilettante and novice). My goal is to create devices with a price point of $50-100. much in line with what is going on here. As a newbie my abilities are severely limited, but growing. I am following this thread to try to keep up with this very robust discussion for the benefit of my patients and friends. Thank you all for your expertise and work - hopefully I can be a contributing member to the discussion in the future as my experience allows. Thanks!

Scott

Hello, Scott, and hopefully a good New Year.
There are many facets to The vOICe, as well as a great diversity of potential users.
You must come across traumatically blind , gradually blind, all sorts of people. Some will be passive, others really motivated.
If you dig into The vOICe download and, with a laptop, try out all of the drop down menu options you will get an idea of its potential.
The Raspberry-Pi adds the dimensions of screen reading.... in fact whatever a cellphone might offer could be made available.
You should scrutinise the ability of a webcam to maintain focus down to inches. Inspect and compare textures and colours of fabrics,
check the colour of fruit and stuff in glass jars, all sorts of domestic tricks.
This is awkward with spectacle mounted cameras to say the least.
However, the Pi has both USB (normally connected to head cam) and its own ribbon camera interface
The latter can be adapted with a TINDIE product to HDMI (Misuse of HDMI ) adaptor kit , giving a quite robust realisation of a finger probe camera for really close sensing , finding things under furniture, and so on. I used a cigar tube and it worked.
So, with camera source switchable and some anti-crash medicine a Pi user could take advantage of both existential and precision perception. johnf.

[Gavinmc42]

A couple of new sensors are out now.
They seem to be optical mouse technology with higher resolution.

Basically 60x60 pixel cameras which detect optical flow.
Pixart and Broadcom are two I am aware of.
Normally use for gesture sensing, the Pixart one found a use in a drone to keep it from drifting from it's hovering spot.
https://www.bitcraze.io/flow-breakout/

As a bonus you get the laser range finder.
The APDS-9500 from Broacom is similar
https://www.broadcom.com/products/optic ... /apds-9500
The image output from both looks to be SPI master, which means the Pi needs to be a slave.
But the on chip processed data is available via i2c.

Mounted on a pair of glasses, when head is stationary they would detect moving object.
By scanning with the head they would then detect stationary objects as well.
2x2 binning gives a 30x30 image.

Not going to be the same as a Pi3 running ML based object recog but could at least be used to walk around without dumping into things
Could also be used as a aiming sensor for the Pi3 camera, as in I am looking at something - what is it?

[johnf]

Hello, Folks...
Pimoroni have come up with a Pi Zero with official Pi camera connector.. at a very low price.
This unit is now shaved down as thin as possible.. with a single core processor.
So maybe a lightweight head mounted basic vOICe without extras could be realised?
john

[gordon77]

Hello, Folks...
Pimoroni have come up with a Pi Zero with official Pi camera connector.. at a very low price.
This unit is now shaved down as thin as possible.. with a single core processor.
So maybe a lightweight head mounted basic vOICe without extras could be realised?
john

Can you provide a link to this?

[PiGraham]

Hello, Folks...
Pimoroni have come up with a Pi Zero with official Pi camera connector.. at a very low price.
This unit is now shaved down as thin as possible.. with a single core processor.
So maybe a lightweight head mounted basic vOICe without extras could be realised?
john

Can you provide a link to this?

This is surely just a regular Pi Zero / ZeroW

[PiGraham]

The camera module for the pi is OK, but it has a narrow field of view. something like 65-80 degrees whereas a wide angle lens will get 120+ degrees.

You can buy wide angle lenses for the raspberry pi module, or you could use another webcam that has a wide angle lens built in. That is what we did for this project (though the smaller field of view cameras also work). You may not require a wide field of view for your task. If so, the camera module is a good choice.

Pimoroni sell a wide angle camera module that connects direct to the Pi Zero CSI port.
https://shop.pimoroni.com/products/rasp ... 3069416458

5MP sensor, 2592x1944 pixels (same as version 1 official Pi camera)
1080p video at 30 FPS (or 60 FPS at 720p, 90 FPS at 480p)
f2.9 lens, 3.60 mm focal length
53.50 degrees horizontal, 41.41 degrees vertical field of view (standard and no IR filter versions)
Approx. dimensions of circuit and camera: 60 x 11.4 x 5.1mm

UPDATE: now available with a 120° wide-angle lens!
UPDATE: now also available with a 160° wide-angle, variable-focus lens! You can adjust the focus on this lens to focus reaaaaaally close!!

Are those too wide?

[gordon77]

Hello, Folks...
Pimoroni have come up with a Pi Zero with official Pi camera connector.. at a very low price.
This unit is now shaved down as thin as possible.. with a single core processor.
So maybe a lightweight head mounted basic vOICe without extras could be realised?
john

Can you provide a link to this?

This is surely just a regular Pi Zero / ZeroW

That was my assumption.

[ctrlw]

Discussing technology solutions for low vision was often frustrating due to the high cost of, in my opinion, primitive CC TV-type devices being >$1000-$2500 US. For elderly patients with fixed incomes this is a non-starter. Researching online has led me here and I have now a few pi3 with camera modules to begin prototyping, but progress has been slow. I have also just started to learn coding (I am admittedly a dilettante and novice). My goal is to create devices with a price point of $50-100. much in line with what is going on here.

I realise this was asked one year ago, but you could follow my description of a video magnifier at http://www.fhack.org/2018/12/19/raspber ... fier-2018/

Other interesting projects are a Raspberry Pi soldering magnifier (including image streaming over Wifi!) at https://www.instructables.com/id/Raspbe ... icroscope/ and a project with netbook and boroscope instead of Pi: https://www.instructables.com/id/Readin ... y-Sighted/

[blindzagm]

Ok, I had to pass my raspberry pi 2, that was running a version of the after-sight/raspiVoice implementation on it on to another person, to let them try out the concept, etc., and, I now would like to get it up and running on my raspberry pi 3 again, since I still have the technaxx video camera sunglasses here, but, the version I pulled off after-sight.com doesn't seem to be compatible = ASM1Piv3wUpdaterFeb7th2017.rar - and, if I install other OS'es on this unit, it does boot up, and handle speech output, etc., but, with this unit, while I can hear that the earphones are getting powered up, nothing else seems to happen. Unfortunately, I don't currently have any sort of display I can hook up to this unit, to try get more info, but, thought would ask if that .img file is indeed the correct OS instance for the pi 3?

Thanks

Jacob

[blindzagm]

And, yes, just to confirm, I did extract the .img file from that archive, and, tried writing it to the micro SDCard using both win32 disk imager, and winFlashtool - same result with both

[mikey11]

When that file was created, it was with the very first release of the Raspberry pi 3.

As time goes on, hardware revisions might be to blame here, I can't say for sure. These days, the only raspberry pi I can even get locally is the 3B+ and I haven't tested that image file on the 3B+.

Attaching a display will likely not help much. I always use SSH sessions via network to interact with the device for pretty well any purpose.

Unfortunately, I have not had time nor inclination to continue working on this in a concerted fashion. I have closed down the after sight not for profit corporation, and have been keeping the after-sight.com site up just to preserve links to the image files for people such as yourself.

I do hope one day to return to this, and I hope that by the time I do, the raspberry pi zero will be capable of running the vOICe at a reasonable speed.

My goal at that time will be to make a wireless device that connects to bluetooth headsets and uses the raspberry pi camera module with a wide angle lens.

My problem with the larger boards has been the extra wires, and the large size.

moving to the pi zero sized board makes it smaller, using the camera module eliminates compatibility issues for video, and bluetooth for audio gets rid of wires completely (except for a charging cable I guess).

If I do get the chance to work again on this, I will of course be posting here first.

[blindzagm]

Ok, I will possibly try hook it up to a PC using something like a LAN cross-over cable, to see if can SSH into it - on that note, what would the default root password be?

Besides that, no worries, and, understand that you haven't had time to carry on with this effort right at the moment, but, know that we do really appreciate your efforts, and, the amount of work you already put into it - thank you.

Jacob

[mikey11]

pretty sure the username is 'pi' and the password is 'raspberry'

[Mario_NL]

Question for mikey11,

Which version Raspberry Pi 2 board did you make the V2 image for?
Was it the V1.1 or V1.2 ?

I ask because I don't want to make the same mistake as I did with purchasing a V3B+ and after that find out the V3 image is not working for the "+" version.

Thanks in advance,

Mario

[mikey11]

Hi Mario,

I'm fairly certain that for the RPI V 2, that either was acceptable.

I hope you have some success with it.