mikey11
Posts: 355
Joined: Tue Jun 25, 2013 6:18 am
Location: canada
Contact: Website

Re: Sight for the Blind for <100$

Tue Sep 29, 2015 4:11 pm

I have ordered single quantities but have not received anything yet.

Just confirmed my PCB order. This time it will be professional grade with the masking and legend.

This will make collaborative hardware troubleshooting easier.

I am always looking for lower priced glasses, but haven't had much luck lately.

I currently need seven pairs.

mikey11
Posts: 355
Joined: Tue Jun 25, 2013 6:18 am
Location: canada
Contact: Website

Re: Sight for the Blind for <100$

Fri Oct 02, 2015 8:15 pm

Good news,

The newest PCB revision arrived, and everything is working as expected. I have placed an extra spot to test out an alternative power switch, but I am so happy with the case mounted one that I don't think I will pursue this.

The Major change to this revision is that the horizontal sliding power switch was removed, and three larger diameter holes were made for the wires from the case mounted switch, and a pad of six holes were placed for the power switch I am probably no going to test out (a board mounted pushbutton switch).

In addition, an additional logic line was added from the power module to the raspberry pi which monitors the state of the external battery power plug.

I have modified the rangefinder2.py code to include notes for this connection. When run from the command line it prints the state to the screen. If I get time, I will make a change of state at the plug trigger an espeak of the state change.

Code: Select all

import time
import RPi.GPIO as GPIO
import serial
import datetime
from subprocess import call

time.sleep(5)
GPIO.setmode(GPIO.BCM)  #setup for PWM
GPIO.setup(20, GPIO.OUT)   #Define pin 20 as output
p = GPIO.PWM(20, 25)  #channel = 20 , frequency = 25 hz
p.start(5) #5% duty cycle to start. should immediately change as the rangefinder reads
GPIO.setup(27, GPIO.IN, pull_up_down=GPIO.PUD_UP)
GPIO.setup(25, GPIO.IN, pull_up_down=GPIO.PUD_UP)
GPIO.setup(9, GPIO.IN, pull_up_down=GPIO.PUD_DOWN)
GPIO.setup(10, GPIO.IN, pull_up_down=GPIO.PUD_DOWN)
maxbotix = serial.Serial("/dev/ttyAMA0",baudrate=9600,timeout=5) # other examples that didnt specify baudrate and timeout did not work for me
time.sleep(0.5)
t1=0
t2=0
t3=0
t4=0
vibration=-1
timesinceflip = 0
while 1:
    maxbotix.flushInput() #clear buffer to get fresh values if you don't do this, you won't get responsive readings
    currdistance = maxbotix.readline(10) #Take ten characters worth of the serial buffer that accumulates since the flush
    #print currdistance # comment out after debugging
    stripstart = currdistance.find("R") #Look for the character "R", find out where it occurs in the string first
    stripend = stripstart + 5 #Define the end of the character length we need to grab the numeric info
    currdistance = currdistance[stripstart+1:stripend] #strip out the numeric info
    #print currdistance #comment this out after debugging
    currmm = float(currdistance) #Now make the info a number instead of a string
    #print currmm #comment this out after debugging
    battstate = GPIO.input(27)
    switchstate = GPIO.input(9)
    externalpowerstate = GPIO.input(10)
    if (externalpowerstate == 1):
        print ('External Power Connected')
    if (externalpowerstate == 0):
        print ('External Power Disconnected, Running from Internal Battery')
    if (switchstate == 1):
        print ('Power Switch Turned Off, System Shutdown Initiated')
        call (["sudo", "espeak", "shutdown"])
        call (["sudo", "shutdown", "-h", "now"])
    if (switchstate == 0):
        print ('Power Switch Is On, Keep Working')
    if (battstate == 1):
        print ('Battery OK, keep system up')
    if (battstate == 0):
        print ('Battery Low, System Shutdown')
    if (vibration == 1):
        #print "vibration on"
        if (currmm > 0 and currmm < 1000):
                p.ChangeFrequency(5)
                p.ChangeDutyCycle(50)
        if (currmm > 1000 and currmm < 2000):
                p.ChangeFrequency(4)
                p.ChangeDutyCycle(50)
        if (currmm > 2000 and currmm < 3000):
                p.ChangeFrequency(3)
                p.ChangeDutyCycle(50)
        if (currmm > 3000 and currmm < 4000):
                p.ChangeFrequency(2)
                p.ChangeDutyCycle(50)
        if currmm > 4000:
                p.ChangeFrequency(1)
                p.ChangeDutyCycle(50)
    elif (vibration == -1):
        p.ChangeDutyCycle(0)
        #print "Vibration Off
    if GPIO.input(25):
        #print('Button Released')
        #print vibration
        t1=0
        t2=0
        t3=0
        t4=0
    elif t1 == 0:
        t1 = time.time()
    elif (t1 > 1 and t3 < 3):
        t2 = time.time()
        t3 = t2 - t1
        print ">1<3",t3
    elif (t3 > 3 and t3 <4):
        print ">3<4",t3
        vibration = -vibration
        t3 = 5.1
        t4=5.1
        #print "Vibration Flipped"
        #print "=5.1",t4
    elif (t4 > 4 and t4 < 7):
        t2=time.time()
        t4 = t2+1 - t1
        #print ">4<7",t4
        #print "between vibration change and shutdown"
    elif t4 > 7:
        p.ChangeDutyCycle(0)
        print "shutdown",t4
        #call (["sudo","killall","raspivoice"])
        #call (["sudo","killall","espeak"])
        #call (["sudo", "espeak", "Shutdown"])
        call (["sudo", "shutdown", "-h", "now"])
        call (["sudo", "espeak","Shutdown"])
        exit
I have verified this code as working.

This was also the first time I got the professional solder mask and legend printed on the board, and it looks great. I am going to clean up the legend so that the relevant values are printed on the board (if this can be done easily? I don't know yet)

I am finally happy with the hardware situation, and I have only one or two small tweaks to the case and it will be finished for the moment. When the case tweaks are done, the repository will go up.

With the hardware and case design drawing to a close, I will reorient myself on the software side of things, getting teradeep integrated and implementing a better menu system for the rotary encoder.

PranavLal
Posts: 127
Joined: Fri Jun 28, 2013 4:49 pm

Re: Sight for the Blind for <100$

Fri Oct 02, 2015 11:33 pm

Hi Mikey11,

Great news and thanks for the hardware modification. I have been thinking about the range finder. I was walking on the road with the prototype when a motorbike drove by me. I was on the side of the road and the bike was going quite fast. I had the vOICe at the one frame per second speed so did not catch it. Yee, my brain needs more training. Yes, my ears did tell me that something was approaching. However, I wonder if it is worth broadening the beam of the sensor to pick up this kind of information. I would like to know if I am too close to that bike. I do not have an easy answer to this question but would like everyone's opinions.

Pranav

mikey11
Posts: 355
Joined: Tue Jun 25, 2013 6:18 am
Location: canada
Contact: Website

Re: Sight for the Blind for <100$

Sat Oct 03, 2015 5:16 am

For the rangefinder, there are a few different models, and I'm not even 100% sure of which model has shipped with which unit, and there is no easy way to tell.

each model has slightly different beam characteristics ranging in width. I think the best way to resolve this is ship an actual user two versions of the device; one with the widest width, and one with the most narrow.

My own observations are this:

1. wide width can be good for outdoors, but completely renders the function useless indoors.
2. narrow width can be used outdoors and indoors. The indoor function is still not great, but is good for certain types of buildings, especially those like museums or public buildings that are not crowded.


Having played with these devices before this project, I know there is the possibility to take a wide width sensor, and by a trick of placing a cylinder about the dimensions of half a soda can around the sensor, you can attenuate the beam width.

Think of it kind of like a photographer changing lenses from a wide angle to a zoom lens.

It would add parts and complexity, and may in the long run be worthwhile.

PranavLal
Posts: 127
Joined: Fri Jun 28, 2013 4:49 pm

Re: Sight for the Blind for <100$

Sat Oct 03, 2015 6:44 am

Hi Mikey11,
<snip Having played with these devices before this project, I know there is the possibility to take a wide width sensor, and by a trick of placing a cylinder
about the dimensions of half a soda can around the sensor, you can attenuate the beam width.
PL] Sounds interesting. I did not realize you could alter the angle of the beam in this way. I agree about a sensor with a wider width being useless indoors.

We can leave this for version 3 unless there is significant user demand. <smile

mikey11
Posts: 355
Joined: Tue Jun 25, 2013 6:18 am
Location: canada
Contact: Website

Re: Sight for the Blind for <100$

Sat Oct 03, 2015 7:23 pm

I have worked out some code to detect changes to the external power source. The behaviour is now when a plug is inserted or removed, espeak will indicate what has happened. This will only occur when the source has a high enough voltage. If the voltage is too low, it will not be detected.

I have also tried a new scheme for the vibration motor which changes the behaviour from its current mode. Essentially when objects are farther away, the frequency is higher. This leads to behaviour where when objects are closer, the vibration is more intense, and when farther away, less intense. This comes down to the momentumn of the vibration motor.

Anyways, maybe give it a go, and see what it feels like. I think you may like it better than the previous method.


rangefinder2.py:

Code: Select all

import time
import RPi.GPIO as GPIO
import serial
import datetime
from subprocess import call

time.sleep(5) #This sleep spot is meant to let the preamble espeak for raspivoice finish with less intrusion from this process.
GPIO.setmode(GPIO.BCM)  #setup for PWM
GPIO.setup(20, GPIO.OUT)   #Define pin 20 as output, for PWM modulation of vibration motor
p = GPIO.PWM(20, 25)  #channel = 20 , frequency = 25 hz
p.start(5) #5% duty cycle to start. should immediately change as the rangefinder reads
GPIO.setup(27, GPIO.IN, pull_up_down=GPIO.PUD_UP) #GPIO for detecting low battery
GPIO.setup(25, GPIO.IN, pull_up_down=GPIO.PUD_UP) #Rotary Pushbutton Input
GPIO.setup(9, GPIO.IN, pull_up_down=GPIO.PUD_DOWN) # GPIO for detecting Power Switch Position, used to shtudown system
GPIO.setup(10, GPIO.IN, pull_up_down=GPIO.PUD_DOWN) # GPIO for Detecting External Power State
maxbotix = serial.Serial("/dev/ttyAMA0",baudrate=9600,timeout=5) #Open a serial input to recieve from the maxbotix ultrasonic sensor
time.sleep(0.2)#This sleep interval is required to let the serial input open completely before it is read for the first time. Without this pause, random crashes on startup occur
t1=0 #t1-t4 used for timing pushbutton events
t2=0
t3=0
t4=0
vibration=-1 #vibration starts off. Changing this = 1 starts vibration based on distance measurement
timesinceflip = 0
oldexternalpowerstate = 0 # this variable enables an espeak event when the power plug is inserted or removed
while 1:
    maxbotix.flushInput() #clear buffer to get fresh values if you don't do this, you won't get responsive readings
    currdistance = maxbotix.readline(10) #Take ten characters worth of the serial buffer that accumulates since the flush
    stripstart = currdistance.find("R") #Look for the character "R", find out where it occurs in the string first
    stripend = stripstart + 5 #Define the end of the character length we need to grab the numeric info
    currdistance = currdistance[stripstart+1:stripend] #strip out the numeric info
    currmm = float(currdistance) #Now make the info a number instead of a string
    print currmm #comment this out after debugging
    battstate = GPIO.input(27)
    switchstate = GPIO.input(9)
    externalpowerstate = GPIO.input(10)
    if (externalpowerstate != oldexternalpowerstate):
        print ('External Power State Changed')
        if(externalpowerstate == 1):
                print ('External Power Connected')
                call (["sudo","espeak","ExternalPowerConnected"])
        elif (externalpowerstate ==0):
                print ('External Power Disconnected')
                call (["sudo","espeak","ExternalPowerDisconnected"])
    if (externalpowerstate == 0):
        print ('External Power Disconnected, Running from Internal Battery')
    if (switchstate == 1):
        print ('Power Switch Turned Off, System Shutdown Initiated')
        call (["sudo", "espeak", "shutdown"])
        call (["sudo", "shutdown", "-h", "now"])
    if (switchstate == 0):
        print ('Power Switch Is On, Keep Working')
    if (battstate == 1):
        print ('Battery OK, keep system up')
    if (battstate == 0):
        print ('Battery Low, System Shutdown')
    if (vibration == 1):
        #print "vibration on"
        if (currmm > 0 and currmm < 1000):
                p.ChangeFrequency(5)
                p.ChangeDutyCycle(50)
        if (currmm > 1000 and currmm < 2000):
                p.ChangeFrequency(15)
                p.ChangeDutyCycle(50)
        if (currmm > 2000 and currmm < 3000):
                p.ChangeFrequency(75)
                p.ChangeDutyCycle(50)
        if (currmm > 3000 and currmm < 4000):
                p.ChangeFrequency(150)
                p.ChangeDutyCycle(50)
        if currmm > 4000:
                p.ChangeFrequency(100)
                p.ChangeDutyCycle(50)
    elif (vibration == -1):
        p.ChangeDutyCycle(0)
        #print "Vibration Off
    if GPIO.input(25):
        #print('Button Released')
        #print vibration
        t1=0
        t2=0
        t3=0
        t4=0
    elif t1 == 0:
        t1 = time.time()
    elif (t1 > 1 and t3 < 3):
        t2 = time.time()
        t3 = t2 - t1
        print ">1<3",t3
    elif (t3 > 3 and t3 <4):
        print ">3<4",t3
        vibration = -vibration
        t3 = 5.1
        t4=5.1
        #print "Vibration Flipped"
        #print "=5.1",t4
    elif (t4 > 4 and t4 < 7):
        t2=time.time()
        t4 = t2+1 - t1
        #print ">4<7",t4
        #print "between vibration change and shutdown"
    elif t4 > 7:
        p.ChangeDutyCycle(0)
        print "shutdown",t4
        call (["sudo", "shutdown", "-h", "now"])
        call (["sudo", "espeak","Shutdown"])
        exit
    oldexternalpowerstate = externalpowerstate #This captures the current external power state to compare when the loop runs next. critical for knowing when power is plugged in or unplugged
I also added some more comments to the code.

I have not run a certain experiment yet, and would like to get an idea if people think it is worthwhile:

The low battery indicator works, and I can detect the state change when the low battery condition occurs. The problem is this:

If you plug in a pair of glasses that are not charged fully, the low battery indicator can come on prematurely, even when the whole system may have hours of life left.

Initially I wanted to force a shutdown when the low battery condition was indicated to protect the filesystem. Now I think what I would like to do instead is run a 10 minute timer when it is encountered, and then force a shutdown. This kind of behaviour would let a user keep using the system if it was a case of a low charge on the glasses, albeit they would have to reboot from time to time. While inconvenient, this is probably a good compromise between keeping the filesystem safe, and letting the user continue to use the device.

As it stands, I currently don't do anything except detect the state change and print it to the console.

PranavLal
Posts: 127
Joined: Fri Jun 28, 2013 4:49 pm

Re: Sight for the Blind for <100$

Sat Oct 03, 2015 11:29 pm

Hi Mikey11,

Many thanks for the updated script. Could you attach it as a file and send it to me? I am concerned about copy pasting it into Linux directly from a browser due to text wrapping issues which could upset python indentation.

Pranav

mikey11
Posts: 355
Joined: Tue Jun 25, 2013 6:18 am
Location: canada
Contact: Website

Re: Sight for the Blind for <100$

Sun Oct 04, 2015 1:23 am

I have created a github repository located here:

https://github.com/aftersight/After-Sight-Model-1.git

Right now the only file there is the updated rangefinder2.py. This will need to be installed in the home directory for the user pi.

I intend to fork the raspivoice code there as well to include the rotary knob debounce that Mikael completed. If you would like to become a collaborator, PM me your github username.

Thanks,

PranavLal
Posts: 127
Joined: Fri Jun 28, 2013 4:49 pm

Re: Sight for the Blind for <100$

Tue Oct 06, 2015 12:02 am

Thanks Mikey11. I have the file. I'll upload it and test. I do have an account on github and will send you my user name.

TypeRA
Posts: 1
Joined: Mon Oct 12, 2015 7:03 pm

Re: Sight for the Blind for <100$

Mon Oct 12, 2015 7:54 pm

Hi

I am a cognitive scientist with experience in visual perception and semantic representation.

I have a b+ new in box that I wish to use for this project. I've never used a Pi, can't program worth a damn, but I have access to a large blind population.

What can I do, what do I buy, and how can I help?

J.D.

PranavLal
Posts: 127
Joined: Fri Jun 28, 2013 4:49 pm

Re: Sight for the Blind for <100$

Tue Oct 13, 2015 12:04 am

Hi J.D.,

One of the biggest things we need is more blind people using the vOICe so your blind population should help there. I will leave Michael and or Peter to tell you about hooking the setup together and research possibilities.. You will not need to do any programming. There is an image which you burn on to the sd card of the pi. You then plugin a UVC compliant Camera, a pair of headphones, activate the pi and you are good to go. There is of course the custom board which you will need to attach to the pi but I will leave those instructions for Michael to give because he has designed it. There are however plenty of diagrams if you go back through this thread.

I am a user of the technology and am happy to help your people get up to speed with the vOICe.

Pranav

mikey11
Posts: 355
Joined: Tue Jun 25, 2013 6:18 am
Location: canada
Contact: Website

Re: Sight for the Blind for <100$

Tue Oct 13, 2015 2:46 pm

Greetings J.D.

First, a few critical questions:

1. Is your raspberry pi B+ version 2?

Version 1 raspberry pis are too slow. The new version has more cpu cores and the software runs with acceptable performance. I have tested the old versions and never got acceptable performance.

2. Where are you located?

I am in Canada. The closer we are the cheaper the shipping would be, and if you are in Canada, there are other practical reasons this would be ideal.


As Pranav said, there is a link in this forum to a working SD card image that can get your raspberry pi with a UVC camera and headphones running the software overnight.

The custom electronics are what I have been producing, along with software to use them.

The electronics include: a sonar rangefinder, battery, and vibration motor along with miscellaneous parts to make it all operate together. I also produce a case to house everything.

This makes the unit wearable and portable.

As for directions in the future, my observations are that of a layperson. Cognitive science is not my domain, and I believe that Peter Meijer or Michael Proulx would be better to consult. However, I will opine:

In a word, Statistics. There has never been a high enough population of blind users using visual to auditory sensory substitution at the same time to generate really valuable information and answer some really basic questions. Of course, I haven't read every paper and may be unaware of some of the progress out there. If a cognitive scientist wanted to answer some questions, I think I have a few good starting points:

1. On the device a resolution of 176x64 is used. Of this level of detail, how much detail can users actually learn to see? How much do new users see? How much do veteran users see? How can this be tested?

2. Are there relevant personal characteristics that better enable people to achieve a high level of understanding of sensory substitution? Does sex matter? With women have more interhemisphere communication, I suspect they may do much better in the early learning phases. How does age affect learning? I suspect children may do well, and perhaps with those who just lose their vision, if they start sensory substitution as the transition time, the reorganization of the brain that occurs may aid in acquiring the sensory substitution understanding skills.

3. How does hearing damage affect skills acquisition? I used to stand feet away from punk rock drummers when I was younger. I have deficits in my high and mid frequency hearing. I find that I can't understand what people say if there is any significant peripheral noise. I suspect this would hamper my ability to use the technology. How much though? Can people be screened for suitability by a hearing test? Can we quantify or even put people into bins that give us the ability to meaningfully recommend the technology or not recommend it?

My guts tells me that an old male with hearing damage would not be a good candidate, but I would rather get a large population, train them, and then do the stats. My gut feeling is irrelevant.

I see the next logical step being producing 100 devices and getting a researcher or three to work with 100 visually impaired people to start answering these questions.

As I say, I am a layperson in these areas, and may be unaware of academic progress on the questions I have posed.

If you like, I can send an assembled custom electronics board, battery, and case. This would get you mobile operation to evaluate the device. Send me a personal message to make arrangements.

We recommend using the technaxx tx-25 video glasses. These are available from a company in Germany. Any UVC usb camera will work though (in theory). I can't afford to provide glasses as I am currently providing everything out of pocket. If you want cheaper glasses, there has been some mixed results searching for the Chinese producer of the tx-25's. I have on one occasion got a set for $95, and a recent order I placed was for $66/each(has not arrived yet). However, my success rate with Chinese suppliers is not good. About half the orders I make never arrive. For this reason, I can only recommend ordering from technaxx in germany.

I guess what is important is finding out how you want to contribute, and what you would like to achieve? Maybe as you get more exposure here and using the device this will become more clear over time.

User avatar
seeingwithsound
Posts: 165
Joined: Sun Aug 28, 2011 6:07 am
Contact: Website

Re: Sight for the Blind for <100$

Tue Oct 13, 2015 7:53 pm

TypeRA wrote:What can I do, what do I buy, and how can I help?

J.D.
Hi J.D.,

Several good answers have been given already by Pranav and Michael. You can find scientific publications based on use of The vOICe at http://www.seeingwithsound.com/literature.htm ; that list includes several cognitive science papers (one of those actually came out just today!).

Most valuable would be to arrange for training of blind people in using The vOICe. A starting point can be the training manual at http://www.seeingwithsound.com/manual/T ... Manual.htm

Since I know nothing about you short of what is in your introduction message, it is hard to make further suggestions.

Best regards,

Peter Meijer


Seeing with Sound - The vOICe
http://www.seeingwithsound.com

mikey11
Posts: 355
Joined: Tue Jun 25, 2013 6:18 am
Location: canada
Contact: Website

Re: Sight for the Blind for <100$

Wed Oct 21, 2015 1:28 am

I have shipped four units to the group in Russia. Some of these were the latest PCB revision and code revisions which now indicate when external power is plugged in/unplugged, and includes a modification to the rangefinder/vibration behaviour to make the signal more intense with reduced distance.

It goes from a diffuse buzz to an insistent vibration now.

The hardware on the latest versions is much nicer than the older case models. There are rounded edges, a robust and reliable power switch, and slight adjustments to the electronics to make everything have a much more perfect fit.

I am syncing the latest filesystem image. Unfortunately, the image size will now require an 8GB SD card. I made the decision to move up from 4 to 8 GB based on preliminary work on getting teradeep/deepbelief operating. Installing either of these software packages required the step up. Given that this is one of the next logical steps in development, it is inevitable.

Perhaps one day, someone will build everything from the ground up in a minimized distribution to keep in on 4GB cards, but I suspect that this will be a move of diminishing returns as I have a difficult time even finding 4GB cards for sale anymore.

With the hardware looking better and being much more reliable, I am going to re-devote myself to getting object recognition implemented.

I also plan on implementing tesseract OCR to read road signs / printed media. I have no experience with this yet, but I have watched a few videos. It looks very time consuming on dense text such as a book (a few minutes to process an image). I am interested to see how it operates on simple text and larger fonts such as road signs.

I have recently done a costing of the units and come up with a value near $220 CAD. This seems like a large amount until you consider that the CAD has been taking a beating on the foreign exchange. In USD it's ~$170.

These figures reflect single quantities. On my next break from work, I will be consulting with a few providers of electronics to see what kind of volume discounts I can achieve on quantities of 100 and 1000. I will also try to do the costing myself to see if I can do better than the electronics suppliers. I often lose because of shipping when I try to do this type of exercise.

If anyone has had progress on the object recognition front, I would love some install guides for teradeep/deepbelief so I can get a new filesystem image with one of them operating.

mikey11
Posts: 355
Joined: Tue Jun 25, 2013 6:18 am
Location: canada
Contact: Website

Re: Sight for the Blind for <100$

Wed Oct 21, 2015 2:48 am

I have received my 'Orange Pi' board which claimed to be pin compatible with the RPi.

I did not find this to be the case.

It may well be GPIO compatible, but the power pins and power management circuits are not compatible. This renders my custom PCB design useless.

While I do want to find a cheaper board than the RPI V2, the Orange Pi will not be the one. The reason I say this is I looked further into the performance based on computation vs. energy consumption and found that the best candidate which is still price competitive is the intel edison. The Orange pi, and the bannana pi are just too close in specs.

The intel edison is a different chip architecture, so there would be loads of work to do with it as well, but I think the efforts would be best served in that direction.

To me, being able to run on both ARM and x86 systems would be a good capability to have, and Intel is generally a few generations ahead on chip fab feature size (and thus power consumption).

Ultimately, getting some of the desired features is looking like it will require more horsepower than the ARM single board computers are offering.

For instance, the ability to run the vOICe at the same time as an object recognition app looks discouragingly impossible on ARM systems at the moment.

I delved into the deepbelief story a bit, and found that one guy has basically done a very thorough optimization using the GPU, and gets results only once every three seconds.

I'm left with the feeling that some of the more advanced features a person would want may have to wait until the performance/watt ratio improves by an order of magnitude.

Or until dedicated coding geniuses want to start working on a charitable effort.

User avatar
seeingwithsound
Posts: 165
Joined: Sun Aug 28, 2011 6:07 am
Contact: Website

Re: Sight for the Blind for <100$

Wed Oct 21, 2015 3:52 pm

Good work, Mike! I'll try to support the Russian team from my side where I can. Later this week they will be presenting about The vOICe at TechTrends Expo in Moscow http://techtrendsexpo.com. It may also be interesting to mention that an exciting new scientific publication based on use of The vOICe appeared just this week in Nature Scientific Reports: 'Visual’ parsing can be taught quickly without visual experience during critical periods http://www.nature.com/articles/srep15359, written by researchers from Amir Amedi's neuroscience group at the Hebrew University of Jerusalem.

Thanks!

Peter


Seeing with Sound - The vOICe
http://www.seeingwithsound.com
Last edited by seeingwithsound on Sat Oct 24, 2015 7:40 am, edited 1 time in total.

User avatar
seeingwithsound
Posts: 165
Joined: Sun Aug 28, 2011 6:07 am
Contact: Website

Re: Sight for the Blind for <100$

Sat Oct 24, 2015 7:35 am

Returning to viewtopic.php?p=813463#p813463
mikey11 wrote:https://drive.google.com/file/d/0B30mp6 ... sp=sharing

New filesystem image with the debounce integrated
Hi Mike, what happens if people flash this image on a vanilla Raspberry Pi 2 device without the custom rotary knob, sonar device and other custom features? Does it break or degrade gracefully? I ask because sometimes people already have a Raspberry Pi lying around and first want to give raspivoice a quick try with what they have, if possible. If things do not degrade gracefully now then it may be something to strive for in the future.

Thanks,

Peter

P.S. BTW, there's even a second paper this week based on use of The vOICe in Nature Scientific Reports! It is titled "Auditory sensory substitution is intuitive and automatic with texture stimuli" http://www.nature.com/articles/srep15628, written by Noelle Stiles and Shinsuke Shimojo of Caltech.

mikey11
Posts: 355
Joined: Tue Jun 25, 2013 6:18 am
Location: canada
Contact: Website

Re: Sight for the Blind for <100$

Sat Oct 24, 2015 1:22 pm

I have just tested what you described.

A Rasbperry Pi V2 with no electronics board, with glasses, powered through the normal power port on the rpi board works fine.

I guess that is to say it degrades gracefully.

Only caveat here is that the image will only work on the V2 pi's because of the change in processor means all the compiled stuff won't be backwards compatible with V1 pi's.

This shouldn't be too much trouble though, people can get the vOICe going for very cheap just to mess about with it.

It's lovely to see some higher profile publications happening too.

User avatar
seeingwithsound
Posts: 165
Joined: Sun Aug 28, 2011 6:07 am
Contact: Website

Re: Sight for the Blind for <100$

Sat Oct 24, 2015 1:31 pm

mikey11 wrote:I have just tested what you described.

A Rasbperry Pi V2 with no electronics board, with glasses, powered through the normal power port on the rpi board works fine.

I guess that is to say it degrades gracefully.

Only caveat here is that the image will only work on the V2 pi's because of the change in processor means all the compiled stuff won't be backwards compatible with V1 pi's.
Excellent, and thank you for trying! I'll use this in various communications.

Peter

mikey11
Posts: 355
Joined: Tue Jun 25, 2013 6:18 am
Location: canada
Contact: Website

Re: Sight for the Blind for <100$

Sat Oct 24, 2015 2:34 pm

After reading the crossmodal/intuitive paper, I imagine this kind of research is really valdating for how you chose the conversion algorithm to operate.

The left to right and bright to loud setup was superior to the other variations.

I can also see how a v2 raspberry pi is pretty much a commodity device. If we were able to find widely distributed UVC video glasses, researchers could get cheap equipment for research.

PranavLal
Posts: 127
Joined: Fri Jun 28, 2013 4:49 pm

Re: Sight for the Blind for <100$

Sun Oct 25, 2015 2:39 am

Hi all,


I am back from my Thailand vacation where I can report that my unit performed flawlessly in terms of rendering visual input. We were next to the beech and I took the unit with me out on to the sand and very close to the waves. All in the interest of good photography and getting more pictures. There were no problems whatsoever. I did not get any strange looks and there was almost no problem with customs. My 50000 MAH battery pack was a disappointment. I had the pi connected to it. I felt the pack and USB connect heat up but I still got 4 hours of battery life per charge. I do not have a talking multimeter to debug what is happening.

The careful reader may have noticed my use of the word "may" when talking about customs. When exiting Thailand, my 50000 MAH battery pack was confiscated by customs. It appears that only battery packs of 32000 MAH are allowed. Yes, it is strange, why was I allowed to bring the pack in in the first place? Moral of the story, take high capacity battery packs as checked in baggage. I have a spare so this is not a significant problem.

I saw plenty of shops, more than my fair share of plants, a temple or two, the outside of a lot of nightclubs, photograph my dad sitting on a chair on demand etc. Photographs will follow once I sort them. The bone conduction headphones were a disadvantage. Stand on a road with people, vehicles and music blaring from speakers and you want regular headphones because it is almost impossible to hear the vOICe. <chuckle Store fronts are interesting especially when they are clustered. I was in a van so did plenty of looking out its side window. It was hard to tell what I was looking at except that there were a lot of lit objects. Shacks are also interesting structures. You can usually tell them apart by looking for 2 vertical lines on either side of the image. The roofs are worth looking at because they are made of thatch and are not uniform which makes for varying soundscapes.

I was also able to navigate between 2 service apartments by looking out for the alcove where their doors were. I was with my cousins and we were in 2 apartments that were next to each other. There was an insect infested bit of wall between them which had windows too. My job was to distinguish between the windows and the door on the side while continuing to look ahead because you could have things like cleaning carts in the way. The wide angle of view really helps in these situations. I was able to find the door by the softening of the texture and the door was a darker more uniform surface.

The other thing was that I did the head tilt for identifying faces. This was during a Thai massage. I was trying to look around while being flat on my back. I was better able to see the features of the masseur. I found it easier to differentiate between different components of her face or so I think. This will need more testing.

Pranav

mikey11
Posts: 355
Joined: Tue Jun 25, 2013 6:18 am
Location: canada
Contact: Website

Re: Sight for the Blind for <100$

Thu Oct 29, 2015 9:04 pm

Hi Pranav,

I'm glad your machine worked well enough for you on your vacation, and I am eager to ship a new model to both you and Peter, as I have made major improvements.

I can't say what would be happening that you would only get 4 hours from the external battery, but I made a discovery today that should help immensely in the area of power management. The power management module was always irritating in that it would sometimes come on to indicate low battery even when it wasn't the case. This would happen if glasses were not fully charged, even if the main battery in the device was charged.

I ran a device to exhaustion today, and noticed something I had not noticed before as my previous tests to exhaustion were always initiated before I left for extended periods of time (day job). Today though, I got to observe it, and the power module actually will begin flashing the low battery indicator when the voltage gets critical instead of just keeping the low battery light going all the time.

All I need to do now to get proper management is look for the blinking as opposed to the constant signal.

The big problem with lithium polymer batteries is that if they get completely run down, they become damaged and lose capacity (or so my learning on google tells me). It's possible that the lack of power management has damaged your battery, but be assured that in the very near future, I will have a routine written that will not let this happen again.

I never saw that this behaviour was covered in the documents about the power management module, or I just missed that detail.

So to summarize, on the newest PCB (which neither Pranav or Peter has yet), you will get notifications of external power being plugged in or removed.

Once the code is rewritten for the low power indicator (which will catch the flashing bahaviour, I will force a shutdown at the time and give an audio indicator as to why such as "low battery shutdown". This should work on the older versions if I remember correctly.

I am waiting on a few supplies to arrive, and I will get new hardware versions out to Peter and Pranav once they have arrived. I think you will mostly like the improvements.

User avatar
seeingwithsound
Posts: 165
Joined: Sun Aug 28, 2011 6:07 am
Contact: Website

Re: Sight for the Blind for <100$

Thu Oct 29, 2015 9:34 pm

Hi Mike,

Thanks for all the great work! There are many reports on the web that cheap Chinese battery packs often have a much lower capacity than claimed by the vendor (even down to 10%!), so that could have been another factor in Pranav's shorter-than-expected battery life.

Peter

PranavLal
Posts: 127
Joined: Fri Jun 28, 2013 4:49 pm

Re: Sight for the Blind for <100$

Thu Oct 29, 2015 11:43 pm

Hi Mikey11 and Peter,

Many thanks for the explanation and hard work. I suspect Peter is right about my battery pack having lower capacity as the ODROID ran on it for only 12 hours or so.

I am excited about the new units and look forward to them.

Pranav

mikey11
Posts: 355
Joined: Tue Jun 25, 2013 6:18 am
Location: canada
Contact: Website

Re: Sight for the Blind for <100$

Fri Oct 30, 2015 5:16 am

My patience for getting an object recognition program to run is wearing thin. I've followed what install guides I can find, but can't seem to get deepbelief or teradeep going.

With deepbelief, I get an error about functions being previously defined in buffer.o, and this prevents jpcnn from compiling.
With teradeep, I can't even remember the errors I've got, as it's all a haze from the numerous attempts.

So, to let my patience rebuild I am going to try a smaller, easier goal: implement tesseract OCR with espeak and add it as a menu option.

I've seen enough out there to believe I should at least have tesseract speaking the words of images without too much trouble. I just hope that the time for computation is low. I've heard a page of text can take on the order of minutes, but these quotes were all based on the old raspberry pi with the single core. I'm hoping its been optimized if extra cores are available. Given that it has longstanding support from google, I think thats a good guess.

I am hoping that a street sign takes just a moment or two. Then hopefully some less complicated things like a sign at a bus stop with a schedule take make a few more seconds, and maybe a restaurant menu will only take 15-20 seconds... A guy can dream anyways.

Return to “Assistive technology and accessibility”