mr_indoj
Posts: 42
Joined: Wed Jul 01, 2015 9:28 am

Re: Sight for the Blind for <100$

Tue Feb 02, 2016 11:08 pm

I'm rewriting bits of the webcamvideo module, it will check the camera when start is called and if its not ok then camera.start() will return False. Also, there will be a function to check if the camera is still ok, and also when camera.stop() is called the camera will be released.
Will post it later to the repository.

mr_indoj
Posts: 42
Joined: Wed Jul 01, 2015 9:28 am

Re: Sight for the Blind for <100$

Thu Feb 04, 2016 8:45 pm

Ok, i have made an update.
Firstly, the reason for the device not responding after holding the rotary to kill the apps, was a line:

Code: Select all

p.ChangeDutyCycle(0) #If the vibration motor was interrupted in an energetic config this should quiet it
that was crashing the menu. I disabled that, and also for now disabled the "stopping application", since we can stop them from the menu. Of course if you continue to press, the shutdown will happen.
I've also added files to the update script.
I fixed the resolution when foveal mapping is enabled, it needs to be higher, according to the code in Raspivoice.cpp.

Camera:
- Firstly, if no camera is detected on boot, there is a spoken message about that. You can connect a camera at this point, it will be checked for proper function when you try to start Raspivoice/teradeep the next time.

- If you try to start raspivoice/teradeep with no working camera, there will be a message and it will not try to start the app.
- Lastly, if you unplug the camera for some reason when apps are running, there will be a message that the camera was disconnected, and the running apps will be stopped and the camera structures will be released.
So, now it should be possible to disconnect and reconnect the camera, and just start raspivoice/teradeep again.
To get the code, just run in the terminal:

Code: Select all

./a-update.sh
(WARNING: This next step will overwrite the core files (menu, confmanager etc)
And to install just run:

Code: Select all

After-Sight-Model-1/a-update_core.sh
/Mikael

PranavLal
Posts: 124
Joined: Fri Jun 28, 2013 4:49 pm

Re: Sight for the Blind for <100$

Fri Feb 05, 2016 12:36 am

Hi /Mikael,
The update went smoothly as far as I can tell. You need to be root to update so the command should be

Code: Select all

sudo After-Sight-Model-1/a-update_core.sh
You then need to reboot with

Code: Select all

sudo reboot
The best way to test is then to unplug the camera and launch raspivoice.

Very very nice.

By the way, now that I have the right image, what has happened to the tactile vibrator? Why has it been disabled? Sorry if I missed this explanation. I like the audible distance reading. Are the distances taken from the camera in audible distance reading?

mr_indoj
Posts: 42
Joined: Wed Jul 01, 2015 9:28 am

Re: Sight for the Blind for <100$

Fri Feb 05, 2016 7:47 am

Hi Pranav,
You shouldn't actually need to sudo on the update command, since it does that on some parts internally. But, that can depend on other changes beeing made, so i think the best way to put it is that if you get error message when running After-Sight-Model-1/a-update_core.sh, put sudo.
Also, sorry about forgetting the "sudo reboot", that's very important.

The distance reading uses the ultrasonic sensor to get the distance. The problem is that only one process can access that at the same time, so currently you need to disable audible distance reading in settings to get tactile vibration.
I however have a fix-plan for that, rewrite the rangefinder code into a class instead of a process, then other apps can get the distance from there.

Mikael

User avatar
seeingwithsound
Posts: 165
Joined: Sun Aug 28, 2011 6:07 am
Contact: Website

Re: Sight for the Blind for <100$

Fri Feb 05, 2016 6:24 pm

Thanks Mikael, the update procedure works very nicely! Being able to swap cameras is also very nice. Like Pranav I had to apply sudo in the "sudo After-Sight-Model-1/a-update_core.sh" command, or else I would get the error messages

Code: Select all

cp: cannot create regular file `/home/pi/keyPress.py': Permission denied
cp: cannot create regular file `/home/pi/rangefinder.py': Permission denied
Peter

mikey11
Posts: 355
Joined: Tue Jun 25, 2013 6:18 am
Location: canada
Contact: Website

Re: Sight for the Blind for <100$

Fri Feb 05, 2016 7:46 pm

I've been away for a few days, and am just running the new code. It seems to be running well, and does nice detection of when the camera is accidentally pulled out. This is something that I found happens all too often. In the next case models I am definitely removing more plastic near the USB plugs.

I like it! Nice work! This is definitely the direction I was driving in when I started investigating how to handle the camera more nicely, but you did a better job than I could have.

One thing I would like to do is add an update function to the settings menu that does the update scripts with sudo on a button press. This is easy to implement. This way, as we code, those that are not capable of dealing with filesystem images may be able to update just by plugging their unit into a router and selecting update. It can also force the reboot.

This in theory should also in general allow for people to recover from bad code, as we replace it with good code again.


The line that was removed that was causing crashes was a hangover from when the rangefinder was operating on pulse width modulation. It is highly likely that I still have to add something similar back in, but it would just turn the vibration pin to off as the motor is now being run using purely digital control.

I'm not sure I'm going to have time for all of this in the next few days, but I will see what I can muster.

I've also had a request that I want to put before everyone:

I've had an outreach from an interested person who used to use something called the mowatt sensor, but he can't get it anymore, and the After Sight is cheaper, and has more capabilities. He used to be active on one of the mailing lists for the vOICe, but has been away from it for a while.

In any case, the mowatt sensor is very similar in operation to the vibration/distance routines we are using, but it also made an audible feedback for distance that was a tone rather than an english reading of distance. This allows for faster response times.

I would like to propose that since the vOICe is using the 0-5khz range, if I experimented with a tone based distance feedback in the 7-11khz range, this should in theory allow for using the vOICe, and a fast response distance cue without using vibration.

If I do this, current users of the mowatt sensor who are hit with the end of life of that device can be welcomed to a familiar interface if then want to use the After Sight as a distance sensor to begin with, then as time goes by, I would venture that they won't be able to resist at least trying the other options by getting a pair of videoglasses.

User avatar
seeingwithsound
Posts: 165
Joined: Sun Aug 28, 2011 6:07 am
Contact: Website

Re: Sight for the Blind for <100$

Fri Feb 05, 2016 8:24 pm

Hi Mike,
mikey11 wrote:I've had an outreach from an interested person who used to use something called the mowatt sensor, but he can't get it anymore
Are you sure this person used a Mowat Sensor? AFAIK, the original Mowat Sensor had only vibration feedback. There are (were) many similar devices, such as the Miniguide which is still available (at some $500) and which provides both tactile and auditory feedback, http://www.gdp-research.com.au/minig_1.htm BTW, where speed matters some users will likely prefer a handheld sonar device that they can quickly point around. It will be good to further discuss with this person what exactly is desired, for what purposes, and what trade-offs are considered acceptable.

Peter


Seeing with Sound - The vOICe
http://www.seeingwithsound.com

User avatar
seeingwithsound
Posts: 165
Joined: Sun Aug 28, 2011 6:07 am
Contact: Website

Re: Sight for the Blind for <100$

Fri Feb 05, 2016 9:36 pm

I understand that switching platforms is rarely easy, and I am not at all suggesting that one should, but I can imagine that using something like the below Odroid-C2 to test software on can at least help to reduce and minimize any dependencies that the After-Sight software may have on Raspberry Pi? Experimental porting can help ensure that things are and remain implemented in a mostly platform-independent manner.

Make a $40 Linux or Android PC with new Raspberry Pi 2 rival
http://www.pcworld.com/article/3030635/ ... rival.html

Peter

mikey11
Posts: 355
Joined: Tue Jun 25, 2013 6:18 am
Location: canada
Contact: Website

Re: Sight for the Blind for <100$

Sat Feb 06, 2016 2:53 am

I have checked, there is indeed an add on that the mowatt sensor used which made an audible version of the vibration signal (most likely for the unfortunate diabetics with advanced neuropathy that can't feel their hands anymore).

I have made my own version of this capability, and tested it out. It works, and works alongside the vOICe/Teradeep no problem. I have added a settings menu for this to allow the user to cycle between vibration, tones for distance, and english reading (which still runs within teradeep).

The only unfortunate bit is that I had to install pyaudio in order to drive sine waves in response to distance changes. This means the update scripts won't install pyaudio for you. However I shall sync my changes with github anyhow. If you want to use them, you will need to install pyaudio or wait for the next filesystem image. pyaudio is a binder for pulseaudio which is among the most widely supported sound system things available.

pyaudio also lets you record if there is a microphone. I could see how one day users might be able to take a photo and tag it with an audible description of what they thought they were photographing (useful for a far future feature of facial recognition of friends/acquaintances)

I did have to add a line to turn the vibration to off which should have been the replacement for the deleted line that was causing the crashes earlier.

My next items are:

1. get contrast back on the menu for raspivoice settings, and find a better default setting. Peter and Pranav both noticed that the default contrast doesn't seem to be where it needs to be.
2. Save money for the next hardware iteration.

With all the fantastic work Mikael has been pumping out I don't have much to do. Cheers to Mr. Indoj!

mikey11
Posts: 355
Joined: Tue Jun 25, 2013 6:18 am
Location: canada
Contact: Website

Re: Sight for the Blind for <100$

Sat Feb 06, 2016 3:05 am

I do like that odroid board. It has nice specs. The extra ram/cpu/gpu/64bit is very attractive.

I'm of two minds on this subject.

On the one hand, I fully agree that it would be valuable to develop an alternative just to ensure as much single board computer cross compatibility as can be found.

On the other hand, the decision of which alternative board to choose keeps me awake at night; Part of me has been hoping for something with clearly superior specs (this odroid board looks quite good by the way), but I also want more functions built in with wearables in mind, as this will drive down prices in a dramatic fashion. For instance one of the Pine64 boards. Some boards have wifi/bluetooth, and I think all have a lithium polymer battery charging function. Having these extras included can have a massive impact on a $/unit basis, and ultimately economics is one of the masters I want to serve.

In fact I generally want better economics at the expense of features to a certain degree.

It's probably almost to the point where I should compile a table of the best offerings to make comparisons easier, and then have a method of showing what it would cost to get minimal functions from them. (ie. for the odroid board, I have to add money for the battery management feature that the pine64 has built in).

I guess what I'm trying to say is that I'm hearing what Peter has said, and I want to address that at the same time as making a serious evaluation of an alternative platform to the rpi rather than making them separate efforts.

In making a comittment to tread that path, I would of course put the development on that as a lower priority than getting everything working as best as possible on the current effort. It does appear with Mikaels recent contributions that the stability is getting better as the features get richer. This argues that the time for selecting an alternative is sooner than I felt previously.

I'm going to think about this for a few more days, then I will probably start a spreadsheet that I can share on google. If you see a candidate, enter its specs on the spreadsheet? Then after a month or so, I will go with whatever seems to be a good combination of performance/price.

mr_indoj
Posts: 42
Joined: Wed Jul 01, 2015 9:28 am

Re: Sight for the Blind for <100$

Sat Feb 06, 2016 9:25 am

Thank you Mike, i'm glad that you like my updates!

I noticed that an updated confmanager.py was missing from the commit, i fixed that .
pyaudio can be installed by doing:

Code: Select all

sudo apt-get install python-pyaudio

User avatar
seeingwithsound
Posts: 165
Joined: Sun Aug 28, 2011 6:07 am
Contact: Website

Re: Sight for the Blind for <100$

Sat Feb 06, 2016 9:55 am

Hi Mike,
mikey11 wrote:On the other hand, the decision of which alternative board to choose keeps me awake at night
Another possible take is to assume that a good alternative for the Raspberry Pi is not on the market yet, but that it will be easier to move over once a great opportunity arises if one ensures that the software remains mostly device-independent by occasionally porting it to another device. Some devices have greater specs at an attractive price and yet may often prove less "stable" than the Raspberry Pi in terms of market lifetime. Many devices come and go at a rate that simply does not warrent switching over in view of the time, effort and cost involved. Most if not all of the available circuit boards are still too bulky and power-hungry for truly wearable use anyway. So I would take a more relaxed approach of merely ensuring - through occasional porting experiments - that one does not get locked in due to inadvertently using more and more Raspberry Pi specific properties without really noticing.

By analogy, I sometimes buy new Android devices just to check compatibility with The vOICe for Android and if possible make changes to improve compatibility. In this manner I've had The vOICe for Android running on an Android smartwatch (overheated in minutes), on Google Glass (a dead end for now as we all know, and also suffering from overheating issues), an Android TV stick, and will be trying the upcoming Optinvent ORA-X smart headphones, without having the expectation that any of these devices will be the one that really catches on. However, it helps to keep the probability that The vOICe for Android will run (perhaps with minor changes) on new and still untested Android devices high, and blind end users can just buy the device that best suits them in terms of price, performance, availability, appearance, whatever.

Also ensure that the After-Sight software stack keeps working with graceful degradation of functionality on vanilla Raspberry Pi devices, that is, on devices that are not under your control but that may assist in market penetration and in convincing people that the core functionality is there to stay even if the supply of dedicated devices would dry up. It is a form of building trust.

Build a Gaussian where one (hardware) slice is under your control but the tails of the Gaussian are not but help anchor the work in society. After all, the work is non-profit, so there is no need to restrict availability to a single hardware supplier. In a way, your are creating and distributing a reference design that others can modify ("improve") through making different trade-offs, e.g. higher performance at a cost, or improved wearability and unobtrusiveness, or still lower cost for developing countries, and so on and so forth.

Peter


Seeing with Sound - The vOICe
http://www.seeingwithsound.com

User avatar
seeingwithsound
Posts: 165
Joined: Sun Aug 28, 2011 6:07 am
Contact: Website

Re: Sight for the Blind for <100$

Sat Feb 06, 2016 10:51 am

Thanks Mikael, the pyaudio install worked like a charm. Adding an update function to the menu as proposed by Mike is also a great idea. I wondered though, in case an update inadvertently breaks something, is there a corresponding means for the user to revert to a previous revision or something, or what is the best way then for a user to recover and keep using the device? After all, stability is key once users start to rely on the technology in daily use.

Thanks,

Peter

mr_indoj
Posts: 42
Joined: Wed Jul 01, 2015 9:28 am

Re: Sight for the Blind for <100$

Sat Feb 06, 2016 4:22 pm

Having an update function from the menu is of course great.
To do this cleanly, we need to first check for internet connectivity, and bug the user if that's not available . And, also check if there's actually new code or not, and what parts are updated (like core raspivoice etc) to avoid compiling when we don't need to.
I'm looking into a package called gitPython

Code: Select all

sudo apt-get install python-git
to possibly find out what's changed.

Reverting would need to have a backup somewhere on the device, maybe when a full image is created we keep a directory with the original files. for menu, raspivoice binaries etc But then it would have to exist a way to revert (if something got really bad so that the menu doesn't work for example). Say hold the rotary pushed in while booting.
That could be checked for in the startup shell scripts.

While we are talking packages, there's an python-espeak package, that i'm thinking could be used to create a speech class to handle the speaking, instead of calling the espeak command line binary.

Lastly, one thing to consider with different development boards is the audio output. The odroid, for example does not have 3.5 mm output.

mikey11
Posts: 355
Joined: Tue Jun 25, 2013 6:18 am
Location: canada
Contact: Website

Re: Sight for the Blind for <100$

Sat Feb 06, 2016 5:45 pm

Peter,

I agree with you about keeping it as functional as possible on the broadest range of hardware possible, that is why I added the support for USB keypads to mimic the encoder.

As I add more hardware, it will just mean more error checking routines to keep things in check for those without. If all goes well, this will allow other assistive hardware makers to use the After Sight as a base computer to run their hardware/software on.

I have just placed an order for my next hardware development which will include a better battery, new case, and accel/gyro to test out an idea for straight line guidance. This will also involve some trial and error with making circuit board changes if it is successful.

I will also experiment with an external sensor to make a finger mounted color sensor.

After that set, I have an obligation to provide units to the other directors of the After Sight corporation in time for our achievement of charitable status (likely in April). Time is starting to move quickly, and my spare time is not growing.

On the bright side, I expect that fundraising no matter how successful will offload the financial burden for production to a degree.

User avatar
seeingwithsound
Posts: 165
Joined: Sun Aug 28, 2011 6:07 am
Contact: Website

Re: Sight for the Blind for <100$

Sat Feb 06, 2016 5:55 pm

Hi Mikael,
mr_indoj wrote:I'm looking into a package called gitPython
I noticed that the After-Sight Raspberry Pi image comes with the Mercurial source control management tool pre-installed: just type the command "hg" and it returns a list of basic commands. Perhaps Mercurial can be used to handle source code commit and revert actions. There is for instance an "hg revert" command, but of course one must first create a repository. The initial setup for using Mercurial will take some effort and some getting used to. Unless an update really messed up the system, I imagine that a simple "revert update" user option in the menu would suit most use cases.

Peter

mikey11
Posts: 355
Joined: Tue Jun 25, 2013 6:18 am
Location: canada
Contact: Website

Re: Sight for the Blind for <100$

Tue Feb 09, 2016 7:46 pm

I'm a little bit conflicted about what to use as a respository.

I jumped in with both feet to use github because it was something that I had seen successful projects use. I respected those projects, and the price isn't too bad (about $7/month).

However I'm not too impressed with some of the changes in within the github corporate structure lately. If there is a repository with better services and features, I would switch and pay double. I've started to do some research, but I haven't come to any conclusions as this kind of thing is relatively new to me. As a consequence its harder for me to make a good judgement of the offerings.

I'm open to alternatives, if you could highlight features they have that others do not, that is what I'm looking for.

I gather from what Peter has written that mercurial is a system that has a more robust method for reversions. This is something we want.

However, I think with github, we could have a stable branch and a development branch, and that way people could avoid most of the bugs that crop up by just using the stable branch for updates. This could be an additional menu choice on an upgrade menu.


Anyhow, if you have direct experience with better service providers, please tell us about them.

User avatar
seeingwithsound
Posts: 165
Joined: Sun Aug 28, 2011 6:07 am
Contact: Website

Re: Sight for the Blind for <100$

Tue Feb 09, 2016 8:34 pm

As a developer I use TortoiseSVN for local proprietary projects, and was thinking of Mercurial for possible similar use with an in-image local source repository after seeing a tutorial http://www.linuxuser.co.uk/tutorials/ma ... l-tutorial However, I do not know how to combine Mercurial with cloud-based repositories such as GitHub or BitBucket http://tortoisehg.bitbucket.org/, and cannot make recommendations in that area. When Googling for TortoiseHg and GitHub quite a few potentially relevant pages turn up, but I lack the time to delve into that. Perhaps your idea of working with stable and development branches is easier to cope with and more practical.

Peter

mikey11
Posts: 355
Joined: Tue Jun 25, 2013 6:18 am
Location: canada
Contact: Website

Re: Sight for the Blind for <100$

Thu Feb 11, 2016 8:30 am

seeingwithsound wrote:Yes, I recently referred Rich to the After-Sight project when he was inquiring about an ultrasonic emitter. Rather than having yet another open source project for a device for the blind, I suggested joining forces if possible and meaningful. Maybe a Sonic-Eye-like device could be used to offer a crude depth map via small tactor array or something in order to avoid ambiguity and confusion with The vOICe image-to-sound mapping while offering more than a single distance indication.

Peter


Seeing with Sound - The vOICe
http://www.seeingwithsound.com
I looked into this particular problem in depth. I'm excited to see what Rich has to contribute over time. In my investigation, the best possible technological solution for the lowest price (I love the low $$$) was refurbishing the microsoft kinect sensor. Unfortunately the power dissipation is >15 watts for the sensor alone.

For your reference, the dissipation of the After Sight is maybe between 2-3 watts.

As a result, you would need too large of a battery to use the kinect sensor for extended periods of time. It's really a show stopper. You would need a battery the size of the sensor bar to last more than an hour or so. It would drain the After Sight device in minutes (if it could).



The ultrasonic can work well when executed correctly. I am eager to see what Rich comes up with.

mikey11
Posts: 355
Joined: Tue Jun 25, 2013 6:18 am
Location: canada
Contact: Website

Re: Sight for the Blind for <100$

Sun Feb 14, 2016 1:59 am

I have recently been looking into facial recognition for an unrelated application (a kid recognizer for my kids to activate a photo frame with family photos). I purchased a book about python and computer vision, and I'm trying to use the examples to learn about some aspects of computer vision.

It occurred to me that in detecting faces, and placing a bounding box around the face I can assign a two dismensional six position location to the bounding box in english (ie top-left, bottom mid, middle etc). This could help users locate where a face is relative to the camera they are wearing to better face towards people while communicating with them. If a bounding box were very large, it is likely the person is quite close (or the face is very large!) so there is even the outside chance of doing some very coarse depth information.

Of course, this can also be coupled with more interesting possibilities like facial recognition (I have already made a principal component analysis method work for me in the unrelated application with good performance with 11 faces remembered). Users could create databases of their friends and then when they are in mixed company, they could have an idea of who they are looking at. It would be somewhat tedious as you would need 10 or so pictures of each person before reasonable success rate would be had, and it would require a microphone to assign their name.

I'm tempted to try to implement the face detector and 2d location reporting.

Thoughts?

Is this something that would be a useful application for blind/visually impaired users? Or am I just being too enthralled with technology. I can personally see some utility in group situations, and possibly even while out and about.

PranavLal
Posts: 124
Joined: Fri Jun 28, 2013 4:49 pm

Re: Sight for the Blind for <100$

Sun Feb 14, 2016 2:26 am

Hi Mikey11,

A face recognizer saying face at top left etc is certainly handy. This is how the iPhone does it. The vOICe for Android also has this capability.

A person identifier is certainly useful. The way I would expect it to work would be for the program to go to my linkedin profile or my facebook profile, pull down the images of my contacts etc., and add them to a database. It should then report if that person enters the camera frame. Getting 10 images of a person is going to be difficult. The social media approach also has the benefit of not requiring a microphone. An micro expression detector is also handy. Something that says
"face at center looking board"

Yes, you can after a bit of work detect some expressions with the vOICe but it takes a lot of focus.

The object recognizer is really cool. I demonstrated the device to a few people yesterday and the combination of the vOICe and teradeep and the distance sensor is quite effective. We need to really train that classifier and get the vOICe to be more like the Android version.

Yesterday's demonstration was to a blind musician who was immediately able to sense the change in the scene. We were in an open area with a fair number of people and many objects. So, shall we target blind musicians who need to travel? This was why this person approached me. He wanted a tool to handle unknown spaces.

mikey11
Posts: 355
Joined: Tue Jun 25, 2013 6:18 am
Location: canada
Contact: Website

Re: Sight for the Blind for <100$

Sun Feb 14, 2016 3:25 am

can you make a list of differences between the android version and what we are running?

you have indicated that there is a bit more lag,
you and Peter both indicated that the contrast settings need to be changed, and a multiple selection should be added.

what are the other standout features that are different? I don't have an android device, so I can't even test it out. For instance, I had no idea it did face detection and localization.

User avatar
seeingwithsound
Posts: 165
Joined: Sun Aug 28, 2011 6:07 am
Contact: Website

Re: Sight for the Blind for <100$

Sun Feb 14, 2016 11:46 am

Hi Mike,

Although it is not a replacement for actually playing with The vOICe for Android on an Android device, you can find its supported features, including its "talking face detector", listed and described at http://www.seeingwithsound.com/android.htm

Peter

mikey11
Posts: 355
Joined: Tue Jun 25, 2013 6:18 am
Location: canada
Contact: Website

Re: Sight for the Blind for <100$

Mon Feb 15, 2016 4:21 am

I added a face detector and early attempt at localization on the repo on github.

I don't think I installed anything strange along the way, but I sometimes get caught up and forget.

If you install:

webcam_face_detection folder plus contents

and facedetect.py and the updated version of menu.py it all works for me.

By no means perfect, but it updates every 2 seconds or so and runs with raspivoice and teradeep.

I'm clearly violating some of my if statements for the 6x6 grid from time to time, but it seemed pretty stable.

PranavLal
Posts: 124
Joined: Fri Jun 28, 2013 4:49 pm

Re: Sight for the Blind for <100$

Mon Feb 15, 2016 2:04 pm

Will Mikael's scripts work to apply these updates?

Return to “Assistive technology and accessibility”