Sight for the Blind for <100$

[mikey11]

I feel a little silly. All those warning were irrelevant, and all I had to do was chase down and install libGL. that was a challenge. As with raspivoice, a lot of installing/uninstalling/reinstalling dependencies.

For anyone facing this trouble in the future, it requires two versions of libavcodec53 and libavutils51 and you have to go back and forth between them (4 files total).

So now onto the execution of one time code based on update number and I'm ready for some remote testers.

I only have one raspberry piv2 device left on hand for myself, so I will also work on this. One way to handle this that I think makes sense is for me to install one of the recent piv2 images on mine, and then get updated to the autoupdater and have everything work. By this method, I can ensure that all piv2 users will be able to keep getting updates by installing that image, and going from there. This is better than trying to meet the needs of the many different image versions out there. I know this will be inconvenient for the pi v 2 users out there, but with all good luck, this will be the last inconvenience.

[mikey11]

I'm finished the updater and confirmed it working on the pi v 3.

Now I am working on my last pi v 2. I should not have to change the stuff submitted to git at this point to make it work. I may have to do a bit of fighting with installing dependencies though, so...

What I propose is this:

I will get a working image file prepared for both hardware versions. This will be the last image you have to download and install. It's probably easier to burn the image than have to run a bunch of shell scripts and do the dependency fights that I have been through.

from that point on, it's all menu driven updates.


Once I have it working, the first 'actual' update will be to add the contrast default setting to raspivoice, and make a contrast increment function in the raspivoice settings menu.

[mikey11]

I have completed both images for the rpi v2 and v3. I am not in a good place to upload though, so it will be a few days before I can get them online. In that time, I will attempt to get the next update prepared which will be the contrast setting.

This way, once you download the image, you can plug into ethernet and run the update. Pretty excited for all of this to come together.

[PranavLal]

Hi Mikey11,

I share that excitement. Could you also get everything working such as the vibration feedback for distance? It could be an interesting test along with the contrast update.

Pranav

[mc007ibi]

hi, sorry for bumping here a little out of the context of the previous posts.

Along with soundscape over opencv, would it make sense to augment GPS data. Lets say you could a make a little plate with at least 12x12 grid of height variable bumpers and so simulating something what the blind already know from pharma products, just that this little plate is 'dynamic' and you wear it at the arm, with lets say some buttons, switching modes:
- my direction (arrow)
- direction to goal (2 arrows)
- distance from goal (number)
- and distances to walls (numbers/stripes, needs then a bigger grid of course)
- with more 3d geo data, you could also 'print' floors or room numbers

not sure how to make that bumpers in real but i am sure it might be possible, cheap. You could probably also make stronger bumpers and twist that arm band inside out, so its bumping in your arm instead you read it with your finger (just tried sticking my arm with a pen but I can't really imagine i could read an arrow there, yet).

well, just an idea but technically i don't see huge problems to get this done, cheap. I will volunteer for coding

[mc007ibi]

I just wanted to add some reality to the idea mentioned above. It seems quite difficult to make that bumpers: https://www.youtube.com/watch?time_cont ... fIg4rI4cDw .Its pneumatic based and consumes a hell of space. I would be happy if someone could come up with an alternative idea. If anyone is interested in this project, please open another thread.

sorry for disturbing here

[seeingwithsound]

I have completed both images for the rpi v2 and v3.

Hi Mike,

I'll try your new updating image for RPi2 once it is ready.

In the following new video from Russia, The vOICe for Raspberry Pi is used by a blind man to go bowling https://www.youtube.com/watch?v=U9fHbW8ylTM More info about this (in Russian) at https://vk.com/wall-96731906_137

Peter


Seeing with Sound - The vOICe
http://www.seeingwithsound.com

[mikey11]

Hi mc700bi

We welcome intrusions and new ideas.

At the moment our device does not have gps. But I would be excited to explore methods of creating high resolution depth representations that are portable and low power.

My first intuition is a 3d printed micro fluidic array with a flexible membrane. My only mental block is how to make extremely small electrically actuated valves.


I also don't own a good enough 3d printer yet. Mine is ok for printing cases, but fluid proof channels are not possible with mine yet. This doesn't prevent one from making designs and having a third party print them though.

Peter:

I am now at the better internet location and will get the images uploaded later tonight if all goes well. I will post links here.

[mc007ibi]

Hi mc700bi

We welcome intrusions and new ideas...

yeah, I got kind of stuck too, reading the traditional braile cell manufactoring: http://www3.imperial.ac.uk/pls/portalli ... 449703.PDF . I was thinking to create a kind of mini-plotter from old DVD drives, and push the cells up seriell (one by one). that could work for simple symbols like arrows, probably still slow,...still thinking. Ah, I've got 3D-Printer, a CNC and carpentry here. Just in case...

[seeingwithsound]

Its pneumatic based and consumes a hell of space. I would be happy if someone could come up with an alternative idea.

Hi mc007ibi, what about a tongue electrode display instead of an electro-mechanical tactile display? https://dustinvenini.wordpress.com/rese ... stitution/

Regards,

Peter

Seeing with Sound - The vOICe
http://www.seeingwithsound.com

[mc007ibi]

Its pneumatic based and consumes a hell of space. I would be happy if someone could come up with an alternative idea.

Hi mc007ibi, what about a tongue electrode display instead of an electro-mechanical tactile display? https://dustinvenini.wordpress.com/rese ... stitution/

couldn't resist to search pictures on it. seems pretty hard for both, builder & consumer. but yeah, ideally you would inject them under the skin and stimulate that cells from something on the arm.

isn't there a wiki for rpi?

[mikey11]

My hope is a tongue system will be made as a USB device. Then you could program it for all kinds of input.

[mikey11]

https://drive.google.com/file/d/0B30mp6 ... sp=sharing

https://drive.google.com/file/d/0B30mp6 ... sp=sharing

Try the above link for the new rpi v 2 image.

It includes the updater. Please mess around with it.

In an ideal world, it will only proceed if you have

1. external power plugged in
2. a valid internet connection

In less than 7 days I expect to put out the raspivoice contrast update so we can have the update system tested for real.

Thanks,

[dvenini]

My hope is a tongue system will be made as a USB device. Then you could program it for all kinds of input.

Thanks to Peter for the acknowledgment. Our tongue system is actually a USB device currently. As long as you can send a string to the device with a corresponding 0 or 1 for each pixel you can use anything for input. It currently presents thresholded camera images at about 50Hz with image updates at about 12Hz.

[seeingwithsound]

In less than 7 days I expect to put out the raspivoice contrast update so we can have the update system tested for real.

Hi Mike,

Thanks, I'm currently downloading the 3.2GB image. For contrast enhancement be sure to check out my http://www.seeingwithsound.com/hificode_OpenCV.cpp sample code, specifically the parts

Code: Select all

#define CONTRAST 2   /* Contrast enhancement, 0=none */

int px;
double avg;
avg = 0.0;
for (i=0; i<M; i++) {
   for (j=0; j<N; j++) {
      avg += gray.at<uchar>(M-1-i,j);
   }
}
avg = avg / (N * M);
for (i=0; i<M; i++) {
   for (j=0; j<N; j++) {
      px = gray.at<uchar>(M-1-i,j);
      px += CONTRAST*(px - avg);
      if (px > 255) px = 255;
      if (px <   0) px =   0;
      gray.at<uchar>(M-1-i,j) = px;
      if (px == 0) A[i][j] = 0; 
      else A[i][j] = pow(10.0,(px/16-15)/10.0);  /* 2dB steps */
   }
}

because the existing raspivoice fixed linear scaling for brightness and contrast in RaspiVoice.cpp is definitely not a good way to do it for a simple yet effective contrast enhancement. The existing raspivoice brightness option is then actually best removed because it only adds noise to the soundscapes. After porting the above code snippet you will only be left with an "if (opt.contrast != 0.0) { }" branch where the default value for opt.contrast will now be 2.0.

Peter


Seeing with Sound - The vOICe
http://www.seeingwithsound.com

[mikey11]

Dustin,

How much is the array? How often do they need to be replaced?

I can see some very useful possibilities.

[seeingwithsound]

After porting the above code snippet you will only be left with an "if (opt.contrast != 0.0) { }" branch where the default value for opt.contrast will now be 2.0.

Hi Mike,

Below you find the ported code that can simply replace the "if ((opt.contrast != 1.0) || (opt.brightness != 0)) { }" branch in RaspiVoice.cpp. The opt.brightness setting can be removed (also from the menu), and the default value for opt.contrast can be set to 2.0. You should definitely notice the more "crisp" soundscapes...

Code: Select all

/* Set default value for opt.contrast to 2.0 */

if (opt.contrast != 0.0)
{
   float avg = 0.0;    
   for (int y = 0; y < processedImage.rows; y++)
   {
      for (int x = 0; x < processedImage.cols; x++)
      {
         avg += processedImage.at<uchar>(y, x);
      }
   }
   avg = avg / (processedImage.rows * processedImage.cols);
   for (int y = 0; y < processedImage.rows; y++)
   {
      for (int x = 0; x < processedImage.cols; x++)
      {
         int mVal = processedImage.at<uchar>(y, x);
         processedImage.at<uchar>(y, x) = cv::saturate_cast<uchar>(mVal + opt.contrast*(mVal - avg));
      }
   }
}

Peter


The vOICe Image to Sound Mapping
http://www.seeingwithsound.com/im2sound.htm

[mikey11]

I have created the update with contrast setting.

Peter:

I did not use your code because raspivoice already has a command line flag for changing the contrast. I used this.

Please see that it meets the needs. If it is vastly inferior we can rework the code directly. I played around with the values, and I found that the range I used seems pretty good.

Also, as this will be the first "real" update, please let me know of bugs and errors.

The best way to do this would be to start the device connected to your local network.

ssh in and kill menu.py

then restart it in a terminal window

then select the update command using the rotary encoder on the device, and look at the text output. As it will be a full update there will be a lot of output. I am already aware of some error messages that come while installing dependencies, and while compiling both raspivoice and teradeep. If the update fails for any reason it would be very helpful to have the entire block of text to troubleshoot. I use putty from windows for my ssh sessions, and I had to increase my screen memory size to keep all the lines from the update available.

or, you know. don't do any of that, and just have faith. I don't think you will be disappointed. If it doesn't work, you can always redo the image, and then do the procedure above to capture the errors. I don't expect you should have any though.

Thanks much.

The more feedback on the updater in the near term the better. I will soon be giving devices away to the reps for the national blindness orgs in Canada, and I want to have any huge bugs ironed out. basically, I want to be able to have them continue to get updates in the near future flawlessly. I'm probably just being paranoid.

[seeingwithsound]

I did not use your code because raspivoice already has a command line flag for changing the contrast. I used this. Please see that it meets the needs.

No, it doesn't, or else I would not have bothered. Algorithms are my specialty.

I'm currently flashing your image but probably have no time today to try it out.

Peter

[mikey11]

fair enough. I will take a stab at yours tonight.

I'm just far less confident in cpp than python. I will do it if I can though.

[seeingwithsound]

Also, as this will be the first "real" update, please let me know of bugs and errors.

Hi Mike,

What I did:

PuTTY settings: Session | Logging | Printable output, in order to get output in a log file.

From PuTTY window:
sudo killall "python menu.py"
sudo python menu.py

Using rotary encoder on the device: Settings | Update software

Device finally rebooted itself, with the usual PuTTY messagebox: Server unexpectedly closed network connection

After the update process and subsequent reboot, things appear to work, but I do not know what to check to make sure that the update worked as intended. The PuTTY log file is attached, such that you can check out if there are any serious errors that need fixing?

Thanks,

Peter

[mikey11]

Thanks, when I get home I will look it over.

Basically, at this time it recompiless raspivoice and teradeep each time an update takes place. This will be helpful later tonight when I try to add your contrast
Code.

if they are still working, then everything compiled fine.

From what I gather, the steps I took to get the command line flags for contrast cycling will still be useful even with your code in place. I would have tried earlier but naptime is only so long. I get to try things I might not pull off on the first go at night when the kids sleep deeper and longer.

[seeingwithsound]

Thanks Mike. I have also created a patch for the github code at https://github.com/seeingwithsound/rasp ... ee/patch-1 and issued a pull request, proposing also to cycle the contrast parameter (whose values now have slightly different semantics) among values 0.0 (no contrast enhancement), 1.0, 2.0 and 3.0, with a default value of 2.0.

Peter

[mikey11]

I have integrated the contrast code, and it is much nicer. I have noticed though that when contrast is set to 0 that I get nothing. I had added the zero setting to the cycling in the menu so that it was 0,1,2,3

at the next opportunity, dropping the 0 seems like a plan. The other settings have a nice range.

It should be available via the update from the menu.

[PranavLal]

Hi all,

I ran the update from the pi. I did not get any error messages. I have not had a chance to test anything. Mike, did you add the contrast code? I ask because I was told that update 11 had been applied.

The internet connection checking routine is working nicely. I had connected a faulty network cable to the pi by accident. We should mention this possibility in the error message. It has a bigger probability of occurrence as opposed to the update servers being offline. <smile