PranavLal
Posts: 124
Joined: Fri Jun 28, 2013 4:49 pm

Re: Sight for the Blind for <100$

Sat Jan 23, 2016 7:45 am

Hi Mikey11,

If memory serves, there is indeed a while loop that tracks key presses in the raspivoice code. The idea being you can hit escape to quit raspivoice.

mr_indoj
Posts: 42
Joined: Wed Jul 01, 2015 9:28 am

Re: Sight for the Blind for <100$

Sat Jan 23, 2016 9:36 am

Nice progress!
I can take a look at the raspivoice problem when i get the image, and also sync the new files to Github.

mikey11
Posts: 354
Joined: Tue Jun 25, 2013 6:18 am
Location: canada
Contact: Website

Re: Sight for the Blind for <100$

Sat Jan 23, 2016 3:21 pm

Unrelated:

Aftershockz has a sale on at the moment - 31.4% off. Code is 'PiDay'

User avatar
seeingwithsound
Posts: 165
Joined: Sun Aug 28, 2011 6:07 am
Contact: Website

Re: Sight for the Blind for <100$

Sat Jan 23, 2016 9:05 pm

mr_indoj wrote:Ok, so now the new image is ready to download.
The url is:
https://onedrive.live.com/redir?resid=D ... file%2czip
The description of progress sounds truly great, but does this image work for anyone? For me the camera turns on maybe half of the time after a device startup. The menu speaks just fine in response to the rotary knob, but toggling either raspivoice or teradeep results in nothing, no soundscapes, no spoken object recognition. Only the menu speaks and I can toggle raspivoice and teradeep on and off, but the apps themselves give no sound. I flashed twice, and tried the After-Sight model 1 and 2. I'm using a regular Logitech C270 webcam that I've used without problems before, and headphones with the 3mm audio connector. I had no such problems with the January 4 image.

Thanks,

Peter

mikey11
Posts: 354
Joined: Tue Jun 25, 2013 6:18 am
Location: canada
Contact: Website

Re: Sight for the Blind for <100$

Sat Jan 23, 2016 9:10 pm

Peter,

I have had no problems such as you have encountered, but I can hazard a guess why: I noticed some of the new camera code that lets both raspivoice and teradeep operate simultaneously has a hard coded resolution in one of the files.

I believe this hard coded resolution is 1280x720. Obviously if a camera can't reach that resolution, it will fail. I will try to track down that code on my next dive in.

I think we have to make the assumption that any UVC camera may be plugged in, and that we need to somehow do a max resolution getting routine. I'm not familiar with this at the moment, but I will add it to my list.

mr_indoj
Posts: 42
Joined: Wed Jul 01, 2015 9:28 am

Re: Sight for the Blind for <100$

Sat Jan 23, 2016 9:14 pm

Hi Peter,
Does that camera support 1280X720? I changed the resolution in this image, because i got better results with the 2 cameras a tested, the Technaxx glasses, and another non branded camera board.
Have you tried the Technaxx? If this causes isues, we might need a setting for the main camera resolution.

mikey11
Posts: 354
Joined: Tue Jun 25, 2013 6:18 am
Location: canada
Contact: Website

Re: Sight for the Blind for <100$

Sat Jan 23, 2016 9:18 pm

Unfortunately, this may be a logitech issue:
I have the same problem as you, that I cant get a higher resolution in opencv different "application". But with the Logitech Software I am able to record videos in 720p (camera C270 ). After some days of research, I came to the same explanation as @break that they restricted the resolution in the driver. I gave it up and will buy a different, non Logitech, one..
I found this while researching how to get max resolutions from here:

http://stackoverflow.com/questions/1944 ... resolution

So what I get from this is that actual 720p support on the logitech c270 is a little bit flakey when not using the logitech software. An unfortunate development.

edit:

However: reading slightly further:
I got it to work, so this post is for others experiencing the same problem:

I am running on the Logitech C270 as well. For some reason it would only show 640x480 even though the webcam supports 1280x720. Same issue persists with the built-in webcam in my laptop.

If I set it to 800x600 in the code it shows 640x480. However, if I set it to 1024x768 it becomes 800x600. And if I set it to something silly like 2000x2000 it becomes 1280x720.

This is in C++ on OpenCV 3.0, but perhaps it applies to Python as well.
It seems if we find that the C270 is used, we can trick it into acting more reliably. I'm sure there is a device string we can query and make an if statement based on that.

User avatar
seeingwithsound
Posts: 165
Joined: Sun Aug 28, 2011 6:07 am
Contact: Website

Re: Sight for the Blind for <100$

Sat Jan 23, 2016 9:36 pm

mr_indoj wrote:Hi Peter,
Does that camera support 1280X720? I changed the resolution in this image, because i got better results with the 2 cameras a tested, the Technaxx glasses, and another non branded camera board.
Have you tried the Technaxx? If this causes isues, we might need a setting for the main camera resolution.
Hi mr_indoj. Yes, I just checked using Amcap on Windows, and my Logitech (HD 720p) C270 webcam supports a long list of resolutions including 1280 x 720 and 1280 x 960 at 30fps. Both these C270 resolutions look fine on my PC's monitor. Note that neither The vOICe (raspivoice) nor Teradeep benefit from high resolutions. Just tried my Technaxx glasses as well: same problems as with the Logitech C270. Both the Logitech C270 and the Technaxx glasses work fine when I check them on my Windows PC using Amcap. Since the menu speech system works fine on the Raspberry Pi devices I do not immediately suspect a problem with flashing the image. I wiped the earlier January 4 image content before flashing.

Thanks,

Peter
Last edited by seeingwithsound on Sat Jan 23, 2016 10:27 pm, edited 1 time in total.

mr_indoj
Posts: 42
Joined: Wed Jul 01, 2015 9:28 am

Re: Sight for the Blind for <100$

Sat Jan 23, 2016 9:46 pm

Do you have the camera plugged in when you boot, or do you plug it in later?
I choose that resolution, because i found that i got a bit unclean soundscapes (like the broken Technaxx that i have) with my other camera as well, this did not happen with the higher resolution.

User avatar
seeingwithsound
Posts: 165
Joined: Sun Aug 28, 2011 6:07 am
Contact: Website

Re: Sight for the Blind for <100$

Sat Jan 23, 2016 10:05 pm

mikey11 wrote:Unfortunately, this may be a logitech issue:
Not sure but it might also be an issue with OpenCV 3.0 and/or Python's linking to OpenCV. I tried the OpenCV version of The vOICe on the PC and it ran fine if I used my Logitech C270 right away in 1280 x 720 resolution. However, there I'm using C/C++ (no Python) with OpenCV 2.4.11 (not 3.0). This using the code from https://www.seeingwithsound.com/extra/vOICeOpenCV.zip and with hard-coded capture resolution changed from 176 x 144 to 1280 x 720. If I try my Python 3 port https://www.seeingwithsound.com/hificode_OpenCV.py and change the hard-coded capture resolution from 176 x 144 to 1280 x 720 it also works fine on the PC, showing the expected resolution on the screen (but it runs painfully slowly with interpreted Python).

Peter

User avatar
seeingwithsound
Posts: 165
Joined: Sun Aug 28, 2011 6:07 am
Contact: Website

Re: Sight for the Blind for <100$

Sat Jan 23, 2016 10:13 pm

mr_indoj wrote:Do you have the camera plugged in when you boot, or do you plug it in later?
I choose that resolution, because i found that i got a bit unclean soundscapes (like the broken Technaxx that i have) with my other camera as well, this did not happen with the higher resolution.
I have the camera plugged in when I boot. Yeah, I had the same problem with a garbled image slice on my original Technaxx glasses when using regular VGA (640x480), which is a bug in the glasses (a later cheaper look-alike of these glasses from eBay did not have this problem). However, it is a waste of bandwidth to capture in 1280 x 720, especially since we have to discard 6 out of every 7 frames to get a new key frame and avoid serious content lag with raspivoice. Where can I change the resolution to, say, 640 x 480 or 176 x 144 to see if that works?

Peter

mr_indoj
Posts: 42
Joined: Wed Jul 01, 2015 9:28 am

Re: Sight for the Blind for <100$

Sat Jan 23, 2016 10:31 pm

We no longer need to pull 7 frames, since the frames are pulled continueously in the background.
Change this in menu.py
there is the line:
camera = WebcamVideoStream(src=camera_port, width=1280, height=720) #define where I dump camera
This will change the camera property using opencv's mechanism for that.
But, don't set it lower than 640x480, since teradeep is coded to use that.

User avatar
seeingwithsound
Posts: 165
Joined: Sun Aug 28, 2011 6:07 am
Contact: Website

Re: Sight for the Blind for <100$

Sat Jan 23, 2016 11:08 pm

mr_indoj wrote:camera = WebcamVideoStream(src=camera_port, width=1280, height=720) #define where I dump camera
Thanks, I changed it to width=640, height=480, but still no luck (here using my Logitech C270 which in all my other experiments has proven rock solid).The camera light turns on after shutdown and restart, but neither raspivoice nor teradeep seems to "see" any image frames. I have to call it a day (it's past midnight here). Thanks for your help!

Peter

mikey11
Posts: 354
Joined: Tue Jun 25, 2013 6:18 am
Location: canada
Contact: Website

Re: Sight for the Blind for <100$

Sun Jan 24, 2016 6:49 am

While sitting here waiting for my synced filesystem link, I was updating the after-sight.com website information.

I discovered something that looks even more interesting, and I would like to integrate this when possible.

Found on github here:

https://github.com/teradeep/demo-apps
Please download files: model.net, categories.txt and stat.t7 from: https://www.dropbox.com/sh/u3bunkfm0dzj ... jrrMa?dl=0 This is our new 5000 categories network from January 2016.
a 5000 category network from people who already made a fantastic 1000 category network! I'm pretty excited right now, but wont have the chance to check this out for a while as life is going to be busy for the next few weeks.

mr_indoj
Posts: 42
Joined: Wed Jul 01, 2015 9:28 am

Re: Sight for the Blind for <100$

Sun Jan 24, 2016 11:00 am

To me it looks like we just have to replace the model with the new one. Will chek it out!

mr_indoj
Posts: 42
Joined: Wed Jul 01, 2015 9:28 am

Re: Sight for the Blind for <100$

Sun Jan 24, 2016 1:55 pm

Need to do more work on that - looking at getting the new model working.

PranavLal
Posts: 124
Joined: Fri Jun 28, 2013 4:49 pm

Re: Sight for the Blind for <100$

Sun Jan 24, 2016 2:29 pm

mr_indoj,

This is really exciting. I took the pi to my drive way and it, after a bit of fiddling was able to identify that I was looking at a treat which is correct because I was pointing it to the road beyond my front gate.

Pranav

mikey11
Posts: 354
Joined: Tue Jun 25, 2013 6:18 am
Location: canada
Contact: Website

Re: Sight for the Blind for <100$

Sun Jan 24, 2016 2:58 pm

My latest filesystem image.

https://drive.google.com/file/d/0B30mp6 ... sp=sharing

https://drive.google.com/file/d/0B30mp6 ... sp=sharing

This does not include the 5000 object network.

You should be able to use keyboards with the +, -, and enter keys to navigate and get audible distances read from the rangefinder

This still has the bug with raspivoice where the keyboard usage will randomly shut it down, but everything else seems fine.

although the file has a weird extension, I've written it to cards and everything worked out. I was able to shrink it back to under 8gb.

User avatar
seeingwithsound
Posts: 165
Joined: Sun Aug 28, 2011 6:07 am
Contact: Website

Re: Sight for the Blind for <100$

Sun Jan 24, 2016 3:08 pm

Today again tried the January 19 image with my Technaxx camera glasses, and raspivoice worked! Added teradeep to the mix, and that worked too (cool!). Then did more testing with my trusted Logitech C270 HD Webcam, but it worked not once, no matter what I tried in terms of adapted resolutions like 640 x 480 or 176 x 144. Reflashed the January 4 image, and raspivoice worked fine with the C270. Reflashed the January 19 image, and raspivoice again appeared to be doing nothing with the C270. However, I noticed that /dev/shm/opencv.jpg upon every toggle of a raspivoice run gets updated with a properly scaled 176 x 64 version of the 1280 x 720 camera view of my Logitech C270. Not sure where things are broken in the chain with the January 19 image.

Also tried 2 other UVC compliant cameras: my Minuro 3D webcam turned on but did not work with raspivoice, just like the Logitech C270, but the cheap webcam from eBay at http://www.ebay.com/itm/291480706254 worked fine. Beats me, but it looks like the current January 19 image is somehow too picky about cameras as compared to the earlier images. The C270 has served me well for a gazillion devices on various platforms, including for instance Android TV sticks, so I find it strange that this workhorse now consistently fails with the January 19 image.

Thanks,

Peter

User avatar
seeingwithsound
Posts: 165
Joined: Sun Aug 28, 2011 6:07 am
Contact: Website

Re: Sight for the Blind for <100$

Sun Jan 24, 2016 4:26 pm

mikey11 wrote:a 5000 category network from people who already made a fantastic 1000 category network!
Sounds great! I knew that it was coming but not that it had already arrived. Demo videos at
https://www.youtube.com/watch?v=qVwLnYrzScc
https://www.youtube.com/watch?v=za8DzsHUgbw

I am a little concerned that more categories is not necessarily better for us, because forced abstraction (due to having less categories) can also mean less mistakes, e.g., "plant" versus "hibiscus", or "laptop" versus "Acer" (the 1,000 category network already consistently mistook my Asus laptop to be an Acer, and should better just have said "laptop" or similar).

Peter

User avatar
seeingwithsound
Posts: 165
Joined: Sun Aug 28, 2011 6:07 am
Contact: Website

Re: Sight for the Blind for <100$

Sun Jan 24, 2016 5:30 pm

mikey11 wrote:My latest filesystem image.

https://drive.google.com/file/d/0B30mp6 ... sp=sharing
Same pattern of camera successes and failures: Logitech C270 HD webcam always fails, Minoru 3D webcam (turns on but) fails, nameless cheap eBay UVC webcam works OK, Technaxx camera glasses work OK. The added distance indicator with Teradeep is very cool!

Peter

mr_indoj
Posts: 42
Joined: Wed Jul 01, 2015 9:28 am

Re: Sight for the Blind for <100$

Sun Jan 24, 2016 6:24 pm

Peter
Can you run these 2 commands from a terminal:

Code: Select all

sudo killall "python menu.py"
sudo python menu.py
This should relaunch the menu so you get terminal output, it will output a lot of stuff.
But, i'm wondering what happens if you try to start raspivoice in this mode, are you getting any form of error when starting raspivoice?

User avatar
seeingwithsound
Posts: 165
Joined: Sun Aug 28, 2011 6:07 am
Contact: Website

Re: Sight for the Blind for <100$

Sun Jan 24, 2016 8:56 pm

mr_indoj wrote:Peter
Can you run these 2 commands from a terminal:

Code: Select all

sudo killall "python menu.py"
sudo python menu.py
Even when not starting raspivoice, the system attempts to start the frame grabbing loop. With the problematic Logitech C270 I get

Code: Select all

ALSA lib pcm_dmix.c:1018:(snd_pcm_dmix_open) unable to open slave
aplay: main:682: audio open error: No such file or directory
and about a dozen lines later 14 lines reading

Code: Select all

VIDIOC_QUERYMENU: Invalid argument
Instead of the above, if I have the Technaxx glasses connected at startup, the initial lines read

Code: Select all

Playing WAVE 'stdin' : Signed 16 bit Little Endian, Rate 22050 Hz, Mono
while the lines containing VIDIOC_QUERYMENU do not show up.

So the difference appears to be the initial error message about a problem with the audio mixer, and the VIDIOC_QUERYMENU messages. Starting raspivoice does not give additional error messages with the Logitech C270. The nameless eBay webcam (which works with raspivoice) show the same "Playing WAVE 'stdin'" message as the Techaxx glasses, but also shows the VIDIOC_QUERYMENU error messages, so overall I do not spot a clear pattern yet except for the mixer error message with the failing cameras. The Minoru 3D (which fails with raspivoice) shows the same mixer error message and VIDIOC_QUERYMENU error messages as the Logitech C270. It almost looks as if audio output fails with some cameras, rather than their frame grabbing, but how could that be?

Thanks!

Peter

mr_indoj
Posts: 42
Joined: Wed Jul 01, 2015 9:28 am

Re: Sight for the Blind for <100$

Sun Jan 24, 2016 9:23 pm

Maybe, these cameras have audio support, that may create isues?
One thing i did was enable USB audio to be the default if plugged.
Try this:
in /etc/modprobe.d/alsa-base.conf
there is the lines:

Code: Select all

# Keep snd-usb-audio from beeing loaded as first soundcard
#options snd-usb-audio index=-2
Edit this file and remove the # on the 2nd line, and see if that works.

User avatar
seeingwithsound
Posts: 165
Joined: Sun Aug 28, 2011 6:07 am
Contact: Website

Re: Sight for the Blind for <100$

Sun Jan 24, 2016 9:43 pm

mr_indoj wrote:Maybe, these cameras have audio support, that may create isues?
...
Edit this file and remove the # on the 2nd line, and see if that works.
I think you nailed it! Both the Logitech C270 and the Minoru 3D webcam have a built-in microphone. I uncommented the line in /etc/modprobe.d/alsa-base.conf that you mentioned, and both the Logitech C270 and the Minoru 3D webcam now work fine with raspivoice! So something in the area of USB audio apparently needs fixing for the next image...

Thanks!

Peter

Return to “Assistive technology and accessibility”