mr_indoj
Posts: 42
Joined: Wed Jul 01, 2015 9:28 am

Re: Sight for the Blind for <100$

Mon Sep 07, 2015 9:30 pm

All this sounds good. I consider my current design as testing.
I have now a working teradeep/cloudsightapi combination as i wrote about before.
I have modified teradeep so that it reads image from file, analyze and speaks with espeak, then exit. I have created a script that is spawned from raspivoice, control then returns to raspivoice. Detection takes around 3-5 secs. As i said, this is just a test to see how it can work.

mikey11
Posts: 355
Joined: Tue Jun 25, 2013 6:18 am
Location: canada
Contact: Website

Re: Sight for the Blind for <100$

Mon Sep 07, 2015 11:34 pm

It does make sense that both could access a lower lever camera governing process through OpenCV, and this is a clever way of avoiding headaches from using the multiple languages.

Is it possible? Will it be easy? Who knows.

It also opens the door for any other camera based application to run concurrently.

I do wonder though, trying to run those processes over top of each other will surely incur some time penalties, I expect we will drop below the 1 frame/sec goal but this may be acceptable if it is being used in a training context.

Good thinking Peter

mikey11
Posts: 355
Joined: Tue Jun 25, 2013 6:18 am
Location: canada
Contact: Website

Re: Sight for the Blind for <100$

Mon Sep 07, 2015 11:35 pm

Also mr indoj, I don't suppose you could post a start to finish procedure for installing teradeep including the conversion and config steps you took?

mr_indoj
Posts: 42
Joined: Wed Jul 01, 2015 9:28 am

Re: Sight for the Blind for <100$

Tue Sep 08, 2015 7:59 pm

@mikey11: I will have to go to a fresh version of your image, and reproduce a working teradeep setup so that i can document the process. Then i guess the easiest solution is to pack up the needed files/install scripts and post a link here. At the time i did the torch7 install, the install scripts had to be modified to work on the pi, maybe they have fixed that since then.
I'll look into this.

mikey11
Posts: 355
Joined: Tue Jun 25, 2013 6:18 am
Location: canada
Contact: Website

Re: Sight for the Blind for <100$

Wed Sep 09, 2015 4:21 am

I'm currently syncing my latest image from July 5th which includes the code for the shutdown detection from the power switch.

Not every prototype has the circuitry for this, but I did test it one those that didn't, and it did not make a difference to operation.

When the file is synced, I will paste the link here.

I agree that making a lower level camera function is an ideal situation. We will investigate this. As long as we can still achieve the once/second framerate with low to no lag on the vOICe, I'm game.

We will see where this ends up.

In the meantime, I'm excited to start playing with teradeep. Some of the videos I saw indicated that you should be able to pull positional information (x/y location) from images, and possibly average the x/y value.

This would allow for a frequency assignment based on y position for the speech playback, and for the x position you could play the words back one after another for a rougher idea of position.

In an ideal world, playing back overlapping words could give better x position information, although it might be a little confusing at first.

mikey11
Posts: 355
Joined: Tue Jun 25, 2013 6:18 am
Location: canada
Contact: Website

Re: Sight for the Blind for <100$

Wed Sep 09, 2015 4:46 pm

The filesystem image from July 5th is located here:

https://drive.google.com/file/d/0B30mp6 ... sp=sharing

User avatar
seeingwithsound
Posts: 165
Joined: Sun Aug 28, 2011 6:07 am
Contact: Website

Re: Sight for the Blind for <100$

Thu Sep 10, 2015 6:42 pm

mikey11 wrote:The filesystem image from July 5th is located here: https://drive.google.com/file/d/0B30mp6 ... sp=sharing
@mikey11 Thanks, I flashed this image today and it basically appears to work fine upon a quick check. However, isn't there supposed to be a debounce filter on the rotary knob now? I still often get two commands spoken in slow succession when turning the know by one step or pressing it once. For instance, in the latter case a single press often gives "muted on" and "muted off". So either the debounce filter is not there yet or it is not reliable. On June 21 I proposed "Blocking button responses for 500ms after a detected change should suffice as a software debounce", but it looks like the current filter, if present, does not work like that.

Thanks,

Peter

mikey11
Posts: 355
Joined: Tue Jun 25, 2013 6:18 am
Location: canada
Contact: Website

Re: Sight for the Blind for <100$

Thu Sep 10, 2015 7:27 pm

The debounce has not been implemented on that image. I want to get that and teradeep on the next image.

On the bright side, I am very close to being finished a more mechanically stable case. I am going to replace the horizontal switch with a pushbutton, and am just waiting on the part to come back in stock.

I got permission to remix a model used in the Atlas3D scanner. It uses screws and nuts to anchor the pi, allowing for better disassembly for repairs. It also has a 'perfect' fit for the various ports and protects them better.

I expect these to be the last hardware tweaks for the near future.

Then I can refocus myself on the software and get the improvements integrated.

mikey11
Posts: 355
Joined: Tue Jun 25, 2013 6:18 am
Location: canada
Contact: Website

Re: Sight for the Blind for <100$

Sat Sep 12, 2015 1:45 am

I've tried to apply mikael's diff to the raspivoice code, but I just get errors.

Code: Select all

pi@raspberrypi ~/raspivoice $ sudo patch < mikael2.patch
patching file rotaryencoder.cpp
Hunk #2 FAILED at 25.
Hunk #3 FAILED at 32.
Hunk #4 FAILED at 43.
Hunk #5 FAILED at 69.
4 out of 5 hunks FAILED -- saving rejects to file rotaryencoder.cpp.rej
patching file rotaryencoder.h
Hunk #2 FAILED at 18.
1 out of 2 hunks FAILED -- saving rejects to file rotaryencoder.h.rej
He sent me a PM with another version of the diff, which yields results similar to the one listed on page 13 of this thread.

I'm new to using patch/diff.

I assumed that the information in diff format uses patch to apply the information.

I have started from a fresh git clone of raspivoice. I have tried numerous times, and have also tried the flag -p1 in case there are any references to subdirectories other than ~/raspivoice/

I will keep on whacking at this for a while, but anyone who thinks they know what I'm doing wrong can save me from myself.

PranavLal
Posts: 129
Joined: Fri Jun 28, 2013 4:49 pm

Re: Sight for the Blind for <100$

Sat Sep 12, 2015 3:18 am

Hi all,

I am not sure why but I was not getting any e-mail notification about updates to this thread. The protype is working quite well.
1. Menu navigation is still a tad tricky probably due to the debounce filter issue.
2. The switch is nice to have. The push button is ok but the switch gives me positional information about the device being on or off. This is handy with my wired aftershokz bone conduction headphones. If I turn the device on, I can be sure that the device is running if the switch is set to the on position. If I do not hear any sound, then that means that the headphone batteries are dead. I currently use the sonic obstacle detector for detecting this. If the detector powers up, great, if it does not, then I know the battery is dead.
3. Michael, what image did you send me? If I need to update, please let me know and I will be happy to do so.
4. The sonic obstacle detector is giving me more audio feedback than tactile feedback. The noise has to be masked yet I should be able to feel the vibration. I suspect case design and material would need to be modified. This is not a show stopper but that detector is noisy. <chuckle
5. How long does the internal battery take to charge?

mikey11
Posts: 355
Joined: Tue Jun 25, 2013 6:18 am
Location: canada
Contact: Website

Re: Sight for the Blind for <100$

Sat Sep 12, 2015 3:34 am

When the horizontal switch is replaced with the pushbutton, there will still be difference in button height to the case top, so you should still be able to tell.

You have the latest image installed. Once I get the debounce patch to work I will post a link to the updated image to this forum.

Regarding the rangefinder/vibrator, I agree. The method I used with it is based on pulse width modulation. The vibration motors tend to be the loudest the longer its activated. Currently this results in the opposite of what I would want; when at 5 meters distance, it is loudest, but less frequent. At less than one meter it is quietest but more frequent. I would want the volume to stay the same ideally, but I have not invested time into solving this, but I had a few ideas regarding changing the frequency and duty cycle to achieve this.

As the time for hardware development at this stage is almost completed (I expect less than three weeks), I will be able to focus on taking contributions and integrating them. While I am doing this I will review the vibration motor behaviour as well.

The internal battery is 2000 mAh. The power module can charge at 1000 mA, so if efficiency was 100%, with a one amp charger, you would have it done in roughly two hours. The reality isn't that nice, but with a one amp charger, I have charged it in under three hours. Many chargers are rated for less, and take longer. Most USB ports on computers only supply 500 mA, so would take about 5 hours.

The nice thing about this power module (which is improved from the version Peter and Ares recieved) is that you can hook up an external battery and it will charge the internal battery while the device is in use. I have achieved very long run times with a 10,000 mAh external battery that fits in my pocket. More than enough for a whole day.

I saw in one of your posts that you had a 50,000 mAh pack if I remember correctly. With that, you should get the better part of a week.

In future versions, I am considering using a larger capacity battery by enlarging the case. The 2000 mAh has been fine at this level of prototyping, but for full time extended use, I want to get at least 6000 mAh onboard, maybe more. This will be something I put off until I have more disposable income, or fundraising takes place.

User avatar
seeingwithsound
Posts: 165
Joined: Sun Aug 28, 2011 6:07 am
Contact: Website

Re: Sight for the Blind for <100$

Sat Sep 12, 2015 7:39 am

mikey11 wrote:anyone who thinks they know what I'm doing wrong can save me from myself.
@mikey11 I never use 'patch' myself, but would suspect that the version of raspivoice code that you try to apply the patch to is different from the one that Mikael used to generate the diff. Since there are only a few errors, you can try to patch manually as suggested at http://stackoverflow.com/questions/1428 ... n/14282775. In fact I made these manual changes myself, based on the patch posted at viewtopic.php?p=794388#p794388, and the result after recompilation gave me a great improvement to the debounce issue (perhaps it is gone completely, but I did only some brief testing moments ago). I'll post the details of the patched rotaryencoder.cpp and rotaryencoder.h files later today under separate cover.

@PranavLal To get e-mail notifications, you need to visit the forum via the link in one of the forum e-mails which read "You can use the following link to view the replies made, no more notifications will be sent until you visit the topic", e.g. http://www.raspberrypi.org/forums/viewt ... 3&e=814543

Peter


Seeing with Sound - The vOICe
http://www.seeingwithsound.com

User avatar
seeingwithsound
Posts: 165
Joined: Sun Aug 28, 2011 6:07 am
Contact: Website

Re: Sight for the Blind for <100$

Sat Sep 12, 2015 8:03 am

The patched source files are fairly small, so I'll post them here inline.

Patched rotaryencoder.cpp:

Code: Select all

/*
Updated version including press switch functionality
based on:
rotaryencoder by astine
http://theatticlight.net/posts/Reading-a-Rotary-Encoder-from-a-Raspberry-Pi/
https://github.com/astine/rotaryencoder
*/
#include <wiringPi.h>

#include <stdio.h>
#include <stdlib.h>
#include <stdint.h>

#include "rotaryencoder.h"
#include <chrono>
void updateEncoders(void);
void updateSwitch(void);

//Pre-allocate encoder objects on the stack so we don't have to 
//worry about freeing them
struct encoder encoders[max_encoders];

int numberofencoders = 0;

void updateEncoders()
{
	struct encoder *encoder = encoders;
   unsigned long nowTime = std::chrono::system_clock::now().time_since_epoch() /
                           std::chrono::milliseconds(1);   
	for (; encoder < encoders + numberofencoders; encoder++)
	{
		int MSB = digitalRead(encoder->pin_a);
		int LSB = digitalRead(encoder->pin_b);

		int encoded = (MSB << 1) | LSB;
		int sum = (encoder->lastEncoded << 2) | encoded;
      if ((nowTime - encoder->lasttime) < 200)
      {

         encoder->lasttime = nowTime;
         encoder->lastEncoded = encoded;
         break;
      }
      encoder->lasttime = nowTime;

		if (sum == 0b1101 || sum == 0b0100 || sum == 0b0010 || sum == 0b1011) encoder->value++;
		if (sum == 0b1110 || sum == 0b0111 || sum == 0b0001 || sum == 0b1000) encoder->value--;

		encoder->lastEncoded = encoded;
	}
}

void updateSwitch()
{
	struct encoder *encoder = encoders;
   unsigned long nowTime = std::chrono::system_clock::now().time_since_epoch() /
                           std::chrono::milliseconds(1);
   
	for (; encoder < encoders + numberofencoders; encoder++)
	{
		int pin_state = digitalRead(encoder->pin_switch);
		if (pin_state == LOW)
		{
			//TODO: Debounce with millis() ? 
         if ((nowTime - encoder->lasttime) > 200)
         {

            encoder->switchpresscount++;
         }
         encoder->lasttime = nowTime;
		}
	}
}

struct encoder *setupencoder(int pin_a, int pin_b, int pin_switch = -1)
{
	if (numberofencoders > max_encoders)
	{
		printf("Maximum number of encodered exceded: %i\n", max_encoders);
		return NULL;
	}

	struct encoder *newencoder = encoders + numberofencoders++;
	newencoder->pin_a = pin_a;
	newencoder->pin_b = pin_b;
	newencoder->pin_switch = pin_switch;
	newencoder->value = 0;
	newencoder->lastEncoded = 0;
	newencoder->switchpresscount = 0;
   newencoder->lasttime = 0;

	pinMode(pin_a, INPUT);
	pinMode(pin_b, INPUT);
	pullUpDnControl(pin_a, PUD_UP);
	pullUpDnControl(pin_b, PUD_UP);
	wiringPiISR(pin_a, INT_EDGE_BOTH, updateEncoders);
	wiringPiISR(pin_b, INT_EDGE_BOTH, updateEncoders);
	if (pin_switch != -1)
	{
		pinMode(pin_switch, INPUT);
		pullUpDnControl(pin_switch, PUD_UP);
		wiringPiISR(pin_switch, INT_EDGE_FALLING, updateSwitch);
	}

	return newencoder;
}
Patched rotaryencoder.h:

Code: Select all

/*
Updated version including press switch functionality
based on:
rotaryencoder by astine
http://theatticlight.net/posts/Reading-a-Rotary-Encoder-from-a-Raspberry-Pi/
https://github.com/astine/rotaryencoder
*/

#pragma once

//17 pins / 3 pins per encoder = 5 maximum encoders
#define max_encoders 5

struct encoder
{
	int pin_a;
	int pin_b;
	int pin_switch;
	volatile long value;
	volatile long switchpresscount;
	volatile int lastEncoded;
   volatile unsigned long lasttime;
};

extern struct encoder encoders[max_encoders];

/*
Should be run for every rotary encoder you want to control
Returns a pointer to the new rotary encoder structer
The pointer will be NULL is the function failed for any reason
*/
struct encoder *setupencoder(int pin_a, int pin_b, int pin_switch);
I then recompiled raspivoice using "make -j 4 CONFIG=release_rpi2" as described in your instructions in ~/raspivoice/INSTALL.

Peter


Seeing with Sound - The vOICe
http://www.seeingwithsound.com

User avatar
seeingwithsound
Posts: 165
Joined: Sun Aug 28, 2011 6:07 am
Contact: Website

Re: Sight for the Blind for <100$

Sat Sep 12, 2015 9:53 am

For your information: I found a new publication about another open source sensory substitution project named "LibreAudioView", with the provisional PDF at http://journal.frontiersin.org/article/ ... .00020/pdf. It looks like it is based on some Android TV box, together with earbuds that do not block the ears for environmental sound, and a UVC camera mounted in a 3D-printed glasses frame. It might be interesting to check out. The LibreAudioView web page at http://leadserv.u-bourgogne.fr/en/membe ... eaudioview currently states that it is under construction.

Peter


Seeing with Sound - The vOICe
http://www.seeingwithsound.com

PranavLal
Posts: 129
Joined: Fri Jun 28, 2013 4:49 pm

Re: Sight for the Blind for <100$

Sat Sep 12, 2015 10:26 am

Hi Michael and Peter,

Thanks for the information.

Michael, I do have a 50K MAH battery pack but I am not sure if that rating is accurate.

I did try the pi with a car charger. The internal battery was dead but the pi did not run off the USB port of the charger. I will be testing with my battery pack and will report on what happens.

mr_indoj
Posts: 42
Joined: Wed Jul 01, 2015 9:28 am

Re: Sight for the Blind for <100$

Sat Sep 12, 2015 3:15 pm

seeingwithsound wrote: @mikey11 I never use 'patch' myself, but would suspect that the version of raspivoice code that you try to apply the patch to is different from the one that Mikael used to generate the diff.
The patch is made against the git version of the files, just tried my original patch file and it works. So i guess the patch easily gets messed up when posted. So it seems best to avoid posting diffs, unless prepared to apply them manually.
Thanks Peter for posting the patched files, i guess its the easiest solution for now.
These issues will be gone when we have a repository for the project.

mikey11
Posts: 355
Joined: Tue Jun 25, 2013 6:18 am
Location: canada
Contact: Website

Re: Sight for the Blind for <100$

Sat Sep 12, 2015 3:31 pm

I am syncing the new filesystem image now

mikey11
Posts: 355
Joined: Tue Jun 25, 2013 6:18 am
Location: canada
Contact: Website

Re: Sight for the Blind for <100$

Sat Sep 12, 2015 8:35 pm

https://drive.google.com/file/d/0B30mp6 ... sp=sharing

New filesystem image with the debounce integrated

PranavLal
Posts: 129
Joined: Fri Jun 28, 2013 4:49 pm

Re: Sight for the Blind for <100$

Sun Sep 13, 2015 2:30 am

Hi,

How do I access the memory card of the unit?

Pranav

User avatar
seeingwithsound
Posts: 165
Joined: Sun Aug 28, 2011 6:07 am
Contact: Website

Re: Sight for the Blind for <100$

Sun Sep 13, 2015 6:11 am

PranavLal wrote:How do I access the memory card of the unit?
If your casing is similar to mine, there will be a horizontal slit in the casing opposite to the side with the multiple USB connectors. The microSD card is in a sunken position behind that slit, and is held with a spring mechanism. It can be clicked in and out using something like a screw driver, because it lies too deep to reach with your fingernail. The microSD card contacts point upward. The above-mentioned slit is centered near the bottom of the casing, in my case below a set of 45-degree vent openings of the 3D-printed casing.

Peter

PranavLal
Posts: 129
Joined: Fri Jun 28, 2013 4:49 pm

Re: Sight for the Blind for <100$

Sun Sep 13, 2015 5:11 pm

Hi Peter and all,

Peter, many thanks for the precise description. I was able to get the card out using a screw driver. The problem came in inserting it back in. That took a good bit of doing though in some ways, it is easier than taking it out. More case redesign.

The best thing would be not to have to take the card out at all. This may be doable if we can update using ansible. I have got ansible installed in a Linux virtual machine but do not know how to deploy images. I have however asked this question on
https://julien.ponge.org/blog/scalable- ... d-vagrant/
I hope I get a response.

In other news, my cameras from China which I used with my immersive vision kit work nicely with this device. It was plug and play.

Michael, what does the threshold option do? I found increasing the threshold helped with image clarity.

mikey11
Posts: 355
Joined: Tue Jun 25, 2013 6:18 am
Location: canada
Contact: Website

Re: Sight for the Blind for <100$

Mon Sep 14, 2015 12:10 am

I actually hadn't used the threshold option before.

I set up a preview window to observe it, and it appears to be simplifying the image from greyscale to black and white, and at the same time with each increasing step of threshold setting, it makes areas of lower brightness black.

This looks like a useful way to find out where the light sources in a room are located, or what in a room is reflecting the most light.

PranavLal
Posts: 129
Joined: Fri Jun 28, 2013 4:49 pm

Re: Sight for the Blind for <100$

Mon Sep 14, 2015 1:17 am

Hi Michael,

Hmm, as far as I know, the algorithm of the vOICe only uses grayscale so I do not think the black and white image would play a significant role. Yes, filtering out dark pixels may though which explains why objects are clearer.

Pranav

mikey11
Posts: 355
Joined: Tue Jun 25, 2013 6:18 am
Location: canada
Contact: Website

Re: Sight for the Blind for <100$

Tue Sep 29, 2015 1:51 am

Gentlepeople,

I have made a major revision to the PCB which I have just sent off for production.

On this version, the major features added were to address:

1. The power switch: It used to be a horizontal movement, but the 3D printed parts were not designed well to handle the lateral forces. the whole implementation of that switch was bad. I have replaced the switch with two possibilities, first, a pushbutton switch located on the circuit board that is located above the rotary knob. This switch is a little bit easy to trigger, so I'm not convinced it is a good idea yet, but it solves the lateral forces problem. I have also obtained a case mount pushbutton switch that is considerably harder to press. I like this because it will be harder to accidentally turn the unit on/off without meaning to.

2. I have added (but not tested) a detection mechanism for telling when external power is plugged into the unit. This will be accessible by another GPIO. When I get the new boards assembled, I will add a few commands to the rangefinder2.py code to monitor the pin and then you can do whatever you want with that information. After testing, I will probably comment them out when making new filesystem images.

I expect to have this work completed within 10 days if all goes well.

If it all passes testing, I will proceed with creating a repository for this project which includes all files, and the up-to-date raspivoice with mikaels debounce. Everything from the 3D models to the PCB. I will also put together an assembly video to take the electronics from start to finish.

Thanks,

PranavLal
Posts: 129
Joined: Fri Jun 28, 2013 4:49 pm

Re: Sight for the Blind for <100$

Tue Sep 29, 2015 4:05 pm

Hi Mikey11,

Great news and thanks for the changes. Have you had a chance to look at any raspberry pi compatible boards such as the banana pi or the orange pi?

The case mounted push button switch sounds nice. There were times when I was not sure if the switch was on or off. I have left the unit switched on by mistake and have found it drained.

Pranav

Return to “Assistive technology and accessibility”