Hi Derek ,
I have not posted for a while so I thought I would tell you what I have been up to.
Building the latest version of opencv fixed the issue where the window sizes were changing.
I ran into problems detecting symbols when my robot was moving, so I decided to stream the processed frame back to my iPad.
That has been an interesting diversion, looking at several solutions from CoderBot, Dexter industry and others. I could not find anyone who had done it from opencv until I came across a blog from Miguel Grinberg who used flask. His example streams an io.byteio image, so I am getting an error when I try and stream an opencv frame!
If nothing else, your magpi article has increased my knowledge in all sorts of areas.
Steve
Re: PiTeR, OpenCV and MagPi issue 28
Yes absolutely, I don't know if it came across in the articles, but I would recommend anyone to build a robot, simply for the amount of new knowledge you acquire while doing it!
I thought about streaming the data back to a host. Like you, I wasn't satisfied with the deadlines available when processing the images on board the robot. I was just using filesystem based streaming to mplayer (I think it was) on the host. I haven't yet experimented with getting the data from that format into OpenCV on a host. Getting command data back certainly shouldn't be a problem.
As you can see above, I'm planning on fitting PiTeR with a Pi 2 which I think will make a huge difference in performance, since the computer vision processing can be run on separate parallel threads by the quad core processor. I'm thinking that will make all the difference in the world if Pythons threading takes advantage of the multi-core processor as I am expecting it to. I really want the design to be fully autonomous so that it is not dependent on a wi-fi or other wireless link.
There are so many places I take PiTeR that don't have easy access to such infrastructure. Sometimes, I don't even have a host computer with me.
I thought about streaming the data back to a host. Like you, I wasn't satisfied with the deadlines available when processing the images on board the robot. I was just using filesystem based streaming to mplayer (I think it was) on the host. I haven't yet experimented with getting the data from that format into OpenCV on a host. Getting command data back certainly shouldn't be a problem.
As you can see above, I'm planning on fitting PiTeR with a Pi 2 which I think will make a huge difference in performance, since the computer vision processing can be run on separate parallel threads by the quad core processor. I'm thinking that will make all the difference in the world if Pythons threading takes advantage of the multi-core processor as I am expecting it to. I really want the design to be fully autonomous so that it is not dependent on a wi-fi or other wireless link.
There are so many places I take PiTeR that don't have easy access to such infrastructure. Sometimes, I don't even have a host computer with me.
Guzunty: A fully programmable peripheral you build yourself! https://github.com/Guzunty/Pi/wiki
Re: PiTeR, OpenCV and MagPi issue 28
I promised a few weeks ago that I would grab some performance statistics from the symbol tracking OpenCV programs I presented in the MagPi.
Well, now I have and I think they make for some interesting reading. First up some background info; I wasn't happy with the frame rates and latencies I was seeing. In particular PiTeR can't steer himself very well if the data he is using is several seconds old.
I started a Google campaign to see if I could find ways to improve matters. I have heard various hypotheses about buffering and frame rate and size settings for the camera. Also, some ideas about choosing a different pixel format. I experimented with all these and here's what I found:
Frame Rate: I saw suggestions that reducing the frame rate from the camera would help. It does, but the slowest possible rate in practice of 3 fps still leaves a significant lag, especially on slower Pis. I read some other comments that suggested a technique where you leave the frame rate at the default 30fps but deal with the buffering that builds up by looking at how long it took to grab a frame. If its too quick, under 10ms, the frame came from the buffer and not from the camera so we can throw it away and grab another until we have to wait longer (33ms would be the frame period at 30 fps). This works quite well, so I included that in the tracking python code.
Frame size: The default frame size is 640 x 480. I think PiTeR can do quite well at 320 x 240. Unfortunately OpenCV 2.4.9 has something broken in the capture settings code and refuses to set the correct values. I tried OpenCV 2.4.10 and blogged that it too had this problem. I tried 2.4.10 again this week and found that I was either wrong or it has been fixed by a minor release. So I have changed the test programs to use the smaller frame size. This makes a significant difference to the frame rate.
Pixel format: The camera produces JPEG encoded frames while OpenCV requires an uncompressed internal format. We are after all slicing images up and doing processing for edge detection and so on. No matter how you try to do it, the frame needs to be uncompressed in memory for OpenCV. There was suggestion that we should set the camera driver to provide a format that OpenCV can work with natively. I tried this and found no significant difference. I suspect that the decompression/buffer copy overheads are such that you pay the price somewhere along the pipeline no matter what you do. So I concluded that sticking with the standard compressed format is best and OpenCV decompresses it as it captures the frame.
So that sums up the changes to the test programs. I will push these up to GitHub in the next day or two so that you can reproduce my findings. Feel free to comment and experiment of course.
I took measurements of every significant step during the symbol recognition flow and will present all the data I gathered. I collected the following data:
Acquisition time (AQU) - The time taken to grab a frame from the camera. This doesn't include the time spent throwing buffered frames away described above. You could measure that too if you want to.
Conversion Time (CVT) - The time taken to convert the frame into Hue/Sat/Value format used by inRange to locate the colour of the symbol.
In Range Time (IRG) - The time taken to compute the pixels in the required colour range.
Mask time (MSK) - The time taken to eliminate noise and holes from the mask. This includes the erode and dilate calls.
Contour Time (CTR) - The time taken to identify all the contours in the mask.
Rectangle Time (RCT) - The time taken identify the largest contour and compute the smallest rectangle that contains it.
Crop time (CRP) - The time taken to crop the frame to just the area of colour found.
Histogram time (HST) - The time taken to normalise the exposure and contrast ready for recognition.
Detect time (DCT) - The time taken to detect the significant features in the cropped image.
Match time (MCH) - The time taken to match the detected features with the ones computed in the reference image.
Decoration time (DEC) - The time taken to decorate the original frame with the region of interest rectangle and the transformed target boundary.
Display time (DSP) - The time taken to display the decorated image in the Camera View window.
Total time (TTL) - The total time taken for processing the image.
Seconds per frame (SPF) - The bottom line time we're all interested in.
I only estimated lag by putting my hand into the path of the camera and observing how long it took to appear. We could probably figure out a way to measure this accurately, but I wanted to focus on the frame rate and processing times for now.
I ran the test on three Pi models, A+, B+ and Pi 2. I didn't test older Pis because I think they will show similar times to the older models tested here. I performed the test in single threaded and multi threaded modes for each model.
You're probably champing at the bit to see the results now, so I'll post them below.
In the interests of readability on the forum, I edited out the less interesting data points, those which were under 10ms most of the time. I'm aware that this will in some cases hide interesting features in the data, so I will put the full measurements up on GitHub for your examination.
Well, now I have and I think they make for some interesting reading. First up some background info; I wasn't happy with the frame rates and latencies I was seeing. In particular PiTeR can't steer himself very well if the data he is using is several seconds old.
I started a Google campaign to see if I could find ways to improve matters. I have heard various hypotheses about buffering and frame rate and size settings for the camera. Also, some ideas about choosing a different pixel format. I experimented with all these and here's what I found:
Frame Rate: I saw suggestions that reducing the frame rate from the camera would help. It does, but the slowest possible rate in practice of 3 fps still leaves a significant lag, especially on slower Pis. I read some other comments that suggested a technique where you leave the frame rate at the default 30fps but deal with the buffering that builds up by looking at how long it took to grab a frame. If its too quick, under 10ms, the frame came from the buffer and not from the camera so we can throw it away and grab another until we have to wait longer (33ms would be the frame period at 30 fps). This works quite well, so I included that in the tracking python code.
Frame size: The default frame size is 640 x 480. I think PiTeR can do quite well at 320 x 240. Unfortunately OpenCV 2.4.9 has something broken in the capture settings code and refuses to set the correct values. I tried OpenCV 2.4.10 and blogged that it too had this problem. I tried 2.4.10 again this week and found that I was either wrong or it has been fixed by a minor release. So I have changed the test programs to use the smaller frame size. This makes a significant difference to the frame rate.
Pixel format: The camera produces JPEG encoded frames while OpenCV requires an uncompressed internal format. We are after all slicing images up and doing processing for edge detection and so on. No matter how you try to do it, the frame needs to be uncompressed in memory for OpenCV. There was suggestion that we should set the camera driver to provide a format that OpenCV can work with natively. I tried this and found no significant difference. I suspect that the decompression/buffer copy overheads are such that you pay the price somewhere along the pipeline no matter what you do. So I concluded that sticking with the standard compressed format is best and OpenCV decompresses it as it captures the frame.
So that sums up the changes to the test programs. I will push these up to GitHub in the next day or two so that you can reproduce my findings. Feel free to comment and experiment of course.
I took measurements of every significant step during the symbol recognition flow and will present all the data I gathered. I collected the following data:
Acquisition time (AQU) - The time taken to grab a frame from the camera. This doesn't include the time spent throwing buffered frames away described above. You could measure that too if you want to.
Conversion Time (CVT) - The time taken to convert the frame into Hue/Sat/Value format used by inRange to locate the colour of the symbol.
In Range Time (IRG) - The time taken to compute the pixels in the required colour range.
Mask time (MSK) - The time taken to eliminate noise and holes from the mask. This includes the erode and dilate calls.
Contour Time (CTR) - The time taken to identify all the contours in the mask.
Rectangle Time (RCT) - The time taken identify the largest contour and compute the smallest rectangle that contains it.
Crop time (CRP) - The time taken to crop the frame to just the area of colour found.
Histogram time (HST) - The time taken to normalise the exposure and contrast ready for recognition.
Detect time (DCT) - The time taken to detect the significant features in the cropped image.
Match time (MCH) - The time taken to match the detected features with the ones computed in the reference image.
Decoration time (DEC) - The time taken to decorate the original frame with the region of interest rectangle and the transformed target boundary.
Display time (DSP) - The time taken to display the decorated image in the Camera View window.
Total time (TTL) - The total time taken for processing the image.
Seconds per frame (SPF) - The bottom line time we're all interested in.
I only estimated lag by putting my hand into the path of the camera and observing how long it took to appear. We could probably figure out a way to measure this accurately, but I wanted to focus on the frame rate and processing times for now.
I ran the test on three Pi models, A+, B+ and Pi 2. I didn't test older Pis because I think they will show similar times to the older models tested here. I performed the test in single threaded and multi threaded modes for each model.
You're probably champing at the bit to see the results now, so I'll post them below.
In the interests of readability on the forum, I edited out the less interesting data points, those which were under 10ms most of the time. I'm aware that this will in some cases hide interesting features in the data, so I will put the full measurements up on GitHub for your examination.
Last edited by guzunty on Fri Feb 27, 2015 2:49 pm, edited 3 times in total.
Guzunty: A fully programmable peripheral you build yourself! https://github.com/Guzunty/Pi/wiki
Re: PiTeR, OpenCV and MagPi issue 28
So here's the data for the A+:
The first thing to note here is that the Multi threaded implementation is actually slower than the single threaded. On a single core processor, this is not surprising. Also not surprising is that feature detection eats a significant part of the flow time, it's a complex job. What is surprising is what is not in the list, I had expected the time to match would have been significant and the time to transfer the image to the X widget would have been more too.
Note that the best case values for detection etc. are misleading. Sometimes a frame comes from the camera with its exposure far enough out that nothing can be seen, so the program wastes no time processing it.
Code: Select all
A+ Single Thread
AVG -> AQU: 0.016508, IRG: 0.016409, MSK: 0.112361, DCT: 0.186243, DEC: 0.017875, TTL: 0.377341, SPF: 0.623902
MAX -> AQU: 0.518836, IRG: 0.036090, MSK: 0.165004, DCT: 0.403119, DEC: 0.046400, TTL: 1.142532, SPF: 1.661459
MIN -> AQU: 0.010041, IRG: 0.007995, MSK: 0.085699, DCT: 0.000801, DEC: 0.000000, TTL: 0.114781, SPF: 0.223755
A+ Multi Thread
AVG -> AQU: 0.017280, IRG: 0.015037, MSK: 0.154035, DCT: 0.385814, DEC: 0.034976, TTL: 0.464018, SPF: 0.690040
MAX -> AQU: 0.051271, IRG: 0.058608, MSK: 0.323409, DCT: 0.505442, DEC: 0.117074, TTL: 1.168557, SPF: 1.368286
MIN -> AQU: 0.010015, IRG: 0.007453, MSK: 0.088895, DCT: 0.287913, DEC: 0.000000, TTL: 0.347180, SPF: 0.403973
Note that the best case values for detection etc. are misleading. Sometimes a frame comes from the camera with its exposure far enough out that nothing can be seen, so the program wastes no time processing it.
Last edited by guzunty on Fri Feb 27, 2015 1:22 pm, edited 1 time in total.
Guzunty: A fully programmable peripheral you build yourself! https://github.com/Guzunty/Pi/wiki
Re: PiTeR, OpenCV and MagPi issue 28
Now the B+:
The B+ is a bit quicker than the A+, the most likely explanation for this is that the system is memory constrained and some paging is happening. Not a huge amount though. Again, the multi threaded data shows that adding threading is not advantageous in a single core processor.
Code: Select all
B+ Single Thread
AVG -> AQU: 0.017099, IRG: 0.013638, MSK: 0.102850, DCT: 0.180537, DEC: 0.018630, TTL: 0.357363, SPF: 0.600208
MAX -> AQU: 0.623729, IRG: 0.032538, MSK: 0.147921, DCT: 0.268542, DEC: 0.068043, TTL: 1.185496, SPF: 1.809293
MIN -> AQU: 0.010018, IRG: 0.008008, MSK: 0.086739, DCT: 0.141226, DEC: 0.000000, TTL: 0.293192, SPF: 0.366759
B+ Multi Thread
AVG -> AQU: 0.017886, IRG: 0.013282, MSK: 0.142662, DCT: 0.368923, DEC: 0.038965, TTL: 0.447509, SPF: 0.680788
MAX -> AQU: 0.047882, IRG: 0.040596, MSK: 0.237612, DCT: 0.468348, DEC: 0.192369, TTL: 1.008803, SPF: 1.189154
MIN -> AQU: 0.010009, IRG: 0.007347, MSK: 0.089492, DCT: 0.255277, DEC: 0.000000, TTL: 0.311580, SPF: 0.388996
Last edited by guzunty on Fri Feb 27, 2015 1:22 pm, edited 1 time in total.
Guzunty: A fully programmable peripheral you build yourself! https://github.com/Guzunty/Pi/wiki
Re: PiTeR, OpenCV and MagPi issue 28
Finally, the Pi 2:
As expected, the multi threaded implementation is faster, because two cores are in use. The single threaded implementation is also slightly faster than the B+. The figures are a definite improvement, about a 25% improvement when comparing the B+ single threaded case with the Pi 2 multi threaded case. However, it's not as dramatic a gain as one might have expected. I suspect that this is because the Pi shares the same GPU as the older Pis and the GPU is handling a lot of the camera load.
On the plus side, I noted that the cpu load, as shown by htop, is much reduced. This is also visible in the above figures; TTL is the time to process the image, excluding acquisition time. It is over 300ms on the B+ and less than 90ms on the Pi 2. That's a pretty dramatic difference. We therefore have additional resources at our disposal. In PiTeRs case, he needs to run the detect and match steps multiple times to differentiate between the symbols. This could now be done on multiple threads with each match result reaching the decision deadline that much faster.
My conclusion is therefore that your robot will definitely benefit from switching to a Pi 2. If you are tracking coloured objects as opposed to doing full symbol or face recognition, the difference will be less noticeable, but faster is always better.
Code: Select all
Pi 2 Single Thread
AVG -> AQU: 0.024807, IRG: 0.004034, MSK: 0.062391, DCT: 0.120350, DEC: 0.009744, TTL: 0.210951, SPF: 0.511227
MAX -> AQU: 0.531965, IRG: 0.009787, MSK: 0.095889, DCT: 0.168232, DEC: 0.034217, TTL: 0.530957, SPF: 1.062964
MIN -> AQU: 0.010046, IRG: 0.003695, MSK: 0.058993, DCT: 0.059665, DEC: 0.004077, TTL: 0.144755, SPF: 0.188589
Pi 2 Multi Thread
AVG -> AQU: 0.017908, IRG: 0.003730, MSK: 0.061656, DCT: 0.075101, DEC: 0.007501, TTL: 0.089679, SPF: 0.431032
MAX -> AQU: 0.613057, IRG: 0.010669, MSK: 0.108700, DCT: 0.126946, DEC: 0.022022, TTL: 0.619609, SPF: 1.386138
MIN -> AQU: 0.010028, IRG: 0.003486, MSK: 0.059037, DCT: 0.057723, DEC: 0.000000, TTL: 0.071066, SPF: 0.081818
On the plus side, I noted that the cpu load, as shown by htop, is much reduced. This is also visible in the above figures; TTL is the time to process the image, excluding acquisition time. It is over 300ms on the B+ and less than 90ms on the Pi 2. That's a pretty dramatic difference. We therefore have additional resources at our disposal. In PiTeRs case, he needs to run the detect and match steps multiple times to differentiate between the symbols. This could now be done on multiple threads with each match result reaching the decision deadline that much faster.
My conclusion is therefore that your robot will definitely benefit from switching to a Pi 2. If you are tracking coloured objects as opposed to doing full symbol or face recognition, the difference will be less noticeable, but faster is always better.
Last edited by guzunty on Sat Feb 28, 2015 3:58 pm, edited 2 times in total.
Guzunty: A fully programmable peripheral you build yourself! https://github.com/Guzunty/Pi/wiki
Re: PiTeR, OpenCV and MagPi issue 28
The modified tracking.py and the new multithreaded programs are now in GitHub for anyone who would like to reproduce my measurements.
In case it is not obvious, tracking.py was used to produce the single threaded results and trackingWithThreads.py produced the multi threaded results. The latter uses a Thread based utility class, symbolFinder to acquire and pre-process the image to locate the area of the image that contains the symbol to be identified. The main thread meanwhile performs the feature detection and matching while the utility class loops back to grab and pre-process the next image in parallel.
The version of OpenCV used was 2.4.11. The point increment (which I just noticed) explains how the frame capture settings regression got fixed.
I built this from source pulled from the OpenCV GitHub, branch "2.4", not master. The default master branch will give you 3.0.0 which may or may not work, there are quite a lot of changes I believe so it would likely require some work to the demo code to get it working with 3.0.0. Build time is just over 1 hour on the Pi 2. I'm told it's over 4 hours on a single core Pi.
Enjoy
In case it is not obvious, tracking.py was used to produce the single threaded results and trackingWithThreads.py produced the multi threaded results. The latter uses a Thread based utility class, symbolFinder to acquire and pre-process the image to locate the area of the image that contains the symbol to be identified. The main thread meanwhile performs the feature detection and matching while the utility class loops back to grab and pre-process the next image in parallel.
The version of OpenCV used was 2.4.11. The point increment (which I just noticed) explains how the frame capture settings regression got fixed.
I built this from source pulled from the OpenCV GitHub, branch "2.4", not master. The default master branch will give you 3.0.0 which may or may not work, there are quite a lot of changes I believe so it would likely require some work to the demo code to get it working with 3.0.0. Build time is just over 1 hour on the Pi 2. I'm told it's over 4 hours on a single core Pi.
Enjoy
Guzunty: A fully programmable peripheral you build yourself! https://github.com/Guzunty/Pi/wiki
-
pepelepoisson
- Posts: 1
- Joined: Thu May 14, 2015 4:18 pm
Re: PiTeR, OpenCV and MagPi issue 28
Hi Derek ,
I discoved PiTeR in MagPi 28 and thought "Wahoooo, this will be a super duper project to learn robotics, OpenCV, Arduino, Guzunti all together
". Well, all together might be a little bit too much so I'm attempting to concentrate on the software side first (and recuce cost apparently) by starting from sainsmart upright rover kit (V3) rather than buying driver + motors + wheels separately.
http://www.sainsmart.com/robotics/sains ... t-kit.html
The rover is somewhere between China and Canada... a few more week to wait!
In the meanwhile, I've assembled and tested Guzunty. All seems to work fine (although some components had been slightly damaged in shipping).
I'm starting to play with the Opencv PiTeR python code from your github repository BUT I'm using Pi 2 + Pi NoIR (I recall you tried this cam as well). I'm having difficulties finding robust settings for for the InRange function so I'm wondering if I should try using other colors (rather than green) to print signs or change something else (including switching to another camera!). Do you have more information or recommendations concerning PiTeR with Pi NoIR????
Best regards from Montreal.
Pascal.
I discoved PiTeR in MagPi 28 and thought "Wahoooo, this will be a super duper project to learn robotics, OpenCV, Arduino, Guzunti all together
". Well, all together might be a little bit too much so I'm attempting to concentrate on the software side first (and recuce cost apparently) by starting from sainsmart upright rover kit (V3) rather than buying driver + motors + wheels separately.http://www.sainsmart.com/robotics/sains ... t-kit.html
The rover is somewhere between China and Canada... a few more week to wait!
In the meanwhile, I've assembled and tested Guzunty. All seems to work fine (although some components had been slightly damaged in shipping).
I'm starting to play with the Opencv PiTeR python code from your github repository BUT I'm using Pi 2 + Pi NoIR (I recall you tried this cam as well). I'm having difficulties finding robust settings for for the InRange function so I'm wondering if I should try using other colors (rather than green) to print signs or change something else (including switching to another camera!). Do you have more information or recommendations concerning PiTeR with Pi NoIR????
Best regards from Montreal.
Pascal.
Re: PiTeR, OpenCV and MagPi issue 28
Hi Pascal,
I'm afraid haven't had time to go back and experiment more with the Pi NoIR camera. As I recall, I encountered the same issue as you have; being unable to get good range parameters. The behaviour was such that I concluded that there was some subtle difference in the data coming from the Pi NoIR camera that rendered OpenCV incompatible.
I find that conclusion hard to believe, since my understanding is the Pi NoIR is the same as the regular camera but without the IR filter. However, it is the only possibility that I haven't eliminated. It is possible that the Pi exposure and balance firmware behaves differently with the Pi NoIR and that is enough to upset OpenCV.
I therefore recommend that unless you specifically need IR for your project, you just go get yourself a regular Pi Camera.
As for using different colours on the symbols, yes, any colour will work, you just need to find suitable hue and saturation settings. Also note that the conversion to monochrome for the recognition step does it by grabbing the red channel. This is optimised for a green image, so if you switch to a different colour you'll likely need to take a different channel to get the necessary contrast.
Hope that helps and enjoy your robot. You will learn a great deal I'm sure.
I'm afraid haven't had time to go back and experiment more with the Pi NoIR camera. As I recall, I encountered the same issue as you have; being unable to get good range parameters. The behaviour was such that I concluded that there was some subtle difference in the data coming from the Pi NoIR camera that rendered OpenCV incompatible.
I find that conclusion hard to believe, since my understanding is the Pi NoIR is the same as the regular camera but without the IR filter. However, it is the only possibility that I haven't eliminated. It is possible that the Pi exposure and balance firmware behaves differently with the Pi NoIR and that is enough to upset OpenCV.
I therefore recommend that unless you specifically need IR for your project, you just go get yourself a regular Pi Camera.
As for using different colours on the symbols, yes, any colour will work, you just need to find suitable hue and saturation settings. Also note that the conversion to monochrome for the recognition step does it by grabbing the red channel. This is optimised for a green image, so if you switch to a different colour you'll likely need to take a different channel to get the necessary contrast.
Hope that helps and enjoy your robot. You will learn a great deal I'm sure.
Guzunty: A fully programmable peripheral you build yourself! https://github.com/Guzunty/Pi/wiki
-
il_diavolo
- Posts: 149
- Joined: Mon Dec 02, 2013 7:56 pm
Re: PiTeR, OpenCV and MagPi issue 28
I am seeking help (nay, pleading for help!) with the programs on the Guzunty website, the github repository and opencv. Nothing seems to work satisfactorily. I see from the posts here that others have had success so I can only assume that I'm doing something wrong on a grand scale.
Firstly I am unable to clone the MagPi folder from github, I get a "403" error. I have however managed to get copies of the programs by cutting and pasting from the github web site.
I have followed the instructions on the Guzunty web site in conjunction with the Magpi arcticles for upgrading to opencv 2.4.9, managing to clone the "Nolaan" opencv library but failing to complete the last task. As I am not being able to clone the MagPi folder I can't copy the new "cv2.so" file as per the instructions. I did manage to cut and paste the file from the github web site but when I overwrite the old copy of the file in /usr/lib/pythonshared/python2.7 with the new version I get error messages saying that the files cannot be found. Reverting back to the old copy of "cv2.so" and things appear to work.
I say "appear" because when I run "imageCap.py" and/or "colour.py" neither work properly. "imageCap" runs for a few seconds and then stalls and needs a reboot to turn the camera off.
Similarly with "colour.py". mostly it just produces 4 blank windows and then requires a reboot to stop the camera. Occasionally it produces images and sliders but these fail to continue working for more that a few seconds.
I have tried this on a Pi "B" and a Pi "2". Modprobe is started so it's not that.
As you can tell from the above I am completely out of my depth when it comes to solving these problems my self, hence the pleas for help!
Il Diavolo
Firstly I am unable to clone the MagPi folder from github, I get a "403" error. I have however managed to get copies of the programs by cutting and pasting from the github web site.
I have followed the instructions on the Guzunty web site in conjunction with the Magpi arcticles for upgrading to opencv 2.4.9, managing to clone the "Nolaan" opencv library but failing to complete the last task. As I am not being able to clone the MagPi folder I can't copy the new "cv2.so" file as per the instructions. I did manage to cut and paste the file from the github web site but when I overwrite the old copy of the file in /usr/lib/pythonshared/python2.7 with the new version I get error messages saying that the files cannot be found. Reverting back to the old copy of "cv2.so" and things appear to work.
I say "appear" because when I run "imageCap.py" and/or "colour.py" neither work properly. "imageCap" runs for a few seconds and then stalls and needs a reboot to turn the camera off.
Similarly with "colour.py". mostly it just produces 4 blank windows and then requires a reboot to stop the camera. Occasionally it produces images and sliders but these fail to continue working for more that a few seconds.
I have tried this on a Pi "B" and a Pi "2". Modprobe is started so it's not that.
As you can tell from the above I am completely out of my depth when it comes to solving these problems my self, hence the pleas for help!
Il Diavolo
Re: PiTeR, OpenCV and MagPi issue 28
Hi diavolo,
Sorry you're having a bit of trouble.
I just cloned the Guzunty code without any errors.
A quick Google about GH 403 gave me this:
https://help.github.com/articles/https-cloning-errors/
I think trying to cut/copy/paste the '.so' file from the website will never work since its a binary. It's best to clone the repo from the root rather than trying to grab just a subtree. A subtree probably works, but I never tried it.
I don't think you're doing anything wrong on a massive scale It's probably something simple.
See if that link helps and if not, get back to us.
HTH,
Derek
Sorry you're having a bit of trouble.
I just cloned the Guzunty code without any errors.
A quick Google about GH 403 gave me this:
https://help.github.com/articles/https-cloning-errors/
I think trying to cut/copy/paste the '.so' file from the website will never work since its a binary. It's best to clone the repo from the root rather than trying to grab just a subtree. A subtree probably works, but I never tried it.
I don't think you're doing anything wrong on a massive scale
It's probably something simple.See if that link helps and if not, get back to us.
HTH,
Derek
Guzunty: A fully programmable peripheral you build yourself! https://github.com/Guzunty/Pi/wiki
-
il_diavolo
- Posts: 149
- Joined: Mon Dec 02, 2013 7:56 pm
Re: PiTeR, OpenCV and MagPi issue 28
Thanks Derek
I have managed to clone the Guzunty code now, I was trying the whole path to MagPi earlier which, of course, didn't work.
However I still couldn't get the "so" file to work, as after copying it to .../pythonshared/python2.7 I get the same errors as before. I must admit to being out of my depth here, I can follow a recipe but when it goes wrong I'm stuck.
"imageCap.py" and "colour.py" are still not working, they will run for a short length of time and then stall with some kernel errors displayed followed by "Select timeout" scrolling continuously.
I introduced an incremental count variable in the loop in "imageCap.py" which shows that the length of time that it will run is highly variable, from 10 loops to 500 or so.
I have tried a new install of Rasbian, updated and upgraded, and a careful step by step implementation of your recipe but still get the same problems.
Although I hate giving up on anything I think I may try a different approach to guide my robot!
Colin
I have managed to clone the Guzunty code now, I was trying the whole path to MagPi earlier which, of course, didn't work.
However I still couldn't get the "so" file to work, as after copying it to .../pythonshared/python2.7 I get the same errors as before. I must admit to being out of my depth here, I can follow a recipe but when it goes wrong I'm stuck.
"imageCap.py" and "colour.py" are still not working, they will run for a short length of time and then stall with some kernel errors displayed followed by "Select timeout" scrolling continuously.
I introduced an incremental count variable in the loop in "imageCap.py" which shows that the length of time that it will run is highly variable, from 10 loops to 500 or so.
I have tried a new install of Rasbian, updated and upgraded, and a careful step by step implementation of your recipe but still get the same problems.
Although I hate giving up on anything I think I may try a different approach to guide my robot!
Colin
Re: PiTeR, OpenCV and MagPi issue 28
Hmmm,
I can't say that I have seen behaviour exactly like that. I have noticed that sometimes the windows can take a long time to start showing images, particularly the colour.py tuning code. I put that down to some stability problems inside OpenCV itself. The demo code is closely based on code from the OpenCV tutorials, I don't think I introduced any issues.
You could try installing and running the demos on a PC host to see if the same issues are shown there? IIRC, when I did that, there were no delays.
Not sure what might be causing it to hang completely. I will try to make some time to re-run the demo code with updated software. It is possible that a Raspbian or OpenCV update has introduced new issues.
Sorry, I can't say when I'll be able to get to it though, I don't have a great deal of free time at the moment.
I can't say that I have seen behaviour exactly like that. I have noticed that sometimes the windows can take a long time to start showing images, particularly the colour.py tuning code. I put that down to some stability problems inside OpenCV itself. The demo code is closely based on code from the OpenCV tutorials, I don't think I introduced any issues.
You could try installing and running the demos on a PC host to see if the same issues are shown there? IIRC, when I did that, there were no delays.
Not sure what might be causing it to hang completely. I will try to make some time to re-run the demo code with updated software. It is possible that a Raspbian or OpenCV update has introduced new issues.
Sorry, I can't say when I'll be able to get to it though, I don't have a great deal of free time at the moment.
Guzunty: A fully programmable peripheral you build yourself! https://github.com/Guzunty/Pi/wiki
Re: PiTeR, OpenCV and MagPi issue 28
Ok, couldn't resist having a quick look.
Having upgraded and dist-upgraded, I am seeing what looks like a hang with trackingWithThreads on a Model B+. The Pi 2 looks to be working normally, and the B+ seems to be working OK with the single threaded version.
Since I'd recommend the single threaded version for all non Pi2's anyway, I'm not seeing any serious new problems. I am curious as to why the B+ is hanging in the multi-threaded demo though. I'll set up a debugging session on this when I get a chance.
Definitely not seeing any kernel panics or anything. colour.py is still pretty slow on a B+, but that's not new.
Sorry, I can't be more help than that just now. You might consider starting your install process over again from a clean install. I have found that sometimes clears problems in the past.
Having upgraded and dist-upgraded, I am seeing what looks like a hang with trackingWithThreads on a Model B+. The Pi 2 looks to be working normally, and the B+ seems to be working OK with the single threaded version.
Since I'd recommend the single threaded version for all non Pi2's anyway, I'm not seeing any serious new problems. I am curious as to why the B+ is hanging in the multi-threaded demo though. I'll set up a debugging session on this when I get a chance.
Definitely not seeing any kernel panics or anything. colour.py is still pretty slow on a B+, but that's not new.
Sorry, I can't be more help than that just now. You might consider starting your install process over again from a clean install. I have found that sometimes clears problems in the past.
Guzunty: A fully programmable peripheral you build yourself! https://github.com/Guzunty/Pi/wiki
-
il_diavolo
- Posts: 149
- Joined: Mon Dec 02, 2013 7:56 pm
Re: PiTeR, OpenCV and MagPi issue 28
I made a Google search on "Select timeout" and found a possible solution on the Stackoverflow site relating to Ubantu. I haven't had time to try the suggested solution yet but it seems it could be V4L playing up. I'll try it out in a couple of days and post the result.
Thanks
Colin
Thanks
Colin
-
il_diavolo
- Posts: 149
- Joined: Mon Dec 02, 2013 7:56 pm
Re: PiTeR, OpenCV and MagPi issue 28
Finally got everything working. A clean install of the latest Rasbian and actually reading Nolaan's README files did the trick. The results are very impressive.
Now for the really difficult bit, trying to understand opencv and your code and writing a script that will guide my robot.
Colin
Now for the really difficult bit, trying to understand opencv and your code and writing a script that will guide my robot.
Colin
Re: PiTeR, OpenCV and MagPi issue 28
Great news! Have fun and keep those questions coming if you see something you don't understand in the sample code.
Guzunty: A fully programmable peripheral you build yourself! https://github.com/Guzunty/Pi/wiki
-
il_diavolo
- Posts: 149
- Joined: Mon Dec 02, 2013 7:56 pm
Re: PiTeR, OpenCV and MagPi issue 28
It's been a while since my last post (July 20) and I've made some progress. My robot can now look around for and then move towards a target. I'm getting better at understanding how Python and CV2 work but, for me at least, it's not simple stuff despite (or because of) years of programming in BBC Basic for Windows. I can almost understand how threading works.
Most of my scripts are hacked versions of the Piter programs and the similar scripts downloaded from the PiBorg site (My robot is a modified DiddyBorg).
Where I am having serious problems is in the recognition of the symbols once the robot reaches them. Running either "tracking.py" or "trackingWithThreads.py" I can (very) occasionally get good matches but when I do I also get good matches with the wrong symbols. It's very frustrating. Improving the lighting seems to make little difference, I even tried a light box to back-illuminate a translucent target.
Any pointers to achieving a better performance would be appreciated.
Colin
Most of my scripts are hacked versions of the Piter programs and the similar scripts downloaded from the PiBorg site (My robot is a modified DiddyBorg).
Where I am having serious problems is in the recognition of the symbols once the robot reaches them. Running either "tracking.py" or "trackingWithThreads.py" I can (very) occasionally get good matches but when I do I also get good matches with the wrong symbols. It's very frustrating. Improving the lighting seems to make little difference, I even tried a light box to back-illuminate a translucent target.
Any pointers to achieving a better performance would be appreciated.
Colin
Re: PiTeR, OpenCV and MagPi issue 28
Hi, sorry for the slow response.
Hmmm, not sure what to suggest beyond the advice in the original article. Getting the colour mask value just right was very important, but I get the impression you have persevered with that.
One thing I haven't previously mentioned was that I found the bounding rectangle was less helpful than I expected. Because all the symbols have it, there are fewer distinguishing differences between them. You might try symbols without the bounding box. That said, I got the recognition working adequately with and without the box.
Another thing is to add python code to show the actual images the robot is processing including where the masks are etc. If the images are defocused, blurred or mis exposed the vision system won't cope. You need to be prepared to retune for every new lighting situation. Feel free to post your images here for feedback.
You might try OpenCV 3 or using a faster Pi 2 might help.
Finally, more complex symbols (even perhaps hand drawn ones) may work better, but will process more slowly.
Sorry I don't have more to suggest.
Hmmm, not sure what to suggest beyond the advice in the original article. Getting the colour mask value just right was very important, but I get the impression you have persevered with that.
One thing I haven't previously mentioned was that I found the bounding rectangle was less helpful than I expected. Because all the symbols have it, there are fewer distinguishing differences between them. You might try symbols without the bounding box. That said, I got the recognition working adequately with and without the box.
Another thing is to add python code to show the actual images the robot is processing including where the masks are etc. If the images are defocused, blurred or mis exposed the vision system won't cope. You need to be prepared to retune for every new lighting situation. Feel free to post your images here for feedback.
You might try OpenCV 3 or using a faster Pi 2 might help.
Finally, more complex symbols (even perhaps hand drawn ones) may work better, but will process more slowly.
Sorry I don't have more to suggest.
Guzunty: A fully programmable peripheral you build yourself! https://github.com/Guzunty/Pi/wiki
-
il_diavolo
- Posts: 149
- Joined: Mon Dec 02, 2013 7:56 pm
Re: PiTeR, OpenCV and MagPi issue 28
Thanks for your reply. I can report that in the last couple of weeks I have made significant progress and can now get reliable symbol recognition. Careful tweaking of the lighting, nice crisp A4 prints of your symbols (they work better than the alternative design I had made!) and tinkering with the numbers for spotFilter and maskMorph, using higher numbers for the initial shape detection and then decrease the numbers (increasing the resolution) for recognition when the robot gets close enough.
I have also increased the contrast of the image being recognised using cv2.THRESH_BINARY which seems to improve matching with the templates.
Currently I loop through the matching code (originally "tracking.py") 5 times and take the average of each template match and then sort to get the best match. It gets it right about 90% of the time at the moment, which I think is pretty impressive. On a Pi 2 it takes about 25 secs to make a choice. It would probably be quicker if I took out all the print statements but I like to see it "thinking".
Incidentally, I use a version of "colour.py" to set the HSV ranges but I have also tweaked it to save the values to a file which is then read back in by the main program.
Colin
I have also increased the contrast of the image being recognised using cv2.THRESH_BINARY which seems to improve matching with the templates.
Currently I loop through the matching code (originally "tracking.py") 5 times and take the average of each template match and then sort to get the best match. It gets it right about 90% of the time at the moment, which I think is pretty impressive. On a Pi 2 it takes about 25 secs to make a choice. It would probably be quicker if I took out all the print statements but I like to see it "thinking".
Incidentally, I use a version of "colour.py" to set the HSV ranges but I have also tweaked it to save the values to a file which is then read back in by the main program.
Colin
Re: PiTeR, OpenCV and MagPi issue 28
thank you for offer your help my friend
i have been installed open CV 2.4.10 and i did face detection , i need your help in two point
1- how i detect any object i want for example if i want to detect (stop sign) , i am beginner in this point so please i need your help
2- if i want to use raspberry pi camera module to detect instead of USB camera ,what i should do
thanks in advance
i have been installed open CV 2.4.10 and i did face detection , i need your help in two point
1- how i detect any object i want for example if i want to detect (stop sign) , i am beginner in this point so please i need your help
2- if i want to use raspberry pi camera module to detect instead of USB camera ,what i should do
thanks in advance
Re: PiTeR, OpenCV and MagPi issue 28
Hi moayedov,
Sorry for the slow response, I've been away on business.
> i did face detection
Did you follow the OpenCV tutorial on face detection? If so, there is another tutorial in the same series that shows how to detect a sample image in a scene:
http://docs.opencv.org/2.4/doc/tutorial ... homography
My article in issue 28 of the MagPi also discusses another example of how to do this.
The source code that goes with this article is here:
https://github.com/Guzunty/Pi/tree/mast ... iter/MagPi
All the example code there uses the Raspberry Pi camera, and there is a page on the wiki here:
https://github.com/Guzunty/Pi/wiki/PiTeR-and-OpenCV
that shows how to set up your Pi to install the necessary drivers to use the Pi camera under Debian Linux/Python/OpenCV control.
Just a disclaimer: These pages go back to the time of the original article and there may be better, more up to date ways of enabling the camera for control, I recommend using Google to see if anything more recent is available before following exactly with my steps.
HTH
Sorry for the slow response, I've been away on business.
> i did face detection
Did you follow the OpenCV tutorial on face detection? If so, there is another tutorial in the same series that shows how to detect a sample image in a scene:
http://docs.opencv.org/2.4/doc/tutorial ... homography
My article in issue 28 of the MagPi also discusses another example of how to do this.
The source code that goes with this article is here:
https://github.com/Guzunty/Pi/tree/mast ... iter/MagPi
All the example code there uses the Raspberry Pi camera, and there is a page on the wiki here:
https://github.com/Guzunty/Pi/wiki/PiTeR-and-OpenCV
that shows how to set up your Pi to install the necessary drivers to use the Pi camera under Debian Linux/Python/OpenCV control.
Just a disclaimer: These pages go back to the time of the original article and there may be better, more up to date ways of enabling the camera for control, I recommend using Google to see if anything more recent is available before following exactly with my steps.
HTH
Guzunty: A fully programmable peripheral you build yourself! https://github.com/Guzunty/Pi/wiki
Re: PiTeR, OpenCV and MagPi issue 28
A good blog on using opencv on the raspberry PI is pyimagesearch.com, there are posts covering everything you need to know.moayedov wrote:thank you for offer your help my friend
i have been installed open CV 2.4.10 and i did face detection , i need your help in two point
1- how i detect any object i want for example if i want to detect (stop sign) , i am beginner in this point so please i need your help
2- if i want to use raspberry pi camera module to detect instead of USB camera ,what i should do
thanks in advance