hurkj
Posts: 7
Joined: Sun Aug 04, 2013 7:57 am

Challenge: Bitmap comparison

Sun Aug 04, 2013 8:09 am

Anyone experience with making a bitmap comparison with Rpi ?

My goal:
Detect differences in images.

Way of working:
I want to store a Bitmap (1920×1080) in the memory of the Rpi as a reference image.
Than whenever I push a button I want the camera to take a snapshot and compare the newly taken image with the bitmap in the memory.

Actions:
When no differences are detected a Green Lamp will start Flashing
When differences are detected I want Rpi to display an image on a monitor where differences are pointed out

Anyone who can help me with this challenge.

itimpi
Posts: 1090
Joined: Sun Sep 25, 2011 11:44 am
Location: Potters Bar, United Kingdom
Contact: Website

Re: Challenge: Bitmap comparison

Sun Aug 04, 2013 8:37 am

I suspect that the chances of there being a single bit difference between successive images is very high - if only due to slight noise or a very minor change in ambient light. I am assuming that in fact you are interest in SIGNIFICANT differences which might be much harder to determine.

Perhaps you should consider looking at how the software that tries to detect motion between image actually operates.

hurkj
Posts: 7
Joined: Sun Aug 04, 2013 7:57 am

Re: Challenge: Bitmap comparison

Sun Aug 04, 2013 7:30 pm

Mm, understand your thought but no,, changes will we be very little...

In fact I want to detect if I have forgotten to place a screw in a assy.
I assemble regularly small products and want to be 100% that I don't forget to place all the screw (>50)
A snapshot of the assy would immediately detect if I have placed them all.

So, brightness etc will al be 100% the same for every snapshot.

please advise..

itimpi
Posts: 1090
Joined: Sun Sep 25, 2011 11:44 am
Location: Potters Bar, United Kingdom
Contact: Website

Re: Challenge: Bitmap comparison

Sun Aug 04, 2013 7:37 pm

It seems to me that if you used something like the raspistill application to save images to files then you could use the 'cmp' command to compare two saved files as that returns an exit code telling you if two files are identical.

If you tried that manually then you would be able to test if two consecutive pictures really are identical down to the bit level - I strongly suspect you will find this is not the case but the proof will be to try it.

If it DOES do what you want it would then be quite easy to automate the process.

User avatar
Burngate
Posts: 6328
Joined: Thu Sep 29, 2011 4:34 pm
Location: Berkshire UK Tralfamadore
Contact: Website

Re: Challenge: Bitmap comparison

Mon Aug 05, 2013 8:23 am

itimpi's method - raspistill, then cmp - is probably the way to go. Maybe some processing of the saved image before comparrison could reduce the lighting and noise problems.

But for your application, I see some problems, apart from noise and varying lighting.

Can you be certain the assembly will be in exactly the same place each time, down to a pixel?
How about if a screw is in, but the slot is at a different angle?

So the hard part is going to be deciding how to throw away information you don't want

PiGraham
Posts: 4019
Joined: Fri Jun 07, 2013 12:37 pm
Location: Waterlooville

Re: Challenge: Bitmap comparison

Mon Aug 05, 2013 9:46 am

This is a much harder problem than it may appear. Similar images of similar objects will have numerous differences all over. If the only thing that varies between image frames is the item of interest it can work. Usually you cannot adequately control for lighting and placement and the object itself is different.

You will have a better chance if you can define region of interest around the features you want to to check and perform some sort of pattern match for those features in the expected locations. The quality of the search result will indicate that the feature is as it should be. See normalised greyscale correlation as a relatively simple technique.
If features you want to check can be rotated or vary significantly in position, things get much more complex. Screws won't line up perfectly and any angular misalignment will show as pixels that don't match.

There are techniquies that work for rotated or scaled outlines. Search for rotation invariant pattern matching.

If all you want to do is check for absence of screws you may do better to check for the hole than the screw.

If you arrange the lighting so that the hole is shadowed and the screw bright you could measure the average pixel intensity in the region around the hole. Bright - screw, dark = hole.

PiGraham
Posts: 4019
Joined: Fri Jun 07, 2013 12:37 pm
Location: Waterlooville

Re: Challenge: Bitmap comparison

Mon Aug 05, 2013 10:01 am

hurkj wrote:Mm, understand your thought but no,, changes will we be very little...

In fact I want to detect if I have forgotten to place a screw in a assy.
I assemble regularly small products and want to be 100% that I don't forget to place all the screw (>50)
A snapshot of the assy would immediately detect if I have placed them all.

So, brightness etc will al be 100% the same for every snapshot.

please advise..

Yu can try this out easily enough. Get a photo editor (gimp, photoshop, paintshop pro) and set any digital camera on a tripod or bracket. Arrange what you think will be 100% identical lighting (excluduing all ambient light and precisely controlling a light source.
Take a series of photos with various parts inserted in your very accurate location jig.
Open the photos in the image editor and subtracte them. Use the histogram / image stats feature of the editor to measure the differences. See if it is obvious from the numbers which images have all the screws and which have one missing.

You can repeat this test but looks at the numbers from small regions around the features.You have a much better chance of success that way, if you can control the lighting.

Twinkletoes
Posts: 210
Joined: Fri May 25, 2012 9:44 pm

Re: Challenge: Bitmap comparison

Mon Aug 05, 2013 11:47 am

Your comparison will be much faster if you first mark the areas of the original "master" image that might be incorrect (e.g. the bounding box around a screw) and your comparison algorithm only checks these areas. You could have a different comparison for each (e.g. dark/bright for a screw, or avg colour for a label)

hurkj
Posts: 7
Joined: Sun Aug 04, 2013 7:57 am

Re: Challenge: Bitmap comparison

Mon Aug 05, 2013 7:46 pm

thanks for all the thoughts...

Indeed several of the problems you mentioned are realistic...like exactly the same picture cannot be managed

So,... conclusion.... I have to detect "black holes" rather than screws.

I guess I need to make a black mask with cleared out the points of interest.
Than put the newly taken picture behind the mask and look for black holes..

Still my Q remains... anyone have an idea how to realize this in practice..?

Please advice,

Regards and thanks in advance

Jeroen

User avatar
Jim Manley
Posts: 1600
Joined: Thu Feb 23, 2012 8:41 pm
Location: SillyCon Valley, California, and Powell, Wyoming, USA, plus The Universe
Contact: Website

Re: Challenge: Bitmap comparison

Tue Aug 06, 2013 2:57 am

Even in a room with no windows and fixed lighting with no presence of people to cast shadows or reflections, itimpi is correct in that there will be differences in the lesser significant bits between any two images just due to thermal noise in both the atmosphere (actual optical effects) and the sensor (electro-mechanical effects due to vibration, heat dynamics, etc.). However, if the images are only compared for differences in the more significant bits, those effects will become less of an issue, although there will likely still be differences in some pixels.

Your application suggests that you will only need to compare the most significant few bits between images in order to detect a fairly gross difference of a screw present/missing. Experimentation will probably reveal that just comparing the most significant bit in each pixel between images is sufficient (i.e., each image would be one bit-plane deep, actually a pure black-and-white image with no color or gray scale).

Here's how to convert an image to monochrome in Python (it will work with PNG, JPEG, GIF, BMP, and TIFF):

Code: Select all

from PIL import Image 

# open color image
image_file = Image.open("original_image.png")

# convert image to black and white
# to convert an image to gray scale, change the '1' to 'L'
image_file = image_file.convert('1')

image_file.save('black-and-white.png')
There are a bunch of Python examples for how to compare two images using the Python Imaging Library (PIL - Pillow is recommended as a less problematic updated fork) at this link:

http://stackoverflow.com/questions/1927 ... -linux-way

The bounding box comments above are valid for vastly reducing the areas to being compared, which will speed things up as well as improve the success rate. IIRC, PIL/Pillow provide the means for extracting subsets of images based on offsets to the bounding boxes' outlines.
The best things in life aren't things ... but, a Pi comes pretty darned close! :D
"Education is not the filling of a pail, but the lighting of a fire." -- W.B. Yeats
In theory, theory & practice are the same - in practice, they aren't!!!

bbmonster
Posts: 22
Joined: Thu Jul 11, 2013 8:11 am

Re: Challenge: Bitmap comparison

Tue Aug 06, 2013 4:04 am

hurkj wrote:thanks for all the thoughts...

Indeed several of the problems you mentioned are realistic...like exactly the same picture cannot be managed

So,... conclusion.... I have to detect "black holes" rather than screws.

I guess I need to make a black mask with cleared out the points of interest.
Than put the newly taken picture behind the mask and look for black holes..

Still my Q remains... anyone have an idea how to realize this in practice..?

Please advice,

Regards and thanks in advance

Jeroen
If you don't need that green LED mentioned earlier, You could try blinking a reference picture with the updated pictures. Instead of creating a mask, just take a picture of an unpopulated assembly as your reference picture. Take a picture of your completed assembly. Then on screen show each picture for a second each, the differences usually stand out. Adjust the interval if a second is too fast or slow.

This works pretty well even when the two pictures are slightly off whether it be zoom, lighting, centering, etc.

Hope this helps.

bbmonster
Posts: 22
Joined: Thu Jul 11, 2013 8:11 am

Re: Challenge: Bitmap comparison

Tue Aug 06, 2013 4:36 am

Here are a couple more ideas if all you want to do is check if you missed a few screws.

Use some sort of colored or bright light under the assembly. Build a jig so only light passes through the holes and not the sides.

Then you could just display the board on screen and look for the lighted holes.

Or, write software that detects the burst of light peeking through and flashes that LED you mentioned earlier to alert you and display the image on screen so you know which holes do not have screws.

Or really crazy and expensive idea that has nothing to do with images, build like a robotic checker that probes the holes.

User avatar
Hove
Posts: 1205
Joined: Sun Oct 21, 2012 6:55 pm
Location: Cotswolds, UK
Contact: Website

Re: Challenge: Bitmap comparison

Tue Aug 06, 2013 7:04 am

This isn't really going to help much, but it requires very complex software to do image matching - there's a company call tineye which makes it's money tracking down copyright images across the net - a photographer submits an image, and they look for any derivatives - even if they've been resized, clipped, edited, re-coloured, copyright statements removed etc.

Even for your much simplified requirements, the slightest wobble on the camera between shots or change of light can cause a huge difference in the stream of bits; add to that, the jpeg compression algorithm takes squares of data (8 x 8 pixels I think), and performs a Discrete Cosine Transform to encode the changes in bit values - changes compress better than absolute values for the brightness - they do this for the 3 colour channels. Even images apparently identical to the eye will have completely different data if in jpg format.

It is possible in a very simple scenario, which may be all you need. The lighting must be 100% under your control (not sunlight as the sun moves and therefore so do the shadows), and the camera is solid as a rock (not even the slightest wobbles from the machine), the images must be the same size, and you use RAW not JPEG compressed images and you do the suggested convertion to B&W (and I mean black and white - no greys) using a contrast detection algorithm (i.e. walk the picture pixel by pixel, and depending on what the surrounding pixels look like, paint it black). Once you have the contrast filtered B&W photo, bit mapping stands a chance, though I'd still reckon you need to aim for a target % rather than an exact match.

Alternatively, you'll need to find out how tineye (or others) do it.
www.pistuffing.co.uk - Raspberry Pi and other stuffing!

PiGraham
Posts: 4019
Joined: Fri Jun 07, 2013 12:37 pm
Location: Waterlooville

Re: Challenge: Bitmap comparison

Tue Aug 06, 2013 7:08 am

bbmonster wrote:If you don't need that green LED mentioned earlier, You could try blinking a reference picture with the updated pictures. Instead of creating a mask, just take a picture of an unpopulated assembly as your reference picture. Take a picture of your completed assembly. Then on screen show each picture for a second each, the differences usually stand out. Adjust the interval if a second is too fast or slow.

This works pretty well even when the two pictures are slightly off whether it be zoom, lighting, centering, etc.

Hope this helps.
This is a very sound idea that definitely works. Human vision is good at spotting changes between aligned image pairs if they are flashed. This is a technique that was widely used before effective machine vision systems were developed. It is still important to have good alignment. Ideally you want no position changes except the missing part.

PiGraham
Posts: 4019
Joined: Fri Jun 07, 2013 12:37 pm
Location: Waterlooville

Re: Challenge: Bitmap comparison

Tue Aug 06, 2013 7:39 am

hurkj wrote:thanks for all the thoughts...

Indeed several of the problems you mentioned are realistic...like exactly the same picture cannot be managed

So,... conclusion.... I have to detect "black holes" rather than screws.

I guess I need to make a black mask with cleared out the points of interest.
Than put the newly taken picture behind the mask and look for black holes..

Still my Q remains... anyone have an idea how to realize this in practice..?

Please advice,

Regards and thanks in advance

Jeroen
You don't need a physical mask, you need specify regions of the image that should be tested. The standard term for this is 'region of interest' and is typically a rectangle defined by coordinates of a corner and width and hight. You then inspect individual pixels within this region, or call an image library function that accepts such a region.
If you use python image library you can extract a ROI using the crop function.

Code: Select all

im.crop(box) ⇒ image
You can then work with this small image to test for the feauture it contains.

Alternatively you can define a region of interest with a 'mask' where pixels of interest are marked by non-zero pixels in a corresponding mask image. You would have one mask image for each feature you want to check.
You might paint the mask image with the mouse using an image editor.
If you call

Code: Select all

ImageStat.Stat(image, mask)
You will get some numbers representing the feature. Mean and standard deviation are a good place to start. Log values for several 'good' parts and compare to the umbers from a 'bad' part. Set a threshold on each value that gives a clear difference between good and bad.

If you find the values from part being tested fall within the thresholds for a good part you can draw a box around the ROI in green, or in red if thresholds are exceeded.
If every ROI is good turn on the green light. If any is bad turun on the red light.

Code: Select all

draw.rectangle(box, options)
PIL info here http://effbot.org/imagingbook/pil-index.htm

More advanced info on computer vision in python here:
http://programmingcomputervision.com/
http://pythonvision.org/

User avatar
Hove
Posts: 1205
Joined: Sun Oct 21, 2012 6:55 pm
Location: Cotswolds, UK
Contact: Website

Re: Challenge: Bitmap comparison

Tue Aug 06, 2013 9:23 am

Just a further thought on my contrast comparison idea:

If you can identify within the photos a reduced area where the error could occur, say 100 bytes square around the missing / present bolt in the 'standard image', then you could take samples from the live image for the same squares, and do a B&W contast match my subtracting byte by byte each pixel in the live and sample image and adding the result together - matching pixels will give zero, which mean the total sum will be near zero except for areas where there are marginal differences. If you do a few runs like this with "a perfect build", and then a few runs with different "faulty builds" I suspect you'll end up with different sums for an correct and faulty build. That will give you a trigger point in your builds to sound an alarm / flash lights or whatever when the scan approaches the 'faulty build' trigger point to go check the mistake.
www.pistuffing.co.uk - Raspberry Pi and other stuffing!

tenochtitlanuk
Posts: 156
Joined: Fri Jul 06, 2012 8:51 pm
Location: Taunton, Somerset, UK
Contact: Website

Re: Challenge: Bitmap comparison

Tue Aug 06, 2013 9:27 am

Flicker comparison is certainly going to be easiest. Lighting, alignment etc are much less critical. See this quick example.
Image

If you go the image-processing route I concur with the comments of others- fixed camera position to near-pixel accuracy is needed unless you'll do a lot of processing!
I've played with feature detection, to detect coloured beads dropping through a bead sorter, and to 'read' seven segment displays by reading selected areas and their average pixel colour. Much more complicated...

User avatar
Hove
Posts: 1205
Joined: Sun Oct 21, 2012 6:55 pm
Location: Cotswolds, UK
Contact: Website

Re: Challenge: Bitmap comparison

Tue Aug 06, 2013 9:34 am

Interesting - can the flicker detection be automated (i.e. coded for) rather than having a human (whose hard coded already for flicker detection for spotted sabre toothed tigers etc) do the monitoring?

I ask as that might play a part on another project I'm starting to must upon.
www.pistuffing.co.uk - Raspberry Pi and other stuffing!

mikerr
Posts: 2826
Joined: Thu Jan 12, 2012 12:46 pm
Location: UK
Contact: Website

Re: Challenge: Bitmap comparison

Tue Aug 06, 2013 9:58 am

Simple python code for detecting differences:
http://www.raspberrypi.org/phpBB3/viewt ... 43&t=45235

User avatar
Burngate
Posts: 6328
Joined: Thu Sep 29, 2011 4:34 pm
Location: Berkshire UK Tralfamadore
Contact: Website

Re: Challenge: Bitmap comparison

Tue Aug 06, 2013 3:58 pm

Hove wrote:Interesting - can the flicker detection be automated (i.e. coded for) rather than having a human (whose hard coded already for flicker detection for spotted sabre toothed tigers etc) do the monitoring?

I ask as that might play a part on another project I'm starting to must upon.
Looks like an interesting project. Where can you find sabre toothed tigers - spotted or striped?

User avatar
Hove
Posts: 1205
Joined: Sun Oct 21, 2012 6:55 pm
Location: Cotswolds, UK
Contact: Website

Re: Challenge: Bitmap comparison

Tue Aug 06, 2013 4:42 pm

Burngate wrote:
Hove wrote:Interesting - can the flicker detection be automated (i.e. coded for) rather than having a human (whose hard coded already for flicker detection for spotted sabre toothed tigers etc) do the monitoring?

I ask as that might play a part on another project I'm starting to must upon.
Looks like an interesting project. Where can you find sabre toothed tigers - spotted or striped?
Oops - it was meant to read along the lines of "humans whose brains have been hard-coded for millenia with flicker detection to spot sabre toothed tigers". The only way I'll get to see them is when I finish my Tardis, but I'm having too much fun with the RPi to waste my time on that ;-)
www.pistuffing.co.uk - Raspberry Pi and other stuffing!

gestahl
Posts: 4
Joined: Tue Aug 06, 2013 9:01 pm

Re: Challenge: Bitmap comparison

Tue Aug 06, 2013 9:59 pm

Having tried something similar before, I can tell you that even with two pictures taken seconds apart automatically with nobody in the room will have vast differences. Your best bet if you're really bent on doing it without visually evaluating the images is identifying say the 25x25 pixel area each screw is in, grabbing each pixel color in that box, and seeing if a majority of them are in whatever range of values makes up the color gray (or whatever color your screws are) or not.
I'd say your best bet is whoever suggested flashing the images between the known correct one and the one in question, a missing screw should jump right out at you. If you plan to do this for more than 20min or so at a time, be aware of the security guard effect ;)

tenochtitlanuk
Posts: 156
Joined: Fri Jul 06, 2012 8:51 pm
Location: Taunton, Somerset, UK
Contact: Website

Re: Challenge: Bitmap comparison

Wed Aug 07, 2013 9:50 am

Anyone happening on this thread would find helpful advice on the ImageMagick -al way of doing this kind of thing. You can script a direct compare; mask area of interest; do a difference- or do from a terminal command line.
http://www.imagemagick.org/Usage/compare/
Given that my flicker-example was a careless crop from a higher resolution image, taken hand-held with no controlled lighting, here's the kind of comparison that you get--
Image
The 'missing' component stands out but so do all the mis-alignments and tiny variations...

User avatar
rurwin
Forum Moderator
Forum Moderator
Posts: 4257
Joined: Mon Jan 09, 2012 3:16 pm
Contact: Website

Re: Challenge: Bitmap comparison

Wed Aug 07, 2013 10:33 am

I would have thought that the fastest way to get a good result would be to crop to the area that is certain to contain the feature of interest, and then calculate the average pixel colour. You'll get two ranges of values -- those with a screw and those without-- add a bit to those and test against them. Anything outside both ranges needs to be manually checked.

That will not necessarily work if someone inserts a black screw, there are large surface features on the panel or your placement is too inaccurate to crop tightly, but it's probably good enough.

It would work with the subtracted image above too. If the piece is in place it will be mostly black. If it is missing then it will be mostly it's colour.

marcelokappa
Posts: 1
Joined: Thu Apr 07, 2016 4:29 pm

Re: Challenge: Bitmap comparison

Thu Apr 07, 2016 4:31 pm

Dear Hurkl, can you tell me if you finally did it? and how?

Best regards

Marcelo.-

Return to “General discussion”