Hatmpatn
Posts: 13
Joined: Wed Oct 24, 2018 6:11 pm

Project: Sorting robot using camera

Wed Oct 24, 2018 6:42 pm

Hi there!

I will start a project for a robot who can pick one type of small metal part travelling on a conveyor belt.

It should be able to:

-Recognize a specific shape of an object on the conveyor belt.
-Stop the conveyor belt which is powered by a stepper motor.
-Identify the X-Y-coordinates on the conveyor belt.
-Move an arm equipped with a vacuum suction cup to the coordinate.
-Pick the part, put it in a magazine.

I won't be able to do this without your help. The robot's design is pretty much already in order, but when it comes to the programming etc. I am very novice.

Here is a drawing of the project;

Image


Any tips and pointers, before I start? I basically have all the hardware, but which camera is more suitable? And is there any guide or page out there that talks about pretty much this stuff?

Hatmpatn
Posts: 13
Joined: Wed Oct 24, 2018 6:11 pm

Re: Project: Sorting robot using camera

Wed Oct 24, 2018 9:08 pm

Here is the metal part I want to pick up from the conveyor belt.

Image

And this is basically what I want to program to think. Red is the pickable ones, and the rest should just move on, until it returns again.
As you can see I just want the ones coming solo to be picked up, and the clusters are of no interest.

Image

User avatar
Joel_Mckay
Posts: 288
Joined: Mon Nov 12, 2012 10:22 pm
Contact: Website

Re: Project: Sorting robot using camera

Thu Oct 25, 2018 1:57 am

1. Pick up a book about OpenCV
https://www.amazon.com/Learning-OpenCV- ... 0596516134

2. Control your camera lighting to make the camera input consistent.

3. Control your camera glare with filters

4. cvPosit or solvePnp with dlib example
http://dlib.net/
https://www.learnopencv.com/head-pose-e ... -and-dlib/


I would highly recommend talking with your local university industry outreach coordinator, as supervised grad students can solve these types of trivial problems rather quickly.

Best of luck,
J

Hatmpatn
Posts: 13
Joined: Wed Oct 24, 2018 6:11 pm

Re: Project: Sorting robot using camera

Thu Oct 25, 2018 5:31 am

Thanks for the reply! It sure would be nice to send the job to the university but not as rewarding for me ;). I’ll have to try it out first myself, and see if I can make it work together with your help.

Could this be something?

https://docs.opencv.org/3.4.0/d4/dc6/tu ... ching.html

What camera should I buy? Is a raspberry pi enough for the task?

User avatar
Joel_Mckay
Posts: 288
Joined: Mon Nov 12, 2012 10:22 pm
Contact: Website

Re: Project: Sorting robot using camera

Thu Oct 25, 2018 6:41 am

Template matching is usually used for looking at closely matching image tiles, and is not designed to account for different orientations.
This is a classic problem, and a common assignment for an undergraduate level coarse.

In general, the camera used for high-speed lines runs at lower resolutions (720p), with a global shutter (expensive), and ups the frames-per-second as high as the host computer bus can handle (usually displaying every 30th frame buffer copy for the users).

In some ways vibratory-feeders are better at some types of problems, than trying to get a magnetic conveyor to pay nice with a Visual-Servoing algorithm.
https://en.wikipedia.org/wiki/Visual_servoing
https://visp.inria.fr/

I don't do this kind of work anymore... best of luck. ;-)
~J~

Hatmpatn
Posts: 13
Joined: Wed Oct 24, 2018 6:11 pm

Re: Project: Sorting robot using camera

Thu Oct 25, 2018 8:37 am

Hi there!
I was more thinking in the lines of rolling in a batch of metal parts into the camera view, take a picture, perform the action with the part picker, and then roll the batch away and a new batch incoming.
So no need of a high speed camera, the program will only handle images.

Hatmpatn
Posts: 13
Joined: Wed Oct 24, 2018 6:11 pm

Re: Project: Sorting robot using camera

Mon Dec 03, 2018 12:44 am

One month later and the program is up and running and working flawlessly. Using openCV was the key to everything. To all you who are looking to do similar stuff, dont be discouraged by pessimists. I am glad I wasnt! :)

Thanks to the helpful people in this thread!

User avatar
tlfong01
Posts: 1312
Joined: Sat Jun 02, 2018 1:43 pm
Location: Hong Kong

Re: Project: Sorting robot using camera

Mon Dec 03, 2018 6:11 am

Hatmpatn wrote:
Mon Dec 03, 2018 12:44 am
Using openCV was the key to everything.

openCv vs TensorFlow

I have no experience in either but I am interested to start playing AI things. Is TensorFlow many times harder than openCv?

Appendices - Rpi Tensor Flow to Recognize Hand Written Digits

How To Build a Neural Network to Recognize Handwritten Digits with TensorFlow 2018
https://www.digitalocean.com/community/ ... tensorflow

Google TensorFlow 1.9 Officially Supports Raspberry Pi Stretch Python 3 By Pete Warden 2018aug02
https://medium.com/tensorflow/tensorflo ... 91669b0aa0

How to install Google Tensorflow on Rpi - MagPi 2018Aug
https://www.raspberrypi.org/magpi/tenso ... pberry-pi/
There is a link to an example that uses the Pi Camera Module directly.
I am an electronics and smart home hobbyist.

User avatar
tlfong01
Posts: 1312
Joined: Sat Jun 02, 2018 1:43 pm
Location: Hong Kong

Re: Project: Sorting robot using camera

Mon Dec 03, 2018 8:55 am

Hatmpatn wrote:
Wed Oct 24, 2018 6:42 pm
a robot who can pick one type of small metal part travelling on a conveyor belt.
Any tips and pointers, before I start? I basically have all the hardware, but which camera is more suitable? And is there any guide or page out there that talks about pretty much this stuff?

TensorFlow Cucumber Sorter

You might like to start with cucumbers. :lol:

Makoto Koike TensorFlow Cucumber Sorter - 2016
https://www.raspberrypi.org/blog/now-added-cucumbers/
https://www.youtube.com/watch?v=4HCE1P-m1l8

Eshet Eilon Cucumber Sorting
https://www.youtube.com/watch?v=eTG21jUZ6Ak

3 cool machine learning projects using TensorFlow and the Raspberry Pi 2017
https://opensource.com/article/17/2/mac ... spberry-pi
Last edited by tlfong01 on Mon Dec 03, 2018 9:54 am, edited 1 time in total.
I am an electronics and smart home hobbyist.

User avatar
tlfong01
Posts: 1312
Joined: Sat Jun 02, 2018 1:43 pm
Location: Hong Kong

Re: Project: Sorting robot using camera

Mon Dec 03, 2018 9:46 am

Hatmpatn wrote:
Thu Oct 25, 2018 8:37 am
So no need of a high speed camera, the program will only handle mages.

TensorFlow Input Requirement

I read that TensorFlow is blind, it cannot see anything. It only wants a very long list of numbers. And it does not know any algorithm, not to mention any image analysis or pattern matching.

In short, TensorFlow wants a one dimensional number list, not even a two dimension matrix. In other words, you don't need any camera, just 100 cheap weight sensors can do a good job.

And you don't need to teach TensorFlow, it can learn by it self like AlphaGo, and sooner or later, will be smart enough to discover that their creators are so stupid and should be terminated! :lol:

BX120-3AA High precision resistance full bridge weight sensor US US$6/10 pieces

https://www.aliexpress.com/item/BX120-3 ... 82566.html

https://www.aliexpress.com/item/10pcs-l ... 93327.html

size 2.5mm x 3.3mm Size 122 BX120-0.5AA
...
Attachments
weight_sensor_matrix_2018dec0301.jpg
weight_sensor_matrix_2018dec0301.jpg (197.42 KiB) Viewed 1750 times
Last edited by tlfong01 on Fri Dec 07, 2018 11:57 am, edited 1 time in total.
I am an electronics and smart home hobbyist.

User avatar
tlfong01
Posts: 1312
Joined: Sat Jun 02, 2018 1:43 pm
Location: Hong Kong

Re: Project: Sorting robot using camera

Mon Dec 03, 2018 12:22 pm

Hatmpatn wrote:
Wed Oct 24, 2018 9:08 pm
As you can see I just want the ones coming solo to be picked up, and the clusters are of no interest.

Automatic Sorting and Orientating Machine

If what you want is to pick up the solo guys, then it is easy to DIY an vibrating bowl feeder type auto electro-mechanical sorting and orientating machine. Not even electronics is needed! :lol:

Design of an Automated Sorting and Orienting Machine for Electronic Pins by MS Chang, MIT 2011
https://dspace.mit.edu/bitstream/handle ... sequence=2
...
Attachments
sort_orient_machine_2018dec0302.jpg
sort_orient_machine_2018dec0302.jpg (171.94 KiB) Viewed 1737 times
I am an electronics and smart home hobbyist.

User avatar
tlfong01
Posts: 1312
Joined: Sat Jun 02, 2018 1:43 pm
Location: Hong Kong

Re: Project: Sorting robot using camera

Mon Dec 03, 2018 12:51 pm

tlfong01 wrote:
Mon Dec 03, 2018 12:22 pm
Automatic Sorting and Orientating Machine
If what you want is to pick up the solo guys, then it is easy to DIY an vibrating bowl feeder type auto electro-mechanical sorting and orientating machine. Not even electronics is needed!

Orienting and Facing Up

And since the little guys have a big head and small tail, also left right arm asymmetrical, it should not be that difficult to design a vibration feeder falling out gate and collecting grove to get the right orientation and also correct face up.
...
Attachments
orienting_machine_2018dec0301.jpg
orienting_machine_2018dec0301.jpg (77.42 KiB) Viewed 1730 times
I am an electronics and smart home hobbyist.

User avatar
tlfong01
Posts: 1312
Joined: Sat Jun 02, 2018 1:43 pm
Location: Hong Kong

Re: Project: Sorting robot using camera

Mon Dec 03, 2018 1:51 pm

tlfong01 wrote:
Mon Dec 03, 2018 12:51 pm
Orienting and Facing Up
And since the little guys have a big head and small tail, also left right arm asymmetrical, ...

Cashew Nuts for TensorFlow

In case your test metal parts are too big to handle, you might like to start with smaller size cashew nuts, which also have big head little tail, and left right arm asymmetrical.
...
Attachments
cashew_nuts_2018dec0301.jpg
cashew_nuts_2018dec0301.jpg (165.98 KiB) Viewed 1720 times
I am an electronics and smart home hobbyist.

Hatmpatn
Posts: 13
Joined: Wed Oct 24, 2018 6:11 pm

Re: Project: Sorting robot using camera

Tue Dec 04, 2018 1:17 am

Thanks for the reply but everything is all sorted and is now working! :)

Return to “Automation, sensing and robotics”