D3R0
Posts: 3
Joined: Tue Sep 18, 2018 11:44 am

Taking pictures in realtime

Wed Sep 19, 2018 6:57 am

Hello everyone,

I work on a project where I try to take an image of moving objects on a conveyor belt. The objects will be detected by a sensor.
The sensor is connected to the gpio via an optocoupler.
The programm is written in Python and the Camera is an USB Global Shutter from Hardkernel (the oCam-1CGN-U).

I use the libocam library from hardkernel for accessing the camera and taking pictures.
The whole setup is working, but not as I expected. The camera is taking pictures on signal, but the position of the objects changes between the pictures.
My expectation is that the object position in the image is nearly constant.
I've changed the setup from an Odroid to the Raspberry Pi, because the RPi.GPIO library supports hardware interrupts (not accessible on the Odroid because of WiringPi). But the changes were not noticeable.
I've also tried to reduce and increase the speed of the conveyor belt, but the result stays the same.

Here is my Code:

Code: Select all

import liboCams
from optparse import OptionParser
import RPi.GPIO as GPIO
from datetime import datetime
import time
import cv2
import os
from time import sleep

path = '/home/pi/Odroid_Scripte/Camera_GUI/pictures'

##Wiring Pi Setup
inputPin = 40
GPIO.setmode(GPIO.BOARD)
GPIO.setup(inputPin, GPIO.IN, pull_up_down = GPIO.PUD_DOWN)



Blc_Red   = 9963790  #min: 0 max: 250  White Balance Red Component
Blc_Blue  = 9963791  #min: 0 max: 250  White Balance Blue Component
Gain 	  = 9963795  #min: 0 max: 255  Gain
Exp	  = 10094850 #min: 1 max: 625  Exposure (Absolute)

def cam_init():
    parser = OptionParser()
    parser.add_option("-a", "--alltest", dest="alltest",
                                      action='store_true', help="test all resolution in playtime")
    parser.add_option("-t", "--time", dest="playtime", default = 1, type = "int",
                                      help="playtime for streaming [sec] intValue, 0 means forever")
    parser.add_option("-i", "--index", dest="index", default = 0, type = "int",
                                      help="index of resolusion mode")

    (options, args) = parser.parse_args()
              
    devpath = liboCams.FindCamera('oCam')
    if devpath is None:
      exit()
      
    global test  #Achtung Global
    test = liboCams.oCams(devpath, verbose=1)
    fmtlist = test.GetFormatList()
    ctrlist = test.GetControlList()
    test.Close()

    if options.alltest is True:
      len_range = range(len(fmtlist))
    else:
      if options.index >= len(fmtlist):
            options.index = 0  
      len_range = { options.index }  

    for i in len_range:
        test = liboCams.oCams(devpath)
        test.Set(fmtlist[i])
        name = test.GetName()
        test.Start()

    test.SetControl(Exp, 15)
    test.SetControl(Gain, 128)
    test.SetControl(Blc_Blue, 117)
    test.SetControl(Blc_Red, 197)
    
    test.GetFrame()


img_name = 0

def getimage(channel):
    global img_name
    frame = test.GetFrame()
    pic = cv2.imwrite(os.path.join(path,'img'+str(img_name)+'.jpg'), frame)
    img_name += 1
    print "saved: "+'img'+str(img_name)+'.jpg'
    

cam_init()

GPIO.add_event_detect(inputPin, GPIO.RISING, callback= getimage, bouncetime = 500)
#GPIO.add_event_detect(inputPin, GPIO.RISING, callback= run_cap1)

try:
    while True:
        sleep(0.5)
except KeyboardInterrupt:
    GPIO.cleanup()
    print "\nBye"

Meanwhile i'm not sure how to continue.

Hopefully my description is understandable, if not feel free to ask.
Thank you for your support

Sincerely D3R0

Edit: I am working on a Raspberry Pi 3 with Python 2.7

User avatar
topguy
Posts: 5007
Joined: Tue Oct 09, 2012 11:46 am
Location: Trondheim, Norway

Re: Taking pictures in realtime

Wed Sep 19, 2018 11:27 am

Based on the differences in position in the pictures and the speed of the conveyor belt what would you estimate the time variances between the pictures to be ?

Can we be sure the sensor on the belt is not the root cause of the variations ?

User avatar
RaTTuS
Posts: 10123
Joined: Tue Nov 29, 2011 11:12 am
Location: North West UK

Re: Taking pictures in realtime

Wed Sep 19, 2018 11:41 am

???
https://www.hardkernel.com/main/product ... 0091011568 ?
you are not going to be getting massice fram rates from this camera

the raspberry camera will be much faster
How To ask Questions :- http://www.catb.org/esr/faqs/smart-questions.html
WARNING - some parts of this post may be erroneous YMMV

1QC43qbL5FySu2Pi51vGqKqxy3UiJgukSX
Covfefe

D3R0
Posts: 3
Joined: Tue Sep 18, 2018 11:44 am

Re: Taking pictures in realtime

Wed Sep 19, 2018 11:57 am

topguy wrote: Based on the differences in position in the pictures and the speed of the conveyor belt what would you estimate the time variances between the pictures to be ?

Can we be sure the sensor on the belt is not the root cause of the variations ?
There will be around 200ms, between two activations of the sensor. If the Process works constantly, there would'nt be any variance between the positon of the object in the picture, because the object itself activates the sensor.

I am pretty sure that the problem does not depend on the light sensor, because we use them already in our production systems and there are no variations in activation time.

Edit: https://www.sick.com/gb/en/w8/wtb8-p2211/p/p210645 This is the light sensor and the specification of it.
RaTTuS wrote:
Wed Sep 19, 2018 11:41 am
???
https://www.hardkernel.com/main/product ... 0091011568 ?
you are not going to be getting massice fram rates from this camera

the raspberry camera will be much faster
The raspberry pi camera would'nt work for me, because of the moving conveyor belt, I need a camera with global shutter.
Also as you can read in the first part of this post, there will be one object in 200ms or 5 in one second, so, if I am not wrong, I don't need a high frame rate.

User avatar
topguy
Posts: 5007
Joined: Tue Oct 09, 2012 11:46 am
Location: Trondheim, Norway

Re: Taking pictures in realtime

Wed Sep 19, 2018 12:50 pm

but the position of the objects changes between the pictures.
I was more asking about what the timing "jitter" would be based on the observed different positions in the pictures. If you halve the speed of the belt the variance in positions in the pictures should also be halved.

Example:
- Object is 20cm long (in belt direction), the difference between two pictures is about 10% of length, 2cm.
- Belt moves at 100cm a second so "jitter" is 2cm/100cm and around 20ms.

Knowing this value will allow us to discuss how low jitter is plausible to achieve on a multitasking OS using a USB webcam.

Also..
A rolling shutter can probably be compensated for with a simple image transformation if you orient the camera correctly in reference to the belt and know the speed of the object.

D3R0
Posts: 3
Joined: Tue Sep 18, 2018 11:44 am

Re: Taking pictures in realtime

Fri Sep 21, 2018 8:58 am

Well we've made some tests and came up with a variation of 16% (32 mm) maximum at a speed of 0,96 m/s.
I have to get a variation of something around 5mm to be able to work further with the images.

User avatar
topguy
Posts: 5007
Joined: Tue Oct 09, 2012 11:46 am
Location: Trondheim, Norway

Re: Taking pictures in realtime

Fri Sep 21, 2018 9:33 am

So that calculates to a difference of 33ms. And you need something around 7 times better than that, 4-5 ms.

I think the main obstacle here is the USB part, if the trigger and the image are both sent over USB, and if there is also network traffic ( ethernet chip also on USB ) you cant really predict that well when things are going to happen. The USB protocol is probably not designed for this, and there is a chipset and driver in Linux that do most of the hidden work for you.

Return to “Advanced users”