fishbulb
Posts: 4
Joined: Thu Jul 12, 2018 8:00 pm

rapidly taking pictures from PiCam and stepping motor

Mon Jul 16, 2018 6:42 pm

Hello everyone,

I am trying to build a project where I move two stepper motors one step each than take and image in a loop. I would like the loop to go as quickly as possible (maybe 20 loops per second). Eventually, I will do image processing on these images. I've used sample code that I've found online for both the steppers and the PiCam, but I've been very surprised how slowly the pi is taking images. If I only run the steppers and comment out the camera code, the Python reports about 20 loops per second, though it seems to slow with time. With only the camera taking images in the loop, the rate goes down to between 2 -3 loops per second, whether or not I display the images. I fear there is something that I am doing wrong.

I am using Python 2.7.9 on a Raspberry Pi 3.

This is the stepper HAT that I am using.
https://www.amazon.com/gp/product/B0721 ... UTF8&psc=1

I am using a standard PiCam for the camera

Here is my code. You can see that for now I have commented out the stepper motors.

Any help is greatly appreciated.
FB

Code: Select all

#!/usr/bin/python
#import Raspi_MotorHAT, Raspi_DCMotor, Raspi_Stepper 
from Raspi_MotorHAT import Raspi_MotorHAT, Raspi_DCMotor, Raspi_StepperMotor
import time
import atexit

from picamera.array import PiRGBArray
from picamera import PiCamera
import cv2

import datetime
import numpy as np

#initialize the camera and grab a reference to the raw camera caputure
camera = PiCamera()
camera.resolution = (640, 480)
rawCapture = PiRGBArray(camera, size=(640, 480))

#allow the camera to warm up
time.sleep(1)

# create a default object, no changes to I2C address or frequency
mh = Raspi_MotorHAT(0x6F)

# recommended for auto-disabling motors on shutdown!
def turnOffMotors():
    mh.getMotor(1).run(Raspi_MotorHAT.RELEASE)
    mh.getMotor(2).run(Raspi_MotorHAT.RELEASE)
    mh.getMotor(3).run(Raspi_MotorHAT.RELEASE)
    mh.getMotor(4).run(Raspi_MotorHAT.RELEASE)

atexit.register(turnOffMotors)

myStepper1 = mh.getStepper(200, 1)  	# 200 steps/rev, motor port #1
myStepper2 = mh.getStepper(200, 2)  	# 200 steps/rev, motor port #1


t_end = time.time() + 10

print('Starting Trial')
start_time = datetime.datetime.now()

while time.time() < t_end:

    #Perform minor timing calculations to get timestamps
    doneFrameTime=datetime.datetime.now()
    diffTime = doneFrameTime-start_time
    frameRate = 1/diffTime.total_seconds()
    start_time=datetime.datetime.now()
    print frameRate
    
    #myStepper1.oneStep(Raspi_MotorHAT.FORWARD,  Raspi_MotorHAT.MICROSTEP)
    #myStepper2.oneStep(Raspi_MotorHAT.FORWARD,  Raspi_MotorHAT.MICROSTEP)

    #grab an image from the camera
    camera.capture(rawCapture, format="rgb")
    image = rawCapture.array

    #display the image on the screen and wait for a keypress
    #cv2.imshow("Image", image)
    #cv2.waitKey(1)
    rawCapture.truncate(0)
    #time.sleep(.01)







    
turnOffMotors()


pootle
Posts: 249
Joined: Wed Sep 04, 2013 10:20 am
Location: Staffordshire
Contact: Website

Re: rapidly taking pictures from PiCam and stepping motor

Tue Jul 17, 2018 8:21 am

I spent time trying to minimise the gap betwen images, although I was doing fairly long exposures, the technique may help with your problem.

I used capture_continuous - see the sample code here with a detailed example here.

Use a stream or at least set up a small ram disc and capture to a cyclic set of filenames. Use a separate process to do further work on the images.

Oh and switch to Python3 as well ;).

fishbulb
Posts: 4
Joined: Thu Jul 12, 2018 8:00 pm

Re: rapidly taking pictures from PiCam and stepping motor

Wed Jul 18, 2018 2:20 pm

Thank you for taking the time to reply tootle. I appreciate it. I went back to some old code to get an idea of what the problem might be. In this instance I'm using a USB webcam and the cv2.VideoCapture function as opposed to the PiCam and camera.capture function. I don't believe the camera is the issue, so is VideoCapture just so much faster than camera.capture and picamera library? Is it possible and a good idea to use VideoCapture with the pi Camera.

Thanks again,
FB

Here is the modified code using VideoCapture. It easily goes 20 fps even with displaying the image and image processing on the Pi 3.

Code: Select all

# import the necessary packages
from Raspi_MotorHAT import Raspi_MotorHAT, Raspi_DCMotor, Raspi_StepperMotor
import time
import cv2
import atexit
import datetime
import numpy as np

# initialize the camera and grab a reference to the raw camera capture
font = cv2.FONT_HERSHEY_SIMPLEX
cap = cv2.VideoCapture(0) #Identifies camera. Can identify multiple

# allow the camera to warmup

# create a default object, no changes to I2C address or frequency
mh = Raspi_MotorHAT(0x6F)

# recommended for auto-disabling motors on shutdown!
def turnOffMotors():
    mh.getMotor(1).run(Raspi_MotorHAT.RELEASE)
    mh.getMotor(2).run(Raspi_MotorHAT.RELEASE)
    mh.getMotor(3).run(Raspi_MotorHAT.RELEASE)
    mh.getMotor(4).run(Raspi_MotorHAT.RELEASE)

atexit.register(turnOffMotors)

#myStepper1 = mh.getStepper(200, 1)  	# 200 steps/rev, motor port #1
myMotor = mh.getMotor(1)
myMotor.setSpeed(50)
myMotor.run(Raspi_MotorHAT.FORWARD);

myStepper2 = mh.getStepper(200, 2)  	# 200 steps/rev, motor port #1
time.sleep(0.1)

t_end = time.time() + 10
print('Starting Trial')
start_time = datetime.datetime.now()

# capture frames from the camera
while time.time() < t_end:

	ret, image = cap.read()
	doneFrameTime=datetime.datetime.now()
	diffTime = doneFrameTime-start_time
	frameRate = 1/diffTime.total_seconds()
	start_time=datetime.datetime.now()
	print frameRate
	myStepper2.oneStep(Raspi_MotorHAT.FORWARD,  Raspi_MotorHAT.MICROSTEP)

	# show the frame
	cv2.imshow("Frame", image)
	key = cv2.waitKey(1)



turnOffMotors()

gordon77
Posts: 3270
Joined: Sun Aug 05, 2012 3:12 pm

Re: rapidly taking pictures from PiCam and stepping motor

Fri Jul 20, 2018 12:45 pm

Deleted
Last edited by gordon77 on Sat Jul 21, 2018 4:08 pm, edited 2 times in total.

pootle
Posts: 249
Joined: Wed Sep 04, 2013 10:20 am
Location: Staffordshire
Contact: Website

Re: rapidly taking pictures from PiCam and stepping motor

Fri Jul 20, 2018 4:24 pm

Ah! I didn't realise you were using a USB webcam - your title implied you were using a standard raspberry pi camera. This will be a lot slower than a raspberry pi camera as USB interface is a lot slower than the special interface to the pi camera.

I don't know about how cv2 works with a webcam, and the camera itself may limit what you can do. It is likely though that streaming video, and throwing away irrelevant frames, will work a lot better than taking individual pictures, as webcams are often optimized for low lag on streaming video. You may have to restrict the video size and you will have to try out how many frames to discard after you trigger the stepper before you get a stable frame - there will be delay between the frames as captured by the camera and when you get them on Raspberry pi.

fishbulb
Posts: 4
Joined: Thu Jul 12, 2018 8:00 pm

Re: rapidly taking pictures from PiCam and stepping motor

Fri Jul 20, 2018 5:19 pm

Thanks again pootle. No, it seems like I've caused more confusion. I only switched back to the USB camera when I tested the old code using the cv2.VideoCapture function.

So with older code and USB camera using cv2.VideoCapture, I get about 30 FPS.

Using the pi camera module (not the USB camera) and PiCam camera.capture, I'm getting the 2 -3 FPS.

I am trying to switch over to your code with Python 3, but it doesn't look like the Raspi_MotorHAT library that I have works with Python 3. So I will try to first implement your PiCam library streaming option first in Python 2 until I can figure out the stepper motor part.

So I take it your opinion is that it's definitely NOT the Raspberry pi camera that's limiting? Hopefully changing the code over will work. And you would also recommend to work with the PiCam library and abandon the cv2.VideoCapture correct?

Thanks,
FB

pootle
Posts: 249
Joined: Wed Sep 04, 2013 10:20 am
Location: Staffordshire
Contact: Website

Re: rapidly taking pictures from PiCam and stepping motor

Sat Jul 21, 2018 8:44 am

The pi camera in single photo mode is very slow, but as the sample code using capture continuous shows, that mode is pretty fast.

gordon77
Posts: 3270
Joined: Sun Aug 05, 2012 3:12 pm

Re: rapidly taking pictures from PiCam and stepping motor

Sun Jul 22, 2018 10:24 am

I experimented with some of the code pootle linked to. It goes quicker but as expected you get some blurry images as I couldn't get the image taking to sync to the stepper motor moving. I tried with the camera hardware sync pulses ( viewtopic.php?p=1316237 ) but still didn't cure it.
I tell it to stop after 10 images but it runs longer than that for some reason , and reports an error, which I don't understand.

Code: Select all

#!/usr/bin/python

import os
import pygame, sys
import time
import subprocess, glob
import signal
from Adafruit_MotorHAT import Adafruit_MotorHAT, Adafruit_DCMotor, Adafruit_StepperMotor
import RPi.GPIO as GPIO
GPIO.setmode (GPIO.BOARD)
GPIO.setwarnings(False)
GPIO.setup(11,GPIO.IN,pull_up_down = GPIO.PUD_UP)
import io
import threading
import picamera
from PIL import Image


#initialise stepper motor
mh = Adafruit_MotorHAT(addr = 0x60)
myStepper = mh.getStepper(200, 1)       
myStepper.setSpeed(120)

class ImageProcessor(threading.Thread):
    def __init__(self, owner):
        super(ImageProcessor, self).__init__()
        self.stream = io.BytesIO()
        self.event = threading.Event()
        self.terminated = False
        self.owner = owner
        self.start()

    def run(self):
        # This method runs in a separate thread
        x = 0
        while not self.terminated:
            # Wait for an image to be written to the stream
            if self.event.wait(1):
                try:
                    self.stream.seek(0)
                    # move stepper motor
                    myStepper.step(1, Adafruit_MotorHAT.FORWARD, Adafruit_MotorHAT.SINGLE)
                    # wait some time for stepper motor to move
                    time.sleep(.05)
                    # Read the image and do some processing on it
                    im = Image.open(self.stream)
                    x +=1
                    im.save("/run/shm/Image" + str(x), "JPEG")
                    # Set done to True if you want the script to terminate
                    # at some point
                    # stop after 10 captures
                    if x > 10:
                        self.owner.done=True
                finally:
                    # Reset the stream and event
                    self.stream.seek(0)
                    self.stream.truncate()
                    self.event.clear()
                    # Return ourselves to the available pool
                    with self.owner.lock:
                        self.owner.pool.append(self)
        
class ProcessOutput(object):
    def __init__(self):
        self.done = False
        # Construct a pool of 4 image processors along with a lock
        # to control access between threads
        self.lock = threading.Lock()
        self.pool = [ImageProcessor(self) for i in range(4)]
        self.processor = None

    def write(self, buf):
        if buf.startswith(b'\xff\xd8'):
            # New frame; set the current processor going and grab
            # a spare one
            if self.processor:
                self.processor.event.set()
            with self.lock:
                if self.pool:
                    self.processor = self.pool.pop()
                else:
                    # No processor's available, we'll have to skip
                    # this frame; you may want to print a warning
                    # here to see whether you hit this case
                    self.processor = None
        if self.processor:
            self.processor.stream.write(buf)

    def flush(self):
        # When told to flush (this indicates end of recording), shut
        # down in an orderly fashion. First, add the current processor
        # back to the pool
        if self.processor:
            with self.lock:
                self.pool.append(self.processor)
                self.processor = None
        # Now, empty the pool, joining each thread as we go
        while True:
            with self.lock:
                try:
                    proc = self.pool.pop()
                except IndexError:
                    pass # pool is empty
            proc.terminated = True
            proc.join()

with picamera.PiCamera(resolution='VGA') as camera:
    camera.start_preview()
    time.sleep(2)
    output = ProcessOutput()
    camera.start_recording(output, format='mjpeg')
    while not output.done:
        camera.wait_recording(1)
    camera.stop_recording()

Return to “Python”

Who is online

Users browsing this forum: Google [Bot] and 14 guests