Image overlay on live video feed

[Church Lenard]

Hey im a complete Noob to this thing and ive tried following tutorials from opencv and many others but ive only seen them work for photo overlays

What im essentially trying to make is the scope from halo with a png overlay to make it legit but all ive found is photo editing and not live feed editing.

Pls help

[gordon77]

Can you show us an example of what it will look like?

[Church Lenard]

https://www.google.com/url?sa=i&source= ... 9016093105

This is the vid of it

He said he used opencv but i cant figure it out

[gordon77]

I think you need to use addweighted..

https://stackoverflow.com/questions/408 ... ther-image

Where your video image will be the background image and the target (overlay) a saved image.

[gordon77]

Try this... You need to save the target image below.

Code: Select all

#!/usr/bin/env python3
import cv2
import os
import time

# setup pi camera 
if os.path.exists('/dev/video0') == False:
  path = 'sudo modprobe bcm2835-v4l2'
  os.system (path)
  time.sleep(1)
path = 'v4l2-ctl --set-ctrl=auto_exposure=0'
os.system (path)
  
# start video
cam = cv2.VideoCapture(0)
# setup window
winName = "Scope"
cv2.namedWindow(winName)

# read foreground image
foreground = cv2.imread('target.jpg')

while True:
    # take video frame
    ok, img = cam.read()
    # add the 2 images
    added_image = cv2.addWeighted(img,0.9,foreground,0.4,0.2)
    # show image
    cv2.imshow( winName,added_image)
    # wait
    key = cv2.waitKey(10)
    # press Esc to EXIT
    if key == 27:
       cv2.destroyWindow(winName)
       break

[6by9]

OpenCV is a very heavyweight way to go about this if you are only after it for the live feed.

The Pi has compositing hardware (the Hardware Video Scalar, or HVS), which can compose multiple planes with alpha blending as it generates the HDMI (or DPI, DSI, or composite) output. This saves the ARM cores reading back all the image data, modifying the pixels, and then feeding it back to be rendered.

Some access is available via PiCamera, and you'd want format=rgba or format=bgra with add_overlay. IIRC you can nearly directly a PNG into a numpy array, and pass that into add_overlay.

Similar threads
viewtopic.php?p=702598
https://github.com/waveform80/picamera/issues/199

[Church Lenard]

Foreground error

```the operation is neither 'array op array'```

And the list goes on but there is a red light on the camera and a blank window

[gordon77]

Assuming you are running the code l posted can you upload the exact errors.

Have you had the camera working before with opencv?

Did you save image as target.jpg in /home/pi/?

[Church Lenard]

Code: Select all

Traceback (most recent call last):
  File "/home/pi/test.py", line 27, in <module>
    added_image = cv2.addWeighted(img, 0.9, foreground, 0.4, 0.2)
cv2.error: Opencv(3.4.4) /home/pi/packaging/opencv-python/opencv/modules/core/src/arithm.cpp:660:
error: (-209: sizes of input arguments do not match) The operationis neither 'array op array' (where arrays have the same size and the same number of channels), nor 'array op scalar', nor 'scalar op array' in functio 'arithm_op'

this is the complete error i get

[gordon77]

Interesting, my system produces video 640*480, and the target.jpg is as well

After the line ' # add the 2 images' add the line 'print (img.shape, foreground.shape)'

And tell us the result.

[Church Lenard]

Code: Select all

File "/home/pi/test.pi", in <module>
  print  (img.shape,  foreground.shape)
AtributeError: 'NoneType' object has no attribute 'shape'

i am using the 3b+ 2015 model

[gordon77]

Does your Pi camera work ok?

If you do ls /dev in a terminal does it list video0? Or video1?

[Church Lenard]

i use raspivid and the module runs fine also it lists video0

[gordon77]

Sorry but l don't know what's wrong. I use a pi 3b running full raspbian, and Python3.

[Church Lenard]

is there anything i should try or have i just not installed openCV correctly?

[gordon77]

I used this...

Code: Select all

 
sudo pip3 install opencv-python 
sudo apt-get install libcblas-dev
sudo apt-get install libhdf5-dev
sudo apt-get install libhdf5-serial-dev
sudo apt-get install libatlas-base-dev
sudo apt-get install libjasper-dev 
sudo apt-get install libqtgui4 
sudo apt-get install libqt4-test

[Church Lenard]

Ok redownloaded now its import error on numpy.core.multiarry

Im going to try updating it


K nothing works right now im going to reset everything and start fresh

[Church Lenard]

Try this... You need to save the target image below.

Code: Select all

#!/usr/bin/env python3
import cv2
import os
import time

# setup pi camera 
if os.path.exists('/dev/video0') == False:
  path = 'sudo modprobe bcm2835-v4l2'
  os.system (path)
  time.sleep(1)
path = 'v4l2-ctl --set-ctrl=auto_exposure=0'
os.system (path)
  
# start video
cam = cv2.VideoCapture(0)
# setup window
winName = "Scope"
cv2.namedWindow(winName)

# read foreground image
foreground = cv2.imread('target.jpg')

while True:
    # take video frame
    ok, img = cam.read()
    # add the 2 images
    added_image = cv2.addWeighted(img,0.9,foreground,0.4,0.2)
    # show image
    cv2.imshow( winName,added_image)
    # wait
    key = cv2.waitKey(10)
    # press Esc to EXIT
    if key == 27:
       cv2.destroyWindow(winName)
       break

ok so i just ran this again with opencv and heres the error i get:

Code: Select all

Traceback (most recent call last):
	File "/home/pi/test.py", line 27, in <module>
		added_image = cv2.addWeighted(img,0.9,foreground,0.4,0.2)
cv2.error: OpenCV(4.1.2) /home/pi/opencv-4.1.2/modules/core/src/arithm.cpp:663:
error: (-209:Sizes of input arguments do not match) The operation is neither 'array op array'
(where arrays have the same sized and number of channels), nor 'array op scalar', nor 'scalar op array'
in function 'arithm_op'

[gordon77]

This appears to be saying the video and the jpg are not the same size.

[Church Lenard]

This appears to be saying the video and the jpg are not the same size.

and how exactly do i find the size of the video or is there a way to work around this?

[gordon77]

This will show you the video size, and set it to 640 x 480...

Also a new mask.

Code: Select all

#!/usr/bin/env python3
import cv2
import os
import time

# setup pi camera 
if os.path.exists('/dev/video0') == False:
  path = 'sudo modprobe bcm2835-v4l2'
  os.system (path)
  time.sleep(1)
path = 'v4l2-ctl --set-ctrl=auto_exposure=0'
os.system (path)
  
# start video
cam = cv2.VideoCapture(0)

# show video size
print (cam.get(3)  , cam.get(4))

# set video size
cam.set(3,640)
cam.set(4,480)

# setup window
winName = "Scope"
cv2.namedWindow(winName)

# read foreground image
img2 = cv2.imread('Reticle2.jpg')
img2gray = cv2.cvtColor(img2,cv2.COLOR_BGR2GRAY)
ret, mask = cv2.threshold(img2gray, 10, 255, cv2.THRESH_BINARY)

while True:
    # take video frame
    ok, img = cam.read()
    # add the 2 images
    added_image = cv2.bitwise_and(img,img,mask = mask)
    # show image
    cv2.imshow( winName,added_image)
    # wait
    key = cv2.waitKey(10)
    # press Esc to EXIT
    if key == 27:
       cv2.destroyWindow(winName)
       break

and a version with black or white reticle with a click of the mouse...

Code: Select all

#!/usr/bin/env python3
import cv2
import os
import time

invert_mask = 0

# mouse action
def point(event, x, y, flags, param):
    global invert_mask
    if event == cv2.EVENT_LBUTTONDOWN:
        if invert_mask == 0:
            invert_mask = 1
        else:
            invert_mask = 0
      
# setup pi camera 
if os.path.exists('/dev/video0') == False:
  path = 'sudo modprobe bcm2835-v4l2'
  os.system (path)
  time.sleep(1)
path = 'v4l2-ctl --set-ctrl=auto_exposure=0'
os.system (path)
  
# start video
cam = cv2.VideoCapture(0)

# show video size
print (cam.get(3),cam.get(4))

# set video size
cam.set(3,640)
cam.set(4,480)

# setup window
winName = "Scope"
cv2.namedWindow(winName)

# setup mouse callback               
cv2.setMouseCallback('Scope',point)

# read foreground image
img2 = cv2.imread('Reticle2.jpg')
img3 = (255-img2)
img2gray = cv2.cvtColor(img2,cv2.COLOR_BGR2GRAY)
ret, mask = cv2.threshold(img2gray, 10, 255, cv2.THRESH_BINARY)

while True:
    # take video frame
    ok, img = cam.read()
    # add the 2 images
    if invert_mask == 0:
        added_image = cv2.bitwise_and(img,img,mask = mask)
    else:
        added_image = cv2.addWeighted(img,0.99,img3,0.7,0.2)
    # show image
    cv2.imshow( winName,added_image)
    # wait
    key = cv2.waitKey(10)
    # press Esc to EXIT
    if key == 27:
       cv2.destroyWindow(winName)
       break

[HermannSW]

Thank you for this cool and fast updating script!
I tested it, and took a smartphone video.
I had installed OpenCV according this instruction from scratch:
https://www.pyimagesearch.com/2019/09/1 ... an-buster/
All that was needed to execute ovl.py script was:

Code: Select all

workon cv
./ovl.py

"gordon77's cool OpenCV live video reticle overlay python scipt"
https://youtu.be/P6bvyHwtIqQ

[gordon77]

If you can live with an older version of opencv, which l do, and don't want to do a full build then try..

Code: Select all

sudo apt-get install python3-opencv
sudo apt-get install libhdf5-dev
sudo apt-get install libhdf5-serial-dev
sudo apt-get install libatlas-base-dev
sudo apt-get install libjasper-dev 
sudo apt-get install libqtgui4 
sudo apt-get install libqt4-test

[Church Lenard]

This will show you the video size, and set it to 640 x 480...

Also a new mask.

Code: Select all

#!/usr/bin/env python3
import cv2
import os
import time

# setup pi camera 
if os.path.exists('/dev/video0') == False:
  path = 'sudo modprobe bcm2835-v4l2'
  os.system (path)
  time.sleep(1)
path = 'v4l2-ctl --set-ctrl=auto_exposure=0'
os.system (path)
  
# start video
cam = cv2.VideoCapture(0)

# show video size
print (cam.get(3)  , cam.get(4))

# set video size
cam.set(3,640)
cam.set(4,480)

# setup window
winName = "Scope"
cv2.namedWindow(winName)

# read foreground image
img2 = cv2.imread('Reticle2.jpg')
img2gray = cv2.cvtColor(img2,cv2.COLOR_BGR2GRAY)
ret, mask = cv2.threshold(img2gray, 10, 255, cv2.THRESH_BINARY)

while True:
    # take video frame
    ok, img = cam.read()
    # add the 2 images
    added_image = cv2.bitwise_and(img,img,mask = mask)
    # show image
    cv2.imshow( winName,added_image)
    # wait
    key = cv2.waitKey(10)
    # press Esc to EXIT
    if key == 27:
       cv2.destroyWindow(winName)
       break

and a version with black or white reticle with a click of the mouse...

Code: Select all

#!/usr/bin/env python3
import cv2
import os
import time

invert_mask = 0

# mouse action
def point(event, x, y, flags, param):
    global invert_mask
    if event == cv2.EVENT_LBUTTONDOWN:
        if invert_mask == 0:
            invert_mask = 1
        else:
            invert_mask = 0
      
# setup pi camera 
if os.path.exists('/dev/video0') == False:
  path = 'sudo modprobe bcm2835-v4l2'
  os.system (path)
  time.sleep(1)
path = 'v4l2-ctl --set-ctrl=auto_exposure=0'
os.system (path)
  
# start video
cam = cv2.VideoCapture(0)

# show video size
print (cam.get(3),cam.get(4))

# set video size
cam.set(3,640)
cam.set(4,480)

# setup window
winName = "Scope"
cv2.namedWindow(winName)

# setup mouse callback               
cv2.setMouseCallback('Scope',point)

# read foreground image
img2 = cv2.imread('Reticle2.jpg')
img3 = (255-img2)
img2gray = cv2.cvtColor(img2,cv2.COLOR_BGR2GRAY)
ret, mask = cv2.threshold(img2gray, 10, 255, cv2.THRESH_BINARY)

while True:
    # take video frame
    ok, img = cam.read()
    # add the 2 images
    if invert_mask == 0:
        added_image = cv2.bitwise_and(img,img,mask = mask)
    else:
        added_image = cv2.addWeighted(img,0.99,img3,0.7,0.2)
    # show image
    cv2.imshow( winName,added_image)
    # wait
    key = cv2.waitKey(10)
    # press Esc to EXIT
    if key == 27:
       cv2.destroyWindow(winName)
       break

For the video size code i get

Code: Select all

img2gray = cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)
		error code (-215:Assertion Failed) in function cvtCOLOR

and in the code to switch it from black to white i get

Code: Select all

img3 = (255-img2)
TypeError: unsupported operand types for -: 'int' and 'NoneType'

also ive tried downloading those devs that you suggested and since it is version 4.1.2 i cant install them without purging the build

[gordon77]

Is the camera working? The error message could suggest there is nothing in img. This may also explain your earlier problem.

Did it print a size?

Did you save the reticle as Reticle2.jpg?

If you do ls /dev do you see Video0?