Church Lenard
Posts: 34
Joined: Wed May 01, 2019 4:45 pm

Image overlay on live video feed

Wed May 01, 2019 4:53 pm

Hey im a complete Noob to this thing and ive tried following tutorials from opencv and many others but ive only seen them work for photo overlays

What im essentially trying to make is the scope from halo with a png overlay to make it legit but all ive found is photo editing and not live feed editing.

Pls help

gordon77
Posts: 4292
Joined: Sun Aug 05, 2012 3:12 pm

Re: Image overlay on live video feed

Wed May 01, 2019 5:14 pm

Can you show us an example of what it will look like?

Church Lenard
Posts: 34
Joined: Wed May 01, 2019 4:45 pm

Re: Image overlay on live video feed

Wed May 01, 2019 6:23 pm

https://www.google.com/url?sa=i&source= ... 9016093105

This is the vid of it

He said he used opencv but i cant figure it out

gordon77
Posts: 4292
Joined: Sun Aug 05, 2012 3:12 pm

Re: Image overlay on live video feed

Wed May 01, 2019 8:01 pm

I think you need to use addweighted..

https://stackoverflow.com/questions/408 ... ther-image

Where your video image will be the background image and the target (overlay) a saved image.

gordon77
Posts: 4292
Joined: Sun Aug 05, 2012 3:12 pm

Re: Image overlay on live video feed

Thu May 02, 2019 8:28 am

Try this... You need to save the target image below.

Code: Select all

#!/usr/bin/env python3
import cv2
import os
import time

# setup pi camera 
if os.path.exists('/dev/video0') == False:
  path = 'sudo modprobe bcm2835-v4l2'
  os.system (path)
  time.sleep(1)
path = 'v4l2-ctl --set-ctrl=auto_exposure=0'
os.system (path)
  
# start video
cam = cv2.VideoCapture(0)
# setup window
winName = "Scope"
cv2.namedWindow(winName)

# read foreground image
foreground = cv2.imread('target.jpg')

while True:
    # take video frame
    ok, img = cam.read()
    # add the 2 images
    added_image = cv2.addWeighted(img,0.9,foreground,0.4,0.2)
    # show image
    cv2.imshow( winName,added_image)
    # wait
    key = cv2.waitKey(10)
    # press Esc to EXIT
    if key == 27:
       cv2.destroyWindow(winName)
       break
Attachments
target.jpg
target.jpg (41.63 KiB) Viewed 1482 times

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 7523
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Image overlay on live video feed

Thu May 02, 2019 10:14 am

OpenCV is a very heavyweight way to go about this if you are only after it for the live feed.

The Pi has compositing hardware (the Hardware Video Scalar, or HVS), which can compose multiple planes with alpha blending as it generates the HDMI (or DPI, DSI, or composite) output. This saves the ARM cores reading back all the image data, modifying the pixels, and then feeding it back to be rendered.

Some access is available via PiCamera, and you'd want format=rgba or format=bgra with add_overlay. IIRC you can nearly directly a PNG into a numpy array, and pass that into add_overlay.

Similar threads
viewtopic.php?p=702598
https://github.com/waveform80/picamera/issues/199
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

Church Lenard
Posts: 34
Joined: Wed May 01, 2019 4:45 pm

Re: Image overlay on live video feed

Thu May 02, 2019 3:47 pm

Foreground error

```the operation is neither 'array op array'```

And the list goes on but there is a red light on the camera and a blank window

gordon77
Posts: 4292
Joined: Sun Aug 05, 2012 3:12 pm

Re: Image overlay on live video feed

Thu May 02, 2019 4:36 pm

Assuming you are running the code l posted can you upload the exact errors.

Have you had the camera working before with opencv?

Did you save image as target.jpg in /home/pi/?

Church Lenard
Posts: 34
Joined: Wed May 01, 2019 4:45 pm

Re: Image overlay on live video feed

Thu May 02, 2019 6:53 pm

Code: Select all

Traceback (most recent call last):
  File "/home/pi/test.py", line 27, in <module>
    added_image = cv2.addWeighted(img, 0.9, foreground, 0.4, 0.2)
cv2.error: Opencv(3.4.4) /home/pi/packaging/opencv-python/opencv/modules/core/src/arithm.cpp:660:
error: (-209: sizes of input arguments do not match) The operationis neither 'array op array' (where arrays have the same size and the same number of channels), nor 'array op scalar', nor 'scalar op array' in functio 'arithm_op'
this is the complete error i get

gordon77
Posts: 4292
Joined: Sun Aug 05, 2012 3:12 pm

Re: Image overlay on live video feed

Thu May 02, 2019 6:58 pm

Interesting, my system produces video 640*480, and the target.jpg is as well

After the line ' # add the 2 images' add the line 'print (img.shape, foreground.shape)'

And tell us the result.

Church Lenard
Posts: 34
Joined: Wed May 01, 2019 4:45 pm

Re: Image overlay on live video feed

Thu May 02, 2019 7:12 pm

Code: Select all

File "/home/pi/test.pi", in <module>
  print  (img.shape,  foreground.shape)
AtributeError: 'NoneType' object has no attribute 'shape'
i am using the 3b+ 2015 model

gordon77
Posts: 4292
Joined: Sun Aug 05, 2012 3:12 pm

Re: Image overlay on live video feed

Thu May 02, 2019 7:20 pm

Does your Pi camera work ok?

If you do ls /dev in a terminal does it list video0? Or video1?

Church Lenard
Posts: 34
Joined: Wed May 01, 2019 4:45 pm

Re: Image overlay on live video feed

Thu May 02, 2019 7:22 pm

i use raspivid and the module runs fine also it lists video0

gordon77
Posts: 4292
Joined: Sun Aug 05, 2012 3:12 pm

Re: Image overlay on live video feed

Thu May 02, 2019 7:31 pm

Sorry but l don't know what's wrong. I use a pi 3b running full raspbian, and Python3.

Church Lenard
Posts: 34
Joined: Wed May 01, 2019 4:45 pm

Re: Image overlay on live video feed

Thu May 02, 2019 7:39 pm

is there anything i should try or have i just not installed openCV correctly?

gordon77
Posts: 4292
Joined: Sun Aug 05, 2012 3:12 pm

Re: Image overlay on live video feed

Thu May 02, 2019 7:47 pm

I used this...

Code: Select all

 
sudo pip3 install opencv-python 
sudo apt-get install libcblas-dev
sudo apt-get install libhdf5-dev
sudo apt-get install libhdf5-serial-dev
sudo apt-get install libatlas-base-dev
sudo apt-get install libjasper-dev 
sudo apt-get install libqtgui4 
sudo apt-get install libqt4-test

Church Lenard
Posts: 34
Joined: Wed May 01, 2019 4:45 pm

Re: Image overlay on live video feed (edit)

Fri May 03, 2019 3:56 pm

Ok redownloaded now its import error on numpy.core.multiarry

Im going to try updating it


K nothing works right now im going to reset everything and start fresh

Church Lenard
Posts: 34
Joined: Wed May 01, 2019 4:45 pm

Re: Image overlay on live video feed

Sat Nov 16, 2019 8:30 pm

gordon77 wrote:
Thu May 02, 2019 8:28 am
Try this... You need to save the target image below.

Code: Select all

#!/usr/bin/env python3
import cv2
import os
import time

# setup pi camera 
if os.path.exists('/dev/video0') == False:
  path = 'sudo modprobe bcm2835-v4l2'
  os.system (path)
  time.sleep(1)
path = 'v4l2-ctl --set-ctrl=auto_exposure=0'
os.system (path)
  
# start video
cam = cv2.VideoCapture(0)
# setup window
winName = "Scope"
cv2.namedWindow(winName)

# read foreground image
foreground = cv2.imread('target.jpg')

while True:
    # take video frame
    ok, img = cam.read()
    # add the 2 images
    added_image = cv2.addWeighted(img,0.9,foreground,0.4,0.2)
    # show image
    cv2.imshow( winName,added_image)
    # wait
    key = cv2.waitKey(10)
    # press Esc to EXIT
    if key == 27:
       cv2.destroyWindow(winName)
       break
ok so i just ran this again with opencv and heres the error i get:

Code: Select all

Traceback (most recent call last):
	File "/home/pi/test.py", line 27, in <module>
		added_image = cv2.addWeighted(img,0.9,foreground,0.4,0.2)
cv2.error: OpenCV(4.1.2) /home/pi/opencv-4.1.2/modules/core/src/arithm.cpp:663:
error: (-209:Sizes of input arguments do not match) The operation is neither 'array op array'
(where arrays have the same sized and number of channels), nor 'array op scalar', nor 'scalar op array'
in function 'arithm_op'

gordon77
Posts: 4292
Joined: Sun Aug 05, 2012 3:12 pm

Re: Image overlay on live video feed

Sat Nov 16, 2019 8:45 pm

This appears to be saying the video and the jpg are not the same size.

Church Lenard
Posts: 34
Joined: Wed May 01, 2019 4:45 pm

Re: Image overlay on live video feed

Sat Nov 16, 2019 8:47 pm

gordon77 wrote:
Sat Nov 16, 2019 8:45 pm
This appears to be saying the video and the jpg are not the same size.
and how exactly do i find the size of the video or is there a way to work around this?

gordon77
Posts: 4292
Joined: Sun Aug 05, 2012 3:12 pm

Re: Image overlay on live video feed

Sun Nov 17, 2019 9:17 am

This will show you the video size, and set it to 640 x 480...

Also a new mask.

Code: Select all

#!/usr/bin/env python3
import cv2
import os
import time

# setup pi camera 
if os.path.exists('/dev/video0') == False:
  path = 'sudo modprobe bcm2835-v4l2'
  os.system (path)
  time.sleep(1)
path = 'v4l2-ctl --set-ctrl=auto_exposure=0'
os.system (path)
  
# start video
cam = cv2.VideoCapture(0)

# show video size
print (cam.get(3)  , cam.get(4))

# set video size
cam.set(3,640)
cam.set(4,480)

# setup window
winName = "Scope"
cv2.namedWindow(winName)

# read foreground image
img2 = cv2.imread('Reticle2.jpg')
img2gray = cv2.cvtColor(img2,cv2.COLOR_BGR2GRAY)
ret, mask = cv2.threshold(img2gray, 10, 255, cv2.THRESH_BINARY)

while True:
    # take video frame
    ok, img = cam.read()
    # add the 2 images
    added_image = cv2.bitwise_and(img,img,mask = mask)
    # show image
    cv2.imshow( winName,added_image)
    # wait
    key = cv2.waitKey(10)
    # press Esc to EXIT
    if key == 27:
       cv2.destroyWindow(winName)
       break
and a version with black or white reticle with a click of the mouse...

Code: Select all

#!/usr/bin/env python3
import cv2
import os
import time

invert_mask = 0

# mouse action
def point(event, x, y, flags, param):
    global invert_mask
    if event == cv2.EVENT_LBUTTONDOWN:
        if invert_mask == 0:
            invert_mask = 1
        else:
            invert_mask = 0
      
# setup pi camera 
if os.path.exists('/dev/video0') == False:
  path = 'sudo modprobe bcm2835-v4l2'
  os.system (path)
  time.sleep(1)
path = 'v4l2-ctl --set-ctrl=auto_exposure=0'
os.system (path)
  
# start video
cam = cv2.VideoCapture(0)

# show video size
print (cam.get(3),cam.get(4))

# set video size
cam.set(3,640)
cam.set(4,480)

# setup window
winName = "Scope"
cv2.namedWindow(winName)

# setup mouse callback               
cv2.setMouseCallback('Scope',point)

# read foreground image
img2 = cv2.imread('Reticle2.jpg')
img3 = (255-img2)
img2gray = cv2.cvtColor(img2,cv2.COLOR_BGR2GRAY)
ret, mask = cv2.threshold(img2gray, 10, 255, cv2.THRESH_BINARY)

while True:
    # take video frame
    ok, img = cam.read()
    # add the 2 images
    if invert_mask == 0:
        added_image = cv2.bitwise_and(img,img,mask = mask)
    else:
        added_image = cv2.addWeighted(img,0.99,img3,0.7,0.2)
    # show image
    cv2.imshow( winName,added_image)
    # wait
    key = cv2.waitKey(10)
    # press Esc to EXIT
    if key == 27:
       cv2.destroyWindow(winName)
       break
Attachments
Reticle2.jpg
Reticle2.jpg (11.39 KiB) Viewed 361 times
Last edited by gordon77 on Sun Nov 17, 2019 4:38 pm, edited 2 times in total.

User avatar
HermannSW
Posts: 1649
Joined: Fri Jul 22, 2016 9:09 pm
Location: Eberbach, Germany
Contact: Website Twitter YouTube

Re: Image overlay on live video feed

Sun Nov 17, 2019 12:24 pm

Thank you for this cool and fast updating script!
I tested it, and took a smartphone video.
I had installed OpenCV according this instruction from scratch:
https://www.pyimagesearch.com/2019/09/1 ... an-buster/
All that was needed to execute ovl.py script was:

Code: Select all

workon cv
./ovl.py

"gordon77's cool OpenCV live video reticle overlay python scipt"
https://youtu.be/P6bvyHwtIqQ
Image
⇨https://stamm-wilbrandt.de/en/Raspberry_camera.html

https://github.com/Hermann-SW/Raspberry_v1_camera_global_external_shutter
https://stamm-wilbrandt.de/github_repo_i420toh264
https://github.com/Hermann-SW/fork-raspiraw
https://twitter.com/HermannSW

gordon77
Posts: 4292
Joined: Sun Aug 05, 2012 3:12 pm

Re: Image overlay on live video feed

Sun Nov 17, 2019 2:20 pm

If you can live with an older version of opencv, which l do, and don't want to do a full build then try..

Code: Select all

sudo apt-get install python3-opencv
sudo apt-get install libhdf5-dev
sudo apt-get install libhdf5-serial-dev
sudo apt-get install libatlas-base-dev
sudo apt-get install libjasper-dev 
sudo apt-get install libqtgui4 
sudo apt-get install libqt4-test

Church Lenard
Posts: 34
Joined: Wed May 01, 2019 4:45 pm

Re: Image overlay on live video feed

Sun Nov 17, 2019 6:31 pm

gordon77 wrote:
Sun Nov 17, 2019 9:17 am
This will show you the video size, and set it to 640 x 480...

Also a new mask.

Code: Select all

#!/usr/bin/env python3
import cv2
import os
import time

# setup pi camera 
if os.path.exists('/dev/video0') == False:
  path = 'sudo modprobe bcm2835-v4l2'
  os.system (path)
  time.sleep(1)
path = 'v4l2-ctl --set-ctrl=auto_exposure=0'
os.system (path)
  
# start video
cam = cv2.VideoCapture(0)

# show video size
print (cam.get(3)  , cam.get(4))

# set video size
cam.set(3,640)
cam.set(4,480)

# setup window
winName = "Scope"
cv2.namedWindow(winName)

# read foreground image
img2 = cv2.imread('Reticle2.jpg')
img2gray = cv2.cvtColor(img2,cv2.COLOR_BGR2GRAY)
ret, mask = cv2.threshold(img2gray, 10, 255, cv2.THRESH_BINARY)

while True:
    # take video frame
    ok, img = cam.read()
    # add the 2 images
    added_image = cv2.bitwise_and(img,img,mask = mask)
    # show image
    cv2.imshow( winName,added_image)
    # wait
    key = cv2.waitKey(10)
    # press Esc to EXIT
    if key == 27:
       cv2.destroyWindow(winName)
       break
and a version with black or white reticle with a click of the mouse...

Code: Select all

#!/usr/bin/env python3
import cv2
import os
import time

invert_mask = 0

# mouse action
def point(event, x, y, flags, param):
    global invert_mask
    if event == cv2.EVENT_LBUTTONDOWN:
        if invert_mask == 0:
            invert_mask = 1
        else:
            invert_mask = 0
      
# setup pi camera 
if os.path.exists('/dev/video0') == False:
  path = 'sudo modprobe bcm2835-v4l2'
  os.system (path)
  time.sleep(1)
path = 'v4l2-ctl --set-ctrl=auto_exposure=0'
os.system (path)
  
# start video
cam = cv2.VideoCapture(0)

# show video size
print (cam.get(3),cam.get(4))

# set video size
cam.set(3,640)
cam.set(4,480)

# setup window
winName = "Scope"
cv2.namedWindow(winName)

# setup mouse callback               
cv2.setMouseCallback('Scope',point)

# read foreground image
img2 = cv2.imread('Reticle2.jpg')
img3 = (255-img2)
img2gray = cv2.cvtColor(img2,cv2.COLOR_BGR2GRAY)
ret, mask = cv2.threshold(img2gray, 10, 255, cv2.THRESH_BINARY)

while True:
    # take video frame
    ok, img = cam.read()
    # add the 2 images
    if invert_mask == 0:
        added_image = cv2.bitwise_and(img,img,mask = mask)
    else:
        added_image = cv2.addWeighted(img,0.99,img3,0.7,0.2)
    # show image
    cv2.imshow( winName,added_image)
    # wait
    key = cv2.waitKey(10)
    # press Esc to EXIT
    if key == 27:
       cv2.destroyWindow(winName)
       break
For the video size code i get

Code: Select all

img2gray = cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)
		error code (-215:Assertion Failed) in function cvtCOLOR
and in the code to switch it from black to white i get

Code: Select all

img3 = (255-img2)
TypeError: unsupported operand types for -: 'int' and 'NoneType'
also ive tried downloading those devs that you suggested and since it is version 4.1.2 i cant install them without purging the build

gordon77
Posts: 4292
Joined: Sun Aug 05, 2012 3:12 pm

Re: Image overlay on live video feed

Sun Nov 17, 2019 7:01 pm

Is the camera working? The error message could suggest there is nothing in img. This may also explain your earlier problem.

Did it print a size?

Did you save the reticle as Reticle2.jpg?

If you do ls /dev do you see Video0?

Return to “Camera board”