Page 1 of 3

Image overlay on live video feed

Posted: Wed May 01, 2019 4:53 pm
by Church Lenard
Hey im a complete Noob to this thing and ive tried following tutorials from opencv and many others but ive only seen them work for photo overlays

What im essentially trying to make is the scope from halo with a png overlay to make it legit but all ive found is photo editing and not live feed editing.

Pls help

Re: Image overlay on live video feed

Posted: Wed May 01, 2019 5:14 pm
by gordon77
Can you show us an example of what it will look like?

Re: Image overlay on live video feed

Posted: Wed May 01, 2019 6:23 pm
by Church Lenard
https://www.google.com/url?sa=i&source= ... 9016093105

This is the vid of it

He said he used opencv but i cant figure it out

Re: Image overlay on live video feed

Posted: Wed May 01, 2019 8:01 pm
by gordon77
I think you need to use addweighted..

https://stackoverflow.com/questions/408 ... ther-image

Where your video image will be the background image and the target (overlay) a saved image.

Re: Image overlay on live video feed

Posted: Thu May 02, 2019 8:28 am
by gordon77
Try this... You need to save the target image below.

Code: Select all

#!/usr/bin/env python3
import cv2
import os
import time

# setup pi camera 
if os.path.exists('/dev/video0') == False:
  path = 'sudo modprobe bcm2835-v4l2'
  os.system (path)
  time.sleep(1)
path = 'v4l2-ctl --set-ctrl=auto_exposure=0'
os.system (path)
  
# start video
cam = cv2.VideoCapture(0)
# setup window
winName = "Scope"
cv2.namedWindow(winName)

# read foreground image
foreground = cv2.imread('target.jpg')

while True:
    # take video frame
    ok, img = cam.read()
    # add the 2 images
    added_image = cv2.addWeighted(img,0.9,foreground,0.4,0.2)
    # show image
    cv2.imshow( winName,added_image)
    # wait
    key = cv2.waitKey(10)
    # press Esc to EXIT
    if key == 27:
       cv2.destroyWindow(winName)
       break

Re: Image overlay on live video feed

Posted: Thu May 02, 2019 10:14 am
by 6by9
OpenCV is a very heavyweight way to go about this if you are only after it for the live feed.

The Pi has compositing hardware (the Hardware Video Scalar, or HVS), which can compose multiple planes with alpha blending as it generates the HDMI (or DPI, DSI, or composite) output. This saves the ARM cores reading back all the image data, modifying the pixels, and then feeding it back to be rendered.

Some access is available via PiCamera, and you'd want format=rgba or format=bgra with add_overlay. IIRC you can nearly directly a PNG into a numpy array, and pass that into add_overlay.

Similar threads
viewtopic.php?p=702598
https://github.com/waveform80/picamera/issues/199

Re: Image overlay on live video feed

Posted: Thu May 02, 2019 3:47 pm
by Church Lenard
Foreground error

```the operation is neither 'array op array'```

And the list goes on but there is a red light on the camera and a blank window

Re: Image overlay on live video feed

Posted: Thu May 02, 2019 4:36 pm
by gordon77
Assuming you are running the code l posted can you upload the exact errors.

Have you had the camera working before with opencv?

Did you save image as target.jpg in /home/pi/?

Re: Image overlay on live video feed

Posted: Thu May 02, 2019 6:53 pm
by Church Lenard

Code: Select all

Traceback (most recent call last):
  File "/home/pi/test.py", line 27, in <module>
    added_image = cv2.addWeighted(img, 0.9, foreground, 0.4, 0.2)
cv2.error: Opencv(3.4.4) /home/pi/packaging/opencv-python/opencv/modules/core/src/arithm.cpp:660:
error: (-209: sizes of input arguments do not match) The operationis neither 'array op array' (where arrays have the same size and the same number of channels), nor 'array op scalar', nor 'scalar op array' in functio 'arithm_op'
this is the complete error i get

Re: Image overlay on live video feed

Posted: Thu May 02, 2019 6:58 pm
by gordon77
Interesting, my system produces video 640*480, and the target.jpg is as well

After the line ' # add the 2 images' add the line 'print (img.shape, foreground.shape)'

And tell us the result.

Re: Image overlay on live video feed

Posted: Thu May 02, 2019 7:12 pm
by Church Lenard

Code: Select all

File "/home/pi/test.pi", in <module>
  print  (img.shape,  foreground.shape)
AtributeError: 'NoneType' object has no attribute 'shape'
i am using the 3b+ 2015 model

Re: Image overlay on live video feed

Posted: Thu May 02, 2019 7:20 pm
by gordon77
Does your Pi camera work ok?

If you do ls /dev in a terminal does it list video0? Or video1?

Re: Image overlay on live video feed

Posted: Thu May 02, 2019 7:22 pm
by Church Lenard
i use raspivid and the module runs fine also it lists video0

Re: Image overlay on live video feed

Posted: Thu May 02, 2019 7:31 pm
by gordon77
Sorry but l don't know what's wrong. I use a pi 3b running full raspbian, and Python3.

Re: Image overlay on live video feed

Posted: Thu May 02, 2019 7:39 pm
by Church Lenard
is there anything i should try or have i just not installed openCV correctly?

Re: Image overlay on live video feed

Posted: Thu May 02, 2019 7:47 pm
by gordon77
I used this...

Code: Select all

 
sudo pip3 install opencv-python 
sudo apt-get install libcblas-dev
sudo apt-get install libhdf5-dev
sudo apt-get install libhdf5-serial-dev
sudo apt-get install libatlas-base-dev
sudo apt-get install libjasper-dev 
sudo apt-get install libqtgui4 
sudo apt-get install libqt4-test

Re: Image overlay on live video feed (edit)

Posted: Fri May 03, 2019 3:56 pm
by Church Lenard
Ok redownloaded now its import error on numpy.core.multiarry

Im going to try updating it


K nothing works right now im going to reset everything and start fresh

Re: Image overlay on live video feed

Posted: Sat Nov 16, 2019 8:30 pm
by Church Lenard
gordon77 wrote:
Thu May 02, 2019 8:28 am
Try this... You need to save the target image below.

Code: Select all

#!/usr/bin/env python3
import cv2
import os
import time

# setup pi camera 
if os.path.exists('/dev/video0') == False:
  path = 'sudo modprobe bcm2835-v4l2'
  os.system (path)
  time.sleep(1)
path = 'v4l2-ctl --set-ctrl=auto_exposure=0'
os.system (path)
  
# start video
cam = cv2.VideoCapture(0)
# setup window
winName = "Scope"
cv2.namedWindow(winName)

# read foreground image
foreground = cv2.imread('target.jpg')

while True:
    # take video frame
    ok, img = cam.read()
    # add the 2 images
    added_image = cv2.addWeighted(img,0.9,foreground,0.4,0.2)
    # show image
    cv2.imshow( winName,added_image)
    # wait
    key = cv2.waitKey(10)
    # press Esc to EXIT
    if key == 27:
       cv2.destroyWindow(winName)
       break
ok so i just ran this again with opencv and heres the error i get:

Code: Select all

Traceback (most recent call last):
	File "/home/pi/test.py", line 27, in <module>
		added_image = cv2.addWeighted(img,0.9,foreground,0.4,0.2)
cv2.error: OpenCV(4.1.2) /home/pi/opencv-4.1.2/modules/core/src/arithm.cpp:663:
error: (-209:Sizes of input arguments do not match) The operation is neither 'array op array'
(where arrays have the same sized and number of channels), nor 'array op scalar', nor 'scalar op array'
in function 'arithm_op'

Re: Image overlay on live video feed

Posted: Sat Nov 16, 2019 8:45 pm
by gordon77
This appears to be saying the video and the jpg are not the same size.

Re: Image overlay on live video feed

Posted: Sat Nov 16, 2019 8:47 pm
by Church Lenard
gordon77 wrote:
Sat Nov 16, 2019 8:45 pm
This appears to be saying the video and the jpg are not the same size.
and how exactly do i find the size of the video or is there a way to work around this?

Re: Image overlay on live video feed

Posted: Sun Nov 17, 2019 9:17 am
by gordon77
This will show you the video size, and set it to 640 x 480...

Also a new mask.

Code: Select all

#!/usr/bin/env python3
import cv2
import os
import time

# setup pi camera 
if os.path.exists('/dev/video0') == False:
  path = 'sudo modprobe bcm2835-v4l2'
  os.system (path)
  time.sleep(1)
path = 'v4l2-ctl --set-ctrl=auto_exposure=0'
os.system (path)
  
# start video
cam = cv2.VideoCapture(0)

# show video size
print (cam.get(3)  , cam.get(4))

# set video size
cam.set(3,640)
cam.set(4,480)

# setup window
winName = "Scope"
cv2.namedWindow(winName)

# read foreground image
img2 = cv2.imread('Reticle2.jpg')
img2gray = cv2.cvtColor(img2,cv2.COLOR_BGR2GRAY)
ret, mask = cv2.threshold(img2gray, 10, 255, cv2.THRESH_BINARY)

while True:
    # take video frame
    ok, img = cam.read()
    # add the 2 images
    added_image = cv2.bitwise_and(img,img,mask = mask)
    # show image
    cv2.imshow( winName,added_image)
    # wait
    key = cv2.waitKey(10)
    # press Esc to EXIT
    if key == 27:
       cv2.destroyWindow(winName)
       break
and a version with black or white reticle with a click of the mouse...

Code: Select all

#!/usr/bin/env python3
import cv2
import os
import time

invert_mask = 0

# mouse action
def point(event, x, y, flags, param):
    global invert_mask
    if event == cv2.EVENT_LBUTTONDOWN:
        if invert_mask == 0:
            invert_mask = 1
        else:
            invert_mask = 0
      
# setup pi camera 
if os.path.exists('/dev/video0') == False:
  path = 'sudo modprobe bcm2835-v4l2'
  os.system (path)
  time.sleep(1)
path = 'v4l2-ctl --set-ctrl=auto_exposure=0'
os.system (path)
  
# start video
cam = cv2.VideoCapture(0)

# show video size
print (cam.get(3),cam.get(4))

# set video size
cam.set(3,640)
cam.set(4,480)

# setup window
winName = "Scope"
cv2.namedWindow(winName)

# setup mouse callback               
cv2.setMouseCallback('Scope',point)

# read foreground image
img2 = cv2.imread('Reticle2.jpg')
img3 = (255-img2)
img2gray = cv2.cvtColor(img2,cv2.COLOR_BGR2GRAY)
ret, mask = cv2.threshold(img2gray, 10, 255, cv2.THRESH_BINARY)

while True:
    # take video frame
    ok, img = cam.read()
    # add the 2 images
    if invert_mask == 0:
        added_image = cv2.bitwise_and(img,img,mask = mask)
    else:
        added_image = cv2.addWeighted(img,0.99,img3,0.7,0.2)
    # show image
    cv2.imshow( winName,added_image)
    # wait
    key = cv2.waitKey(10)
    # press Esc to EXIT
    if key == 27:
       cv2.destroyWindow(winName)
       break

Re: Image overlay on live video feed

Posted: Sun Nov 17, 2019 12:24 pm
by HermannSW
Thank you for this cool and fast updating script!
I tested it, and took a smartphone video.
I had installed OpenCV according this instruction from scratch:
https://www.pyimagesearch.com/2019/09/1 ... an-buster/
All that was needed to execute ovl.py script was:

Code: Select all

workon cv
./ovl.py

"gordon77's cool OpenCV live video reticle overlay python scipt"
https://youtu.be/P6bvyHwtIqQ
Image

Re: Image overlay on live video feed

Posted: Sun Nov 17, 2019 2:20 pm
by gordon77
If you can live with an older version of opencv, which l do, and don't want to do a full build then try..

Code: Select all

sudo apt-get install python3-opencv
sudo apt-get install libhdf5-dev
sudo apt-get install libhdf5-serial-dev
sudo apt-get install libatlas-base-dev
sudo apt-get install libjasper-dev 
sudo apt-get install libqtgui4 
sudo apt-get install libqt4-test

Re: Image overlay on live video feed

Posted: Sun Nov 17, 2019 6:31 pm
by Church Lenard
gordon77 wrote:
Sun Nov 17, 2019 9:17 am
This will show you the video size, and set it to 640 x 480...

Also a new mask.

Code: Select all

#!/usr/bin/env python3
import cv2
import os
import time

# setup pi camera 
if os.path.exists('/dev/video0') == False:
  path = 'sudo modprobe bcm2835-v4l2'
  os.system (path)
  time.sleep(1)
path = 'v4l2-ctl --set-ctrl=auto_exposure=0'
os.system (path)
  
# start video
cam = cv2.VideoCapture(0)

# show video size
print (cam.get(3)  , cam.get(4))

# set video size
cam.set(3,640)
cam.set(4,480)

# setup window
winName = "Scope"
cv2.namedWindow(winName)

# read foreground image
img2 = cv2.imread('Reticle2.jpg')
img2gray = cv2.cvtColor(img2,cv2.COLOR_BGR2GRAY)
ret, mask = cv2.threshold(img2gray, 10, 255, cv2.THRESH_BINARY)

while True:
    # take video frame
    ok, img = cam.read()
    # add the 2 images
    added_image = cv2.bitwise_and(img,img,mask = mask)
    # show image
    cv2.imshow( winName,added_image)
    # wait
    key = cv2.waitKey(10)
    # press Esc to EXIT
    if key == 27:
       cv2.destroyWindow(winName)
       break
and a version with black or white reticle with a click of the mouse...

Code: Select all

#!/usr/bin/env python3
import cv2
import os
import time

invert_mask = 0

# mouse action
def point(event, x, y, flags, param):
    global invert_mask
    if event == cv2.EVENT_LBUTTONDOWN:
        if invert_mask == 0:
            invert_mask = 1
        else:
            invert_mask = 0
      
# setup pi camera 
if os.path.exists('/dev/video0') == False:
  path = 'sudo modprobe bcm2835-v4l2'
  os.system (path)
  time.sleep(1)
path = 'v4l2-ctl --set-ctrl=auto_exposure=0'
os.system (path)
  
# start video
cam = cv2.VideoCapture(0)

# show video size
print (cam.get(3),cam.get(4))

# set video size
cam.set(3,640)
cam.set(4,480)

# setup window
winName = "Scope"
cv2.namedWindow(winName)

# setup mouse callback               
cv2.setMouseCallback('Scope',point)

# read foreground image
img2 = cv2.imread('Reticle2.jpg')
img3 = (255-img2)
img2gray = cv2.cvtColor(img2,cv2.COLOR_BGR2GRAY)
ret, mask = cv2.threshold(img2gray, 10, 255, cv2.THRESH_BINARY)

while True:
    # take video frame
    ok, img = cam.read()
    # add the 2 images
    if invert_mask == 0:
        added_image = cv2.bitwise_and(img,img,mask = mask)
    else:
        added_image = cv2.addWeighted(img,0.99,img3,0.7,0.2)
    # show image
    cv2.imshow( winName,added_image)
    # wait
    key = cv2.waitKey(10)
    # press Esc to EXIT
    if key == 27:
       cv2.destroyWindow(winName)
       break
For the video size code i get

Code: Select all

img2gray = cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)
		error code (-215:Assertion Failed) in function cvtCOLOR
and in the code to switch it from black to white i get

Code: Select all

img3 = (255-img2)
TypeError: unsupported operand types for -: 'int' and 'NoneType'
also ive tried downloading those devs that you suggested and since it is version 4.1.2 i cant install them without purging the build

Re: Image overlay on live video feed

Posted: Sun Nov 17, 2019 7:01 pm
by gordon77
Is the camera working? The error message could suggest there is nothing in img. This may also explain your earlier problem.

Did it print a size?

Did you save the reticle as Reticle2.jpg?

If you do ls /dev do you see Video0?