splitthediff
Posts: 9
Joined: Mon Jul 19, 2021 9:46 pm

migrating a VS program to RPI

Tue Jul 20, 2021 2:14 pm

My project currently tracks some shapes in the visual space using a USB camera shield, sensor, PC and visual studio 2017. How difficult would it be to port it to RPI for and lose the PC step with an existing openCV code that works? My concerns are whether the raspberry PI is fast enough to handle my openCV app, whether there would be significant latency and my own novice skills in linux (I am moderate in C++/openCV). How do I port the VS program to RPI?

drgeoff
Posts: 12182
Joined: Wed Jan 25, 2012 6:39 pm

Re: migrating a VS program to RPI

Wed Jul 21, 2021 2:37 pm

Visual Studio is an IDE (Integrated Development Environment). Therefore the words "VS Program" don't really make sense. Which programming language is the code written in?
Quis custodiet ipsos custodes?

dbrion06
Posts: 644
Joined: Tue May 28, 2019 11:57 am

Re: migrating a VS program to RPI

Wed Jul 21, 2021 5:20 pm

Opencv is written in C++ (unless you have av 1.xx version) and can be easily used with C++ or python3 (it is just a wrapper: 95% of the job is done by C,C++ functions ... and Fortran -yes- SUBROUTINES). It is system-agnostic (but cameras are not!; neither are sensors)

1rst, you might find useful to know the version of opencv on windows
if you use python on windows, it is very simple (I never dared: works on GNUlinux and python is OS agnostic). It works on GNUlinux

Code: Select all

import cv2
print (cv2.__VERSION__)
in C++, is seems simple, too https://learnopencv.com/how-to-find-ope ... ython-cpp/

Code: Select all

#include "opencv2/opencv.hpp
	using namespace cv;
	using namespace std;
	 
	int main( int argc, char** argv )	{
	  cout << "OpenCV version : " << CV_VERSION << endl;
}
Once you know the version on windows, things can become complicated. I suppose you want a newer version than the windows one:

Debian (hence, RPi) ships the 3.2 version (you must wait until they ship a newer one, or compile from source)
https://packages.debian.org/buster/libopencv-dev
the command

Code: Select all

sudo apt-get install libopencv-dev
will bring you each and everything you need to compile applications.(c++ already exists and is preinstalled on RPIOS)... in command line, if 3.2 is enough.

If you need a newer shinier version you have two solutions:
a) wait until a newRPiOS is ready , read books, enjoy sunshine
b) compile freom source. Best tutorial -I use it and am fully satisfied with- I think is
https://qengineering.eu/install-raspberry-64-os.html (if you need 64 bits: it is a beta version I am very satisfied with ... but never played with cameras...) If you need 32 bits, it is trivial.
and then
https://qengineering.eu/install-opencv- ... -pi-4.html
(and its brother for 64 bits: both worked flawlessly ... for me and generated a c++ library and python3 bindings on my RPI4s. Process lasts about 6 hour (to avoid overheating -in sommer- and swap usage, one can use make -j2 instead of make -j4: it is twice slower, cooler, less RAM eating and one can browse the web while "only" two cores compile)....

Then remain IDE: Sorry, but I am uncompetent (and some of my Pis do not have screens-produce *.jpegs-, if needed... are on a scp link and I manage to have Makefiles...) but:
openengineering ships code::blocks projects I find easy to adapt/convert into Makefiles (just have a verbose output...)

perhaps stripping https://github.com/Qengineering/Face-de ... 32-64-bits of extra DNNs will give you what you need code::blocks is a simple IDE, easy to get -apt-get codeblocks does the job)

I think and https://code.visualstudio.com/docs/setup/raspberry-pi too visual studio can be downloaded for RPi (I have it, but never use it, as , when I have screens, menus are too tiny for my eyes)....

splitthediff
Posts: 9
Joined: Mon Jul 19, 2021 9:46 pm

Re: migrating a VS program to RPI

Thu Jul 22, 2021 11:01 pm

drgeoff wrote:
Wed Jul 21, 2021 2:37 pm
Visual Studio is an IDE (Integrated Development Environment). Therefore the words "VS Program" don't really make sense. Which programming language is the code written in?
Sorry, fatigue : { C++

splitthediff
Posts: 9
Joined: Mon Jul 19, 2021 9:46 pm

Re: migrating a VS program to RPI

Thu Jul 22, 2021 11:13 pm

Its opencv 3.x

You mentioned VS for he PI but my impression was that that was the downside of PI that you couldn't 'use' Windows anything (correct me if I am wrong). I like Linux and yes its better than VS but I am victim f my own programming (no pun intended). I have strong experience with certain aspects of OpenCV and I am hesitant to earn Linux because if its anything like learning c++ it will be a long time before its up and running.

I really like the idea of RPI, but I m afraid of the learning curve for linux. Someone please tell me to get n the water is just fine : )

Heater
Posts: 18527
Joined: Tue Jul 17, 2012 3:02 pm

Re: migrating a VS program to RPI

Fri Jul 23, 2021 1:38 am

splitthediff wrote:
Thu Jul 22, 2021 11:13 pm
Its opencv 3.x

You mentioned VS for he PI but my impression was that that was the downside of PI that you couldn't 'use' Windows anything (correct me if I am wrong).
There is "Visual Studio" and "Visual Studio Code" (commonly known as "VS Code" or just "Code"). They are very different things.

Visual Studio is the IDE you are probably familiar with on Windows.

VS Code is an editor, with lots of IDE features provided by extensions. VS Code is an Open Source product of Microsoft and runs on Linux and Mac as well as Windows. VS Code runs on the Raspberry Pi just fine. It's installable from the Pi OS packaging system, apt.

Some would say that not being able to run Windows software on the Pi is a positive point not a down side. However VS Code is my favourite editor, for C++, Javascript, Rust. It supports many other languages and file formats as well. Being Opens Source, cross platform and generally very good at what it does I think VS Code is Microsoft's best ever product.
splitthediff wrote:
Thu Jul 22, 2021 11:13 pm
I like Linux and yes its better than VS but I am victim f my own programming (no pun intended). I have strong experience with certain aspects of OpenCV and I am hesitant to earn Linux because if its anything like learning c++ it will be a long time before its up and running.

I really like the idea of RPI, but I m afraid of the learning curve for linux. Someone please tell me to get n the water is just fine : )
Luckily learning about Linux is nothing as confusing as learning C++. You certainly don't need to learn a lot about Linux to write a C++ program using OpenCV. Could be as simple as this: https://docs.opencv.org/4.5.2/db/df5/tu ... cmake.html

A google search for something like "C++ OpenCV Linux tutorial" will find you a lot of guidance, tutorials and documentation. There are good presentations on YouTube about this.

So yes, do jump in, the Linux water is just fine.
Memory in C++ is a leaky abstraction .

dbrion06
Posts: 644
Joined: Tue May 28, 2019 11:57 am

Re: migrating a VS program to RPI

Fri Jul 23, 2021 1:18 pm

Re: migrating a VS program to RPI
Unread post by splitthediff » Fri Jul 23, 2021 1:13 am

Its opencv 3.x
well, that is a good news: you do not need to compile from source the library, which is scary -but works: takes a night...-.

To have Heater example working, you only know to type:

Code: Select all

sudo apt install opencv-dev cmake
(
*sudo is a command meaning 'please, givemme root/admin privileges'

* apt invokes a sophisticated package manager which downloads and installs the package you want (opencv header files and cmake make generators) and each and everything one needs to have the package working -if make is lacking, it gets downloaded and installed, too- ...)

Remaining issues are:

* USB camera support

* learning some basic Unix/GNUlinux commands (how to find a file -find- , where one is -pwd-, which executable is executed -which-, how much a process eats memory -top-).

* having some communication between the original windows machine and the RPi (maybe a USB stick: it is ridiculous, but works and backups ones work; scp is easy to use between two GNUlinux machines, but for Windows, I do not know (use cygwin: there are more comfortable ways to begin, but I really do not know)

splitthediff
Posts: 9
Joined: Mon Jul 19, 2021 9:46 pm

Re: migrating a VS program to RPI

Fri Jul 23, 2021 4:21 pm

I understand that installing OpenCV 3.x on the RPI takes 6+ hours or more. If RPI is an SBC than shouldn't it compile in a time comparable to a PC? What is the bottle neck if any?

Also I use cvui as a GUI interface with OpenCV. It is a wrapper, very simple to use:

https://dovyski.github.io/cvui/components/button/

Its not as good-looking as QT and its practically flat as far as learning curve goes. Does anyone know whether this would be compatible with the RPI environment. If not, what GUI code would you recommend for Open CV/c++ on the RPI?

splitthediff
Posts: 9
Joined: Mon Jul 19, 2021 9:46 pm

Re: migrating a VS program to RPI

Fri Jul 23, 2021 8:11 pm

Sorry last question added to the above. Is there any way to estimate whether an RPI would execute a real time tracking app well/with good frame rate. I know you guys aren't prognosticators but any information there would be appreciated. In general is the RPIs performance enough to run most OpenCV 3.x apps? I am not talking reading plates from a satellite image, just some object features sometimes, simple blob tracking sometimes maybe some pose estimations if I'm lucky..

Thanks in advance

Heater
Posts: 18527
Joined: Tue Jul 17, 2012 3:02 pm

Re: migrating a VS program to RPI

Sat Jul 24, 2021 12:04 am

splitthediff wrote:
Fri Jul 23, 2021 4:21 pm
If RPI is an SBC than shouldn't it compile in a time comparable to a PC?
I don't follow the logic there. "SBC" is an acronym that has been in common use for many decades now and stands for "Single Board Computer". The is to say, an entire working computer on a single printed circuit board, as opposed to what was the more typical computer assembled from multiple boards, processor board(s), memory board(s), IO board(s), graphics board(s) etc.

"SBC" says nothing about performance. Indeed typically SBC's have lagged behind typical computers as they have been designed for small size, low power, embedded systems etc. An SBC might well have a lowly 8 bit processor on it, like the Arduino.
splitthediff wrote:
Fri Jul 23, 2021 4:21 pm
What is the bottle neck if any?
In a hand waving way:
Much slower processor.
Much slower storage.

What one would expect of such a low cost machine.
Memory in C++ is a leaky abstraction .

User avatar
davidcoton
Posts: 6522
Joined: Mon Sep 01, 2014 2:37 pm
Location: Cambridge, UK

Re: migrating a VS program to RPI

Sat Jul 24, 2021 9:22 am

Heater wrote:
Sat Jul 24, 2021 12:04 am
splitthediff wrote:
Fri Jul 23, 2021 4:21 pm
What is the bottle neck if any?
In a hand waving way:
Much slower processor.
Much slower storage.

What one would expect of such a low cost machine.
Look at it this way. How much do you have to pay an F1 driver to go fast? And what does a safe F1 track cost?
You have to throw money at your system if you want it to go fast. Bits on a Pi budget just don't do warp speed. :ugeek: 8-) :lol:
Location: 345th cell on the right of the 210th row of L2 cache

dbrion06
Posts: 644
Joined: Tue May 28, 2019 11:57 am

Re: migrating a VS program to RPI

Sat Jul 24, 2021 1:59 pm

I understand that installing OpenCV 3.x on the RPI takes 6+ hours or more.
Well, NO (sorry for my poor English).
for 3.x :it is already packaged (compiled with each and every optimization by skilled Debianero) and ready to install using the following incantation

Code: Select all

sudo apt update # downloads a recent list of packages
sudo apt upgrade # downloads and replaces packages, if needed
sudo apt install libopencv-dev cmake 
Time depends only on the download speed (usually very fast: maybe 10 seconds to copy and paste my incantation, and 1 minute for the package manager -one of the bests- to pickup and seed every package you need). That is binary installation (plus header files, necessary to develop)

Then, one is ready to compile example Heater give you with an incantation like that

Code: Select all

mkdir build && cd build && cmake .. && make
Same incantation apllyes with each and every C++ example you can find in https://github.com/spmallick/learnopencv (the CMakelist.txt is slightly more complicated than the fiveliner Heater linked you to).
Examples are fast to compile.

for 4.x : here, it needs between 1.5 and 6 hours depending on the way you build(make has an option -Jx to set the number of processors you want to use when compiling from source. This is scary, but the link I gave you worked for me many times.
Unless you want the latest and greates (or play with DNNs) you do not need it.

Once library is installed, compiling is rather fast on a RPi 4B (mine has 8G RAM, for DNNs; for usual openCV 2 or 4G are more than enough...IMO)
I installes example Heater linked you to and adapted it slightly (adding a directory/folder named build for clean builds)

Code: Select all


pi@raspberrypi:~/testcvcmake/build $ touch ../DisplayImage.cpp
pi@raspberrypi:~/testcvcmake/build $ time make
Consolidate compiler generated dependencies of target DisplayImage
[ 50%] Building CXX object CMakeFiles/DisplayImage.dir/DisplayImage.cpp.o
[100%] Linking CXX executable DisplayImage
[100%] Built target DisplayImage

real    0m6,998s
user    0m6,265s
sys     0m0,720s

splitthediff
Posts: 9
Joined: Mon Jul 19, 2021 9:46 pm

Re: migrating a VS program to RPI

Mon Jul 26, 2021 10:15 pm

Another circumspect question arising from all of these posts: if the RPI can't keep up with object tracking etc. how did OpenCV become part of its in-package sales and plethora of camera sensors that are meant to be compatible with it? Is the Jetson more or less appropriate than RPI regarding speed, cost and supporting OpenCV? Like one of these posters was saying, you get what you pay for. The Jetson is 3x cost but is it the next RPI regarding real-time machine vision? IDK

dbrion06
Posts: 644
Joined: Tue May 28, 2019 11:57 am

Re: migrating a VS program to RPI

Tue Jul 27, 2021 1:18 pm

if the RPI can't keep up with object tracking etc
Without more details (object tracking in 20,000*20,000 pixels image can be done .... slowly with very expensive computers), this is pure speculation (RPi3 could object track on youtube videos with ca 100 000 pixels in half real time IIRC; RPi4 is faster)...

It would be more efficient to post a link to a youtube video and ask whether it can be tracked in real time...


w/r cvui: I had no difficulty at all in having each and every example working with ocv 4.5.2 compiled with qt (I do not want to reinstall ocv3.x for vague questions). As ocv3 mainly differs from ocv4 with DNNs support (ocv3 does not support them, IIRC) and as cvui was last updated in 2018, it is most likely it will work on RPi4... (at last examples, which can be reproduced...).

w/r other questions : please, post them in distinct threads, and take time to ask precise questions (or links to given videos).
Last edited by dbrion06 on Tue Jul 27, 2021 2:29 pm, edited 1 time in total.

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 29346
Joined: Sat Jul 30, 2011 7:41 pm

Re: migrating a VS program to RPI

Tue Jul 27, 2021 1:38 pm

splitthediff wrote:
Mon Jul 26, 2021 10:15 pm
Another circumspect question arising from all of these posts: if the RPI can't keep up with object tracking etc. how did OpenCV become part of its in-package sales and plethora of camera sensors that are meant to be compatible with it? Is the Jetson more or less appropriate than RPI regarding speed, cost and supporting OpenCV? Like one of these posters was saying, you get what you pay for. The Jetson is 3x cost but is it the next RPI regarding real-time machine vision? IDK
What "in package sales" do you mean? AFAIK, we don't advertise the Pi as an OpenCV engine, although it's certainly is powerful enough to do certain things (he says, watching the guy over the partition doing face tracking with OpenCV using libcamera on Pi4).
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Working in the Applications Team.

RpiName
Posts: 740
Joined: Sat Jul 06, 2013 3:14 am

Re: migrating a VS program to RPI

Tue Jul 27, 2021 2:35 pm

splitthediff wrote:
Mon Jul 26, 2021 10:15 pm
Another circumspect question arising from all of these posts: if the RPI can't keep up with object tracking etc. how did OpenCV become part of its in-package sales and plethora of camera sensors that are meant to be compatible with it? Is the Jetson more or less appropriate than RPI regarding speed, cost and supporting OpenCV? Like one of these posters was saying, you get what you pay for. The Jetson is 3x cost but is it the next RPI regarding real-time machine vision? IDK
"Decent", real-time generic object tracking is possible on the RPi 4 with the help of some hardware accelerator that can run deep neural nets in hardware. I have not seen any other way to do this yet.

dbrion06
Posts: 644
Joined: Tue May 28, 2019 11:57 am

Re: migrating a VS program to RPI

Tue Jul 27, 2021 2:46 pm

Does OP need the latest and greatest DNN to:
track some shapes
(1rst part of this thread)

(correlation methods can do some job and have worked for drones since 5 years).

splitthediff
Posts: 9
Joined: Mon Jul 19, 2021 9:46 pm

Re: migrating a VS program to RPI

Thu Jul 29, 2021 9:16 pm

Excellent posts and advice. I think I'll try both the RPI 4 and Jetson Nano with the same camera (OV9281) and see what the OPENCV blob tracking performance is between them. The Jetson GPU is stronger than the RPI4. So what right? Would this help blob detection batter than the RPI4.? If anybody already has this data or a hypothesis for this specific scenario I would like too know.

My mistake is trying to settle this in a single post! Latency could be a factor as well. Very exciting technology.

Heater
Posts: 18527
Joined: Tue Jul 17, 2012 3:02 pm

Re: migrating a VS program to RPI

Fri Jul 30, 2021 12:49 am

splitthediff wrote:
Thu Jul 29, 2021 9:16 pm
The Jetson GPU is stronger than the RPI4. So what right?
The whole sales pitch of the Jetson Nano is that it's GPU can be used to accelerate Neural Network processing a great deal. That is to say "Artificial Intelligence", image classifiers etc. Both for running trained models and training models. In short it accelerates the Tensor Flow and other NN libraries.

Have a look at the Jetson Nano getting started tutorials "Hello AI World" to get an idea of what it can do with OpenCV and Tensor Flow etc: https://developer.nvidia.com/embedded/t ... o_ai_world

To see what can be done re: object detection, object tracking, face recognition etc have a look at Paul McWorters video series on the Jetson Nano: "AI on the Jetson Nano" : https://www.youtube.com/watch?v=5INy0Fv ... RFFPkOuP4_
(It's a long series, the interesting stuff is in later episodes).

As far as I know the Pi GPI has never been used for NN acceleration, I have no idea if that is even possible.
splitthediff wrote:
Thu Jul 29, 2021 9:16 pm
Would this help blob detection batter than the RPI4.? If anybody already has this data or a hypothesis for this specific scenario I would like too know.
It's not clear to me that the GPU on the Jetson Nano is used by Open CV. I guess it is.
splitthediff wrote:
Thu Jul 29, 2021 9:16 pm
My mistake is trying to settle this in a single post! Latency could be a factor as well. Very exciting technology.
It is a huge topic.

I have some Jetson Nano here that we are going to use in a project. Apart from working through those "Hello AI World" examples I have not done much with its GPU/NN accelerator. What I have found is that the ARM cores of the Nano are 10% or so slower than a Pi 4 at running the compiled C/C++/Rust code I throw at it.
Memory in C++ is a leaky abstraction .

bjtheone
Posts: 1488
Joined: Mon May 20, 2019 11:28 pm
Location: The Frozen North (AKA Canada)

Re: migrating a VS program to RPI

Fri Jul 30, 2021 11:37 am

Both Jetsons and Pis are very popular in FRC Robotics for doing vision "stuff" (game piece tracking, target acquisition, robot positioning). Most teams are using OpenCV pipelines. The consensus seems to be the Jetsons are faster but more expensive, however it really depends on your exact vision task.

If you search for "jetson frc" or "jetson frc opencv" you can find lots of code examples and discussion.

splitthediff
Posts: 9
Joined: Mon Jul 19, 2021 9:46 pm

Re: migrating a VS program to RPI

Fri Jul 30, 2021 6:33 pm

I guess experimentation will bear it out. I found this spec about the RPI4. What exactly does it mean?
...the Raspberry PI offers 4k 60FPS...
Is this a claim to some of the PIs bandwidth for video? What is the 4k specification?

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 29346
Joined: Sat Jul 30, 2011 7:41 pm

Re: migrating a VS program to RPI

Sat Jul 31, 2021 6:45 am

The hdmi can do 4kp60.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Working in the Applications Team.

dbrion06
Posts: 644
Joined: Tue May 28, 2019 11:57 am

Re: migrating a VS program to RPI

Sat Jul 31, 2021 2:07 pm

The Jetson GPU is stronger than the RPI4. So what right?
Yes, but does opencv (edited : the default option Ubuntu offers for opencv) support GPU on jetson (I bet ... you have to recompile https://qengineering.eu/install-opencv- ... -nano.html:
as jetson CPU is slightly slower than RPi 4 one, recompiling will be slightly longer and scarier: gcc /g++ does not need nor use GPU!).
There are other issues with NVIDIA Jetson:

it needs some HDMI screens. I by passed this issue by ssh (with X forwarding) over "inetrnet over USB" to my RPi (which has a screen: I have only 2 eyes and a brain and do not need 2 screens)
it is slightly more expensive.

IMO Best you begin with RPi only (after, porting from one GNU linux to another is trivial, if needed) and play with GNUlinux to get trained...
If is is enough(I have no crystal balls) for you, you will have spared ca 150E$.

splitthediff
Posts: 9
Joined: Mon Jul 19, 2021 9:46 pm

Re: migrating a VS program to RPI

Sat Jul 31, 2021 10:49 pm

Hi Jamesh, I found this on wiki:
4K Resolution and 4K Video Decoding
The Raspberry Pi 4 can decode up to 4K video at 60FPS, encoded in H. 265. Unfortunately, H. 264 decoding is only supported for up to 1080p at 60FPS.Jun 24, 2019
VPU Decoding Capabilities: 4K@60FPS video ...
CPU: 4 × Cortex-A72, 1.5 GHz
USB Ports: 2 x USB 3.0, 2 x USB 2.0
Display Ports: 2 x micro-HDMI, DSI, Composite ...


What flout-out is the best documented frame rate/camera sensor for the RPI4? My app currently uses the PC, and I want to migrate this. For me, Jetson nano has been ruled out for a variety of factors. My app just tracks a few IR beacons to maximize FPS and diminished latency. And I'm pretty sure opencv on the RPI4 doesn't utilize compressed video but "raw".

Best Regards,

feeling like a hack : )

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 29346
Joined: Sat Jul 30, 2011 7:41 pm

Re: migrating a VS program to RPI

Sun Aug 01, 2021 10:16 am

That just the Pi4 spec for HDMI and decoding, not sure what you are getting at with that. Is there a specific question on that stuff?

The V2 camera has a decent frame rate, but on the whole, we are limited by the two lane CSi2 bandwidth rather than the cameras.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Working in the Applications Team.

Return to “General discussion”