EmilWandel
Posts: 2
Joined: Fri Aug 01, 2014 2:05 pm

Cam0 & Cam1 -> HDMI (simultaneously)

Fri Aug 01, 2014 2:18 pm

Hi Forum,

I've been playing around a little with the pi and enjoyed this very much. Now I wonder if the compute module would be suited for the following application.

Is it possible to use the live "--preview"-window of two cameras with the compute module on a single screen at the same time, i.e.

Cam0 ==real time==> left side of screen, Cam1==real time=>right side of screen?

This would be superawesome(tm) for me as could use one camera for beam profiling and the other one to monitor my target through a microscope in a laser experiment. This would make my work life much more comfy, productive and easier.

If it could be done at which frame rate would it be possible? Has anyone tried something similar?

Cheers,

Emil

ShiftPlusOne
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5762
Joined: Fri Jul 29, 2011 5:36 pm
Location: The unfashionable end of the western spiral arm of the Galaxy

Re: Cam0 & Cam1 -> HDMI (simultaneously)

Fri Aug 01, 2014 4:23 pm

Yup. But for whatever reason, you can't have them on the left and right. It needs to be top and bottom.

User avatar
redhawk
Posts: 3465
Joined: Sun Mar 04, 2012 2:13 pm
Location: ::1

Re: Cam0 & Cam1 -> HDMI (simultaneously)

Fri Aug 01, 2014 4:35 pm

Is it possible to rotate the HDMI output and rotate the cam sources by 90 degrees??

Richard S.

gsh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 1421
Joined: Sat Sep 10, 2011 11:43 am

Re: Cam0 & Cam1 -> HDMI (simultaneously)

Fri Aug 01, 2014 4:47 pm

The reason for not playing them left -right will be because of the bandwidth in the HVS...

What resolution are you previewing at and are you writing to the SD card at the same time... What frame rate are you getting from simultaneous output?

I'll do some calculations...

Gordon
--
Gordon Hollingworth PhD
Raspberry Pi - Director of Software Engineering

ShiftPlusOne
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5762
Joined: Fri Jul 29, 2011 5:36 pm
Location: The unfashionable end of the western spiral arm of the Galaxy

Re: Cam0 & Cam1 -> HDMI (simultaneously)

Fri Aug 01, 2014 5:06 pm

gsh wrote:The reason for not playing them left -right will be because of the bandwidth in the HVS...

What resolution are you previewing at and are you writing to the SD card at the same time... What frame rate are you getting from simultaneous output?

I'll do some calculations...

Gordon
Calculations? On a Friday? I think not.

gsh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 1421
Joined: Sat Sep 10, 2011 11:43 am

Re: Cam0 & Cam1 -> HDMI (simultaneously)

Fri Aug 01, 2014 7:20 pm

Yes even on a Friday I do calculations...

I make it out to be around 1.6GByte/s Which out of a perfect maximum of 2.1 GByte/s (i.e. doing reads and writes in the most efficient manor possible) is quite a lot! The main failure in my calculations here is the SDRAM latency does not allow you to do 2x1080P30 encode streams simultaneously so you'll be running quite low framerate and should have an even bigger margin.

So you are going to get a little bit warm (that SDRAM uses a lot of power at this throughput), So I'm fairly sure we should be able to achieve this without overclocking, but it should be much easier if you overclock the GPU a bit...

So I would think the problem here is one of HVS priority / panic priority and how the HVS, ARM, VPU and H264 share the memory bandwidth nicely...

Gordon
--
Gordon Hollingworth PhD
Raspberry Pi - Director of Software Engineering

ShiftPlusOne
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5762
Joined: Fri Jul 29, 2011 5:36 pm
Location: The unfashionable end of the western spiral arm of the Galaxy

Re: Cam0 & Cam1 -> HDMI (simultaneously)

Fri Aug 01, 2014 8:00 pm

gsh wrote:Yes even on a Friday I do calculations...
Heh, actually I misread that as "I'd do some calculations", so I thought you were suggesting that I do that.
After frying my brain chasing race conditions, I'm having none of that. :lol:
gsh wrote:but it should be much easier if you overclock the GPU a bit...
For what it's worth, I've learned not to overclock when debugging... the hard way. Running with the 'turbo' preset would cause all sorts of fun things to happen when the cameras were running.

I'll see if lowering the frame rate allows the preview windows to be side by side. And you're spot on about HVS too - it would complain loudly when trying to display the two images side by side.

How come aligning them vertically works though? Is it because data is processed line by line, so it's effectively only writing one image frame to the display at any one time?

gsh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 1421
Joined: Sat Sep 10, 2011 11:43 am

Re: Cam0 & Cam1 -> HDMI (simultaneously)

Fri Aug 01, 2014 8:19 pm

The Hardware Video Scaler actually composes each line for each display one line at a time...

So it goes through the display list calculating what needs to be done for line N, it then creates that line and pushes it into the line buffer ready for the line to be pushed out to the display (HDMI DPI or DSI)

To create the line it may have to scale input images and write them into the output buffer... If you are outputting the images side by side then it needs to read the whole of the left image line ( or lines if you are vertically decimating) and push it through the scaler to output a few pixels, it then reads the whole of the right image line (or lines) pushing through the scaler and the pixels end up in the output line buffer.

So the most probably place where it is running out of clocks is in the scaler, this is quite limited in the number of pixels you can push through it, but basically you're limited to 1 pixel per clock at either the input or output side...

You see it's all really rather complex and there is a big pipeline of different blocks doing different things at different clocks with different limitations! Guess what I used to spend a rather inordinate amount of time doing at Broadcom!


Gordon
--
Gordon Hollingworth PhD
Raspberry Pi - Director of Software Engineering

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 23065
Joined: Sat Jul 30, 2011 7:41 pm

Re: Cam0 & Cam1 -> HDMI (simultaneously)

Fri Aug 01, 2014 8:21 pm

ShiftPlusOne wrote:
gsh wrote:Yes even on a Friday I do calculations...
Heh, actually I misread that as "I'd do some calculations", so I thought you were suggesting that I do that.
After frying my brain chasing race conditions, I'm having none of that. :lol:
gsh wrote:but it should be much easier if you overclock the GPU a bit...
For what it's worth, I've learned not to overclock when debugging... the hard way. Running with the 'turbo' preset would cause all sorts of fun things to happen when the cameras were running.

I'll see if lowering the frame rate allows the preview windows to be side by side. And you're spot on about HVS too - it would complain loudly when trying to display the two images side by side.

How come aligning them vertically works though? Is it because data is processed line by line, so it's effectively only writing one image frame to the display at any one time?
Hi Shift, on the latest ISYS source tree at Brcm (the one not being used for Raspi!) we found a some race conditions after I ported the ISP code to Linux and ran it through valgrind. So there may be fixes for your problem already in the ISYS tree. I'm off for the next two weeks, but try and get hold of 6x9 who may be able to help out and pin them down, along with possible fixes.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
"My grief counseller just died, luckily, he was so good, I didn't care."

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 23065
Joined: Sat Jul 30, 2011 7:41 pm

Re: Cam0 & Cam1 -> HDMI (simultaneously)

Fri Aug 01, 2014 8:22 pm

gsh wrote: You see it's all really rather complex and there is a big pipeline of different blocks doing different things at different clocks with different limitations! Guess what I used to spend a rather inordinate amount of time doing at Broadcom!
Gordon
Snoring? (well, that's what I heard)
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
"My grief counseller just died, luckily, he was so good, I didn't care."

ShiftPlusOne
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5762
Joined: Fri Jul 29, 2011 5:36 pm
Location: The unfashionable end of the western spiral arm of the Galaxy

Re: Cam0 & Cam1 -> HDMI (simultaneously)

Fri Aug 01, 2014 8:47 pm

gsh wrote: You see it's all really rather complex and there is a big pipeline of different blocks doing different things at different clocks with different limitations! Guess what I used to spend a rather inordinate amount of time doing at Broadcom!
I imagine you were designing the hardware while muttering "you'll see... you'll all see!"

I see no other explanation for it all.
jamesh wrote: Hi Shift, on the latest ISYS source tree at Brcm (the one not being used for Raspi!) we found a some race conditions after I ported the ISP code to Linux and ran it through valgrind. So there may be fixes for your problem already in the ISYS tree. I'm off for the next two weeks, but try and get hold of 6x9 who may be able to help out and pin them down, along with possible fixes.
Ah, interesting. But in the end I found a global pointer to a configuration structure which was cleared and re-written any time a camera was opened. So opening two cameras at once would mean that the first camera would try to use data that the second camera has just cleared. One of your Broadcom guys (who I suspect is 6by9) only took a few minutes to fix it, send a patch for testing and get the fix into the firmware released today, which was all very impressive.

gsh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 1421
Joined: Sat Sep 10, 2011 11:43 am

Re: Cam0 & Cam1 -> HDMI (simultaneously)

Fri Aug 01, 2014 9:06 pm

Yes it was 6x9...

Gordon
--
Gordon Hollingworth PhD
Raspberry Pi - Director of Software Engineering

EmilWandel
Posts: 2
Joined: Fri Aug 01, 2014 2:05 pm

Re: Cam0 & Cam1 -> HDMI (simultaneously)

Fri Aug 01, 2014 9:30 pm

Hi Gordon, Shift and jamesh,

Thank you for the impressive discussion. I must admit that most parts of it sound slightly Star Trek-esque for me.

As of my background I was very impressed how easily I could get a Pi B and the camera to work in python (~20 minutes). While I did some coding in the past I have never programmed anything close to hardware, so I am a novice in this regard.

Maybe you can shed a little light for the novice on the calculations you do, what the players are and how the data flows;

if I were to use two times cameras in the 640x480 60.1-90fps mode (http://www.raspberrypi.org/new-camera-mode-released/) would they both deliver 90fps or would they have to "share" their bandwidth?

Has someone done benchmarking(?) of different modes for two cameras already?

Regarding up and down can you give a cartoon explanation why left and right does not work? What does HVS mean? I'm not convinced I found the right thing in Wikipedia...

However, I would not mind turning the display and/or the cameras -hardware style- if I am concerned with left/right vs. top/bottom.

Again, thank you for the fast response and the vivid discussion!

Cheers,

Emil

EDIT: Gordon, I apparently overlooked your post from Fri Aug 01, 2014 9:19 pm, my apologies. Thanks for the great explanation for the up/down vs. left and right issue.

ShiftPlusOne
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5762
Joined: Fri Jul 29, 2011 5:36 pm
Location: The unfashionable end of the western spiral arm of the Galaxy

Re: Cam0 & Cam1 -> HDMI (simultaneously)

Fri Aug 01, 2014 10:00 pm

Not sure that I can shed any light on it, I'm afraid.

I'll spend a few minutes on Monday seeing how resolution and frame rate affect things though

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 23065
Joined: Sat Jul 30, 2011 7:41 pm

Re: Cam0 & Cam1 -> HDMI (simultaneously)

Fri Aug 01, 2014 10:14 pm

gsh wrote:Yes it was 6x9...

Gordon
He is also available for Bar Mitzvahs.

Actually, it would be worth trying to grab some of the race conditions fixes made on ISYS. They are rare but possible. Grab 6x9 while he is a good mood!
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
"My grief counseller just died, luckily, he was so good, I didn't care."

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 7007
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Cam0 & Cam1 -> HDMI (simultaneously)

Wed Aug 13, 2014 8:29 am

jamesh wrote:He is also available for Bar Mitzvahs.
Mazel tov!
jamesh wrote:Actually, it would be worth trying to grab some of the race conditions fixes made on ISYS. They are rare but possible. Grab 6x9 while he is a good mood!
Never happens!

I've only just picked up on this thread as I don't normally follow the compute module forum.
I know it wasn't the original question, but it may help that I have full stereoscopic working on my desk in side by side mode. I wasn't pushing the resolution limits though as I was in half/half mode. I'll see what happens in the HVS if I push it to full mode.
Top/bottom should work too (I'm just adding all the config options).
I suspect we'll have some significant limitations on modes for video and JPEG encode if we're trying to avoid half/half. JPEG I believe wants side by side which shouldn't be an issue. The H264 block I'm lead to believe has a 1920 wide line buffer, so we may be able to try top/bottom, but there's a max height (1800ish IIRC) before it changes mode too. If doing side by side, then the width of each half will have to be a multiple of 128 (YUVUV column width)
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

dom
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5282
Joined: Wed Aug 17, 2011 7:41 pm
Location: Cambridge

Re: Cam0 & Cam1 -> HDMI (simultaneously)

Wed Aug 13, 2014 2:39 pm

"dispmanx_offline=1" in config.txt may avoid the HVS bandwidth issue of two camera viewfinders, side-by-side.
This makes HVS compose to an offscreen buffer then display that which removes the real-time requirement. (With some memory and framerate cost).

However I supect the real answer is to use a smaller viewfinder.
The ISP typically produces two images, one large and one smaller. The large one is designed for capturing to jpeg/video. The smaller one is designed to display as a viewfinder.
Possibly in the dual camera mode, you need to request a smaller viewfinder image. I suspect 6x9 would know the magic.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 7007
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Cam0 & Cam1 -> HDMI (simultaneously)

Wed Aug 13, 2014 3:01 pm

dom wrote:However I supect the real answer is to use a smaller viewfinder.
The ISP typically produces two images, one large and one smaller. The large one is designed for capturing to jpeg/video. The smaller one is designed to display as a viewfinder.
Possibly in the dual camera mode, you need to request a smaller viewfinder image. I suspect 6x9 would know the magic.
What resolution are you wanting to capture for processing, and how fast?

Dom is technically correct that there are two images produced. Just a minor issue during video encode as the codec needs that lower res image in a particular format, and the display can't accept that format :( For video encode we therefore send the same encode res to both display and codec.
For stills mode, we actually do multiple passes through the hardware to get all the images required, and allow extra processing.
Based on use case, I may be able to advise additional tweaks to allow custom resizes so that encode and display can get different sizes. It hasn't been tried on this branch before, and I know there are limitations as to what is supported, but it may help out.

I have just tried two 720P frames side by side on the display, and that works fine. Two by 1080P side by side does indeed cause badness. Actually 2 by 720P seems to be just on the edge, as 2 by 1280x960 fails, and so does 2 by 1296x720. Are the arbiters set sensibly? I know we've had fun and games with those in the past. This has all been done with two shells doing:
- raspivid -w <width> -h <height> -t 50000 -p 0,0,960,540
- raspivid -w <width> -h <height> -t 50000 -p 960,0,960,540
I can add a -o <filename> to the end to encode both streams, so it does seem to be HVS specific, rather than overall memory bandwidth.

720P gives the advantage of a slightly greater field of view - 1080P actually has to do a crop on the sensor in order to achieve 30fps, and that reduces the FOV. Without knowing a little more of the use case I don't know if that is an issue or not.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

ShiftPlusOne
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5762
Joined: Fri Jul 29, 2011 5:36 pm
Location: The unfashionable end of the western spiral arm of the Galaxy

Re: Cam0 & Cam1 -> HDMI (simultaneously)

Wed Aug 13, 2014 3:06 pm

Interesting. My CM is not set up to try it out any time soon, but it's good to know there are ways to get around that limitation.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 7007
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Cam0 & Cam1 -> HDMI (simultaneously)

Thu Aug 14, 2014 10:32 am

6by9 wrote:I have just tried two 720P frames side by side on the display, and that works fine. Two by 1080P side by side does indeed cause badness. Actually 2 by 720P seems to be just on the edge, as 2 by 1280x960 fails, and so does 2 by 1296x720. Are the arbiters set sensibly? I know we've had fun and games with those in the past. This has all been done with two shells doing:
- raspivid -w <width> -h <height> -t 50000 -p 0,0,960,540
- raspivid -w <width> -h <height> -t 50000 -p 960,0,960,540
I can add a -o <filename> to the end to encode both streams, so it does seem to be HVS specific, rather than overall memory bandwidth.
Me being silly:
- raspivid -w 1280 -h 960 -t 50000 -p 0,0,960,720
- raspivid -w 1280 -h 960 -t 50000 -p 960,0,960,720
works. I hadn't changed the display region before, so it was trying to display more src lines in the same output lines which is obviously more work and exceeds the limit.
It does seem that 2560 src pixels is about the limit per output line without overclocking. I'm sure Gordon would go and dig out the numbers if he really felt so inclined, but I don't think it's necessary for now.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

airvb99
Posts: 7
Joined: Thu Aug 21, 2014 3:06 pm

Re: Cam0 & Cam1 -> HDMI (simultaneously)

Thu Aug 21, 2014 3:54 pm

@gsh:
Hi Gordon,

I have a similar request (maybe do I have to open a new forum thread ?):
I need to stream a h264 3D live video (side-by-side / top-bottom or line interleaved with some simple OSD added) to the network link. RPi-compute would be fine as it has h264 hw-encoder and 2 video CSI input. I understand that 2 x 1080p will overflow the bandwitdh but do you think it will be ok by using 2 x 720p capture input / 1 x 720p output ?
Can I use some C/C++ API to capture the frames, add some OSD and combine the 2 frames before feeding h264 encoder ?
Is there a way to use this API (broadcom closed-source video firmware) on a Xenomai compiled kernel to enable the RPi to process some other real-time stuff too ?

Regards,

Herve

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 7007
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Cam0 & Cam1 -> HDMI (simultaneously)

Fri Aug 22, 2014 7:31 am

airvb99 wrote:I need to stream a h264 3D live video (side-by-side / top-bottom or line interleaved with some simple OSD added) to the network link. RPi-compute would be fine as it has h264 hw-encoder and 2 video CSI input. I understand that 2 x 1080p will overflow the bandwitdh but do you think it will be ok by using 2 x 720p capture input / 1 x 720p output ?
Can I use some C/C++ API to capture the frames, add some OSD and combine the 2 frames before feeding h264 encoder ?
Is there a way to use this API (broadcom closed-source video firmware) on a Xenomai compiled kernel to enable the RPi to process some other real-time stuff too ?
You've already commented on http://www.raspberrypi.org/forums/viewt ... 43&t=85012, so you are aware of the stereoscopic support now available.
I would have thought you're best off making use of that and requesting a single stereoscopic 1080P side-by-side (or top/bottom) decimated (aka half-half) stream. I have been told that the H264 encoder has a hardware limit at about 2000 pixels wide, and other strange things happen at more than 1344 pixels high, so that forces the half-half requirement (1280*2 = 2560 > 2000, and 720*2 = 1440 > 1344).

There is the annotate parameter (http://www.raspberrypi.org/forums/viewt ... 25#p592767) that allows you to add a simple text string to the top middle of the captured frames - is that sufficient OSD for you? I would need to check whether it writes the text to both channels of the image or just one - it wasn't a case I had been thinking about! There are still plans for an overlay component to do OSD type things, but time is a little on the short side so I don't know if it will actually happen. Setting it up for stereoscopic could be entertaining too.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

airvb99
Posts: 7
Joined: Thu Aug 21, 2014 3:06 pm

Re: Cam0 & Cam1 -> HDMI (simultaneously)

Fri Aug 22, 2014 9:37 am

Hi Gordon,

Thank's for your reply, I found 6by9 post last evening, that was a good news. Side by side output @1080p is sufficent for me so the 2000 pixel GPU limit is OK. My project is to use RPi in a quadcopter with wifi link (video, osd and remote control). It seems that adding OSD to the video before streaming is a bad idea. It would be better to stream the OSD data separatly (on another tcp/udp socket port) and display it at play time. That's how I will do.
Today's FPV Quadcopters still use analogic video link and therefore are still limited with PAL/NTSC poor resolutions. Standards video goggles can't display more than 640x480 and the best of them are 800x600 so if RPi can do stereoscopic 1080p ([email protected]) or simple [email protected] it will cover the maximum capacity of theses goggles.
The other question was the usablility of mmal (broadcom firmware) in case of re-compiling the RPi kernel to add RealTime capability. Is the raspbian kernel can be re-compiled (with differents kernel options) without disabling the firmware ? Or is the firmware is doing some control on the kernel version/config to desactivate himself ?

Regards,

Herve

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 23065
Joined: Sat Jul 30, 2011 7:41 pm

Re: Cam0 & Cam1 -> HDMI (simultaneously)

Fri Aug 22, 2014 10:14 am

Not a technical comment, but a stereo camera on a quadcopter played back over Oculus Rift headset would be completely awesome....
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
"My grief counseller just died, luckily, he was so good, I didn't care."

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 7007
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Cam0 & Cam1 -> HDMI (simultaneously)

Fri Aug 22, 2014 10:24 am

airvb99 wrote:Hi Gordon,
Actually it was me (6by9) replying to you, not Gordon! ;)
airvb99 wrote:Thank's for your reply, I found 6by9 post last evening, that was a good news. Side by side output @1080p is sufficent for me so the 2000 pixel GPU limit is OK. My project is to use RPi in a quadcopter with wifi link (video, osd and remote control). It seems that adding OSD to the video before streaming is a bad idea. It would be better to stream the OSD data separatly (on another tcp/udp socket port) and display it at play time. That's how I will do.
Today's FPV Quadcopters still use analogic video link and therefore are still limited with PAL/NTSC poor resolutions. Standards video goggles can't display more than 640x480 and the best of them are 800x600 so if RPi can do stereoscopic 1080p ([email protected]) or simple [email protected] it will cover the maximum capacity of theses goggles.
The other question was the usablility of mmal (broadcom firmware) in case of re-compiling the RPi kernel to add RealTime capability. Is the raspbian kernel can be re-compiled (with differents kernel options) without disabling the firmware ? Or is the firmware is doing some control on the kernel version/config to desactivate himself ?
I still don't know! That's one for one of the kernel experts - Gordon, or Dom. A quick Google for "Xenomai raspberry pi" produces fair number of hits. None directly reference the camera or mmal, but they all seem to be a bundle of extra patches on top of the standard kernel, in which case there should be no issue.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

Return to “Compute Module”