DanR
Posts: 33
Joined: Fri Jan 18, 2013 1:20 pm

ADV7280A-M Image Capture Issues

Tue Aug 07, 2018 11:48 am

Hi All,

I have a working board with an ADV7280A-M and I can grab video fine but single image capture is proving a problem. I can save a raw frame which is UYVY data and when I view it, the very first frame I grab is perfect but each subsequent frame is shifted down. For example the top of the image contains the middle part, the middle has a black line in it and then the actual top part of the image starts. I though it was my application but after running "v4l2-ctl --stream-mmap=3 --stream-count=1 --stream-to=test.raw" and it gives the exact same issue. The final application needs to be able to take a GPIO input and either take a single image or start capturing video until another GPIO input is triggered, this part works fine except for the image roll effect I'm seeing. I've set the video mode to pal-I so it's not the fact that the device is in NTSC mode or anything especially as I can stream video using v4l2rtspserver.

Does anyone have any ideas as to why this is happening?
Also does anyone know of a great way to convert the resulting image data into a 24bit bitmap that Windows can use as well??

Many Thanks, Dan

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 7457
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: ADV7280A-M Image Capture Issues

Tue Aug 07, 2018 12:34 pm

By grab video I assume you mean H264 via MMAL video_encode. That implies that there is no fundamental issue with the drivers or hardware. What application are you using there?

Typically the first frame off these devices is often clipped as they start producing the first frame independently of the incoming frame start sync pulse. Certainly the TC358743 tends to exhibit that problem. However you seem to be describing the reverse situation.
Are you looking at the buffers as they have been returned from V4L2, or having sent them to a file? It sounds like just a file handling issue. What viewer are you using? My recommendation on Linux is Vooya.
(Side question: How are you encoding the video to use with v4l2rtspserver? There are bits missing in order to achieve that).

Do note that the stride of the image will be 736*2 = 1472 bytes, not the standard 720*2 = 1440 that you might otherwise expect. There are alignment requirements in the hardware. Each frame is therefore 847872 bytes in size. Getting that wrong will give a diagonal shift down the image though, not a vertical shift.

Conversion to RGB - use MMAL. I'm trying to sort out a simplified example app for you, but yavta is doing the basics with exporting a DMABUF from V4L2, importing it via vcsm to make a handle that MMAL can use, and then passing the buffer to the component. On the output side you set up a buffer pool, and will get a callback with a pointer to the filled buffer. It'll resize and/or crop too should you so wish.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

DanR
Posts: 33
Joined: Fri Jan 18, 2013 1:20 pm

Re: ADV7280A-M Image Capture Issues

Wed Aug 08, 2018 8:57 am

I was mistaken about the rtspserver, I have two RasPi 3B+'s on my desk on with an official camera on it and the other with the 7280 on it and I was hot, bothered and confused. I can record video with Yavta from the 7280 and I currently have no way of capturing stills. Management also dumped the request that the device also streams the live feed and also tacks on audio to the rtsp stream only for training purposes. Hmm, not a lot of time to do a hell of a lot of work and now even less of an idea of how to do it. Any helpful pointers are greatly appreciated such as take a look at this file or this documentation etc.

Actually I just found some MMAL documentation online, so I'll go through that...

Many Thanks, Dan

DanR
Posts: 33
Joined: Fri Jan 18, 2013 1:20 pm

Re: ADV7280A-M Image Capture Issues

Wed Aug 08, 2018 9:48 am

Am I right in thinking the routes or data paths are as that the ADV7280 data over MIPI goes to V4L2 so I need to set up:

MMAL isp -> MMAL renderer -> MMAL Encoder
Where the Encoder can be 3 encoders, one for MMAL_ENCODING_BMP for still captures, two for MMAL_ENCODING_H264 one of which will record a video based upon the GPIO trigger and the other encoder can supply the RTSP server component.

Does that sound about right? Can 3 encoder components be connected to one renderer component?

Many Thanks, Dan

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 7457
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: ADV7280A-M Image Capture Issues

Wed Aug 08, 2018 10:07 am

MMAL has documentation markup in the headers. For some reason we haven't done the conversion to HTML and stuck it in our documentation, but it has been done at http://www.jvcref.com/files/PI/document ... index.html.

Basic concepts of components having a number of input and output ports. A port can either be connected to another component using a MMAL_CONNECTION_T, or you can enable your own callbacks to handle the buffers on the port. Buffers come from a buffer pool normally.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 7457
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: ADV7280A-M Image Capture Issues

Wed Aug 08, 2018 10:35 am

DanR wrote:
Wed Aug 08, 2018 9:48 am
Am I right in thinking the routes or data paths are as that the ADV7280 data over MIPI goes to V4L2 so I need to set up:

MMAL isp -> MMAL renderer -> MMAL Encoder
Where the Encoder can be 3 encoders, one for MMAL_ENCODING_BMP for still captures, two for MMAL_ENCODING_H264 one of which will record a video based upon the GPIO trigger and the other encoder can supply the RTSP server component.

Does that sound about right? Can 3 encoder components be connected to one renderer component?
The data is coming in via V4L2, so needs some manipulation to get it into MMAL. That is what the code in buffer_export is doing, and then the loop that is calling VIDIOC_DQBUF, finding the matching MMAL buffer header and passing it into MMAL. When MMAL returns the buffer via isp_ip_cb it returns it to V4L2 via video_queue_buffer and VIDIOC_QBUF.

UYVY is only accepted by the ISP, therefore it first needs to be passed to vc.ril.isp to be converted to something more useful. (Actually in the latest firmware it is also supported by video_encode).
video_render only has one input port and no output ports, so you can't take a buffer from it to pass to anywhere else.

What you can do is use a mmal_connection to a video_splitter component to clone the image data to multiple destinations. You can then hook image_encode to one port, video_encode to another, and video_render to a third. That's not ideal as it is doing a memcpy on all the buffers.

The alternative is what yavta is doing in using mmal_buffer_header_replicate. The app is set up to take callbacks from the ISP output into function isp_output_callback. As you can see in the source, depending on which components are enabled if calls mmal_buffer_header_replicate and sends the buffer to render and/or encoder. When the buffer is returned from render or encode via render_encoder_input_callback the replicated header is released, and when all replicated headers are returned the source buffer will also be released back to the pool. The buffers_to_isp function is then returning all available buffers to the ISP for filling.
The output of encode is handled by a separate callback (encoder_buffer_callback) which puts the buffer on another queue which then saves the data in asynchronously. Blocking in the callbacks for a long time will lead to stalls in the pipeline.

You don't really need to have two independent video encodes for saving and RTSP unless you need differing bitrates or other parameters for them. H264 can be split at any IDR frame, and if you set the parameter MMAL_PARAMETER_VIDEO_ENCODE_INLINE_HEADER to MMAL_TRUE on the encoder output port then it will also spit out the H264 header bytes before every IDR frame.
Raspivid has a circular buffer mode that uses this technique and keeps a note of where the header bytes are in the stream. On trigger it saves from the header bytes forward to give the last N seconds of video.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 7457
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: ADV7280A-M Image Capture Issues

Wed Aug 08, 2018 2:25 pm

Significantly pruned version of yavta now at https://github.com/6by9/v4l2_mmal
I'm now trying to make it a little more generic in adding sink components....
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 7457
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: ADV7280A-M Image Capture Issues

Wed Aug 08, 2018 3:31 pm

And now updated to set up the destinations via a relatively simple array of component names and encodings.

BMP drops lots of frames as it is requires a format conversion and is not hardware accelerated. JPEG does OK. I'd suggest you only enable the image_encode path when you want a still, not trying to use it as a video codec.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

DanR
Posts: 33
Joined: Fri Jan 18, 2013 1:20 pm

Re: ADV7280A-M Image Capture Issues

Thu Aug 09, 2018 7:48 am

Great, thanks Dave much appreciated. I've been hounded for support on our other hardware yesterday so I lost a days work yet again and only had a little time to look through it. I'll have a look through the code now.

Many many thanks,

Dan

DanR
Posts: 33
Joined: Fri Jan 18, 2013 1:20 pm

Re: ADV7280A-M Image Capture Issues

Tue Aug 14, 2018 7:44 am

Hi,

I'm having a little trouble with some understanding of MMAL, in particular the vc.ril.video_encode and vc.ril.image_encode and their connection to vc.ril.isp. Can they both be connected at the same time or do I need a splitter between them? The reason I ask is that they're all MMAL_COMPONENT_T and the docs say that they have one input and one output connection and you can only connect one output to one input. Is my understanding correct or am I missing something?

Many Thanks,

Dan

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 7457
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: ADV7280A-M Image Capture Issues

Tue Aug 14, 2018 7:58 am

MMAL_CONNECTION_T will only connect one output to one input.

You can use mmal_buffer_header_replicate to create "slaved" buffer headers from a source one. You need to create your own callbacks rather than relying on MMAL_CONNECTION_T, but I've given you boilerplate - see https://github.com/6by9/v4l2_mmal/blob/ ... mal.c#L786 for the callbacks, and lines 1118-1333 to set up the sinks. You'll want some form of trigger to say when image_encode is required.

If you want to use MMAL_CONNECTION_T then you will have to insert a video_splitter component, but it involves a copy (with optional format conversion) between input and output buffers.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

DanR
Posts: 33
Joined: Fri Jan 18, 2013 1:20 pm

Re: ADV7280A-M Image Capture Issues

Wed Aug 07, 2019 7:50 am

Hiya,

One issue I'm having is it seems, although I can't be certain, is that the ADV7280 is selecting the input as SVideo as the image is there but almost undiscernible due to it being extremely dark. It only does this once in a blue moon and never when I'm about and in a position to debug it further.

How can I force the input modes to select component, svideo or CVBS? I'm guessing this would be via an IOCtl call or several? Sorry to be a pain chap, it's very much appreciated!!

Kind Regards, Dan

Return to “Interfacing (DSI, CSI, I2C, etc.)”