6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 11628
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: TC358743 HDMI to CSI-2 install instructions

Tue Feb 09, 2021 9:54 am

rma153 wrote:
Tue Feb 09, 2021 4:01 am
Well, I meant to type that the HAT is currently at 4.19.127. Nonetheless, when asked about this, the vendor's response back was terribly disappointing - "Your request is noted! Currently, we don’t have plans on updating the kernel version on our image."
So, I have no other choice but to hope your instructions will work on 4.19.127. I hope so. I have to say, the effort and commitment to developing the solution in this thread has been nothing short of extraordinary.
What is this HAT? Even if the vendor's largely ignore you, if it's a kernel driver it must be GPLv2 with source code available, so others can look at updating it.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

rma153
Posts: 53
Joined: Sat Nov 12, 2016 6:17 am

Re: TC358743 HDMI to CSI-2 install instructions

Tue Feb 09, 2021 9:38 pm

EMLID's Navio2 Autopilot HAT for Raspberry Pi https://emlid.com/

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 11628
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: TC358743 HDMI to CSI-2 install instructions

Wed Feb 10, 2021 6:28 pm

rma153 wrote:
Tue Feb 09, 2021 9:38 pm
EMLID's Navio2 Autopilot HAT for Raspberry Pi https://emlid.com/
Well it claims to be a preconfigured Raspbian system, so one would hope that just doing an "apt update" "apt dist-upgrade" would get the latest kernel and firmware, however they may have pinned or otherwise changed that.
I'm downloading the image to see what they have changed. Forking off this thread to viewtopic.php?f=45&t=303477 as it is unrelated to TC358743.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

rma153
Posts: 53
Joined: Sat Nov 12, 2016 6:17 am

Re: TC358743 HDMI to CSI-2 install instructions

Sun Feb 14, 2021 5:37 am

From this thread I understand I need to upgrade to Gstreamer 1.14 or higher on my RPi. Since I send video to my Windows 10 laptop, do I need to also update gstreamer on my Windows laptop?

rma153
Posts: 53
Joined: Sat Nov 12, 2016 6:17 am

Re: TC358743 HDMI to CSI-2 install instructions

Mon Feb 15, 2021 5:38 am

I have closely followed these well crafted TC358743 HDMI to CSI-2 install instructions.
Can you please recommend the two gstreamer pipelines that I should use
1) on the RPi side that I am streaming video from
1) on the Windows side that I am streaming video to from the RPi.

Would this work for the RPi side pipeline:
gst-launch-1.0 v4l2src ! "video/x-raw,framerate=30/1,format=UYVY" ! v4l2h264enc extra-controls="controls,h264_profile=4,h264_level=10,video_bitrate=256000;" ! video/x-h264,profile=high ! h264parse ! queue ! udpsink host=192.168.191.XX port=5600

Regarding the Windows side pipeline:
Although I have just updated the RPi gstreamer to 1.14, I have not also updated the Windows gstreamer. I'm thinking that I need to also update Windows gstreamer, otherwise I will likely get an error like "No such element or plugin 'v4l2h264dec". Am I correct?

miro.rovis
Posts: 57
Joined: Mon Feb 08, 2021 10:31 am

Re: TC358743 HDMI to CSI-2 install instructions

Mon Feb 15, 2021 5:11 pm

Because it is missing in this topic, from page 17 of:
HDMI to CSI-2 via TC358743 on kernel 4.1
viewtopic.php?p=1339178#p1339178
6by9 wrote:
Tue Jul 10, 2018 2:27 pm
[...]
For audio, add

Code: Select all

dtoverlay=tc358743-audio
and wire it up as LRCK/WFS to GPIO 19, BCK/SCK to GPIO 18, and DATA/SD to GPIO 20. (those are 7, 6 and 5 respectively on the B101/B102 boards).
That should add an audio capture device.
[...]
Page 17, take note. Takes a lot of searching, even if you know which topic you read that piece of info in.
BTW, if anybody cares, have I wired those correctly, as seen on:
CM4 on CM4IO with Auvidea B102 play dead
viewtopic.php?f=98&t=303177#p1816630
I'll need it when redeploying my CM4 with B102 setup...

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 11628
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: TC358743 HDMI to CSI-2 install instructions

Mon Feb 15, 2021 9:23 pm

miro.rovis wrote:
Mon Feb 15, 2021 5:11 pm
Because it is missing in this topic, from page 17 of:
HDMI to CSI-2 via TC358743 on kernel 4.1
viewtopic.php?p=1339178#p1339178
6by9 wrote:
Tue Jul 10, 2018 2:27 pm
[...]
For audio, add

Code: Select all

dtoverlay=tc358743-audio
and wire it up as LRCK/WFS to GPIO 19, BCK/SCK to GPIO 18, and DATA/SD to GPIO 20. (those are 7, 6 and 5 respectively on the B101/B102 boards).
That should add an audio capture device.
[...]
Page 17, take note. Takes a lot of searching, even if you know which topic you read that piece of info in.
BTW, if anybody cares, have I wired those correctly, as seen on:
CM4 on CM4IO with Auvidea B102 play dead
viewtopic.php?f=98&t=303177#p1816630
I'll need it when redeploying my CM4 with B102 setup...
It's in the docs which are referenced in the first post - https://www.raspberrypi.org/documentati ... 2-usage.md
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 11628
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: TC358743 HDMI to CSI-2 install instructions

Sun Feb 28, 2021 9:22 pm

(Split rma153's v4l2h264enc issues off into viewtopic.php?f=38&t=305634)
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

shangbinbin
Posts: 2
Joined: Thu Mar 25, 2021 3:18 am

Re: TC358743 HDMI to CSI-2 install instructions

Thu Mar 25, 2021 3:24 am

Hello, I'm one of the main manufacturers of HDMI to CSI adapter board. The price of tc358743 has gone up too high in the last half year, but I didn't want the price of terminal board to go up, so I found another chip tc358749 of the same kind, but my software level is limited, and I won't drive it in raspberry. Can you help me with this

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 11628
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: TC358743 HDMI to CSI-2 install instructions

Thu Mar 25, 2021 2:52 pm

shangbinbin wrote:
Thu Mar 25, 2021 3:24 am
Hello, I'm one of the main manufacturers of HDMI to CSI adapter board. The price of tc358743 has gone up too high in the last half year, but I didn't want the price of terminal board to go up, so I found another chip tc358749 of the same kind, but my software level is limited, and I won't drive it in raspberry. Can you help me with this
Discussion moved to https://github.com/raspberrypi/linux/issues/4235
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

p2p
Posts: 7
Joined: Thu Mar 25, 2021 5:26 pm

Re: TC358743 HDMI to CSI-2 install instructions

Thu Mar 25, 2021 5:55 pm

Hi, I'm currently using tc358743 with cm4, 4lanes is enabled. Input source 1080p 60hz.

Code: Select all

root@raspberrypi:/home/pi# gst-launch-1.0 --version
gst-launch-1.0 version 1.16.3
GStreamer 1.14.4
1, When the capture framerate >30fps, gstreamer shows

Code: Select all

root@raspberrypi:/home/pi# gst-launch-1.0 v4l2src io-mode=5 ! video/x-raw, format=UYVY, framerate=31/1 ! v4l2h264enc output-io-mode=4 ! video/x-h264,profile=high ! h264parse ! queue ! matroskamux name=mux ! filesink location=foo.mkv alsasrc device=hw:1 ! audio/x-raw,rate=48000,channels=2 ! audioconvert ! avenc_aac bitrate=48000 ! aacparse ! queue ! mux.
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
New clock: GstAudioSrcClock
Redistribute latency...
Redistribute latency...
ERROR: from element /GstPipeline:pipeline0/v4l2h264enc:v4l2h264enc0: Failed to process frame.
Additional debug info:
gstv4l2videoenc.c(803): gst_v4l2_video_enc_handle_frame (): /GstPipeline:pipeline0/v4l2h264enc:v4l2h264enc0:
Maybe be due to not enough memory or failing driver
Execution ended after 0:00:00.127976869
Setting pipeline to NULL ...
Freeing pipeline ...
30fps and below works.

Yavta can capture 1080p60fps without any error. But I want the sound be record at the same time.

I think it's the problem of h264 level?



2, When I change format=UYVY to RGB, the file still shows yuv420, but the color of RGB is wrong, some thing turn blue.



3, Why the command given above can't work even below 30fps?

Code: Select all

gst-launch-1.0 v4l2src ! "video/x-raw,framerate=30/1,format=UYVY" ! v4l2h264enc extra-controls="controls,h264_profile=4,h264_level=10,video_bitrate=256000;" ! video/x-h264,profile=high ! h264parse ! queue ! matroskamux name=mux ! filesink location=foo.mkv alsasrc device=hw:1 ! audio/x-raw,rate=48000,channels=2 ! audioconvert ! avenc_aac bitrate=48000 ! aacparse ! queue ! mux.
on my device it shows

Code: Select all

root@raspberrypi:/home/pi# gst-launch-1.0 v4l2src ! "video/x-raw,framerate=20/1,format=UYVY" ! v4l2h264enc extra-controls="controls,h264_profile=4,h264_level=10,video_bitrate=256000;" ! video/x-h264,profile=high ! h264parse ! queue ! matroskamux name=mux ! filesink location=foo.mkv alsasrc device=hw:1 ! audio/x-raw,rate=48000,channels=2 ! audioconvert ! avenc_aac bitrate=48000 ! aacparse ! queue ! mux.
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
New clock: GstAudioSrcClock
Redistribute latency...
Redistribute latency...
ERROR: from element /GstPipeline:pipeline0/v4l2h264enc:v4l2h264enc0: Failed to process frame.
Additional debug info:
gstv4l2videoenc.c(803): gst_v4l2_video_enc_handle_frame (): /GstPipeline:pipeline0/v4l2h264enc:v4l2h264enc0:
Maybe be due to not enough memory or failing driver
Execution ended after 0:00:00.168783357
Setting pipeline to NULL ...
Freeing pipeline ...
Thank you!

proffff
Posts: 2
Joined: Wed Mar 31, 2021 12:25 pm

Re: TC358743 HDMI to CSI-2 install instructions

Wed Mar 31, 2021 5:12 pm

hello gods of CSI
I have issue when following install tutorial
I'm running on pi 4 with standard latest pios

Code: Select all

pi@DVMissionControl:~ $ uname -a
Linux DVMissionControl 5.10.17-v7l+ #1403 SMP Mon Feb 22 11:33:35 GMT 2021 armv7l GNU/Linux
out of box using mmal driver it works in 1080i and not 1080p @30fps

so I followed install steps and all seem to go well until edid step
I have well the unicam device driver loader

Code: Select all

pi@DVMissionControl:~ $ v4l2-ctl --list-devices
bcm2835-codec-decode (platform:bcm2835-codec):
	/dev/video10
	/dev/video11
	/dev/video12

bcm2835-isp (platform:bcm2835-isp):
	/dev/video13
	/dev/video14
	/dev/video15
	/dev/video16

unicam (platform:fe801000.csi):
	/dev/video0
	/dev/video1
i've use the EDIDfile here https://github.com/6by9/CSI2_device_con ... 50EDID.txt and when sending it I got

Code: Select all

v4l2-ctl --set-edid=file=1080P60EDID.txt  --fix-edid-checksums -d /dev/video0

CTA-861 Header
  IT Formats Underscanned: yes
  Audio:                   yes
  YCbCr 4:4:4:             no
  YCbCr 4:2:2:             no

HDMI Vendor-Specific Data Block
  Physical Address:        3.0.0.0
  YCbCr 4:4:4 Deep Color:  no
  30-bit:                  no
  36-bit:                  no
  48-bit:                  no

CTA-861 Video Capability Descriptor
  RGB Quantization Range:  yes
  YCC Quantization Range:  no
  PT:                      Supports both over- and underscan
  IT:                      Supports both over- and underscan
  CE:                      Supports both over- and underscan
that looks a bit empty but why not I'm not a edid specialist after all

the problems start when trying timing quering

Code: Select all

v4l2-ctl --query-dv-timings -d /dev/video0
VIDIOC_QUERY_DV_TIMINGS: failed: Link has been severed
	Active width: 0
	Active height: 0
	Total width: 0
	Total height: 0
	Frame format: progressive
	Polarities: -vsync -hsync
	Pixelclock: 0 Hz
	Horizontal frontporch: 0
	Horizontal sync: 0
	Horizontal backporch: 0
	Vertical frontporch: 0
	Vertical sync: 0
	Vertical backporch: 0
	Standards: 
	Flags: 
and obviously everything else following this turns into nightmare
if anyone can find a small tips to get me on the right way that would be wonderful
thanks a lot

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 11628
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: TC358743 HDMI to CSI-2 install instructions

Wed Mar 31, 2021 5:44 pm

proffff wrote:
Wed Mar 31, 2021 5:12 pm
the problems start when trying timing quering

Code: Select all

v4l2-ctl --query-dv-timings -d /dev/video0
VIDIOC_QUERY_DV_TIMINGS: failed: Link has been severed
	Active width: 0
	Active height: 0
	Total width: 0
	Total height: 0
	Frame format: progressive
	Polarities: -vsync -hsync
	Pixelclock: 0 Hz
	Horizontal frontporch: 0
	Horizontal sync: 0
	Horizontal backporch: 0
	Vertical frontporch: 0
	Vertical sync: 0
	Vertical backporch: 0
	Standards: 
	Flags: 
and obviously everything else following this turns into nightmare
if anyone can find a small tips to get me on the right way that would be wonderful
thanks a lot
So it hasn't detected a source. What have you got connected to the TC358743 as a source? Does it respond correctly to hot plug events? If not then disconnect and reconnect the HDMI lead, or reboot the source.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

jegK
Posts: 5
Joined: Mon Oct 16, 2017 9:46 am

Re: TC358743 HDMI to CSI-2 install instructions

Wed Apr 07, 2021 1:32 pm

Hello,

Thanks for the good setup summary at the top of this thread!
I am testing an Auvidea B101 and able to access the video fine in VLC and record with ffmpeg, using either an old GoPro at 1080p or a laptop as input.

I would like to use this as video source in chromium for a webrtc app, but strangely, the colors are messed up even though they appear fine in VLC, see exmple pictures below.

any explanation? or advice on how to fix?

thks,


chromium:
Image

VLC (correct colors, the step is actually yellow :-) ):
Image

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 11628
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: TC358743 HDMI to CSI-2 install instructions

Wed Apr 07, 2021 4:25 pm

jegK wrote:
Wed Apr 07, 2021 1:32 pm
I am testing an Auvidea B101 and able to access the video fine in VLC and record with ffmpeg, using either an old GoPro at 1080p or a laptop as input.

I would like to use this as video source in chromium for a webrtc app, but strangely, the colors are messed up even though they appear fine in VLC, see exmple pictures below.

any explanation? or advice on how to fix?
It looks like Chromium is misinterpretting the image data as RGB888 instead of BGR888, so red and blue components are swapped.
Using the UYVY format instead may work better, and allows slightly higher capture rates as it is only 16bpp instead of 24bpp.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

jegK
Posts: 5
Joined: Mon Oct 16, 2017 9:46 am

Re: TC358743 HDMI to CSI-2 install instructions

Wed Apr 07, 2021 6:45 pm

I see, thanks for the info.

I tried setting:

Code: Select all

v4l2-ctl -v pixelformat=UYVY
but it made no difference, so looks like chromium is ignoring this... and i havent found a way to set something related in chromium either.

jtaverne
Posts: 2
Joined: Tue Mar 23, 2021 10:57 am

Re: TC358743 HDMI to CSI-2 install instructions

Thu Apr 08, 2021 10:39 am

I have been using the camera module and "raspivid -t 0" to get latency free video on the HDMI output, without running X, framebuffer only and without (re)encoding anything. I am now using the Auvidea B101 and a camera outputting 1080P30. As Raspvid isn't supporting this (garbled video), I switched to using the tc358743 driver. Now I am trying to find a way to get the same result as "raspivid -t 0". All solutions I tried with gstreamer, mplayer, ffmpeg all give delays a low fps. What simple command would give me low latency video on the HDMI output from /dev/video0?

p2p
Posts: 7
Joined: Thu Mar 25, 2021 5:26 pm

Re: TC358743 HDMI to CSI-2 install instructions

Sat Apr 17, 2021 10:55 am

Hello! Thanks for the tutorials, my hdmi to csi bridge is work perfect now.
I just find out the V4L2 codec doesn't allow for levels above 4. Are there any ways to record or stream 1080 50/60fps with audio?
I tried yavta with ffmepg on raspberry pi 4b and cm4, it can record or stream 1080 50/60fps without frame drop, but I don't know how to get sound output without gstreamer.
gstreamer only works when h264 level <= 4
Thanks!

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 11628
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: TC358743 HDMI to CSI-2 install instructions

Sun Apr 18, 2021 6:53 am

p2p wrote:
Sat Apr 17, 2021 10:55 am
Hello! Thanks for the tutorials, my hdmi to csi bridge is work perfect now.
I just find out the V4L2 codec doesn't allow for levels above 4. Are there any ways to record or stream 1080 50/60fps with audio?
I tried yavta with ffmepg on raspberry pi 4b and cm4, it can record or stream 1080 50/60fps without frame drop, but I don't know how to get sound output without gstreamer.
gstreamer only works when h264 level <= 4
Thanks!
https://github.com/raspberrypi/linux/pull/4238 adds levels 4.1 and 4.2 to the codec. I'll nudge people to get it merged.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

p2p
Posts: 7
Joined: Thu Mar 25, 2021 5:26 pm

Re: TC358743 HDMI to CSI-2 install instructions

Sun Apr 25, 2021 4:31 am

6by9 wrote:
Sun Apr 18, 2021 6:53 am
p2p wrote:
Sat Apr 17, 2021 10:55 am
Hello! Thanks for the tutorials, my hdmi to csi bridge is work perfect now.
I just find out the V4L2 codec doesn't allow for levels above 4. Are there any ways to record or stream 1080 50/60fps with audio?
I tried yavta with ffmepg on raspberry pi 4b and cm4, it can record or stream 1080 50/60fps without frame drop, but I don't know how to get sound output without gstreamer.
gstreamer only works when h264 level <= 4
Thanks!
https://github.com/raspberrypi/linux/pull/4238 adds levels 4.1 and 4.2 to the codec. I'll nudge people to get it merged.
Thank you! Now it's work well with cm4, 4b 3b.
I'm trying to get it work on cm3+ now. I already modified dt-blob file

Code: Select all

        pin_define@CAMERA_0_UNICAM_PORT {
              type = "internal";
              number = <4>;
            };
            pin_define@CAMERA_0_I2C_PORT {
              type = "internal";
              number = <0>;
            };
            pin_define@CAMERA_0_SDA_PIN {
              type = "internal";
              number = <28>;
            };
            pin_define@CAMERA_0_SCL_PIN {
              type = "internal";
              number = <29>;
            };

Code: Select all

pin@p28  { function = "input";   termination = "pull_up";    }; // CAMERA 0 I2C0 SDA
            pin@p29 { function = "input";   termination = "pull_up";    }; // CAMERA 0 I2C0 SCL
            pin@p2  { function = "output";  termination = "no_pulling"; }; // CAMERA 0 LED
            pin@p3  { function = "output";  termination = "no_pulling"; }; // CAMERA 0 SHUTDOWN
and connect like this
Image

for the config.txt, I add

Code: Select all

dtoverlay=tc358743,i2c_pins_28_29=1,4lane=1
but when I test it with yavta, I randomly success once, in the normal time I usually get

Code: Select all

We're encoding to file.h264
Device /dev/video0 opened.
Device `unicam' on `platform:3f801000.csi' (driver 'unicam') is a video capture (without mplanes) device.
stride is 0
stride is now 1280
Video format set: UYVY (59565955) 1920x1080 (stride 3840) field none buffer size 4147200
QUERY_DV_TIMINGS returned 1920x1080 pixclk 148500000
Framerate is 60
Video format: UYVY (59565955) 1920x1080 (stride 3840) field none buffer size 4147200
vc.ril.isp:in:0(UYVY)(0x127c120)type: video, fourcc: UYVY bitrate: 0, framed: 0 extra data: 0, (nil) width: 1920, height: 1080, (0,0,1920,1080) pixel aspect ratio: 0/0, frame rate: 0/0
buffers num: 3(opt 1, min 1), size: 4147200(opt 4147200, min: 4147200), align: 0
Created pool of length 3, size 0
Enable encoder....
Create pool of 3 buffers of size 0 for render
Create pool of 3 buffers of size 0 for encode ip
Create pool of 3 buffers of size 3110400 for encode/render
Writing data to file.h264
Create pool of 8 buffers of size 262144
Sent buffer 0x1286a28
Sent buffer 0x1286c00
Sent buffer 0x1286dd8
Sent buffer 0x1286fb0
Sent buffer 0x1287188
Sent buffer 0x1287360
Sent buffer 0x1287538
Sent buffer 0x1287710
3 buffers requested, V4L2 returned 3 bufs.
length: 4147200 offset: 0 timestamp type/source: mono/EoF
Buffer 0/0 mapped at address 0x7090b000.
Importing DMABUF 25 into VCSM...
...done. vcsm_handle 27
Exported buffer 0 to dmabuf 25, vcsm handle 27
Linking V4L2 buffer index 0 ptr 0x1288430 to MMAL header 0x1283190. mmal->data 0xC0000014
length: 4147200 offset: 4149248 timestamp type/source: mono/EoF
Buffer 1/0 mapped at address 0x70121000.
Importing DMABUF 27 into VCSM...
...done. vcsm_handle 29
Exported buffer 1 to dmabuf 27, vcsm handle 29
Linking V4L2 buffer index 1 ptr 0x12884a0 to MMAL header 0x1283368. mmal->data 0xC0000015
length: 4147200 offset: 8298496 timestamp type/source: mono/EoF
Buffer 2/0 mapped at address 0x6f937000.
Importing DMABUF 29 into VCSM...
...done. vcsm_handle 31
Exported buffer 2 to dmabuf 29, vcsm handle 31
Linking V4L2 buffer index 2 ptr 0x1288510 to MMAL header 0x1283540. mmal->data 0xC0000016
select timeout
with gstreamer, it won't start record and I get a 0kb file.

after setting edid, query works

Code: Select all

Active width: 1920
	Active height: 1080
	Total width: 2200
	Total height: 1125
	Frame format: progressive
	Polarities: -vsync -hsync
	Pixelclock: 148500000 Hz (60.00 frames per second)
	Horizontal frontporch: 0
	Horizontal sync: 280
	Horizontal backporch: 0
	Vertical frontporch: 0
	Vertical sync: 45
	Vertical backporch: 0
	Standards: 
	Flags: 

Code: Select all

i2cdetect -l
i2c-1	i2c       	bcm2835 (i2c@7e804000)          	I2C adapter
i2c-11	i2c       	bcm2835 (i2c@7e205000)          	I2C adapter
i2c-0	i2c       	i2c-11-mux (chan_id 0)          	I2C adapter
i2c-10	i2c       	i2c-11-mux (chan_id 1)          	I2C adapter

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 11628
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: TC358743 HDMI to CSI-2 install instructions

Wed Jun 30, 2021 2:51 pm

Thread on issues getting audio to work split off to viewtopic.php?f=38&t=314944&p=1881935
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

natttyyy
Posts: 2
Joined: Thu Jul 22, 2021 7:45 am

Re: TC358743 HDMI to CSI-2 install instructions

Thu Jul 29, 2021 2:08 pm

Thanks 6by9 for the wonderful setup instructions! Finally managed to get it to work... well somewhat xD

I have been trying to just grab the HDMI input, mux it with audio and send it to an nginx server on the pi so that others can stream from it. Sound easy enough and have come up with this pipeline

Code: Select all

gst-launch-1.0 -vvv v4l2src ! "video/x-raw,format=UYVY,framerate=30/1" ! v4l2h264enc extra-controls="controls,h264_profile=4,h264_level=13,video_bitrate=256000;" ! "video/x-h264,profile=high,level=(string)4.2" ! h264parse ! queue ! flvmux streamable=true name=mux ! rtmpsink location="rtmp://localhost/live/test" alsasrc device=hw:2 ! audio/x-raw,rate=48000,channels=2 ! audioconvert ! avenc_aac bitrate=44100 ! aacparse ! queue ! mux.

But somehow, the audio always plays first and the video is out of sync by as bad as 30s some times, audio always leads video.

What am I screwing up?

Thanks so much in advance!



Some additional info:

Code: Select all

$ v4l2-ctl --set-edid=file=EDID.txt --fix-edid-checksums

CTA-861 Header
  IT Formats Underscanned: yes
  Audio:                   yes
  YCbCr 4:4:4:             no
  YCbCr 4:2:2:             no

HDMI Vendor-Specific Data Block
  Physical Address:        3.0.0.0
  YCbCr 4:4:4 Deep Color:  no
  30-bit:                  no
  36-bit:                  no
  48-bit:                  no

CTA-861 Video Capability Descriptor
  RGB Quantization Range:  yes
  YCC Quantization Range:  no
  PT:                      Supports both over- and underscan
  IT:                      Supports both over- and underscan
  CE:                      Supports both over- and underscan
pi@raspberrypi:~ $ v4l2-ctl --query-dv-timings
	Active width: 1920
	Active height: 1080
	Total width: 2640
	Total height: 1125
	Frame format: progressive
	Polarities: -vsync -hsync
	Pixelclock: 148500000 Hz (50.00 frames per second)
	Horizontal frontporch: 0
	Horizontal sync: 720
	Horizontal backporch: 0
	Vertical frontporch: 0
	Vertical sync: 45
	Vertical backporch: 0
	Standards: 
	Flags: 
pi@raspberrypi:~ $ v4l2-ctl --set-dv-bt-timings query
BT timings set
And this is what DEBUG=2 got me:

Code: Select all

0:00:00.317046714   657  0x182e4c0 WARN                    v4l2 gstv4l2object.c:4408:gst_v4l2_object_probe_caps:<v4l2h264enc0:src> Failed to probe pixel aspect ratio with VIDIOC_CROPCAP: Invalid argument
0:00:00.319712905   657  0x17855b0 WARN                 v4l2src gstv4l2src.c:695:gst_v4l2src_query:<v4l2src0> Can't give latency since framerate isn't fixated !
0:00:00.319827476   657  0x17855b0 WARN              aggregator gstaggregator.c:2046:gst_aggregator_query_latency_unlocked:<mux> Latency query failed
Pipeline is live and does not need PREROLL ...
0:00:00.323319943   657  0x17855b0 WARN                 v4l2src gstv4l2src.c:695:gst_v4l2src_query:<v4l2src0> Can't give latency since framerate isn't fixated !
0:00:00.323378385   657  0x17855b0 WARN              aggregator gstaggregator.c:2046:gst_aggregator_query_latency_unlocked:<mux> Latency query failed
0:00:00.324187180   657  0x1788980 WARN                    alsa pcm_hw.c:1355:snd_pcm_hw_get_chmap: alsalib error: Cannot read Channel Map ctl
: No such file or directory
0:00:00.324354898   657  0x17855b0 WARN                 v4l2src gstv4l2src.c:695:gst_v4l2src_query:<v4l2src0> Can't give latency since framerate isn't fixated !
0:00:00.324443525   657  0x17855b0 WARN              aggregator gstaggregator.c:2046:gst_aggregator_query_latency_unlocked:<mux> Latency query failed
0:00:00.324616965   657  0x17855b0 WARN                 v4l2src gstv4l2src.c:695:gst_v4l2src_query:<v4l2src0> Can't give latency since framerate isn't fixated !
0:00:00.324670593   657  0x17855b0 WARN              aggregator gstaggregator.c:2046:gst_aggregator_query_latency_unlocked:<mux> Latency query failed
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
0:00:00.326327903   657  0x17855b0 WARN                 v4l2src gstv4l2src.c:695:gst_v4l2src_query:<v4l2src0> Can't give latency since framerate isn't fixated !
0:00:00.326397864   657  0x17855b0 WARN              aggregator gstaggregator.c:2046:gst_aggregator_query_latency_unlocked:<mux> Latency query failed
/GstPipeline:pipeline0/GstV4l2Src:v4l2src0.GstPad:src: caps = video/x-raw, format=(string)UYVY, framerate=(fraction)30/1, width=(int)1920, height=(int)1080, interlace-mode=(string)progressive, colorimetry=(string)bt601
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = video/x-raw, format=(string)UYVY, framerate=(fraction)30/1, width=(int)1920, height=(int)1080, interlace-mode=(string)progressive, colorimetry=(string)bt601
/GstPipeline:pipeline0/v4l2h264enc:v4l2h264enc0.GstPad:src: caps = video/x-h264, stream-format=(string)byte-stream, alignment=(string)au, level=(string)4.2, profile=(string)high, width=(int)1920, height=(int)1080, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)30/1, interlace-mode=(string)progressive, colorimetry=(string)bt601
/GstPipeline:pipeline0/GstCapsFilter:capsfilter1.GstPad:src: caps = video/x-h264, stream-format=(string)byte-stream, alignment=(string)au, level=(string)4.2, profile=(string)high, width=(int)1920, height=(int)1080, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)30/1, interlace-mode=(string)progressive, colorimetry=(string)bt601
/GstPipeline:pipeline0/GstH264Parse:h264parse0.GstPad:sink: caps = video/x-h264, stream-format=(string)byte-stream, alignment=(string)au, level=(string)4.2, profile=(string)high, width=(int)1920, height=(int)1080, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)30/1, interlace-mode=(string)progressive, colorimetry=(string)bt601
/GstPipeline:pipeline0/GstCapsFilter:capsfilter1.GstPad:sink: caps = video/x-h264, stream-format=(string)byte-stream, alignment=(string)au, level=(string)4.2, profile=(string)high, width=(int)1920, height=(int)1080, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)30/1, interlace-mode=(string)progressive, colorimetry=(string)bt601
Redistribute latency...
/GstPipeline:pipeline0/v4l2h264enc:v4l2h264enc0.GstPad:sink: caps = video/x-raw, format=(string)UYVY, framerate=(fraction)30/1, width=(int)1920, height=(int)1080, interlace-mode=(string)progressive, colorimetry=(string)bt601
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:sink: caps = video/x-raw, format=(string)UYVY, framerate=(fraction)30/1, width=(int)1920, height=(int)1080, interlace-mode=(string)progressive, colorimetry=(string)bt601
New clock: GstAudioSrcClock
/GstPipeline:pipeline0/GstAlsaSrc:alsasrc0: actual-buffer-time = 200000
/GstPipeline:pipeline0/GstAlsaSrc:alsasrc0: actual-latency-time = 10000
Redistribute latency...
/GstPipeline:pipeline0/GstAlsaSrc:alsasrc0.GstPad:src: caps = audio/x-raw, format=(string)S16LE, layout=(string)interleaved, rate=(int)48000, channels=(int)2, channel-mask=(bitmask)0x0000000000000003
/GstPipeline:pipeline0/GstCapsFilter:capsfilter2.GstPad:src: caps = audio/x-raw, format=(string)S16LE, layout=(string)interleaved, rate=(int)48000, channels=(int)2, channel-mask=(bitmask)0x0000000000000003
/GstPipeline:pipeline0/GstAudioConvert:audioconvert0.GstPad:src: caps = audio/x-raw, rate=(int)48000, channels=(int)2, format=(string)F32LE, layout=(string)interleaved, channel-mask=(bitmask)0x0000000000000003
/GstPipeline:pipeline0/avenc_aac:avenc_aac0.GstPad:sink: caps = audio/x-raw, rate=(int)48000, channels=(int)2, format=(string)F32LE, layout=(string)interleaved, channel-mask=(bitmask)0x0000000000000003
/GstPipeline:pipeline0/GstAudioConvert:audioconvert0.GstPad:sink: caps = audio/x-raw, format=(string)S16LE, layout=(string)interleaved, rate=(int)48000, channels=(int)2, channel-mask=(bitmask)0x0000000000000003
/GstPipeline:pipeline0/GstCapsFilter:capsfilter2.GstPad:sink: caps = audio/x-raw, format=(string)S16LE, layout=(string)interleaved, rate=(int)48000, channels=(int)2, channel-mask=(bitmask)0x0000000000000003
0:00:00.375382072   657  0x1788ac0 WARN          v4l2bufferpool gstv4l2bufferpool.c:815:gst_v4l2_buffer_pool_start:<v4l2h264enc0:pool0:src> Uncertain or not enough buffers, enabling copy threshold
/GstPipeline:pipeline0/avenc_aac:avenc_aac0.GstPad:src: caps = audio/mpeg, channels=(int)2, rate=(int)48000, mpegversion=(int)4, base-profile=(string)lc, stream-format=(string)raw, channel-mask=(bitmask)0x0000000000000003, level=(string)2, profile=(string)lc, codec_data=(buffer)119056e500
/GstPipeline:pipeline0/GstAacParse:aacparse0.GstPad:src: caps = audio/mpeg, channels=(int)2, rate=(int)48000, mpegversion=(int)4, base-profile=(string)lc, stream-format=(string)raw, channel-mask=(bitmask)0x0000000000000003, level=(string)2, profile=(string)lc, codec_data=(buffer)119056e500, framed=(boolean)true
/GstPipeline:pipeline0/GstQueue:queue1.GstPad:src: caps = audio/mpeg, channels=(int)2, rate=(int)48000, mpegversion=(int)4, base-profile=(string)lc, stream-format=(string)raw, channel-mask=(bitmask)0x0000000000000003, level=(string)2, profile=(string)lc, codec_data=(buffer)119056e500, framed=(boolean)true
/GstPipeline:pipeline0/GstFlvMux:mux.GstFlvMuxPad:sink_1: caps = audio/mpeg, channels=(int)2, rate=(int)48000, mpegversion=(int)4, base-profile=(string)lc, stream-format=(string)raw, channel-mask=(bitmask)0x0000000000000003, level=(string)2, profile=(string)lc, codec_data=(buffer)119056e500, framed=(boolean)true
/GstPipeline:pipeline0/GstQueue:queue1.GstPad:sink: caps = audio/mpeg, channels=(int)2, rate=(int)48000, mpegversion=(int)4, base-profile=(string)lc, stream-format=(string)raw, channel-mask=(bitmask)0x0000000000000003, level=(string)2, profile=(string)lc, codec_data=(buffer)119056e500, framed=(boolean)true
/GstPipeline:pipeline0/GstAacParse:aacparse0.GstPad:sink: caps = audio/mpeg, channels=(int)2, rate=(int)48000, mpegversion=(int)4, base-profile=(string)lc, stream-format=(string)raw, channel-mask=(bitmask)0x0000000000000003, level=(string)2, profile=(string)lc, codec_data=(buffer)119056e500
0:00:00.415134147   657  0x177fef0 WARN          v4l2bufferpool gstv4l2bufferpool.c:1289:gst_v4l2_buffer_pool_dqbuf:<v4l2h264enc0:pool0:src> Driver should never set v4l2_buffer.field to ANY
/GstPipeline:pipeline0/GstH264Parse:h264parse0.GstPad:src: caps = video/x-h264, stream-format=(string)avc, alignment=(string)au, level=(string)4, profile=(string)high, width=(int)1920, height=(int)1080, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)30/1, interlace-mode=(string)progressive, colorimetry=(string)bt601, chroma-format=(string)4:2:0, bit-depth-luma=(uint)8, bit-depth-chroma=(uint)8, parsed=(boolean)true, codec_data=(buffer)01640028ffe1002427640028ac2b403c0113f2c200000300020000030079c14007a10000bebcdef700f1226a01000528ee01372c
/GstPipeline:pipeline0/GstQueue:queue0.GstPad:sink: caps = video/x-h264, stream-format=(string)avc, alignment=(string)au, level=(string)4, profile=(string)high, width=(int)1920, height=(int)1080, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)30/1, interlace-mode=(string)progressive, colorimetry=(string)bt601, chroma-format=(string)4:2:0, bit-depth-luma=(uint)8, bit-depth-chroma=(uint)8, parsed=(boolean)true, codec_data=(buffer)01640028ffe1002427640028ac2b403c0113f2c200000300020000030079c14007a10000bebcdef700f1226a01000528ee01372c
/GstPipeline:pipeline0/GstQueue:queue0.GstPad:src: caps = video/x-h264, stream-format=(string)avc, alignment=(string)au, level=(string)4, profile=(string)high, width=(int)1920, height=(int)1080, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)30/1, interlace-mode=(string)progressive, colorimetry=(string)bt601, chroma-format=(string)4:2:0, bit-depth-luma=(uint)8, bit-depth-chroma=(uint)8, parsed=(boolean)true, codec_data=(buffer)01640028ffe1002427640028ac2b403c0113f2c200000300020000030079c14007a10000bebcdef700f1226a01000528ee01372c
/GstPipeline:pipeline0/GstFlvMux:mux.GstFlvMuxPad:sink_0: caps = video/x-h264, stream-format=(string)avc, alignment=(string)au, level=(string)4, profile=(string)high, width=(int)1920, height=(int)1080, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)30/1, interlace-mode=(string)progressive, colorimetry=(string)bt601, chroma-format=(string)4:2:0, bit-depth-luma=(uint)8, bit-depth-chroma=(uint)8, parsed=(boolean)true, codec_data=(buffer)01640028ffe1002427640028ac2b403c0113f2c200000300020000030079c14007a10000bebcdef700f1226a01000528ee01372c
/GstPipeline:pipeline0/GstFlvMux:mux.GstAggregatorPad:src: caps = video/x-flv, streamheader=(buffer)< 464c5601050000000900000000, 120001530000000000000002000a6f6e4d65746144617461080000000c000c766964656f636f646563696400401c0000000000000005776964746800409e0000000000000006686569676874004090e00000000000000c417370656374526174696f58003ff0000000000000000c417370656374526174696f59003ff000000000000000096672616d657261746500403e000000000000000d766964656f6461746172617465000000000000000000000c617564696f636f6465636964004024000000000000000d617564696f6461746172617465004045800000000000000f6d6574616461746163726561746f7202001a4753747265616d657220312e31382e3420464c56206d757865720007656e636f64657202001a4753747265616d657220312e31382e3420464c56206d75786572000c6372656174696f6e64617465020018546875204a756c2032392031343a30373a333320323032310000090000015e, 0900003900000000000000170000000001640028ffe1002427640028ac2b403c0113f2c200000300020000030079c14007a10000bebcdef700f1226a01000528ee01372c00000044, 0800000700000000000000af00119056e50000000012 >
/GstPipeline:pipeline0/GstRTMPSink:rtmpsink0.GstPad:sink: caps = video/x-flv, streamheader=(buffer)< 464c5601050000000900000000, 120001530000000000000002000a6f6e4d65746144617461080000000c000c766964656f636f646563696400401c0000000000000005776964746800409e0000000000000006686569676874004090e00000000000000c417370656374526174696f58003ff0000000000000000c417370656374526174696f59003ff000000000000000096672616d657261746500403e000000000000000d766964656f6461746172617465000000000000000000000c617564696f636f6465636964004024000000000000000d617564696f6461746172617465004045800000000000000f6d6574616461746163726561746f7202001a4753747265616d657220312e31382e3420464c56206d757865720007656e636f64657202001a4753747265616d657220312e31382e3420464c56206d75786572000c6372656174696f6e64617465020018546875204a756c2032392031343a30373a333320323032310000090000015e, 0900003900000000000000170000000001640028ffe1002427640028ac2b403c0113f2c200000300020000030079c14007a10000bebcdef700f1226a01000528ee01372c00000044, 0800000700000000000000af00119056e50000000012 >
/GstPipeline:pipeline0/GstFlvMux:mux.GstAggregatorPad:src: caps = video/x-flv, streamheader=(buffer)< 464c5601050000000900000000, 120001530000000000000002000a6f6e4d65746144617461080000000c000c766964656f636f646563696400401c0000000000000005776964746800409e0000000000000006686569676874004090e00000000000000c417370656374526174696f58003ff0000000000000000c417370656374526174696f59003ff000000000000000096672616d657261746500403e000000000000000d766964656f6461746172617465000000000000000000000c617564696f636f6465636964004024000000000000000d617564696f6461746172617465004045800000000000000f6d6574616461746163726561746f7202001a4753747265616d657220312e31382e3420464c56206d757865720007656e636f64657202001a4753747265616d657220312e31382e3420464c56206d75786572000c6372656174696f6e64617465020018546875204a756c2032392031343a30373a333320323032310000090000015e, 0900003900000000000000170000000001640028ffe1002427640028ac2b403c0113f2c200000300020000030079c14007a10000bebcdef700f1226a01000528ee01372c00000044, 0800000700000000000000af00119056e50000000012 >
/GstPipeline:pipeline0/GstRTMPSink:rtmpsink0.GstPad:sink: caps = video/x-flv, streamheader=(buffer)< 464c5601050000000900000000, 120001530000000000000002000a6f6e4d65746144617461080000000c000c766964656f636f646563696400401c0000000000000005776964746800409e0000000000000006686569676874004090e00000000000000c417370656374526174696f58003ff0000000000000000c417370656374526174696f59003ff000000000000000096672616d657261746500403e000000000000000d766964656f6461746172617465000000000000000000000c617564696f636f6465636964004024000000000000000d617564696f6461746172617465004045800000000000000f6d6574616461746163726561746f7202001a4753747265616d657220312e31382e3420464c56206d757865720007656e636f64657202001a4753747265616d657220312e31382e3420464c56206d75786572000c6372656174696f6e64617465020018546875204a756c2032392031343a30373a333320323032310000090000015e, 0900003900000000000000170000000001640028ffe1002427640028ac2b403c0113f2c200000300020000030079c14007a10000bebcdef700f1226a01000528ee01372c00000044, 0800000700000000000000af00119056e50000000012 >
0:00:00.467249035   657  0x17855b0 WARN                  flvmux gstflvmux.c:1249:gst_flv_mux_buffer_to_tag_internal:<mux:sink_0> Got backwards dts! (0:00:00.077000000 < 0:00:00.085000000)
0:00:01.643588070   657  0x17855b0 WARN                  flvmux gstflvmux.c:1249:gst_flv_mux_buffer_to_tag_internal:<mux:sink_0> Got backwards dts! (0:00:01.257000000 < 0:00:01.258000000)
0:00:01.663506935   657  0x17855b0 WARN                  flvmux gstflvmux.c:1249:gst_flv_mux_buffer_to_tag_internal:<mux:sink_0> Got backwards dts! (0:00:01.277000000 < 0:00:01.280000000)
0:00:01.683211010   657  0x17855b0 WARN                  flvmux gstflvmux.c:1249:gst_flv_mux_buffer_to_tag_internal:<mux:sink_0> Got backwards dts! (0:00:01.297000000 < 0:00:01.301000000)
0:00:01.704716650   657  0x17855b0 WARN                  flvmux gstflvmux.c:1249:gst_flv_mux_buffer_to_tag_internal:<mux:sink_0> Got backwards dts! (0:00:01.317000000 < 0:00:01.322000000)
0:00:01.725635453   657  0x17855b0 WARN                  flvmux gstflvmux.c:1249:gst_flv_mux_buffer_to_tag_internal:<mux:sink_0> Got backwards dts! (0:00:01.337000000 < 0:00:01.344000000)
0:00:01.747118705   657  0x17855b0 WARN                  flvmux gstflvmux.c:1249:gst_flv_mux_buffer_to_tag_internal:<mux:sink_0> Got backwards dts! (0:00:01.357000000 < 0:00:01.365000000)
0:00:01.768637512   657  0x17855b0 WARN                  flvmux gstflvmux.c:1249:gst_flv_mux_buffer_to_tag_internal:<mux:sink_0> Got backwards dts! (0:00:01.377000000 < 0:00:01.386000000)
0:00:02.603008661   657  0x17855b0 WARN                  flvmux gstflvmux.c:1249:gst_flv_mux_buffer_to_tag_internal:<mux:sink_0> Got backwards dts! (0:00:02.217000000 < 0:00:02.218000000)
0:00:02.622900273   657  0x17855b0 WARN                  flvmux gstflvmux.c:1249:gst_flv_mux_buffer_to_tag_internal:<mux:sink_0> Got backwards dts! (0:00:02.237000000 < 0:00:02.240000000)
0:00:02.643186542   657  0x17855b0 WARN                  flvmux gstflvmux.c:1249:gst_flv_mux_buffer_to_tag_internal:<mux:sink_0> Got backwards dts! (0:00:02.257000000 < 0:00:02.261000000)
0:00:02.664444731   657  0x17855b0 WARN                  flvmux gstflvmux.c:1249:gst_flv_mux_buffer_to_tag_internal:<mux:sink_0> Got backwards dts! (0:00:02.277000000 < 0:00:02.282000000)
0:00:02.686008117   657  0x17855b0 WARN                  flvmux gstflvmux.c:1249:gst_flv_mux_buffer_to_tag_internal:<mux:sink_0> Got backwards dts! (0:00:02.297000000 < 0:00:02.304000000)
0:00:02.707171549   657  0x17855b0 WARN                  flvmux gstflvmux.c:1249:gst_flv_mux_buffer_to_tag_internal:<mux:sink_0> Got backwards dts! (0:00:02.317000000 < 0:00:02.325000000)
0:00:02.728352684   657  0x17855b0 WARN                  flvmux gstflvmux.c:1249:gst_flv_mux_buffer_to_tag_internal:<mux:sink_0> Got backwards dts! (0:00:02.337000000 < 0:00:02.346000000)
0:00:02.923070730   657  0x17855b0 WARN                  flvmux gstflvmux.c:1249:gst_flv_mux_buffer_to_tag_internal:<mux:sink_0> Got backwards dts! (0:00:02.537000000 < 0:00:02.538000000)
0:00:02.943205175   657  0x17855b0 WARN                  flvmux gstflvmux.c:1249:gst_flv_mux_buffer_to_tag_internal:<mux:sink_0> Got backwards dts! (0:00:02.557000000 < 0:00:02.560000000)
0:00:02.963017591   657  0x17855b0 WARN                  flvmux gstflvmux.c:1249:gst_flv_mux_buffer_to_tag_internal:<mux:sink_0> Got backwards dts! (0:00:02.577000000 < 0:00:02.581000000)
0:00:02.984373024   657  0x17855b0 WARN                  flvmux gstflvmux.c:1249:gst_flv_mux_buffer_to_tag_internal:<mux:sink_0> Got backwards dts! (0:00:02.597000000 < 0:00:02.602000000)
0:00:03.005714642   657  0x17855b0 WARN                  flvmux gstflvmux.c:1249:gst_flv_mux_buffer_to_tag_internal:<mux:sink_0> Got backwards dts! (0:00:02.617000000 < 0:00:02.624000000)
0:00:03.027014799   657  0x17855b0 WARN                  flvmux gstflvmux.c:1249:gst_flv_mux_buffer_to_tag_internal:<mux:sink_0> Got backwards dts! (0:00:02.637000000 < 0:00:02.645000000)
0:00:03.048326918   657  0x17855b0 WARN                  flvmux gstflvmux.c:1249:gst_flv_mux_buffer_to_tag_internal:<mux:sink_0> Got backwards dts! (0:00:02.657000000 < 0:00:02.666000000)
0:00:03.882864260   657  0x17855b0 WARN                  flvmux gstflvmux.c:1249:gst_flv_mux_buffer_to_tag_internal:<mux:sink_0> Got backwards dts! (0:00:03.497000000 < 0:00:03.498000000)
0:00:03.903079833   657  0x17855b0 WARN                  flvmux gstflvmux.c:1249:gst_flv_mux_buffer_to_tag_internal:<mux:sink_0> Got backwards dts! (0:00:03.517000000 < 0:00:03.520000000)
0:00:03.923214244   657  0x17855b0 WARN                  flvmux gstflvmux.c:1249:gst_flv_mux_buffer_to_tag_internal:<mux:sink_0> Got backwards dts! (0:00:03.537000000 < 0:00:03.541000000)
0:00:03.944373150   657  0x17855b0 WARN                  flvmux gstflvmux.c:1249:gst_flv_mux_buffer_to_tag_internal:<mux:sink_0> Got backwards dts! (0:00:03.557000000 < 0:00:03.562000000)
0:00:03.966017100   657  0x17855b0 WARN                  flvmux gstflvmux.c:1249:gst_flv_mux_buffer_to_tag_internal:<mux:sink_0> Got backwards dts! (0:00:03.577000000 < 0:00:03.584000000)
0:00:03.986981566   657  0x17855b0 WARN                  flvmux gstflvmux.c:1249:gst_flv_mux_buffer_to_tag_internal:<mux:sink_0> Got backwards dts! (0:00:03.597000000 < 0:00:03.605000000)
0:00:04.008608572   657  0x17855b0 WARN                  flvmux gstflvmux.c:1249:gst_flv_mux_buffer_to_tag_internal:<mux:sink_0> Got backwards dts! (0:00:03.617000000 < 0:00:03.626000000)
0:00:04.203051184   657  0x17855b0 WARN                  flvmux gstflvmux.c:1249:gst_flv_mux_buffer_to_tag_internal:<mux:sink_0> Got backwards dts! (0:00:03.817000000 < 0:00:03.818000000)
0:00:04.222910141   657  0x17855b0 WARN                  flvmux gstflvmux.c:1249:gst_flv_mux_buffer_to_tag_internal:<mux:sink_0> Got backwards dts! (0:00:03.837000000 < 0:00:03.840000000)
0:00:04.243144495   657  0x17855b0 WARN                  flvmux gstflvmux.c:1249:gst_flv_mux_buffer_to_tag_internal:<mux:sink_0> Got backwards dts! (0:00:03.857000000 < 0:00:03.861000000)
0:00:04.264481508   657  0x17855b0 WARN                  flvmux gstflvmux.c:1249:gst_flv_mux_buffer_to_tag_internal:<mux:sink_0> Got backwards dts! (0:00:03.877000000 < 0:00:03.882000000)
0:00:04.285839243   657  0x17855b0 WARN                  flvmux gstflvmux.c:1249:gst_flv_mux_buffer_to_tag_internal:<mux:sink_0> Got backwards dts! (0:00:03.897000000 < 0:00:03.904000000)
0:00:04.307207403   657  0x17855b0 WARN                  flvmux gstflvmux.c:1249:gst_flv_mux_buffer_to_tag_internal:<mux:sink_0> Got backwards dts! (0:00:03.917000000 < 0:00:03.925000000)
0:00:04.328475233   657  0x17855b0 WARN                  flvmux gstflvmux.c:1249:gst_flv_mux_buffer_to_tag_internal:<mux:sink_0> Got backwards dts! (0:00:03.937000000 < 0:00:03.946000000)
0:00:05.203172337   657  0x17855b0 WARN                  flvmux gstflvmux.c:1249:gst_flv_mux_buffer_to_tag_internal:<mux:sink_0> Got backwards dts! (0:00:04.817000000 < 0:00:04.821000000)
^Chandling interrupt.
Interrupt: Stopping pipeline ...
Execution ended after 0:00:05.327783635
Setting pipeline to NULL ...
Freeing pipeline ...

carldani
Posts: 50
Joined: Wed Jan 11, 2012 12:08 am

Re: TC358743 HDMI to CSI-2 install instructions

Tue Aug 03, 2021 9:02 am

natttyyy wrote:
Thu Jul 29, 2021 2:08 pm
I have been trying to just grab the HDMI input, mux it with audio and send it to an nginx server on the pi so that others can stream from it. Sound easy enough and have come up with this pipeline

Code: Select all

gst-launch-1.0 -vvv v4l2src ! "video/x-raw,format=UYVY,framerate=30/1" ! v4l2h264enc extra-controls="controls,h264_profile=4,h264_level=13,video_bitrate=256000;" ! "video/x-h264,profile=high,level=(string)4.2" ! h264parse ! queue ! flvmux streamable=true name=mux ! rtmpsink location="rtmp://localhost/live/test" alsasrc device=hw:2 ! audio/x-raw,rate=48000,channels=2 ! audioconvert ! avenc_aac bitrate=44100 ! aacparse ! queue ! mux.

But somehow, the audio always plays first and the video is out of sync by as bad as 30s some times, audio always leads video.
I'm also working on a project grabbing HDMI input and streaming it over RTMP with GStreamer.

Two possible problems:
- v4l2h264enc in GStreamer 1.14 has issues. I never got it to run in a stable fashion, so I switched to GStreamer 1.19.
- Try setting the latency for v4l2src. That may or may not help.

I've been building latest GStreamer 1.19 from git for the last few weeks. I can share the resulting debs if you're interested. Centricular has a nice build script which I adapted it to work on Raspberry Pi OS (32bit). Please note that those GStreamer builds are mostly untested, I use them for tracking which of my bugs are fixed. Those builds may also be excessively slow and OMX is not supported. That said, they work reasonably well for my use case.

Out of curiousity, why did you specify a bitrate of 44100 bit/s for avenc_aac? This looks like it was intended as sampling rate, not as bitrate.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 11628
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: TC358743 HDMI to CSI-2 install instructions

Tue Aug 03, 2021 10:19 am

Backporting the FFmpeg and GStreamer fixes to actually make V4L2 codecs work on Buster has been on my long list of jobs for a while, but it's never made it to the top. I suspect Bullseye may happen first.
There's no requirement for it to be an Raspberry Pi employee to create the required patches. If anyone else felt so inclined then backports such as that would be very welcome.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

Bundan
Posts: 26
Joined: Fri Dec 18, 2020 9:25 am

Re: TC358743 HDMI to CSI-2 install instructions

Sat Sep 04, 2021 7:35 am

some questions

I would like to try to reproduce this project viewtopic.php?f=38&t=279612&start=25#p1826483

but capturing in RGB, because I noticed that some shades and other small details are worse in UYVY

I would remember that there is no conversion to h264, the video is viewed as it is with this line

Code: Select all

gst-launch-1.0 -vvvv v4l2src io-mode=4 ! video/x-raw,framerate=50/1,format=UYVY ! v4l2convert output-io-mode=5 capture-io-mode=4 ! kmssink sync=false
then there are some psnr comparisons with imagemagik to analyze small portions of the picture

since cpu usages during the process is at most 25% and the new project is based by capturing 1080p50 in RGB, so there is no need anymore to use "v4l2convert" and it could save cpu usage, do you think it would be possible to use a raspberrypi zero WH?

then, I noticed that there is a lot of excitement among the Chinese versions of the hdmi to csi2 modules, I focused on this product in the c780b version https://it.aliexpress.com/item/1005002861310912.html

do you think in theory it could replicate the performance of the auvidea b102 with audio?

last question is about the current situation of RGB captured via the module camera, I did a test with my auvidea b101 at 720p@50 RGB

Code: Select all

v4l2-ctl -v pixelformat=RGB
gst-launch-1.0 -vvvv v4l2src ! kmssink sync=false
and the red and blue are still swapped, is there any solution at the moment?

Return to “Graphics, sound and multimedia”