breakout34
Posts: 7
Joined: Wed Aug 25, 2021 9:03 am

Re: TC358743 HDMI to CSI-2 install instructions

Mon Sep 13, 2021 1:34 pm

Hello 6by9. First of all thank you.

I came to this post by your reply to viewtopic.php?f=38&t=319421&p=1912181#p1912181

I tried everything as you have instructed here. But I have this error: ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Failed to allocate required memory.

I also looked at whole web to solve this problem. But no. :(

I am really stucked on it and I am ready to pay to anyone who could help me about this.


dmesg | grep cma
cma: Reserved 256 MiB at 0x000000001ec00000

-----------------
v4l2-ctl --set-edid=file=1080P30EDID.txt --fix-edid-checksums

CTA-861 Header
IT Formats Underscanned: yes
Audio: yes
YCbCr 4:4:4: no
YCbCr 4:2:2: no

HDMI Vendor-Specific Data Block
Physical Address: 3.0.0.0
YCbCr 4:4:4 Deep Color: no
30-bit: no
36-bit: no
48-bit: no

CTA-861 Video Capability Descriptor
RGB Quantization Range: yes
YCC Quantization Range: no
PT: Supports both over- and underscan
IT: Supports both over- and underscan
CE: Supports both over- and underscan




-------------
v4l2-ctl --query-dv-timings
Active width: 1920
Active height: 1080
Total width: 2200
Total height: 1125
Frame format: progressive
Polarities: -vsync -hsync
Pixelclock: 148500000 Hz (60.00 frames per second)
Horizontal frontporch: 0
Horizontal sync: 280
Horizontal backporch: 0
Vertical frontporch: 0
Vertical sync: 45
Vertical backporch: 0
Standards:
Flags:



------
v4l2-ctl --set-dv-bt-timings query
BT timings set



-------
v4l2-ctl -V
Format Video Capture:
Width/Height : 1920/1080
Pixel Format : 'UYVY' (UYVY 4:2:2)
Field : None
Bytes per Line : 3840
Size Image : 4147200
Colorspace : SMPTE 170M
Transfer Function : Default (maps to Rec. 709)
YCbCr/HSV Encoding: Default (maps to ITU-R 601)
Quantization : Default (maps to Limited Range)
Flags :

----------------------
v4l2-ctl -v pixelformat=UYVY
---------------
gst-launch-1.0 v4l2src ! "video/x-raw,framerate=30/1,format=UYVY" ! v4l2h264enc extra-controls="controls,h264_profile=4,h264_level=10,video_bitrate=256000;" ! video/x-h264,profile=high ! h264parse ! queue ! matroskamux ! filesink location=foo.mkv
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Redistribute latency...
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Failed to allocate required memory.
Additional debug info:
gstv4l2src.c(658): gst_v4l2src_decide_allocation (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
Buffer pool activation failed
Execution ended after 0:00:00.048726509
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
Setting pipeline to NULL ...
Freeing pipeline ...


-----------------
v4l2-ctl --log-status

Status Log:

[ 6919.841148] unicam fe801000.csi: ================= START STATUS =================
[ 6919.845261] tc358743 10-000f: -----Chip status-----
[ 6919.846787] tc358743 10-000f: Chip ID: 0x00
[ 6919.848311] tc358743 10-000f: Chip revision: 0x00
[ 6919.848329] tc358743 10-000f: Reset: IR: 1, CEC: 1, CSI TX: 0, HDMI: 0
[ 6919.848345] tc358743 10-000f: Sleep mode: off
[ 6919.848360] tc358743 10-000f: Cable detected (+5V power): yes
[ 6919.849657] tc358743 10-000f: DDC lines enabled: yes
[ 6919.850954] tc358743 10-000f: Hotplug enabled: yes
[ 6919.852477] tc358743 10-000f: CEC enabled: no
[ 6919.852492] tc358743 10-000f: -----Signal status-----
[ 6919.852507] tc358743 10-000f: TMDS signal detected: yes
[ 6919.852521] tc358743 10-000f: Stable sync signal: yes
[ 6919.852536] tc358743 10-000f: PHY PLL locked: yes
[ 6919.852551] tc358743 10-000f: PHY DE detected: yes
[ 6919.869230] tc358743 10-000f: Detected format: 1920x1080p60.00 (2200x1125)
[ 6919.869249] tc358743 10-000f: horizontal: fp = 0, -sync = 280, bp = 0
[ 6919.869265] tc358743 10-000f: vertical: fp = 0, -sync = 45, bp = 0
[ 6919.869281] tc358743 10-000f: pixelclock: 148500000
[ 6919.869298] tc358743 10-000f: flags (0x0):
[ 6919.869314] tc358743 10-000f: standards (0x0):
[ 6919.869334] tc358743 10-000f: Configured format: 1920x1080p60.00 (2200x1125)
[ 6919.869350] tc358743 10-000f: horizontal: fp = 0, -sync = 280, bp = 0
[ 6919.869366] tc358743 10-000f: vertical: fp = 0, -sync = 45, bp = 0
[ 6919.869382] tc358743 10-000f: pixelclock: 148500000
[ 6919.869398] tc358743 10-000f: flags (0x0):
[ 6919.869414] tc358743 10-000f: standards (0x0):
[ 6919.869428] tc358743 10-000f: -----CSI-TX status-----
[ 6919.869444] tc358743 10-000f: Lanes needed: 3
[ 6919.869459] tc358743 10-000f: Lanes in use: 3
[ 6919.870982] tc358743 10-000f: Waiting for particular sync signal: no
[ 6919.872505] tc358743 10-000f: Transmit mode: no
[ 6919.874026] tc358743 10-000f: Receive mode: no
[ 6919.875552] tc358743 10-000f: Stopped: no
[ 6919.875568] tc358743 10-000f: Color space: YCbCr 422 16-bit
[ 6919.876866] tc358743 10-000f: -----DVI-D status-----
[ 6919.876881] tc358743 10-000f: HDCP encrypted content: no
[ 6919.876897] tc358743 10-000f: Input color space: RGB full range
[ 6919.878194] unicam fe801000.csi: -----Receiver status-----
[ 6919.878211] unicam fe801000.csi: V4L2 width/height: 1920x1080
[ 6919.878227] unicam fe801000.csi: Mediabus format: 0000200f
[ 6919.878242] unicam fe801000.csi: V4L2 format: 59565955
[ 6919.878258] unicam fe801000.csi: Unpacking/packing: 0 / 0
[ 6919.878272] unicam fe801000.csi: ----Live data----
[ 6919.878287] unicam fe801000.csi: Programmed stride: 0
[ 6919.878303] unicam fe801000.csi: Detected resolution: 0x0
[ 6919.878318] unicam fe801000.csi: Write pointer: 00000000
[ 6919.878336] unicam fe801000.csi: ================== END STATUS ==================

Bundan
Posts: 26
Joined: Fri Dec 18, 2020 9:25 am

Re: TC358743 HDMI to CSI-2 install instructions

Mon Sep 13, 2021 3:01 pm

Pixelclock: 148500000 Hz (60.00 frames per second)

[ 6919.869230] tc358743 10-000f: Detected format: 1920x1080p60.00 (2200x1125)
the problem is that you are trying to capture an hdmi output at 1080p@60, the auvidea b101 can handle max 1080p@50 UYVY

you have to lower the refresh to 30hz or the resolution to 720p, but this on the device with hdmi output you are trying to capture, the raspberry cannot do a down-refresh or down-scaling of the signal that you are providing to capture

breakout34
Posts: 7
Joined: Wed Aug 25, 2021 9:03 am

Re: TC358743 HDMI to CSI-2 install instructions

Mon Sep 13, 2021 3:17 pm

Hello Bundan. Thank you for answer. Unfortunately I can't change resolution or change refresh since I have no soft-access to Source HDMI Computer. Is there any other thing I can do?

In fact there is a project which works with such refresh rate by B101. and I can't learn how they did it since they are my competitor on this project. :(
Bundan wrote:
Mon Sep 13, 2021 3:01 pm
Pixelclock: 148500000 Hz (60.00 frames per second)

[ 6919.869230] tc358743 10-000f: Detected format: 1920x1080p60.00 (2200x1125)
the problem is that you are trying to capture an hdmi output at 1080p@60, the auvidea b101 can handle max 1080p@50 UYVY

you have to lower the refresh to 30hz or the resolution to 720p, but this on the device with hdmi output you are trying to capture, the raspberry cannot do a down-refresh or down-scaling of the signal that you are providing to capture

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 11650
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: TC358743 HDMI to CSI-2 install instructions

Mon Sep 13, 2021 3:54 pm

breakout34 wrote:
Mon Sep 13, 2021 3:17 pm
Hello Bundan. Thank you for answer. Unfortunately I can't change resolution or change refresh since I have no soft-access to Source HDMI Computer. Is there any other thing I can do?

In fact there is a project which works with such refresh rate by B101. and I can't learn how they did it since they are my competitor on this project. :(
You should be loading an EDID into the B101 which tells the HDMI source device what resolutions and frame rates are supported. If the source chooses to ignore that then it isn't complying with the HDMI spec. If you advertise 1080p60 in that EDID then it is your problem.

There is no way of capturing 1080p60 using a B101. You need to use a B102 and Compute Module to achieve 1080p60.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

breakout34
Posts: 7
Joined: Wed Aug 25, 2021 9:03 am

Re: TC358743 HDMI to CSI-2 install instructions

Mon Sep 13, 2021 5:48 pm

Thank you for your support and patience. I really appreciate it and thank you so much.

I loaded 50P. And by status it seems; Detected format: 1920x1080p50.00 (2640x1125) now. I guess it should be okay?

But I still get this error. What could go wrong? :(


gst-launch-1.0 v4l2src ! "video/x-raw,framerate=30/1,format=UYVY" ! v4l2h264enc extra-controls="controls,h264_profile=4,h264_level=10,video_bitrate=256000;" ! video/x-h264,profile=high ! h264parse ! queue ! matroskamux ! filesink location=foo.mkv

Code: Select all

Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Redistribute latency...
ERROR: from element /GstPipeline:pipeline0/v4l2h264enc:v4l2h264enc0: Failed to process frame.
Additional debug info:
gstv4l2videoenc.c(803): gst_v4l2_video_enc_handle_frame (): /GstPipeline:pipeline0/v4l2h264enc:v4l2h264enc0:
Maybe be due to not enough memory or failing driver
Execution ended after 0:00:00.108857721
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
Setting pipeline to NULL ...
Freeing pipeline ...




v4l2-ctl --set-edid=file=1080P50EDID.txt --fix-edid-checksums

Code: Select all

CTA-861 Header
  IT Formats Underscanned: yes
  Audio:                   yes
  YCbCr 4:4:4:             no
  YCbCr 4:2:2:             no

HDMI Vendor-Specific Data Block
  Physical Address:        3.0.0.0
  YCbCr 4:4:4 Deep Color:  no
  30-bit:                  no
  36-bit:                  no
  48-bit:                  no

CTA-861 Video Capability Descriptor
  RGB Quantization Range:  yes
  YCC Quantization Range:  no
  PT:                      Supports both over- and underscan
  IT:                      Supports both over- and underscan
  CE:                      Supports both over- and underscan




and status log;
v4l2-ctl --log-status

Code: Select all

Status Log:

   [  267.279143] unicam fe801000.csi: =================  START STATUS  =================
   [  267.282488] tc358743 10-000f: -----Chip status-----
   [  267.283718] tc358743 10-000f: Chip ID: 0x00
   [  267.284946] tc358743 10-000f: Chip revision: 0x00
   [  267.284964] tc358743 10-000f: Reset: IR: 1, CEC: 1, CSI TX: 0, HDMI: 0
   [  267.284980] tc358743 10-000f: Sleep mode: off
   [  267.285056] tc358743 10-000f: Cable detected (+5V power): yes
   [  267.286119] tc358743 10-000f: DDC lines enabled: yes
   [  267.287165] tc358743 10-000f: Hotplug enabled: yes
   [  267.288393] tc358743 10-000f: CEC enabled: no
   [  267.288408] tc358743 10-000f: -----Signal status-----
   [  267.288423] tc358743 10-000f: TMDS signal detected: yes
   [  267.288438] tc358743 10-000f: Stable sync signal: yes
   [  267.288453] tc358743 10-000f: PHY PLL locked: yes
   [  267.288467] tc358743 10-000f: PHY DE detected: yes
   [  267.301892] tc358743 10-000f: Detected format: 1920x1080p50.00 (2640x1125)
   [  267.301910] tc358743 10-000f: horizontal: fp = 0, -sync = 720, bp = 0
   [  267.301927] tc358743 10-000f: vertical: fp = 0, -sync = 45, bp = 0
   [  267.301943] tc358743 10-000f: pixelclock: 148500000
   [  267.301961] tc358743 10-000f: flags (0x0):
   [  267.301977] tc358743 10-000f: standards (0x0):
   [  267.301996] tc358743 10-000f: Configured format: 1920x1080p50.00 (2640x1125)
   [  267.302012] tc358743 10-000f: horizontal: fp = 0, -sync = 720, bp = 0
   [  267.302028] tc358743 10-000f: vertical: fp = 0, -sync = 45, bp = 0
   [  267.302044] tc358743 10-000f: pixelclock: 148500000
   [  267.302060] tc358743 10-000f: flags (0x0):
   [  267.302076] tc358743 10-000f: standards (0x0):
   [  267.302091] tc358743 10-000f: -----CSI-TX status-----
   [  267.302106] tc358743 10-000f: Lanes needed: 2
   [  267.302121] tc358743 10-000f: Lanes in use: 2
   [  267.303349] tc358743 10-000f: Waiting for particular sync signal: no
   [  267.304576] tc358743 10-000f: Transmit mode: no
   [  267.305809] tc358743 10-000f: Receive mode: no
   [  267.307035] tc358743 10-000f: Stopped: no
   [  267.307051] tc358743 10-000f: Color space: YCbCr 422 16-bit
   [  267.308097] tc358743 10-000f: -----HDMI status-----
   [  267.308113] tc358743 10-000f: HDCP encrypted content: no
   [  267.308129] tc358743 10-000f: Input color space: RGB limited range
   [  267.309177] tc358743 10-000f: AV Mute: off
   [  267.310227] tc358743 10-000f: Deep color mode: 8-bits per channel
   [  267.315202] tc358743 10-000f: HDMI infoframe: Auxiliary Video Information (AVI), version 2, length 13
   [  267.315220] tc358743 10-000f:     colorspace: RGB
   [  267.315237] tc358743 10-000f:     scan mode: Underscan
   [  267.315254] tc358743 10-000f:     colorimetry: ITU601
   [  267.315270] tc358743 10-000f:     picture aspect: 16:9
   [  267.315287] tc358743 10-000f:     active aspect: Same as Picture
   [  267.315303] tc358743 10-000f:     itc: No Data
   [  267.315320] tc358743 10-000f:     extended colorimetry: xvYCC 601
   [  267.315336] tc358743 10-000f:     quantization range: Default
   [  267.315352] tc358743 10-000f:     nups: Unknown Non-uniform Scaling
   [  267.315369] tc358743 10-000f:     video code: 31
   [  267.315385] tc358743 10-000f:     ycc quantization range: Limited
   [  267.315402] tc358743 10-000f:     hdmi content type: Graphics
   [  267.315418] tc358743 10-000f:     pixel repeat: 0
   [  267.315436] tc358743 10-000f:     bar top 0, bottom 0, left 0, right 0
   [  267.315452] unicam fe801000.csi: -----Receiver status-----
   [  267.315468] unicam fe801000.csi: V4L2 width/height:   1920x1080
   [  267.315484] unicam fe801000.csi: Mediabus format:     0000200f
   [  267.315499] unicam fe801000.csi: V4L2 format:         59565955
   [  267.315515] unicam fe801000.csi: Unpacking/packing:   0 / 0
   [  267.315529] unicam fe801000.csi: ----Live data----
   [  267.315545] unicam fe801000.csi: Programmed stride:   3840
   [  267.315561] unicam fe801000.csi: Detected resolution: 0x0
   [  267.315576] unicam fe801000.csi: Write pointer:       f5700000
   [  267.315591] unicam fe801000.csi: ==================  END STATUS  ==================


Bundan
Posts: 26
Joined: Fri Dec 18, 2020 9:25 am

Re: TC358743 HDMI to CSI-2 install instructions

Mon Sep 13, 2021 7:43 pm

since you are capturing at 50fps i think you should specify level=(string)4.2, this line should fix your issue,

adding io-mode etc should also save cpu usages

Code: Select all

gst-launch-1.0 v4l2src io-mode=4 ! "video/x-raw,framerate=50/1,format=UYVY" ! \
  v4l2h264enc output-io-mode=5 capture-io-mode=4 extra-controls="controls,h264_profile=4,h264_level=10,video_bitrate=256000;" ! \
  "video/x-h264,profile=high,level=(string)4.2" ! h264parse ! queue ! matroskamux ! filesink location=foo.mkv

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 11650
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: TC358743 HDMI to CSI-2 install instructions

Tue Sep 14, 2021 6:35 am

Bundan wrote:
Mon Sep 13, 2021 7:43 pm
since you are capturing at 50fps i think you should specify level=(string)4.2, this line should fix your issue,

adding io-mode etc should also save cpu usages

Code: Select all

gst-launch-1.0 v4l2src io-mode=4 ! "video/x-raw,framerate=50/1,format=UYVY" ! \
  v4l2h264enc output-io-mode=5 capture-io-mode=4 extra-controls="controls,h264_profile=4,h264_level=10,video_bitrate=256000;" ! \
  "video/x-h264,profile=high,level=(string)4.2" ! h264parse ! queue ! matroskamux ! filesink location=foo.mkv
Memory says that parsing the caps for the level came in about Gstreamer 1.16, but Buster is on 1.14. Its the h264_level parameter in extra-controls that sets it, which probably needs to go up to 12 (v4l2-ctl --list-ctrls-menu will give you the meaning of each step)
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

Bundan
Posts: 26
Joined: Fri Dec 18, 2020 9:25 am

Re: TC358743 HDMI to CSI-2 install instructions

Tue Sep 14, 2021 8:11 am

6by9 wrote:
Tue Sep 14, 2021 6:35 am
Memory says that parsing the caps for the level came in about Gstreamer 1.16, but Buster is on 1.14. Its the h264_level parameter in extra-controls that sets it, which probably needs to go up to 12 (v4l2-ctl --list-ctrls-menu will give you the meaning of each step)
you right i'm on GStreamer 1.19.1, I had to rebuilt kmssink because it was failing to shows 50fps for a bug with vsync or something like that

@breakout34

you should go up from h264_level=10 to h264_level=12 as @6by9 said

p.s.

@6by9

I was trying to compare the images captured in RGB and UYVY, is it confirmed that at the moment in RGB red and blue are swapped?

Image

is there a way to fix that after in gstreamer?

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 11650
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: TC358743 HDMI to CSI-2 install instructions

Tue Sep 14, 2021 8:37 am

Bundan wrote:
Tue Sep 14, 2021 8:11 am
I was trying to compare the images captured in RGB and UYVY, is it confirmed that at the moment in RGB red and blue are swapped?

Image

is there a way to fix that after in gstreamer?
I really ought to get a hex analyser out on the data to confirm who is in the wrong.

V4L2 defines the byte layout in memory - https://www.kernel.org/doc/html/latest/ ... t-rgb.html
When bringing up TC358743 via Unicam I was using qv4l2 to display the images, and with the mapping at https://github.com/raspberrypi/linux/bl ... cam.c#L222 between MEDIA_BUS and V4L2 formats I got the right colours. Now it is quite possible that qv4l2 was getting it wrong, but we now have all the other codec and conversion components following that same assignment so I really hope that isn't the case.

kmssink has a mapping table at https://github.com/GStreamer/gst-plugin ... tils.c#L45 which changes based on endianness. I don't know what sets that G_BYTE_ORDER define up within GStreamer, and is it correct on the Pi.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

breakout34
Posts: 7
Joined: Wed Aug 25, 2021 9:03 am

Re: TC358743 HDMI to CSI-2 install instructions

Tue Sep 14, 2021 8:45 am

Thank you so much 6by9 and Bundan.

I can capture video to file by this command now. Ura! but I have no idea to how to stream it to by RTMP Server.

only changing rtmpsink with filesink, it wasn't enough. I guess I should do some convertions, like "! rtmpsink location='rtmp://streaming.sandsadsadsads920'"

Code: Select all

gst-launch-1.0 v4l2src device=/dev/video0  ! videocrop top=0 left=0 right=1200 bottom=0 ! videoflip method=counterclockwise ! 'video/x-raw,framerate=30/1,format=UYVY' ! v4l2h264enc ! 'video/x-h264,level=(string)4' ! filesink location = dirrik3.h264

Bundan
Posts: 26
Joined: Fri Dec 18, 2020 9:25 am

Re: TC358743 HDMI to CSI-2 install instructions

Tue Sep 14, 2021 9:36 am

breakout34 wrote:
Tue Sep 14, 2021 8:45 am
Thank you so much 6by9 and Bundan.

I can capture video to file by this command now. Ura! but I have no idea to how to stream it to by RTMP Server.

only changing rtmpsink with filesink, it wasn't enough. I guess I should do some convertions, like "! rtmpsink location='rtmp://streaming.sandsadsadsads920'"

Code: Select all

gst-launch-1.0 v4l2src device=/dev/video0  ! videocrop top=0 left=0 right=1200 bottom=0 ! videoflip method=counterclockwise ! 'video/x-raw,framerate=30/1,format=UYVY' ! v4l2h264enc ! 'video/x-h264,level=(string)4' ! filesink location = dirrik3.h264
you have to setup a NGINX server first, this is one of the million of guide that you can find with google https://www.okdo.com/project/streaming-server/

then this is my line with audio and video, but remember that i'm on gstreamer 1.19.1 you have to use h264_level instead of ",level=(string)4.2"

Code: Select all

gst-launch-1.0 -vvv flvmux streamable=true name=mux ! rtmpsink location="rtmp://localhost/live live=1" \
  v4l2src io-mode=4 ! video/x-raw,framerate=50/1,format=UYVY ! \
  v4l2h264enc output-io-mode=5 capture-io-mode=4 extra-controls="controls,h264_profile=4,h264_level=13,video_bitrate=5M" ! \
  "video/x-h264,profile=high,level=(string)4.2" ! h264parse ! queue ! mux. \
  alsasrc ! audio/x-raw,rate=48000,channels=2 ! \
  audioconvert ! voaacenc bitrate=48000 ! aacparse ! queue ! mux.

Bundan
Posts: 26
Joined: Fri Dec 18, 2020 9:25 am

Re: TC358743 HDMI to CSI-2 install instructions

Tue Sep 14, 2021 12:20 pm

6by9 wrote:
Tue Sep 14, 2021 8:37 am
I really ought to get a hex analyser out on the data to confirm who is in the wrong.

V4L2 defines the byte layout in memory - https://www.kernel.org/doc/html/latest/ ... t-rgb.html
When bringing up TC358743 via Unicam I was using qv4l2 to display the images, and with the mapping at https://github.com/raspberrypi/linux/bl ... cam.c#L222 between MEDIA_BUS and V4L2 formats I got the right colours. Now it is quite possible that qv4l2 was getting it wrong, but we now have all the other codec and conversion components following that same assignment so I really hope that isn't the case.

kmssink has a mapping table at https://github.com/GStreamer/gst-plugin ... tils.c#L45 which changes based on endianness. I don't know what sets that G_BYTE_ORDER define up within GStreamer, and is it correct on the Pi.
sorry I missed your answer, but so just as workaround for my issue do you think that inverting the values of these definitions could fix the kmssink output or I'm going to mess other things?

p.s.

anyway i just noticied that these values are all inverted in the code, tonight I want to try to play with these things

DEF_FMT (ARGB8888, BGRA),
DEF_FMT (XRGB8888, BGRx),
DEF_FMT (ABGR8888, RGBA),
DEF_FMT (XBGR8888, RGBx),
DEF_FMT (BGR888, RGB),
DEF_FMT (RGB888, BGR),

DEF_FMT (ARGB8888, RGBA),
DEF_FMT (XRGB8888, RGBx),
DEF_FMT (ABGR8888, BGRA),
DEF_FMT (XBGR8888, BGRx),
DEF_FMT (BGR888, BGR),
DEF_FMT (RGB888, RGB),

breakout34
Posts: 7
Joined: Wed Aug 25, 2021 9:03 am

Re: TC358743 HDMI to CSI-2 install instructions

Tue Sep 14, 2021 12:30 pm

Hello Bundan. I use Antmedia's Streaming Server.

For example I was able to serve IP Camera stream like this

Code: Select all

ffmpeg  -i rtsp://172.16.2.63/0 -vf crop=1080:720:1080:1200 -c:v h264 -c:a:137 libvorbis -s 1080:720 -tune zerolatency -f flv rtmp://streaming.XXX.net/LiveApp/28AB872048351914078239DS

And as you know, I am able to capture video by B101 by following command

Code: Select all

gst-launch-1.0 v4l2src device=/dev/video0  ! videocrop top=0 left=0 right=1200 bottom=0 ! videoflip method=counterclockwise ! 'video/x-raw,framerate=30/1,format=UYVY' ! v4l2h264enc ! 'video/x-h264,level=(string)4' ! filesink location = temp.h264
I am totally new to gst-launch and other things like that. and I spent hours to use your code, but no result for me.

Code: Select all

gst-launch-1.0 -vvv flvmux streamable=true name=mux ! rtmpsink location="rtmp://localhost/live live=1" \
  v4l2src io-mode=4 ! video/x-raw,framerate=50/1,format=UYVY ! \
  v4l2h264enc output-io-mode=5 capture-io-mode=4 extra-controls="controls,h264_profile=4,h264_level=13,video_bitrate=5M" ! \
  "video/x-h264,profile=high,level=(string)4.2" ! h264parse ! queue ! mux. \
  alsasrc ! audio/x-raw,rate=48000,channels=2 ! \
  audioconvert ! voaacenc bitrate=48000 ! aacparse ! queue ! mux.

I modified your code. It streams to server. And its status as shown to "Broadcasting." But when I click to Antmedia's preview - player, it doesn't show anything. But after I stop streaming, AntMedia automaticly saves as MP4 and I look at saved file. It is normal. But unfortunately it doesn't show anything while I stream. Maybe I send a different format to AntMedia?

Code: Select all

gst-launch-1.0 -vvv flvmux streamable=true name=mux ! rtmpsink location="rtmp://streaming.sanaloyun.net/LiveApp/285287204835191407823920" \
  v4l2src io-mode=4 ! videocrop top=0 left=0 right=1200 bottom=0 ! videoflip method=counterclockwise ! video/x-raw,framerate=30/1,format=UYVY ! v4l2h264enc \
  ! "video/x-h264,level=(string)4" ! h264parse ! queue ! mux.
Maybe you could help me about this issue? Especially I am thinking that I will also need audio streaming together. (I have inserted a USB Soundcard to Raspberry.

Please help me. I would really appreciate and donate you / to any foundation you like as appreciating your help.

Thank you.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 11650
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: TC358743 HDMI to CSI-2 install instructions

Tue Sep 14, 2021 1:31 pm

Bundan wrote:
Tue Sep 14, 2021 12:20 pm
sorry I missed your answer, but so just as workaround for my issue do you think that inverting the values of these definitions could fix the kmssink output or I'm going to mess other things?

p.s.

anyway i just noticied that these values are all inverted in the code, tonight I want to try to play with these things

DEF_FMT (ARGB8888, BGRA),
DEF_FMT (XRGB8888, BGRx),
DEF_FMT (ABGR8888, RGBA),
DEF_FMT (XBGR8888, RGBx),
DEF_FMT (BGR888, RGB),
DEF_FMT (RGB888, BGR),

DEF_FMT (ARGB8888, RGBA),
DEF_FMT (XRGB8888, RGBx),
DEF_FMT (ABGR8888, BGRA),
DEF_FMT (XBGR8888, BGRx),
DEF_FMT (BGR888, BGR),
DEF_FMT (RGB888, RGB),
AIUI:
- DRM/KMS defines their pixel formats as how they pack into a 32bit word, hence needing to consider endianness.
- V4L2 doesn't as it defines the element that goes into each memory location.
- I don't know which approach GStreamer takes - presumably the same as V4L2 as it doesn't do an endianness mapping there.

If you can create a plain red buffer as your HDMI source, run a capture with

Code: Select all

v4l2-ctl --set-dv-bt-timings query
v4l2-ctl --stream-mmap=3 --stream-count=1 --stream-skip=3 --stream-to=file.rgb
VIew with "xxd -g 1 file.rgb" and see whether the big number is the first byte or the third.
Based on https://www.kernel.org/doc/html/latest/ ... -component I'm hoping it is the 3rd byte for red.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

Bundan
Posts: 26
Joined: Fri Dec 18, 2020 9:25 am

Re: TC358743 HDMI to CSI-2 install instructions

Tue Sep 14, 2021 2:15 pm

@6by9

I think that the result it is what you were hoping for
Image

so what now?, do you have an idea for a temp workaround in gstreamer?

p.s.

out of curiosity why does it show #000fe and not #000ff ? my picture is red 255, green 0, blue 0 and the hdmi output is set as Full RGB 4:4:4

the captured buffer reports 0,0,254

@breakout34
sorry but I have no experience with antmedia server

for the audio you can find here viewtopic.php?t=210683 the info to connect the auvidea i2s audio on raspberry pi gpio, then you have to follow the instructions on the first page of this thread to enable the audio driver

just a note, the provided audio to work must be an uncompressed 2ch PCM

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 11650
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: TC358743 HDMI to CSI-2 install instructions

Tue Sep 14, 2021 3:33 pm

Bundan wrote:
Tue Sep 14, 2021 2:15 pm
I think that the result it is what you were hoping for
Image

so what now?, do you have an idea for a temp workaround in gstreamer?
I am truly thankful for that result!
I think that kicks it back into GStreamer's court. My suggestion would be to rebuild GStreamer with the opposite mapping in gstkmsutils.c (sorry, I don't know which is the default).

DRM defines their formats as being little-endian - https://elixir.bootlin.com/linux/latest ... urcc.h#L28

Code: Select all

/*
 * DRM formats are little endian.  Define host endian variants for the
 * most common formats here, to reduce the #ifdefs needed in drivers.
 *
 * Note that the DRM_FORMAT_BIG_ENDIAN flag should only be used in
 * case the format can't be specified otherwise, so we don't end up
 * with two values describing the same format.
 */
so that really depends on what GStreamer's defines mean with regard endianess.
Bundan wrote:p.s.

out of curiosity why does it show #000fe and not #000ff ? my picture is red 255, green 0, blue 0 and the hdmi output is set as Full RGB 4:4:4

the captured buffer reports 0,0,254
Rounding.
Are you SURE you're on full range? HDMI doesn't like using full mode for CEA modes.

Even if so, I suspect that the TC358743 is passing the data through the colour conversion block though probably not needed. Register 0x8573 (VOUT_SET2) bottom 2 bits set the colour mode which is set to MASK_VOUTCOLORMODE_AUTO. Change that to MASK_VOUTCOLORMODE_THROUGH for RGB888 and I'd expect you to get 255. 254 will just be a rounding issue in the conversion block.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

Bundan
Posts: 26
Joined: Fri Dec 18, 2020 9:25 am

Re: TC358743 HDMI to CSI-2 install instructions

Tue Sep 14, 2021 4:04 pm

6by9 wrote:
Tue Sep 14, 2021 3:33 pm
Rounding.
Are you SURE you're on full range? HDMI doesn't like using full mode for CEA modes.

Even if so, I suspect that the TC358743 is passing the data through the colour conversion block though probably not needed. Register 0x8573 (VOUT_SET2) bottom 2 bits set the colour mode which is set to MASK_VOUTCOLORMODE_AUTO. Change that to MASK_VOUTCOLORMODE_THROUGH for RGB888 and I'd expect you to get 255. 254 will just be a rounding issue in the conversion block.
anyway i just rebuilt kmssink by inverting the values that you pointed out in https://github.com/GStreamer/gst-plugin ... tils.c#L45

DEF_FMT (ARGB8888, BGRA),
DEF_FMT (XRGB8888, BGRx),
DEF_FMT (ABGR8888, RGBA),
DEF_FMT (XBGR8888, RGBx),
DEF_FMT (BGR888, RGB),
DEF_FMT (RGB888, BGR),

DEF_FMT (ARGB8888, RGBA),
DEF_FMT (XRGB8888, RGBx),
DEF_FMT (ABGR8888, BGRA),
DEF_FMT (XBGR8888, BGRx),
DEF_FMT (BGR888, BGR),
DEF_FMT (RGB888, RGB),

now the colors are correct in RGB and swapped in UYVY, obviously every filter and/or *sink that I need in gstreamer should be inverted, so the fix should hit at begin when gstreamer receive the source and only with RGB

any attempt for a direct comparison between RGB and UYVY by capturing a frame is impossible

my intent was to have a perfect pass through with 0 differences but even just seeing with naked eye it seems that the captured RGB showed from the pi is a bit different from the original hdmi signal in some gradation, but better than origianl vs captured in UYVY

but to know for sure some captured frame was better for a perfect comparisons

Bundan
Posts: 26
Joined: Fri Dec 18, 2020 9:25 am

Re: TC358743 HDMI to CSI-2 install instructions

Tue Sep 14, 2021 6:52 pm

I have to test better with all the situation but a possible workaround could be with capssetter

gst-launch-1.0 -vvvv v4l2src ! video/x-raw,framerate=50/1,format=RGB ! capssetter caps="video/x-raw,framerate=50/1,format=BGR" ! kmssink sync=false

EUREKA :) it works, and finally a comparison getting a frame from the auvidea b101 at 720p50

RGB
Image

UYVY
Image

also the differences that i saw with the pi in RGB vs the original hdmi signal was due to the raspberrypi that was upscaling from 720p to 1080p

now, in RGB the gradation are smoother and also the saturation is different

so back to my first question

@6by9

I know that no one can know for sure, but I have no experience with the pi 0

will the Raspberrypi 0 WH able to run this simple line in RGB with an auvidea b102 to capture a full rgb 1080p50 or I need a compute module 4?

on top of this my prog does some psnr comparison, with a pi4 the cpu is at 20%, but in that case there is a also a UYVY to RGB conversion that by capturing directly in RGB is not needed anymore

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 11650
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: TC358743 HDMI to CSI-2 install instructions

Wed Sep 15, 2021 7:57 am

Bundan wrote:
Tue Sep 14, 2021 6:52 pm
I know that no one can know for sure, but I have no experience with the pi 0

will the Raspberrypi 0 WH able to run this simple line in RGB with an auvidea b102 to capture a full rgb 1080p50 or I need a compute module 4?
Pi0 uses the 22pin connector, but only has 2 lanes wired up. Only the Compute Modules (all varieties) exposes a 4 lane interface.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

Bundan
Posts: 26
Joined: Fri Dec 18, 2020 9:25 am

Re: TC358743 HDMI to CSI-2 install instructions

Wed Sep 15, 2021 9:41 am

6by9 wrote:
Wed Sep 15, 2021 7:57 am
Pi0 uses the 22pin connector, but only has 2 lanes wired up. Only the Compute Modules (all varieties) exposes a 4 lane interface.
this puts an end on everything, I would prefer to have the latest module and io board, but at the moment the cm4 is unavailable anywhere in my country

in addition, speaking of money, compared to a pi0 it would be perhaps an unjustified expense considering that I already have a working solution (but in UYVY: °)

thx again!

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 11650
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: TC358743 HDMI to CSI-2 install instructions

Wed Sep 15, 2021 9:45 am

Bundan wrote:
Wed Sep 15, 2021 9:41 am
6by9 wrote:
Wed Sep 15, 2021 7:57 am
Pi0 uses the 22pin connector, but only has 2 lanes wired up. Only the Compute Modules (all varieties) exposes a 4 lane interface.
this puts an end on everything, I would prefer to have the latest module and io board, but at the moment the cm4 is unavailable anywhere in my country

in addition, speaking of money, compared to a pi0 it would be perhaps an unjustified expense considering that I already have a working solution (but in UYVY: °)
If you need it as RGB, then use GStreamer's v4l2convert which uses hardware to do that conversion.
YUV is always "fun" as to which colourspace & primaries are being used - BT601, BT709, or BT2020, and full or limited range. Getting it wrong will result in slight errors in rendering the colour.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

Bundan
Posts: 26
Joined: Fri Dec 18, 2020 9:25 am

Re: TC358743 HDMI to CSI-2 install instructions

Wed Sep 15, 2021 2:18 pm

6by9 wrote:
Wed Sep 15, 2021 9:45 am
If you need it as RGB, then use GStreamer's v4l2convert which uses hardware to do that conversion.
YUV is always "fun" as to which colourspace & primaries are being used - BT601, BT709, or BT2020, and full or limited range. Getting it wrong will result in slight errors in rendering the colour.
yes, I am already using v4l2convert when I capture in UYVY

even if the pipeline written in c is a little more complex practically the main line I use is this

Code: Select all

gst-launch-1.0 -vvvv v4l2src io-mode=4 ! video/x-raw,framerate=50/1,format=UYVY ! v4l2convert output-io-mode=5 capture-io-mode=4 ! kmssink sync=false alsasrc ! audio/x-raw,rate=48000,channels=2 ! pulsesink
the differences by capturing RGB vs UYVY can be seen more on graphic elements such as menus, recently I added an hdmi switch and now both the skybox and a firestick are connected behind the auvidea b101, this because football in my country is mostly sent via streamings apps now and as you know my main purpose is to obscure the static elements as banner, scoreboard etc of the image to prevent burn in on the oled tv

anyway on the firestick menu, around the tiles, when there is some red or yellow element, etc... you can see some difference more, but anyway these are small things, which after all I already knew, we all know from millions of examples that we find on google the difference between 4: 2: 2 and 4: 4: 4, but when I bought the rpi4, I had no idea if my project was feasible, the cm4 did not exist yet and I did not know if the computing power of the rp4 compared to the cm3 could be decisive, so I mostly went for pi4 and auvidea b101


anyway this is a frame captured by the firestick and a zoom of a tile

720p RGB vs UYVY
https://screenshotcomparison.com/compar ... /picture:0

Zoomed tile RGB vs UYVY
https://screenshotcomparison.com/compar ... /picture:1

obviously on a 65" TV the differences are more evident, but they remain negligible in real world usages

p.s.

I recently also enabled the ability to stream by encoding in h264 from hdmi with the pi

with this line

Code: Select all

gst-launch-1.0 -vvv flvmux streamable=true name=mux ! rtmpsink location="rtmp://localhost/live live=1" \
  v4l2src io-mode=4 ! video/x-raw,framerate=50/1,format=UYVY ! \
  v4l2h264enc output-io-mode=5 capture-io-mode=4 extra-controls="controls,h264_profile=4,h264_level=13,video_bitrate=5M" ! \
  "video/x-h264,profile=high,level=(string)4.2" ! \
  h264parse ! queue ! mux. \
  alsasrc ! audio/x-raw,rate=48000,channels=2 ! \
  audioconvert ! voaacenc bitrate=48000 ! aacparse ! queue ! mux.

incredibly with this line I can also render with kmsink and stream h264 simultaneously, but in this case after about an hour it happens that it crashes and I have to restart the pi, but that's not the question

Code: Select all

gst-launch-1.0 -vvvv flvmux streamable=true name=mux ! rtmpsink location="rtmp://localhost/live live=1" \
  v4l2src io-mode=4 ! video/x-raw, framerate=50/1,format=UYVY ! tee name=videoTee \
  alsasrc ! audio/x-raw,rate=48000,channels=2 ! tee name=audioTee \
  videoTee. ! queue ! v4l2convert output-io-mode=5 capture-io-mode=4 ! kmssink sync=false \
  audioTee. ! queue ! pulsesink \
  videoTee. ! v4l2h264enc output-io-mode=5 capture-io-mode=4 extra-controls="controls,h264_profile=4,h264_level=13,video_bitrate=5M" ! \
  "video/x-h264,profile=high,level=(string)4.2" ! h264parse ! queue ! mux. \
  audioTee. ! audioconvert ! voaacenc bitrate=48000 ! aacparse ! queue ! mux. 
 
I have some issue by sending the raw video via udp, these are the two lines i use

send (pi)

Code: Select all

gst-launch-1.0 -vvvv v4l2src io-mode=4 ! video/x-raw,framerate=50/1,format=UYVY ! v4l2convert output-io-mode=5 capture-io-mode=4 ! rtpvrawpay ! udpsink host=192.168.1.57 port=5001
receive (windows)

Code: Select all

gst-launch-1.0 udpsrc port=5001 caps = "application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)RAW, sampling=(string)YCbCr-4:2:2, depth=(string)8, width=(string)1920, height=(string)1080, colorimetry=(string)BT601, payload=(int)96, ssrc=(uint)916954793, timestamp-offset=(uint)4142406786, seqnum-offset=(uint)27437, a-framerate=(string)50" ! rtpvrawdepay ! videoconvert ! queue ! autovideosink
when the pi sends via udpsink has the cpu at 100% and the video I receive on windows is jerky, with few frames

shouldn't it be lighter to stream without encoding and a nginx server?

Return to “Graphics, sound and multimedia”