HDMI to CSI-2 via TC358743 on kernel 4.1

[6by9]

@6by9 I am working with the B101 HDMI Input board. I am having trouble wading through this forum on how to get audio to work. What I am trying to accomplish is connecting lets say DVD player > B101 > Raspberry Pi > Computer via Ethernet I would like to take the HDMI video and Audio in through the B101 and send it out via UDP from the PI. I was trying to private message 6by9 but I am new to this forum and could not figure that out. Assistance would be great. I have enabled the camera and verified that I am getting video. Its the audio that I cant seem to find a clear answer on.

https://www.raspberrypi.org/forums/view ... 0#p1339178

PMs are disabled due to child safeguarding concerns.

[TanaPanda]

I have gone through that post you made But I am a little unclear on all of the aspects of it. My experience with the Raspberry Pi has been strictly to preform certain functions. Any chance you have some time to help me? I have teamviewer available on my Pi or step by steps would be great. Just saying i need to do something is great for someone who understands what that means. I am not so fortunate to be like oh ok and able to do it

[danergo]

If you just want a single frame in the native format of the source, then use v4l2-ctl

Code: Select all

v4l2-ctl --set-dv-bt-timings query
v4l2-ctl -v pixelformat=UYVY
v4l2-ctl --stream-mmap=3 --stream-count=1 --stream-to=foo.uyvy

You can add --skip-count=N to skip some frames at the start of the stream. Post process the frame if you want it in a different format.

6by9, I'm also hunting for a single frame. I have a TC358743-based HDMI capturer board with PI0W. I can capture H264 without any problems, but I'd like to get one still image.

I've followed your guidance and uyvy file is actually created but I have no idea on how can I convert it to jpg or png. I also tried online converters, all ran into errors. (But I guess the uyvy file is good, as the h264 video can be played back by VLC).

What's the easiest way to get a still in a jpg, png, or even bmp format? (I'm not sensitive to colors, it can be also black-and-white or grayscale).

Thank you!

[danergo]

My above issue is kindof solved: I thought my HDMI is running on 1920x1080, but this capture script uses 1280x720.

And for converting uyvy to anything you have to specialize the width and height, and with 1920x1080 it was always an early EOF error.
After setting it to 1280x720 imagemagick could convert it, but it always had a green bar at the bottom.

The perfect value for converting was 1280x690, which is very interesting. I'd be interested in this if someone know the reason

Thanks!

[globalc]

@6by9 I am working with the B101 HDMI Input board. I am having trouble wading through this forum on how to get audio to work. What I am trying to accomplish is connecting lets say DVD player > B101 > Raspberry Pi > Computer via Ethernet I would like to take the HDMI video and Audio in through the B101 and send it out via UDP from the PI. [..]

viewtopic.php?f=38&t=120702&start=400#p1339178

I have the same issue here, I read also that post (gone over the whole thread). I have no extra cables/wiring on my setup, apart of that these the config should be ok. My board is the H2C-RPI-B01, https://mzyy94.com/blog/2020/04/10/rasp ... dmi-input/ is from a user who used it (apparently including audio) without additional wiring.

- Raspi4
- Using these in /boot/config.txt:

dtparam=i2c_arm=on
dtparam=i2s=on
dtparam=spi=on
dtparam=i2c_vc=on
dtparam=audio=on
dtoverlay=vc4-fkms-v3d
dtoverlay=dwc2
dtoverlay=tc358743
dtoverlay=tc358743-audio
start_x=1
gpu_mem=128

- EDID is set, recording 720p60Hz files with yuvta works
- "v4l2-ctl --list-ctrls" reports that audio is present, bitrate 48000
- I have gone over the whole thread, tried the 5.4 kernel via "rpi-update", but no change:

Code: Select all

$ arecord -l
**** List of CAPTURE Hardware Devices ****
card 1: tc358743 [tc358743], device 0: bcm2835-i2s-dir-hifi dir-hifi-0 [bcm2835-i2s-dir-hifi dir-hifi-0]
  Subdevices: 1/1
  Subdevice #0: subdevice #0
$  arecord -D hw:1,0 -d 25 -f dat test2.wav
Recording WAVE 'test2.wav' : Signed 16 bit Little Endian, Rate 48000 Hz, Stereo
arecord: pcm_read:2145: read error: Input/output error
$ dmesg|tail -1
[  598.192045] bcm2835-i2s fe203000.i2s: I2S SYNC error!

gst-launch-1.0 exits like this:

Code: Select all

$ gst-launch-1.0 -vvv -e v4l2src ! video/x-h264,width=1280,height=720,framerate=60/1 ! h264parse ! rtph264pay ! udpsink host=192.168.0.2 port=5000
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Internal data stream error.
Additional debug info:
gstbasesrc.c(3055): gst_base_src_loop (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
streaming stopped, reason not-negotiated (-4)
ERROR: pipeline doesn't want to preroll.
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
Setting pipeline to NULL ...
Freeing pipeline ...

Ideas appreciated.

[6by9]

viewtopic.php?f=38&t=120702&start=400#p1339178

I have the same issue here, I read also that post (gone over the whole thread). I have no extra cables/wiring on my setup, apart of that these the config should be ok. My board is the H2C-RPI-B01, https://mzyy94.com/blog/2020/04/10/rasp ... dmi-input/ is from a user who used it (apparently including audio) without additional wiring.

It is possible to send the audio via CSI2 on an alternate data type, but not with any of these drivers.
Using Google translate on that blog it appears to be using raspivid or the firmware V4L2 stack - both the unsupported options, and no audio.
Both do make a monolithic thing that includes the option of H264 encoding. The supported kernel drivers don't, but H264 encoding is available via alternative means.

- Raspi4
- Using these in /boot/config.txt:

dtparam=i2c_arm=on
dtparam=i2s=on
dtparam=spi=on
dtparam=i2c_vc=on
dtparam=audio=on
dtoverlay=vc4-fkms-v3d
dtoverlay=dwc2
dtoverlay=tc358743
dtoverlay=tc358743-audio
start_x=1
gpu_mem=128

- EDID is set, recording 720p60Hz files with yuvta works
- "v4l2-ctl --list-ctrls" reports that audio is present, bitrate 48000
- I have gone over the whole thread, tried the 5.4 kernel via "rpi-update", but no change:

Code: Select all

$ arecord -l
**** List of CAPTURE Hardware Devices ****
card 1: tc358743 [tc358743], device 0: bcm2835-i2s-dir-hifi dir-hifi-0 [bcm2835-i2s-dir-hifi dir-hifi-0]
  Subdevices: 1/1
  Subdevice #0: subdevice #0
$  arecord -D hw:1,0 -d 25 -f dat test2.wav
Recording WAVE 'test2.wav' : Signed 16 bit Little Endian, Rate 48000 Hz, Stereo
arecord: pcm_read:2145: read error: Input/output error
$ dmesg|tail -1
[  598.192045] bcm2835-i2s fe203000.i2s: I2S SYNC error!

You have a wiring issue. I don't have one of the Chinese boards, only the Auvidea ones, therefore I can't really offer any assistance - please query it with your supplier.

gst-launch-1.0 exits like this:

Code: Select all

$ gst-launch-1.0 -vvv -e v4l2src ! video/x-h264,width=1280,height=720,framerate=60/1 ! h264parse ! rtph264pay ! udpsink host=192.168.0.2 port=5000
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Internal data stream error.
Additional debug info:
gstbasesrc.c(3055): gst_base_src_loop (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
streaming stopped, reason not-negotiated (-4)
ERROR: pipeline doesn't want to preroll.
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
Setting pipeline to NULL ...
Freeing pipeline ...

It will because the supported kernel drivers don't include H264 encoding as a monolithic thing. v4l2src has a choice of either UYVY or RGB3 for output with these drivers. Include v4l2h264enc in the pipeline to do the H264 encoding.

[danergo]

Hi, guys.

I have a little problem with getting one frame from the HDMI input, please help!

There is a notebook (Win10) with 2 external HDMI monitors, both are running FHD (1920x1080@60Hz).
Now I want to capture the screen from one of these monitors, so I applied for this TC358743 board, and hooked the HDMI cable to it.

So now the notebook drives one external monitor running FHD@60Hz, and drives this capture card.

I've used this EDID file from 6by9: https://github.com/6by9/CSI2_device_con ... 60EDID.txt.

Code: Select all

v4l2-ctl --fix-edid-checksums --set-edid=file=/boot/hdmi/1080P60EDID.txt
v4l2-ctl --set-dv-bt-timings query
v4l2-ctl --query-dv-timings
v4l2-ctl -v pixelformat=UYVY
v4l2-ctl --device /dev/video0 --stream-mmap --stream-to=/tmp/preview.raw --stream-count=1

As this query command reported 1280x720 resolution, I went to display settings and changed to 1920x1080:

Code: Select all

# v4l2-ctl --query-dv-timings
        Active width: 1280
        Active height: 720
        Total width: 1650
        Total height: 750
        Frame format: progressive
        Polarities: -vsync -hsync
        Pixelclock: 74250000 Hz (60.00 frames per second)
        Horizontal frontporch: 0
        Horizontal sync: 370
        Horizontal backporch: 0
        Vertical frontporch: 0
        Vertical sync: 30
        Vertical backporch: 0
        Standards:
        Flags:

After setting to FHD in Windows settings:

Code: Select all

# v4l2-ctl --query-dv-timings
        Active width: 1920
        Active height: 1080
        Total width: 2200
        Total height: 1125
        Frame format: progressive
        Polarities: -vsync -hsync
        Pixelclock: 148500000 Hz (60.00 frames per second)
        Horizontal frontporch: 0
        Horizontal sync: 280
        Horizontal backporch: 0
        Vertical frontporch: 0
        Vertical sync: 45
        Vertical backporch: 0
        Standards:
        Flags:

I considered this good.

So I captured the frame:

Code: Select all

# v4l2-ctl --device /dev/video0 --stream-mmap --stream-to=/tmp/preview.raw --stream-count=1
New timings found

Note: there is no '<' sign, and /tmp/preview.raw is 0bytes long.
Note2: Windows's display setting marks 1280x720 as 'recommended', but 1920x1080 is also on the list:


Note3: when I change back to 1280x720, I can get one frame:

Code: Select all

# v4l2-ctl --device /dev/video0 --stream-mmap --stream-to=/tmp/preview.raw --stream-count=1
New timings found
<

Then I convert the raw file with imagemagick to png:

Code: Select all

convert -size 1280x720 -depth 8 uyvy:/tmp/preview.raw /home/pi/preview.png

But the result is terrible:



This green bar is variable height, sometimes bigger, sometimes smaller, but I have not been able to get one single full size frame even in 1280x720 resolution.

My goal is to get one single FHD frame, and convert to png. How can I achieve this?
Thank you.

[6by9]

After setting to FHD in Windows settings:

Code: Select all

# v4l2-ctl --query-dv-timings
        Active width: 1920
        Active height: 1080
        Total width: 2200
        Total height: 1125
        Frame format: progressive
        Polarities: -vsync -hsync
        Pixelclock: 148500000 Hz (60.00 frames per second)
        Horizontal frontporch: 0
        Horizontal sync: 280
        Horizontal backporch: 0
        Vertical frontporch: 0
        Vertical sync: 45
        Vertical backporch: 0
        Standards:
        Flags:

I considered this good.

So I captured the frame:

Code: Select all

# v4l2-ctl --device /dev/video0 --stream-mmap --stream-to=/tmp/preview.raw --stream-count=1
New timings found

Note: there is no '<' sign, and /tmp/preview.raw is 0bytes long.

Note all the references here that say the maximum rate that can be achieved down 2 CSI-2 lanes is 1080p50 in UYVY or 1080p30 in RGB3. There will be an error in the kernel log if the source asks for more than the number of lanes present on the device (2 on all normal Pis).
You can capture 1080p60 in either format over 4 CSI-2 lanes, but that requires an Auvidea B102 or equivalent, and using CAM1 on a Compute Module (which has the 4 lanes wired out).

Note3: when I change back to 1280x720, I can get one frame:

Code: Select all

# v4l2-ctl --device /dev/video0 --stream-mmap --stream-to=/tmp/preview.raw --stream-count=1
New timings found
<

Then I convert the raw file with imagemagick to png:

Code: Select all

convert -size 1280x720 -depth 8 uyvy:/tmp/preview.raw /home/pi/preview.png

But the result is terrible:



This green bar is variable height, sometimes bigger, sometimes smaller, but I have not been able to get one single full size frame even in 1280x720 resolution.

My goal is to get one single FHD frame, and convert to png. How can I achieve this?
Thank you.

You've grabbed the first frame the chip produces. It starts sending an output as soon as it is told to start, even if the input HDMI frame is part complete, hence the vertical shift and unused image data at the bottom (green is 0,0,0 is the YUV domain).
Add "--stream-skip=1" to your v4l2-ctl command line and it'll skip the first frame. The second should be complete.

[danergo]

6by9: thanks so much. (this thread is a bit long, and just tried to hunt down the right infos)

After loading the 1080P50EDID, and adding the "stream-skip=1" I got the perfect frame.

Thank you!

[globalc]

It is possible to send the audio via CSI2 on an alternate data type, but not with any of these drivers.
Using Google translate on that blog it appears to be using raspivid or the firmware V4L2 stack - both the unsupported options, and no audio.
Both do make a monolithic thing that includes the option of H264 encoding. The supported kernel drivers don't, but H264 encoding is available via alternative means.
[..]
You have a wiring issue. I don't have one of the Chinese boards, only the Auvidea ones, therefore I can't really offer any assistance - please query it with your supplier.

> Using Google translate on that blog it appears to be using raspivid or the firmware
> V4L2 stack - both the unsupported options, and no audio.
Rereading the users posts, I think that applies. He talks in other blog entries about getting the sound from
the Nintendo Switch, his HDMI source, via usb.

Thanks a bunch, I will approach the vendor of the H2C-RPI-B01. The feature page does
not explicitly state "audio", but one could assume it as part of HDMI.

I have my notes regarding the H2C-RPI-B01 and the TC358743 at https://fluxcoil.net/hardwarerelated/ra ... 4_tc358743.

[6by9]

> Using Google translate on that blog it appears to be using raspivid or the firmware
> V4L2 stack - both the unsupported options, and no audio.
Rereading the users posts, I think that applies. He talks in other blog entries about getting the sound from
the Nintendo Switch, his HDMI source, via usb.

Thanks a bunch, I will approach the vendor of the H2C-RPI-B01. The feature page does
not explicitly state "audio", but one could assume it as part of HDMI.

I have my notes regarding the H2C-RPI-B01 and the TC358743 at https://fluxcoil.net/hardwarerelated/ra ... 4_tc358743.

I have got a couple of Chinese boards on order as I seem to have been killing B101s just recently. I'll see what I can get when those arrive, but there are docs in the process of being merged which detail what is required - https://github.com/raspberrypi/documentation/pull/1533

Most of the Chinese boards I've seen include a silkscreen describing the pinout of the unpopulated header, so combined with the wiring info of

Code: Select all

| Signal   | B101 header | Pi 40 pin header |
|----------|:-----------:|:----------------:|
| LRCK/WFS |     7       |      19          |
| BCK/SCK  |     6       |      18          |
| DATA/SD  |     5       |      20          |
| GND      |     8       |      39          |

you should be able to wire it up sensibly.

[globalc]

Most of the Chinese boards I've seen include a silkscreen describing the pinout of the unpopulated header, so combined with the wiring info of

Code: Select all

| Signal   | B101 header | Pi 40 pin header |
|----------|:-----------:|:----------------:|
| LRCK/WFS |     7       |      19          |
| BCK/SCK  |     6       |      18          |
| DATA/SD  |     5       |      20          |
| GND      |     8       |      39          |

you should be able to wire it up sensibly.

Fantastic, thanks! I've noticed all of these on the identifiers on top of the board:
https://fluxcoil.net/files/tmp/raspi-gr ... I-B01b.jpg .

[6by9]

Most of the Chinese boards I've seen include a silkscreen describing the pinout of the unpopulated header, so combined with the wiring info of

Code: Select all

| Signal   | B101 header | Pi 40 pin header |
|----------|:-----------:|:----------------:|
| LRCK/WFS |     7       |      19          |
| BCK/SCK  |     6       |      18          |
| DATA/SD  |     5       |      20          |
| GND      |     8       |      39          |

you should be able to wire it up sensibly.

Fantastic, thanks! I've noticed all of these on the identifiers on top of the board:
https://fluxcoil.net/files/tmp/raspi-gr ... I-B01b.jpg .

So they relate to the block of 5x2 holes at the top left of the board in that photo. You'll need to solder either wires or a header to those holes, and then connect across to the relevant pins on the 40pin header of the Pi.
Do exercise some caution when soldering to ensure you don't short anything to either 3.3V or GND. IR, INT, RST, OSCL and 3.3V can all be ignored.

[danergo]

Hi!

I'm trying to achieve a simple task again: I'd like to stream the HDMI stream to a VLC player on the same local subnet.

Device is PI0W, HDMI source is 1920x1080, with UYVY to fit into PI0W's CSI. (grabbing one frame works, and saving a raw h264 with yavta also works).

What I've tried:

Code: Select all

./yavta -f UYVY --capture=3000 -n 3 --encode-to=- -m -T /dev/video0 | ffmpeg -i - -c:v copy -an udp://127.0.0.1:23000
#Then on VLC open udp://IPOFPI:23000

./yavta -f UYVY --capture=3000 -n 3 --encode-to=- -m -T /dev/video0 | ffmpeg -i - -c:v copy -an udp://IPOFVLC:23000
#Then on VLC open udp://LOCALHOST:23000

./yavta -f UYVY --capture=3000 -n 3 --encode-to=- -m -T /dev/video0 | ffmpeg -i - -c:v copy -an udp://IPOFVLC:23000
#Then on VLC open udp://IPOFPI:23000

And I also tried with opening rtp on VLC (instead those 'udp' prefixes), but I don't succeed at all.

Then I have tried to remux it:

Code: Select all

 ./yavta -f UYVY --capture=3000 -n 3 --encode-to=- -m -T /dev/video0 | ffmpeg -i - -vcodec mpeg4 -f mpegts udp://{every_combination_from_above}

Do you know any configuration which is working? The main idea is to ask ffmpeg to stream from yavta, then let VLC on a different machine (but same subnet) to play it.

Thanks very much!

[globalc]

I'm trying to achieve a simple task again: I'd like to stream the HDMI stream to a VLC player on the same local subnet.
Device is PI0W, HDMI source is 1920x1080, with UYVY to fit into PI0W's CSI. (grabbing one frame works, and saving a raw h264 with yavta also works).

My board can only 25Hz at 1920x1080, you should also keep an eye on the specs of your board.

This works, but is unfortunately reencoding x264:

Code: Select all

# server/raspi side
./yavta -f UYVY --capture=3000 -n 3 --encode-to=- -m -T /dev/video0 | ffmpeg -i - -preset ultrafast \
  -vcodec libx264 -tune zerolatency -f mpegts udp://<your-cient-ip>:4242
# client side
ffplay -an -sn -i -fflags nobuffer udp://<ip-of-raspi>?listen

Something like

Code: Select all

[..] ffmpeg -i - -f <format> -c:v copy udp://<your-client-ip>:4242

might be quickest, as not reencoding. Apparently yavta can deliver h264 directly. I did not see a format for this unfortunately, "-f mpegts" results in

Code: Select all

[mpegts @ 0x20bb4f0] Timestamps are unset in a packet for stream 0. This is deprecated and will stop working in the future. Fix your code to set the timestamps properly
[mpegts @ 0x20bb4f0] H.264 bitstream malformed, no startcode found, use the video bitstream filter 'h264_mp4toannexb' to fix it ('-bsf:v h264_mp4toannexb' option with ffmpeg)
av_interleaved_write_frame(): Invalid data found when processing input

"-f webm" might be an other idea for streaming. "ffmpeg -formats" shows all formats, "ffmpeg -codecs | grep 'EV'" shows video encoding codecs.

I have some combinations listed at https://fluxcoil.net/hardwarerelated/ra ... _streaming, but did not find a low latency variant when using the kernel drivers.
gst-launch-1.0 or cvlc could also be used to stream, but I did not see a combination with low latency there either.

[danergo]

If you don't need low latency, fragmented mp4 can also be used, it's very common nowadays in online VOD. Params should be like this:

Code: Select all

-i - -c:v copy -an -movflags +frag_keyframe+empty_moov+faststart -f mp4

Latency is measured in seconds, but I did not need it to be any lower.

Thanks very much!

[Lowen_Na]

Hi!

I'm trying to achieve a simple task again: I'd like to stream the HDMI stream to a VLC player on the same local subnet.

Device is PI0W, HDMI source is 1920x1080, with UYVY to fit into PI0W's CSI. (grabbing one frame works, and saving a raw h264 with yavta also works).

What I've tried:

Code: Select all

./yavta -f UYVY --capture=3000 -n 3 --encode-to=- -m -T /dev/video0 | ffmpeg -i - -c:v copy -an udp://127.0.0.1:23000
#Then on VLC open udp://IPOFPI:23000

./yavta -f UYVY --capture=3000 -n 3 --encode-to=- -m -T /dev/video0 | ffmpeg -i - -c:v copy -an udp://IPOFVLC:23000
#Then on VLC open udp://LOCALHOST:23000

./yavta -f UYVY --capture=3000 -n 3 --encode-to=- -m -T /dev/video0 | ffmpeg -i - -c:v copy -an udp://IPOFVLC:23000
#Then on VLC open udp://IPOFPI:23000

And I also tried with opening rtp on VLC (instead those 'udp' prefixes), but I don't succeed at all.

Then I have tried to remux it:

Code: Select all

 ./yavta -f UYVY --capture=3000 -n 3 --encode-to=- -m -T /dev/video0 | ffmpeg -i - -vcodec mpeg4 -f mpegts udp://{every_combination_from_above}

Do you know any configuration which is working? The main idea is to ask ffmpeg to stream from yavta, then let VLC on a different machine (but same subnet) to play it.

Thanks very much!

I am trying to do something very similar but having limited results.

globalc has the right idea but I hit the same issues he did with that format using this command:

Code: Select all

./yavta -f UYVY --capture -n 3 --encode-to=- -m -T /dev/video0 | ffmpeg -i - -c:v copy -an -f mpegts udp://IPADDRESSofMyDesktop:12345

You get this error

Code: Select all

[mpegts @ 0x17837f0] H.264 bitstream malformed, no startcode found, use the video bitstream filter 'h264_mp4toannexb' to fix it ('-bsf:v h264_mp4toannexb' option with ffmpeg)
av_interleaved_write_frame(): Invalid data found when processing input

You get the same error if you add '-bsf:v h264_mp4toannex' so I am not sure what to do about that. I believe that copy is the codec you want to use for the lower resources and latency, how to get it to work.




Here is something else that I tried that got me a little farther but still didn't work:

Code: Select all

./yavta -f UYVY --capture -n 3 --encode-to=- -m -T /dev/video0 | ffmpeg -i - -c:v h264 -an -f mpegts udp://IPADDRESSofMyDesktop:12345

or

Code: Select all

./yavta -f UYVY --capture -n 3 --encode-to=- -m -T /dev/video0 | ffmpeg -i - -c:v h264_omx -an -f mpegts udp://IPADDRESSofMyDesktop:12345

And on the desktop PC, open VLC and have it play a network stream address of

Code: Select all

 udp://:12345

VLC will "play" the stream, but it never displays an image. Even if it did, the CPU usage would be too high for h264 and I have doubts that h264_omx would be low enough latency for what I am trying to do.

Ultimately I want to be able to embed the video stream in a HTTP page hosted on the Pi, so I am going to start trying to work on that. If anyone has any suggestions on any thing else to try, please let me know.

[6by9]

My board can only 25Hz at 1920x1080, you should also keep an eye on the specs of your board.

The (unsupported) firmware drivers were limited at 1080p25 as they always used RGB 24bpp at a different CSI2 link frequency. All boards support up to 1080p50 as UYVY using the kernel drivers.
The manufacturers are all copying Auvidea's documentation, and that was based on the original firmware drivers.

[Lowen_Na]

OK, I got streaming to work using FFMPEG to send and FFPLAY to receive and using copy for the codec

192.168.1.192 is my desktop PC that I am sending the stream to, edit as required

On the Pi, I used this command

Code: Select all

./yavta -f UYVY --capture -n 3 --encode-to=- -m -T /dev/video0 | ffmpeg -i - -c:v copy -an -f rtp rtp://192.168.1.192:4242

In the terminal, FFMPEG willoutput a section that starts with SPD:

Mine looks like this:

Code: Select all

SDP:
v=0
o=- 0 0 IN IP4 127.0.0.1
s=No Name
c=IN IP4 192.168.1.192
t=0 0
a=tool:libavformat 58.20.100
m=video 4242 RTP/AVP 96
a=rtpmap:96 H264/90000
a=fmtp:96 packetization-mode=1; sprop-parameter-sets=J2QAKKwrQDwBE/LAPEiagA==,KO4CXLAA; profile-level-id=640028

Copy that and put init a file on your desktop named stream.spd

Download FFPLAY for what ever OS you are using for your desktop. Put that spd file in the folder with the FFPLAY executable.

Code: Select all

ffplay -i -protocol_whitelist file,udp,rtp test.spd

This got me a video stream with about 1 second of latency and about 2% CPU usage on the Pi.

edit: Forgot to mention that this was with my source video being set to 24Hz. I just found out that if I set the source to 50hz, the latency is Much higher. Many many seconds higher.

[globalc]

Code: Select all

./yavta -f UYVY --capture -n 3 --encode-to=- -m -T /dev/video0 | ffmpeg -i - -c:v copy -an -f rtp rtp://192.168.1.192:4242

Nice! Seems like real copy indeed. Interestingly, even that has many seconds of latency for me, over gbit ethernet, 0.3ms RTT. For latency, the firmware based approaches are better for me than the kernel based ones. When firmware based, then a simple

Code: Select all

mplayer tv:// -tv driver=v4l2:device=/dev/video0:width=1280:height=720:fps=30:outfmt=yuy2

over SSH with X-forwarding works and has low latency (but of course high bandwidth consumption and cpu load).

[globalc]

The (unsupported) firmware drivers were limited at 1080p25 as they always used RGB 24bpp at a different CSI2 link frequency. All boards support up to 1080p50 as UYVY using the kernel drivers.

Tried with custom mode for the HDMI output of my thinkpad, @50Hz works indeed. Thanks!

[velocity00]

Hello all!

So I got the Auvidea B101 earlier in the week and have spent the last 4 days banging my head against a wall trying to make it work. I've read everything from this thread to many others on this forum and it's just not working like it's supposed to. A lot of the information is extremely fragmented and what some guides say should be done, others say it's obsolete (even this thread is more than 4 years old).

I can get vision with raspivid but it's cropped and stretched (think a 640x480 crop and then stretched to 1920x1080). After trying to make it work with v4l2-ctl, (“dtoverlay=tc358743” in config) the device isn't even being registered.

So my questions, is there a definitive guide from the last 6 months or so that summarises the process from start to finish on how to make this work with an RPi 3+ and v4l2?

I'm a broadcast engineer by trade and deal with high-tech equipment on a daily basis, but when it's all outlined in a manual/tech guide it's easy to make it work and fix issues. Not to mention the support you get when you buy $100k pieces of equipment. This isn't my first Pi project either, so not exactly a noob to the whole Linux/Pi environment, but I've found this to be more difficult than it should be.

The information provided from Auvidea is less than helpful, and confusing at best with different details about each revision of the board. I mean the B101 didn't even come with a single piece of paper pointing you in the direction of a guide.

Any help appreciated, cheers!

[globalc]

Hoi,

I can get vision with raspivid but it's cropped and stretched (think a 640x480 crop and then stretched to 1920x1080). [..]

That's using firmware mode. I would also try

Code: Select all

$ mplayer tv:// -tv driver=v4l2:device=/dev/video0:width=1280:height=720:fps=30:outfmt=yuy2

(with a source which offers that resolution/framerate), and the examples in this section: https://fluxcoil.net/hardwarerelated/ra ... mbinations

> After trying to make it work with v4l2-ctl, (“dtoverlay=tc358743” in config) the device isn't even being registered.
With switching to kernel mode, that's expected - based on other comments in this thread and what I see with my setup.

> So my questions, is there a definitive guide from the last 6 months or so that summarises
> the process from start to finish on how to make this work with an RPi 3+ and v4l2?
I'm not aware of any, and it would also need to cover multiple boards - I guess many just got one. I have my pieces on the page linked above. I get lower latency in firmware mode, but can also record/gram in kernel driver mode.

Wiki pages seem like a good format for documenting "a current state", and then having them indexed by search engines. This thread is great and helpful, but from format one can read it completely through, and has along the way also captured information which is no longer valid.
I think the camera docs part of the raspberry wiki currently does not refer to kernel mode, that could be a nice extension.

> The information provided from Auvidea is less than helpful, and confusing at best
> with different details about each revision of the board. I mean the B101 didn't even
> come with a single piece of paper pointing you in the direction of a guide.
I have a different board, but I'm surprised that the vendors do not care more about documenting their pieces for what the customers want to use it with. Maybe they just aim at industrial customers who buy thousands, and have their engineers talk directly?

In my case, audio grabbing is not yet working. I was able to reach the Chinese vendor of the board I got, and was informed that "Audio was not tested, but with connecting 4 pins it should work". (Need now to get hardware from Akihabara to test it.)
So while not easy, one can reach the vendors.

[6by9]

The documentation is being copy edited at the moment - https://github.com/raspberrypi/documentation/pull/1533

[cpetku]

I know this is an old thread, but the issue described in 2016 is still relevant.

Was there ever a workaround when using the CSI input and video formats such as 480p/576p whereby the width of the image is not divisible by 32? The only possible solution I have found in this thread would appear to be to change the width to 1440 then attempt to scale the resultant image back to its native format for display on the HDMI Output of the PI.

My focus is to use the PI to control a TI HDMI/LVDS bridge (like samsonx) as a method to prove out all functions of the TI Bridge. I have created a custom HAT for the PI using the UH940 and verified functionality at 1920x1080@25 and 720P. However I can't demonstrate the lower resolutions due to what I believe has been stated as a Hardware limitation of the GPU (e.g., fixed stride length). Can this be confirmed (HW limitation) or is it a driver issue that requires me to recompile the kernel?