hongyuanlu
Posts: 12
Joined: Mon Jun 12, 2017 9:12 am

What is meant by raw H.264?

Mon Jun 19, 2017 6:53 pm

After research on internet, I found someone said "Pi Camera gives raw H.264 video".

And after another research, I worked out a command line code to run

Code: Select all

raspivid -t 10000 -h 720 -w 1280 -fps 60 -hf -b 2000000 -o -
However, for this 10s 60 fps video generated, it is only 1.5MB. I am wondering why this is so small? I thought it should be 10 * 60 * 1280 * 720 * 3 bytes, but the actual size is clearly far less than this number..

So I am wondering about what exactly the output format of Pi Camera is? Is there any other way than omx264enc in Gstreamer to encode it fast and lossless?

User avatar
topguy
Posts: 3292
Joined: Tue Oct 09, 2012 11:46 am
Location: Trondheim, Norway

Re: What is meant by raw H.264?

Mon Jun 19, 2017 7:30 pm

h264 is the name of "part 10" of the Mpeg4 encoder specification. ( https://en.wikipedia.org/wiki/H.264/MPEG-4_AVC )

Its job is to effectively compress video information in a way that the resulting decompressing is almost exactly the same as the original video.
Your calculation about how much data it is in a "raw" videostream is probably correct, but the Pi camera (together with the GPU) will produce already compressed video.
(Because raw data would be too much for the SD-card to handle btw.)

When we say its raw h264 that means that the videostream is not inside a container file-format. http://www.pitivi.org/manual/codecscontainers.html
One main purpose of container-formats is to synchronise videostream with one or more audiotracks, subtitles etc.
Since the raw videostream is missing some information (like framerate) means that some videoplayers will have problem playing your file correctly.
A solution is to use the programs "avconv", "ffmpeg" or "mp4box" to "mux" (multiplex) the raw stream into the standard container format for mpeg4 video.
( which happen to be part 14 of the standard )

TL:DR
The camera output is already compressed by a "not lossless" codec (h264), and there is no gain trying to reencode it again
Last edited by topguy on Mon Jun 19, 2017 8:48 pm, edited 1 time in total.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 4240
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: What is meant by raw H.264?

Mon Jun 19, 2017 8:42 pm

raspivid produces an H264 elementary stream. That is what is being referred to as a "raw stream" because it is not put in any form of container (as topguy says).

-b 2000000 asks for a stream at 2Mbit/s. *10seconds = 20Mbits or 2.5Mbytes. So it's actually come in slightly under budget.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
Please don't send PMs asking for support - use the forum.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

hongyuanlu
Posts: 12
Joined: Mon Jun 12, 2017 9:12 am

Re: What is meant by raw H.264?

Tue Jun 20, 2017 9:47 am

Thank you topguy, 6by9!

hongyuanlu
Posts: 12
Joined: Mon Jun 12, 2017 9:12 am

Re: What is meant by raw H.264?

Tue Jun 20, 2017 10:52 am

I am also wondering if I want to insert audio with raw elementary stream, is there any other way rather than raspivid?

User avatar
flatmax
Posts: 318
Joined: Thu May 26, 2016 10:36 pm

Re: What is meant by raw H.264?

Tue Jun 20, 2017 11:57 am

It seems that the way to go about this is to install avconv (sudo apt-get install avconv should do it) and pipe the video to avconv. What avconv will let you do is wrap the audio an video into a single stream. Does it matter if you use raspivid ?

I notice from here, and here and from memory that an audio stream can be included like so :
raspivid -t 0 -w 960 -h 480 -fps 24 -b 5000000 -o - | avconv -i - -f alsa -ac 1 -i hw:0,0 -map 0:0 -map 1:0 -vcodec copy -acodec aac -strict -2 test.flv

I haven't tested it, but hope it gets you on your way.
Matt
Zero sound card for the Pi - live now on kickstarter : http://kck.st/2rwkQMH
Sound card for the Raspberry Pi with inbuilt microphone : http://www.audioinjector.net
Audio Inector Octo multitrack GPIO sound card : http://www.audioinjector.net

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 4240
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: What is meant by raw H.264?

Tue Jun 20, 2017 12:58 pm

hongyuanlu wrote:I am also wondering if I want to insert audio with raw elementary stream, is there any other way rather than raspivid?
elementary, ie single element, or one stream.

To handle multiple streams you need a container, eg MP4, MKV, AVI, etc. That also handles timestamping of frames in order to sort synchronisation.

Please note that the elementary stream from raspivid has no timestamp information. You can use "raspivid -pts timecodes.txt -o video.h264" to save them to a second file, such that mkvmerge can put the two together with "mkvmerge -o video.mkv --timecodes 0:timecodes.txt video.h264". Doing that directly with pipes is going to be difficult.
Your other option would be to use V4L2 and something like GStreamer. GStreamer will happily take both audio and video sources simultaneously and use the timestamps that are present. Doing so may be opening up another can of worms though.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
Please don't send PMs asking for support - use the forum.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

hongyuanlu
Posts: 12
Joined: Mon Jun 12, 2017 9:12 am

Re: What is meant by raw H.264?

Tue Jun 20, 2017 2:13 pm

6by9 wrote:
hongyuanlu wrote:I am also wondering if I want to insert audio with raw elementary stream, is there any other way rather than raspivid?
elementary, ie single element, or one stream.

To handle multiple streams you need a container, eg MP4, MKV, AVI, etc. That also handles timestamping of frames in order to sort synchronisation.

Please note that the elementary stream from raspivid has no timestamp information. You can use "raspivid -pts timecodes.txt -o video.h264" to save them to a second file, such that mkvmerge can put the two together with "mkvmerge -o video.mkv --timecodes 0:timecodes.txt video.h264". Doing that directly with pipes is going to be difficult.
Your other option would be to use V4L2 and something like GStreamer. GStreamer will happily take both audio and video sources simultaneously and use the timestamps that are present. Doing so may be opening up another can of worms though.
Thanks!

Code: Select all

raspivid -t 0 -h 720 -w 1280 -fps 60 -hf -b 2000000 -o - | timeout -s INT 20 gst-launch-1.0 -v fdsrc ! video/x-h264,width=1280,height=720,framerate=60/1 ! h264parse ! queue ! qtmux0. alsasrc device=hw:1,0 ! audioconvert ! voaacenc ! aacparse ! queue ! qtmux0. qtmux ! filesink location=/mnt/ramdisk/sample.mp4
I am using this code. The pipeline could be opened, however the output is wrong. This code works with other Webcam. Do you have any idea?

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 4240
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: What is meant by raw H.264?

Tue Jun 20, 2017 2:24 pm

hongyuanlu wrote:I am using this code. The pipeline could be opened, however the output is wrong. This code works with other Webcam. Do you have any idea?
Define "wrong".
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
Please don't send PMs asking for support - use the forum.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

hongyuanlu
Posts: 12
Joined: Mon Jun 12, 2017 9:12 am

Re: What is meant by raw H.264?

Tue Jun 20, 2017 2:31 pm

6by9 wrote:
hongyuanlu wrote:I am using this code. The pipeline could be opened, however the output is wrong. This code works with other Webcam. Do you have any idea?
Define "wrong".
I opened the video file generated by using VLC and I found the video length was 00:00, and nothing was displayed.

hongyuanlu
Posts: 12
Joined: Mon Jun 12, 2017 9:12 am

Re: What is meant by raw H.264?

Tue Jun 20, 2017 3:54 pm

flatmax wrote:It seems that the way to go about this is to install avconv (sudo apt-get install avconv should do it) and pipe the video to avconv. What avconv will let you do is wrap the audio an video into a single stream. Does it matter if you use raspivid ?

I notice from here, and here and from memory that an audio stream can be included like so :
raspivid -t 0 -w 960 -h 480 -fps 24 -b 5000000 -o - | avconv -i - -f alsa -ac 1 -i hw:0,0 -map 0:0 -map 1:0 -vcodec copy -acodec aac -strict -2 test.flv

I haven't tested it, but hope it gets you on your way.
Matt
Thanks! I am testing through it..

-----------------------------

After try, this code seems will output a file that is unplayable. And the file size is twice larger than the .h264 file I got; The video length is correct.

I think G-Streamer could potentially be a better solution so I will investigate that option first! Thanks..

hongyuanlu
Posts: 12
Joined: Mon Jun 12, 2017 9:12 am

Re: What is meant by raw H.264?

Wed Jun 21, 2017 9:37 pm

Solved.. Thank you all guys!

Return to “Camera board”

Who is online

Users browsing this forum: drich and 19 guests