I'm using OpenMAX for decoding an H264 stream (coming from an iOS AirPlay Mirroring connection) under Raspbian on a Raspberry Pi. I'm receiving data in packets, with (AFAICT) one NAL unit per packet. I'm then forwarding the individual NAL units to the OpenMAX H264 decoder.
In my current working solution, I'm not specifying any OMX_IndexParamBrcmDataUnit explicitly, so I believe the default of OMX_DataUnitArbitraryStreamSection is used. I'm manually adding a four-byte 0x00000001 start code to the beginning of every NAL unit I feed the decoder. In the case that the NAL unit doesn't fit into an OMX input buffer, I'm splitting the NAL into multiple buffers and set the OMX_BUFFER_FLAG_ENDOFNAL flag on the last one (I'm not setting any flag on NAL units that fit into a single buffer). This works, but I'm seeing a bit of latency that I'd like to reduce.
Having read through some more information about H264, I've come to the conclusion that by following the procedure described above, I'm using the NAL Byte Stream Format and let the decoder determine NAL boundaries, when in fact I should have all information needed to tell the decoder where the NAL boundaries are.
Now, I tried to adjust my program to use the NAL Transport Stream Format. I discovered the OMX_IndexParamBrcmDataUnit configuration that has a unit type I set to OMX_DataUnitCodedPicture and an encapsulation type I set to OMX_DataEncapsulationRtpPayload. From some threads on this forum, I found I needed to set the OMX_BUFFER_FLAG_ENDOFFRAME flag on buffers when using the OMX_DataUnitCodedPicture data unit, so I set that to all buffers that contained a whole NAL, and to all buffers that contained the last part of a NAL split into multiple buffers. I didn't know how exactly I should handle SPS and PPS NALs, so I just set OMX_BUFFER_FLAG_ENDOFNAL flag on those. I also didn't add the start codes I previously had added. However, I couldn't get the decoder (and attached renderer) to display any frames.
I figured my NALs probably contain only a part of a frame, so the OMX_BUFFER_FLAG_ENDOFFRAME flag is inappropriate for me to set. But still, there must be a way of getting rid of manually prepending the 0x00000001 start code to every NAL.
So, here are my questions:
1) What is the difference between prepending a NAL unit with the start code and setting the OMX_BUFFER_FLAG_ENDOFNAL flag on a buffer?
2) Is there a way to let the decoder figure out frame boundaries, but explicitly tell it NAL boundaries? I imagine that must improve latency a bit at least if my stream doesn't tell me frame boundaries.
3) What is the difference between the OMX_PARAM_DATAUNITTYPE's unit type and encapsulation type?
Any help is highly appreciated!