Orbital6
Posts: 140
Joined: Sat Aug 08, 2015 6:32 pm

I2S/USB Audio input

Thu Nov 12, 2015 3:47 pm

Hi,

So i've scoured the web and these forums and am aware it's become common to output audio from the I2S bus via p5 on the RPi, but i have an IC that passes passes hdmi video through CSI-2, and isolates audio via I2S.

I want the audio output by this IC via I2S to be input to the RPi, by any means possible, provided the additional hardware is kept to a minimum. AV sync would be done manually so that's not an issue, since i know that broadcom make it hard to retrieve audio from CSI-2 via the GPU.

Is it just a matter of connecting the two i2s buses together and doing some kernel magic? I can't seem to figure it out. If no one can help me here, then i'll be pushed to look into passing the i2s audio from the HDMI feed to some line level converter IC, then input that to the RPi through a cheap USB sound card.

Does anyone have any ideas on this?

Thanks.
Last edited by Orbital6 on Wed Nov 18, 2015 5:27 pm, edited 1 time in total.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5691
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: I2S/USB Audio input

Thu Nov 12, 2015 5:22 pm

That ic has already been covered as it is the same one that the foundation used for their prototype hdmi input board, and is the same that auvidea have used in their equivalent board (has no driver support from them on the Pi).
With the csi2 peripheral driver, if you have configured the ic and peripheral correctly you will get the audio as buffers flagged with the codec info flag. The length on those buffers is currently wrong as I had no way of testing it, but if you know the sample rate I would expect them all to be the same length in reality.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
Please don't send PMs asking for support - use the forum.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

Orbital6
Posts: 140
Joined: Sat Aug 08, 2015 6:32 pm

Re: I2S/USB Audio input

Thu Nov 12, 2015 8:19 pm

6by9 wrote:That ic has already been covered as it is the same one that the foundation used for their prototype hdmi input board, and is the same that auvidea have used in their equivalent board (has no driver support from them on the Pi).
With the csi2 peripheral driver, if you have configured the ic and peripheral correctly you will get the audio as buffers flagged with the codec info flag. The length on those buffers is currently wrong as I had no way of testing it, but if you know the sample rate I would expect them all to be the same length in reality.
Thanks a lot for your response, i really appreciate it.

I have that auvidea board and as far as i can see, there's very little support for it. I'm able to get the hdmi feed with raspivid. I'm currently setting up the hardware to control the IC through I2C. Is there any documentation or pointers you can give me in the right direction to get started with (the csi2 peripheral driver/ic configuration/making sense of the audio buffer), i mean it's fantastic if it's possible to get the audio feed from the CSI-2 input, as it means i won't need to hack something together and sync the AV later.

I shall share all and any findings i make publicly if i get anywhere with this, i worry it may require a very specialised approach though.
Last edited by Orbital6 on Wed Nov 18, 2015 5:20 pm, edited 1 time in total.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5691
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: I2S/USB Audio input

Thu Nov 12, 2015 10:19 pm

Your pockets are deeper than mine if you splashed out on the Auvidea board!

viewtopic.php?f=43&t=109137 covers getting CSI-2 data.

Last time I looked at the GPU support for the Toshiba chip, Gordon did seem to have plumbed in all the audio side into the VideoCore audio system.
Comment in the source:

Code: Select all

 WHAT UNICAM DOES TO DATA FROM THE TC358743XB:
 TC**XB encodes the audio data in audio packets, with a particular audio packet ID
 However, Unicam only distinguishes between image packet id and non-image packet id.
 Therefore, all audio packets are simply "data". However, we know that TC**XB will
 only encode 224 bytes of InfoFrame data. Presumably, all remaining data are audio
 packets.

 Hence, in this function, as long as we have more than 224 bytes of data, we copy them into the audio buffer.
(Unicam is the CSI-2 receiver peripheral).

A very quick rereading of the GPU code and it appears the audio side is hooked in to the AudioPlus subsystem running on the GPU, which isn't so accessible to Linux (AFAIK). There is a MMAL/IL component audio_capture, which if told that the source via MMAL_PARAMETER_AUDIO_SOURCE/OMX_IndexConfigBrcmAudioSource is arec_h2c, then it should hook in to the Toshiba audio stream and deliver it to your app via MMAL.
(Seeing as the mappings to MMAL have been added, I suspect that they used MMAL instead of IL. Either should theoretically be possible)

I can advise on what may or may not work, but I don't have hardware to try anything out.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
Please don't send PMs asking for support - use the forum.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

Orbital6
Posts: 140
Joined: Sat Aug 08, 2015 6:32 pm

Re: I2S/USB Audio input

Fri Nov 13, 2015 2:08 pm

6by9 wrote:Your pockets are deeper than mine if you splashed out on the Auvidea board!

viewtopic.php?f=43&t=109137 covers getting CSI-2 data.

Last time I looked at the GPU support for the Toshiba chip, Gordon did seem to have plumbed in all the audio side into the VideoCore audio system.
Comment in the source:

Code: Select all

 WHAT UNICAM DOES TO DATA FROM THE TC358743XB:
 TC**XB encodes the audio data in audio packets, with a particular audio packet ID
 However, Unicam only distinguishes between image packet id and non-image packet id.
 Therefore, all audio packets are simply "data". However, we know that TC**XB will
 only encode 224 bytes of InfoFrame data. Presumably, all remaining data are audio
 packets.

 Hence, in this function, as long as we have more than 224 bytes of data, we copy them into the audio buffer.
(Unicam is the CSI-2 receiver peripheral).

A very quick rereading of the GPU code and it appears the audio side is hooked in to the AudioPlus subsystem running on the GPU, which isn't so accessible to Linux (AFAIK). There is a MMAL/IL component audio_capture, which if told that the source via MMAL_PARAMETER_AUDIO_SOURCE/OMX_IndexConfigBrcmAudioSource is arec_h2c, then it should hook in to the Toshiba audio stream and deliver it to your app via MMAL.
(Seeing as the mappings to MMAL have been added, I suspect that they used MMAL instead of IL. Either should theoretically be possible)

I can advise on what may or may not work, but I don't have hardware to try anything out.
[/size]

I'm borrowing the auvidea board to see if i can add the audio feed :p

I contacted Gordon, he saidthe method you're describing (audio via gpu) is no longer supported, and he advised me to take the I2S route.

I don't mind trying anything you wanted out, i've got the auvidea board here.

I'm not sure if the hardware of the i2s route is as simple as hooking the two buses up - i would need to write a small driver to decode the audio, but i'm a bit stuck on that front (never written a driver before), and it seems most of the guides on the web cover drivers for sending audio OUT of the i2s bus.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5691
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: I2S/USB Audio input

Fri Nov 13, 2015 9:58 pm

Orbital6 wrote:I'm borrowing the auvidea board to see if i can add the audio feed :p

I contacted Gordon, he saidthe method you're describing (audio via gpu) is no longer supported, and he advised me to take the I2S route.
Curious. I'll drop him a line to find out what has been going on there, as all the code still appears to be present. Not supported != doesn't work. There were a load of nasty looking manipulations being performed to the data, but that may just be formatting stuff.
I may try to borrow the prototype HDMI input board too to see what I can get out of it directly via the CSI-2 receiver route. I probably have a slight advantage in that I can see the GPU code that they got working previously, so know their I2C commands. Perhaps a read of the Toshiba datasheet wouldn't go amiss if I have problems sleeping.
Orbital6 wrote:I don't mind trying anything you wanted out, i've got the auvidea board here.

I'm not sure if the hardware of the i2s route is as simple as hooking the two buses up - i would need to write a small driver to decode the audio, but i'm a bit stuck on that front (never written a driver before), and it seems most of the guides on the web cover drivers for sending audio OUT of the i2s bus.
I have never played with I2S so can't really comment. Clocking always seemed to be the slightly tricky one, particularly as in this case you have a fixed data rate coming in via HDMI, so that needs to drive all the clocking. On an ADC, the Pi can produce a clock signal that the ADC then follows. I believe the Pi hardware can run either way, but it may be an issue of finding an example that works in the correct manner.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
Please don't send PMs asking for support - use the forum.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

chaitu
Posts: 1
Joined: Thu Jun 23, 2016 11:22 am

Re: I2S/USB Audio input

Mon Jun 27, 2016 9:49 am

6by9 wrote:That ic has already been covered as it is the same one that the foundation used for their prototype hdmi input board, and is the same that auvidea have used in their equivalent board (has no driver support from them on the Pi).
With the csi2 peripheral driver, if you have configured the ic and peripheral correctly you will get the audio as buffers flagged with the codec info flag. The length on those buffers is currently wrong as I had no way of testing it, but if you know the sample rate I would expect them all to be the same length in reality.

Hi,

We are using TC358743XBG to Convert HDMI to CSI 2,
We implemented the CSI2 connections as per RPIB board.For Audio i have taken I2S signals to different connector. Is it Ok? or the 2 data lines are enough for audio (in same CSI2 connector).

hblanken
Posts: 10
Joined: Fri May 06, 2016 11:43 am

Re: I2S/USB Audio input

Sun Jan 14, 2018 12:13 pm

I also have a Auvidea B101 rev 4 and was hoping to get audio from HDMI board into raspberry.

Did anyone succeed yet? Please help us with the GPIO connections.

I followed this tutorial and with adafruit everything works fine.
https://learn.adafruit.com/adafruit-i2s ... g-and-test

In their manual for B101 Auvidea specifies several pinouts - I tried to match those to adafruit.

BCLK = A-BCK = pin 12
DOUT = A-DATA = pin 38
LRCLK = A-LRCK = pin 35
SEL = ? (not available on Auvidea)
? = A-MCLK (No idea if I need to connect this.)
3.3v
Ground

Auvidea Reset and Cable Pins are unclear - Which GPIO shall I connect those?

Anyone successful in getting audio into Pi?
Thanks!

Return to “Interfacing (DSI, CSI, I2C, etc.)”

Who is online

Users browsing this forum: No registered users and 10 guests