mehrschwein
Posts: 15
Joined: Thu Oct 02, 2014 9:35 am

B101 hdmi input with i2s sound

Wed Apr 11, 2018 11:51 am

Hi,

I'm trying to get the B101 Module(rev4) for hdmi input running. I'm now able to get the video in but kind of desperate when looking at the i2s sound.
In the end I only want to get a video signal with sound into c++ to work with it. From the i2s sound thread I didn't really understand how to hook up the pins and what to with them. In the past I only worked with C#, c++ basics and python, so I'm more of a software guy.

I've attached the only information I got for the hdmi module. I want to understand what the pins 4,5,6,7 mean and what to do with them.
Maybe there already is a good explanation out there what A-MCLK and all that stuff does and I'm just blind.
Attachments
Bildschirmfoto 2018-04-11 um 13.38.59.png
Bildschirmfoto 2018-04-11 um 13.38.59.png (127.82 KiB) Viewed 2639 times
Last edited by mehrschwein on Fri Apr 27, 2018 3:01 am, edited 2 times in total.

mehrschwein
Posts: 15
Joined: Thu Oct 02, 2014 9:35 am

Re: B101 hdmi input with i2s sound

Wed Apr 11, 2018 2:15 pm

or lets start with small steps. how to I get a MCLK clock since raspberry doesn't seam to support that?

fruitoftheloom
Posts: 19483
Joined: Tue Mar 25, 2014 12:40 pm
Location: Delightful Dorset

Re: B101 hdmi input with i2s sound

Wed Apr 11, 2018 2:17 pm

mehrschwein wrote:
Wed Apr 11, 2018 2:15 pm
or lets start with small steps. how to I get a MCLK clock since raspberry doesn't seam to support that?

Is this long Post no help ??

viewtopic.php?f=38&t=120702
adieu

My other Computer is an Asus CS10 ChromeBit running Chrome Operating System.
HP Envy 4500 Wireless Printer supported by HPLIP software in Raspbian Stretch.

mehrschwein
Posts: 15
Joined: Thu Oct 02, 2014 9:35 am

Re: B101 hdmi input with i2s sound

Wed Apr 11, 2018 4:00 pm

fruitoftheloom wrote:
Wed Apr 11, 2018 2:17 pm
mehrschwein wrote:
Wed Apr 11, 2018 2:15 pm
or lets start with small steps. how to I get a MCLK clock since raspberry doesn't seam to support that?

Is this long Post no help ??

viewtopic.php?f=38&t=120702
yes, it was. but if i've read all 15 pages correctly there is noone known who ever got the sound for the b101rev4 running so i tried to make my own approach what until now was mainly looking for tutorials/explanations for similar modules and trying to apply them to my problem. The developer of the board has 0 explanations and doesn't seam to answer any questions/requests anymore.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 6641
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: B101 hdmi input with i2s sound

Wed Apr 11, 2018 4:25 pm

I believe that Orbital6 has managed to get audio running, but I don't know the details.

It is interesting that Auvidea (a hardware company) say the lines are MCLK, DATA, BCK, and LRCK. The datasheet lists it as a 3 wire interface with SCK, WFS (aka LRCK) and SD (aka DATA). There is an A_OSCK (Oversampling clock) line, but I don't see that being particularly useful.

As noted on that other thread I'm trying to get the driver merged for the CSI2 receiver and then I might have a quick play with audio. No guarantees though as priorities change, and I've never played with I2S before.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

mehrschwein
Posts: 15
Joined: Thu Oct 02, 2014 9:35 am

Re: B101 hdmi input with i2s sound

Thu Apr 12, 2018 3:22 pm

so would you say Auvidea provides wrong information or are they just doing some magic we don't know of?

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 6641
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: B101 hdmi input with i2s sound

Thu Apr 12, 2018 4:03 pm

mehrschwein wrote:
Thu Apr 12, 2018 3:22 pm
so would you say Auvidea provides wrong information or are they just doing some magic we don't know of?
Pass. I've only taken a quick look at the datasheet and noted that it doesn't quite match up. I don't have the schematics for the B101/2 so dont know how it has been wired. In theory there could be other circuitry in between, but I doubt it.
When I get to looking at audio it will be a case of sticking the signals on an oscilloscope. At that point it should be fairly obvious what is going on.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

mehrschwein
Posts: 15
Joined: Thu Oct 02, 2014 9:35 am

Re: B101 hdmi input with i2s sound

Mon Apr 23, 2018 1:34 pm

I sadly don't have an oscilloscope. My guess right now would have been that the B101 needs a clock for its sound and doesn't provide its own. So while I get the i2s from data, bit clock and word clock I need to give it a master clock to work with? This is just guessing and I have no Idea what I'm doing.
Now I just put an LED on those pins connecting with GND to see if there is a signal at all. MCLK got an output, the other 3 dont say anything at all. Now i feel like I know even less than before.

mehrschwein
Posts: 15
Joined: Thu Oct 02, 2014 9:35 am

Re: B101 hdmi input with i2s sound

Mon Apr 23, 2018 1:58 pm

so the b101 is a slave. what do i need mclk for? is that information of "got some output" of any use?

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 6641
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: B101 hdmi input with i2s sound

Mon Apr 23, 2018 2:28 pm

I can't post the datasheet due to NDAs, but I see no reason I can't post:
Support only Master Clock option
Since the transmitter and receiver have the same clock signal for data transmission, the transmitter as the master, has to generate the bit clock (SCK), and word-select (WS)
All 3 signals they refer to (SCK, WFS, and SD) show direction arrows going from transmitter (TC358743) to receiver (SoC). The TC358743 is the master of everything I2S.

The V4L2 driver does advertise a couple of custom controls for audio-present and audio-sampling-rate. Again the TC358743 is the master of these parameters, so it is telling you what rate it will be producing the data, and it is up to you to set up the I2S pipeline to accept that. Don't go looking for options on resampling it in the hardware as there aren't any.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

mehrschwein
Posts: 15
Joined: Thu Oct 02, 2014 9:35 am

Re: B101 hdmi input with i2s sound

Fri Apr 27, 2018 2:29 am

6by9 wrote: All 3 signals they refer to (SCK, WFS, and SD) show direction arrows going from transmitter (TC358743) to receiver (SoC). The TC358743 is the master of everything I2S.
Sounds logical, but now I have an oscilloscope. I dont really know how to use it yet, but I got it to work somehow. The MLCK gives a clock at something between 11.18 kHZ and 12.42 kHZ which doesn't make any sense to me and the other 3 (SCK, WFS, and SD) don't have any output at all. If the b101 is the Master, why is there no signal?

mehrschwein
Posts: 15
Joined: Thu Oct 02, 2014 9:35 am

Re: B101 hdmi input with i2s sound

Fri Apr 27, 2018 3:26 am

mehrschwein wrote:The MLCK gives a clock at something between 11.18 kHZ and 12.42 kHZ
ok, now it tells me the same in MHz. its propably supposed to be 12.288MHz. closer to what I expected, but still surprised that i don't have anything on the other 3

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 6641
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: B101 hdmi input with i2s sound

Fri Apr 27, 2018 1:40 pm

Putting a scope on the I2S lines looks fine to me.

Check first that the B101 is seeing HDMI audio - "v4l2-ctl --list-ctrls" will print out the user controls, which includes audio_sampling_frequency, and audio_present. Whilst I'm seeing signals even without audio_present, it makes more sense if you have audio coming in.

Working from the end of the connector closest to the camera connector we have:
- ground.
- 48kHz - L-R clock (WFS)
- 3.2MHz - I2S data clock (SCK)
- audio data (SD)
- 12.8MHz "H2C output OverSampling clock (A_OSCK), A_OSCK frequency equal to 256fs" - 48*256 = 12.28MHz
- I haven't connected to the remaining 3 pins as they aren't audio related.
That all looks fine, and if I trigger the scope off the L-R clock then I can see the audio data bouncing up and down like a basic VU meter.
All the signals look a little ugly with significant ringing, but I haven't rigged it up to the I2S receiver, so that's not so unexpected.

So overall I'd say that
pin 5 = SD -> Pi GPIO20 PCM_DIN
pin 6 = SCK -> Pi GPIO18 PCM_CLK
pin 7 = WFS -> Pi GPIO19 PCM_FS
And you then need to configure the receiver to slave mode so that it treats PCM_CLK as an input.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 6641
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: B101 hdmi input with i2s sound

Fri Apr 27, 2018 1:47 pm

You haven't said what software you're using. I've assumed you're using the unicam V4L2 driver.
- If raspivid then forget it as it routes the audio over the CSI2, and that code is unsupported.
- UV4L2 is a third-party library that I have no idea over.
- raspi_tc358743 will probably spit out the audio over I2S as it copied the kernel driver, but AFAIK it won't be reading the audio sampling frequency or detected registers.

For info I'm actually running on a B102 on a Compute Module 3, but that should make no difference.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 6641
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: B101 hdmi input with i2s sound

Fri Apr 27, 2018 5:10 pm

I've got sound working using the wiring above.
It appears to only be coming through on one channel, but I can't tell where in the chain that is happening. Choosing a different source it may actually be at half speed which may well produce the same result (the first source I wasn't familiar enough with to say if it was at the wrong speed).
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 6641
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: B101 hdmi input with i2s sound

Tue May 01, 2018 10:03 am

Sound is all working now.
I'll update my branch in a bit as I'm in the middle of something else at the moment.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

ahmed mosad
Posts: 16
Joined: Sun Jun 03, 2018 10:41 pm

Re: B101 hdmi input with i2s sound

Sun Jun 03, 2018 10:44 pm

please , urgently can you guide me how you get your B101 MODULE Audio works ?

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 6641
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: B101 hdmi input with i2s sound

Mon Jun 04, 2018 8:23 am

ahmed mosad wrote:
Sun Jun 03, 2018 10:44 pm
please , urgently can you guide me how you get your B101 MODULE Audio works ?
See kernel branch https://github.com/6by9/linux/commits/r ... and-codecs, and mainly commit https://github.com/6by9/linux/commit/2f ... 0c6cb57f41.
Build and install that as your main kernel, and then add

Code: Select all

dtparam=i2c_vc=on
dtoverlay=tc358743-fast
dtoverlay=tc358743-audio
to /boot/config.txt and it should register the devices.

Wiring is as noted before:
pin 5 = SD -> Pi GPIO20 PCM_DIN
pin 6 = SCK -> Pi GPIO18 PCM_CLK
pin 7 = WFS -> Pi GPIO19 PCM_FS
and pin 8 to a ground pin for completeness (not really needed as there is a ground up the flexi cable from the camera connector.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

ahmed mosad
Posts: 16
Joined: Sun Jun 03, 2018 10:41 pm

Re: B101 hdmi input with i2s sound

Mon Jun 04, 2018 10:21 am

Thank you very much that you take time to replay to my Q

sorry but i am not Linux expert , so can you give me more guide through this part

" See kernel branch https://github.com/6by9/linux/commits/r ... and-codecs, and mainly commit https://github.com/6by9/linux/commit/2f ... 0c6cb57f41.
Build and install that as your main kernel, and then add "

as this part very important to me , we can works on it as job i can pay money for you for your time

we can works through team viewer to help me get kernel part done in my side , i hope you accept my offer , this is very important for me

you can connect to me also through :

email : [email protected]
Skybe : ahmed.mohamed6056
Awaiting your kind response,
Best Regards,
Ahmed Mosad

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 6641
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: B101 hdmi input with i2s sound

Mon Jun 04, 2018 10:36 am

ahmed mosad wrote:
Mon Jun 04, 2018 10:21 am
Thank you very much that you take time to replay to my Q

sorry but i am not Linux expert , so can you give me more guide through this part

" See kernel branch https://github.com/6by9/linux/commits/r ... and-codecs, and mainly commit https://github.com/6by9/linux/commit/2f ... 0c6cb57f41.
Build and install that as your main kernel, and then add "

as this part very important to me , we can works on it as job i can pay money for you for your time

we can works through team viewer to help me get kernel part done in my side , i hope you accept my offer , this is very important for me

you can connect to me also through :

email : [email protected]
Skybe : ahmed.mohamed6056
Awaiting your kind response,
Best Regards,
Ahmed Mosad
At the bottom of every sinlgle one of my posts is
"I'm not interested in doing contracts for bespoke functionality - please don't ask."
so I'll give you two guesses as to my answer.

See https://www.raspberrypi.org/documentati ... uilding.md for instructions on building the kernel. Amend the "git clone" line to take my branch instead of the standard Pi one.
Otherwise wait until https://github.com/raspberrypi/linux/pull/2513 is merged.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

ahmed mosad
Posts: 16
Joined: Sun Jun 03, 2018 10:41 pm

Re: B101 hdmi input with i2s sound

Mon Jun 04, 2018 11:32 am

thanks , and sorry i am not noticed your note message in bottom of your post

the mean problem that i have very low experience in Linux or even raspberry pi
so is that possible take image from your raspberry pi SD card ( image that have updated kernel and all stuff )

thank you so much in advance

waiting for your kindly replay

ahmed mosad
Posts: 16
Joined: Sun Jun 03, 2018 10:41 pm

Re: B101 hdmi input with i2s sound

Tue Jun 05, 2018 8:53 am

hello

so you decided stop replay to my messages

i am sorry if i cause you any problems , but i checked your previous posts to another peoples and you ready to help them through your mail and in there side , so please help me get this done

this is very urgent for me

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 6641
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: B101 hdmi input with i2s sound

Tue Jun 05, 2018 10:03 am

ahmed mosad wrote:
Tue Jun 05, 2018 8:53 am
so you decided stop replay to my messages

i am sorry if i cause you any problems , but i checked your previous posts to another peoples and you ready to help them through your mail and in there side , so please help me get this done

this is very urgent for me
  • I'm a software engineer for Raspberry Pi, not full time forum support.
  • I will always respond via the forum rather than PMs or emails as then the information is available for all rather than just the one individual. The only exception is if there is a commercial sensitivity to the information required (pretty rare).
  • The B101 is not a product made by Raspberry Pi, therefore it has no financial benefit to Raspberry Pi Foundation/Trading, and the fact you get any support on it at all should be counted as a bonus (it's mainly as we think it'd be a pretty cool feature to get working). Feel free to go back to Auvidea to get support, but I can predict the response you'll get there as they mainly just make boards.
  • Patience is a virtue. 18 hours may seem a long time to you, but other people have different priorities.
I will help people that are prepared to put some effort in.
I have pointed you at a working branch, and step-by-step instructions on how to build the kernel. If you're not prepared to invest the effort to at least read through what is involved, then you will have to wait for things to be merged.
If you'd read around the subject a bit further, you would have found viewtopic.php?f=38&t=120702&start=325#p1232850 and linking to madnerd's script that should do the full kernel compile and install for you. I have pointed you at the rpi-4.14.y-unicam-and-codecs branch to gain the audio overlay, so line 27 "git clone --depth 1 -b unicam_v4_4.14 https://github.com/6by9/linux/" needs to be amended.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

ahmed mosad
Posts: 16
Joined: Sun Jun 03, 2018 10:41 pm

Re: B101 hdmi input with i2s sound

Tue Jun 05, 2018 10:32 am

hello

thank you for your patient

i am not search for help without do works , i have this module about 2 month ago and i spent more than 1 month try to works on it and find solution for this problem , what make me in harry that i am Master degree student and i am very late in finishing my thesis , my Doctor have very bad impact factor that until now i am not get this prat done

what i am ask here just step by step guide and i will fallow that , i am confused with all those new names and links and i don't know where i should start

so last thing i will ask from you is just put things in order to me and i will works on that

for example :

1- first step go to this link and fallow this step
2- then after that go to this link and do those steps and replace this part by that
3- ..... etc

that is all , i know that i am ask you a lot of things , but you don't know how i am happy to find some one get this module sound works , this refresh my hope to get this works done in my side

and i am very sorry that i am cause any trouble to you , just i am very tense and try to void lost my master degree

thank you very much

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 6641
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: B101 hdmi input with i2s sound

Tue Jun 05, 2018 1:46 pm

Take a clean SD card and install the latest Raspbian image on it. Raspbian Lite would be my recommendation, but it's up to you if you need the GUI.

git clone https://github.com/maditnerd/tc358743.git
cd tc358743
nano install.sh
Edit line 27 to read "git clone --depth 1 -b rpi-4.14.y-unicam-and-codecs https://github.com/6by9/linux/". Save and quit.
./install.sh
Wait for it to complete. It will take several hours.
Follow the steps that it prints at the end. Add "dtoverlay=tc358743-audio" to /boot/config.txt before you reboot.
Wire up the B101 as previously discussed.

cd yavta
wget https://raw.githubusercontent.com/6by9/ ... 50EDID.txt
v4l2-ctl --set-edid=file=1080P50EDID.txt --fix-edid-checksums
v4l2-ctl --list-ctrls should list audio_present reflecting the current status, and audio_sampling_rate reflecting the sampling rate of the audio.
./yavta --capture=1000 -n 3 --encode-to=file.h264 -f UYVY -m -T /dev/video0 will capture 1000 frames of video.
Audio you should be able to capture via ALSA. I can't remember the command I used (probably arecord with some options).
To record video and audio together you'll probably want to use GStreamer or similar. That really is an exercise for the reader. GStreamer can't easily use the hardware acceleration for video_encode, at least not in the format that the B101 is producing it.

Sorry, but you're on your own now. My time is far better used trying to get it merged into the main kernel for use by everyone.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

Return to “Interfacing (DSI, CSI, I2C, etc.)”