Brandon92
Posts: 491
Joined: Wed Jul 25, 2018 9:29 pm
Location: Netherlands

Re: I2C Problems on SDA

Sun Sep 23, 2018 3:05 pm

oweno wrote:
Sun Sep 23, 2018 2:42 pm
The picture where data line (SDA) doesn't go to 0V is the result of using a small value pull-up, and is normal. I was only trying to illustrate what the signal looked like during the ACK phase. clock line (SCL) still has the larger (4k7) pull-up resistor which is why it looks different.
A, okay. I thought you use the same resistors for both lines. Than that issue is solved :)
oweno wrote:
Sun Sep 23, 2018 2:42 pm
Yes, I2S is used for audio data, but this is not the issue. I2C is used to configure the device and is called '2 wire MPU control interface' or something in the data sheet. This might point to a compatibility problem, but most likely they can't use the I2C name in the datasheet because it is trademarked. This is a common situation with I2C.
Okay. So, you trying to confure the IC. I stead of sending the audio to him. And I2C has indeed a lot of different names :?

Maybe a stupid question, but is the mode input of the chip connected to ground?

Maybe you could also connect a scope probe the the DBVDD / DCVDD pin of the device, to see if the voltage drop to much.

oweno
Posts: 25
Joined: Sat Feb 16, 2013 3:40 pm

Re: I2C Problems on SDA

Mon Sep 24, 2018 2:57 am

John Westlake wrote: If we are lucky added a couple of 100nF directly across Pin 1 - Pin 28 and Pin 27 - Pin 28 will resolve the problem ... fingers crossed
Sadly this didn't change the situation :( But thanks for pointing it out, I'll definitely fix those caps on the next board!

Another observation: if I remove the scope probe on SCL, while keeping the SDA one attached, the situation improves. Almost all the NAKs go away, and I don't see any more of those negative going spikes during the ACK cycle...

aBUGSworstnightmare
Posts: 1073
Joined: Tue Jun 30, 2015 1:35 pm

Re: I2C Problems on SDA

Mon Sep 24, 2018 8:19 am

oweno wrote:
John Westlake wrote: If we are lucky added a couple of 100nF directly across Pin 1 - Pin 28 and Pin 27 - Pin 28 will resolve the problem ... fingers crossed
Sadly this didn't change the situation :( But thanks for pointing it out, I'll definitely fix those caps on the next board!

Another observation: if I remove the scope probe on SCL, while keeping the SDA one attached, the situation improves. Almost all the NAKs go away, and I don't see any more of those negative going spikes during the ACK cycle...
What Kind of probes are you using? How big is their capacitive load?

Maybe you should use a logic analyzer for further debugging of your communication. Will also allow you to read the protocol (i.e. https://www.saleae.com/)

John Westlake
Posts: 84
Joined: Thu Nov 09, 2017 4:34 am

Re: I2C Problems on SDA

Mon Sep 24, 2018 4:46 pm

oweno wrote:
Mon Sep 24, 2018 2:57 am
John Westlake wrote: If we are lucky added a couple of 100nF directly across Pin 1 - Pin 28 and Pin 27 - Pin 28 will resolve the problem ... fingers crossed
Sadly this didn't change the situation :( But thanks for pointing it out, I'll definitely fix those caps on the next board!

Another observation: if I remove the scope probe on SCL, while keeping the SDA one attached, the situation improves. Almost all the NAKs go away, and I don't see any more of those negative going spikes during the ACK cycle...
Sadly I suspect as much it - it was a "Cheap hope" that improving the decoupling might help.

Your adding extra capacitance to the SDA line by leaving it attached - which suggest either troublesome ringing - or a timing issue as your delaying the SDA in relation to the SCL by adding the extra capacitance of the probe - its not a bad idea to add 33R to 100R resistor in these positions for series termination....

The improvements brought by adding the Series termination resistors would indicate issues with edge ringing - but also they would add extra delay (when combined with circuit 'parasitic' capacitance)....

Unless your using active probes with "Tip Grounding" your not going to see the real edge transitions and will be working pretty much in the blind - which will drive you nuts, been there and its not nice!

For high fidelity probing I use Tektronix P6245 1.5GHz "Solderable" Active probes.... you can buy an external interface / PSU (TekProbe 1103) so you can use these probes with other gear.... If your scopes good to atleast 500MHz+ EBays your friend here, but with these probes (as with any Active probe) can be damaged with over-voltage - I've blown about 4 over the years... :(

https://www.dropbox.com/s/x7131lpk58oad ... 3.JPG?dl=0

In the above picture notice the short Probe Ground connection "Tip Grounding" its the only way your going to see the "real" edge transitions...

If you where located near the Czech Rep. then you could pop over and i'm sure we would debug / resolve the issue in no time.... If you get really stuck then you can post me a board and I'll resolve the issue for free for you...

TBH, you can be sure of edge ringing so adding series termination is sound engineering - and one that seems to solve your issue...

oweno
Posts: 25
Joined: Sat Feb 16, 2013 3:40 pm

Re: I2C Problems on SDA

Mon Sep 24, 2018 5:41 pm

aBUGSworstnightmare wrote: What Kind of probes are you using? How big is their capacitive load?
They are just regular 1m passive bnc scope probes, so I guess capacitance is around 100pF...
aBUGSworstnightmare wrote: Maybe you should use a logic analyzer for further debugging of your communication.
I have one and it is what led me to discovering all the random NAKs on the line in the first place... that and the horrible screeching full volume noise coming out of the headphones when the codec doesn't get configured properly :o

I was hoping it was a software problem, but everything looked fine on that end, and then noticed those weird negative going spikes on the scope.

oweno
Posts: 25
Joined: Sat Feb 16, 2013 3:40 pm

Re: I2C Problems on SDA

Mon Sep 24, 2018 6:07 pm

John Westlake wrote: Your adding extra capacitance to the SDA line by leaving it attached - which suggest either troublesome ringing - or a timing issue as your delaying the SDA in relation to the SCL by adding the extra capacitance of the probe
I thought (hoped) it was a timing thing too, and it would make sense that if the device was sampling SDA towards the end of the cycle, delaying SDA a little would allow for some extra time and improve things.... but I've been over the timing diagram many times, and the datasheet mentions no minimum hold time (time from a falling SCL clock and SDA changing state). Also I slowed down the bus to 20kHz, but still got NAKs. (slowing the bus also has the effect of increasing the hold time, past the max 900ns listed in datasheet to about 3.5us, but if this really mattered than delaying SDA would make the problem worse not better...)

Also the funny 'attempted NAK' spikes in the first picture of the post hint at something more like ringing than timing....

I guess I won't really know unless I get some better probes, those are expensive! but I'll keep my eye out.
John Westlake wrote: TBH, you can be sure of edge ringing so adding series termination is sound engineering - and one that seems to solve your issue...
In the end, that might be all I needed to hear ;) I've seen the termination resistors with I2C mentioned occasionally, but they don't seem that common... so it is good to know it would be a normal thing to do...
Last edited by oweno on Mon Sep 24, 2018 7:27 pm, edited 1 time in total.

Brandon92
Posts: 491
Joined: Wed Jul 25, 2018 9:29 pm
Location: Netherlands

Re: I2C Problems on SDA

Mon Sep 24, 2018 6:11 pm

I know probes can give some problems with a measurement.
But I think with this setup the probe should not be a issue here. You clk is reasonable low 50kHz. And I expect that your probe is 10:1, and this should normally not be a issue. I used a scope probe all the time on a I2C bus, and had no problem with it. I even had a scope probe and a logic analyses connected at the same time, without a problem.

Did you try a other I2C bus of your module? And did you try to swap the white and purple wire?

John Westlake
Posts: 84
Joined: Thu Nov 09, 2017 4:34 am

Re: I2C Problems on SDA

Mon Sep 24, 2018 6:18 pm

Brandon92 wrote:
Mon Sep 24, 2018 6:11 pm
You clk is reasonable low 50kHz. And I expect that your probe is 10:1, and this should normally not be a issue.
Don't confuse Data rate with Edge speeds, sure I2C Data rate is slow, but its edge speed can be measured in nS... the fact that probe loading and series termination resistors are affecting the performance then this is a decent clue that the issue could be related to setup and hold time or ringing / noise (Ground bounce) issue...
Last edited by John Westlake on Tue Sep 25, 2018 12:25 am, edited 1 time in total.

Brandon92
Posts: 491
Joined: Wed Jul 25, 2018 9:29 pm
Location: Netherlands

Re: I2C Problems on SDA

Mon Sep 24, 2018 7:30 pm

John Westlake wrote:
Mon Sep 24, 2018 6:18 pm
Brandon92 wrote:
Mon Sep 24, 2018 6:11 pm
You clk is reasonable low 50kHz. And I expect that your probe is 10:1, and this should normally not be a issue.
Don't confuse Data rate with Edge speeds, sure I2C Data rate is slow, but its edge speed can be measured in nS... the fact that probe loading and series termination resistors are effecting the performance, then this is a sure sign that the issue is related to setup and hold time or ringing / noise (Ground bounce) issue...
Jup, that is true.

I had a quick look at the I2C standaard and found this table:
I2C standard time.png
I2C standard time.png (95.56 KiB) Viewed 629 times
And at the second image that he posted, the rise time was give a take 1-2us. And that value is at the high side for a I2C bus.

oweno
Posts: 25
Joined: Sat Feb 16, 2013 3:40 pm

Re: I2C Problems on SDA

Tue Sep 25, 2018 1:00 pm

Brandon92 wrote: at the second image that he posted, the rise time was give a take 1-2us. And that value is at the high side for a I2C bus.
Yes, I was thinking about that. certainly seems borderline, and this might help explain why performance improves when pull up R is reduced (faster rise times). But it is not the whole story because even with a pull up of only 1k some transmissions still fail with NAK...

aBUGSworstnightmare
Posts: 1073
Joined: Tue Jun 30, 2015 1:35 pm

Re: I2C Problems on SDA

Tue Sep 25, 2018 1:17 pm

oweno wrote: Yes, I was thinking about that. certainly seems borderline, and this might help explain why performance improves when pull up R is reduced (faster rise times). But it is not the whole story because even with a pull up of only 1k some transmissions still fail with NAK...
Have you tested with a different device/target already? If there is a problem coming from your PCB (i.e. crosstalk issue) you will never figure out this way. If you know your I2C code is working reliable you can re-connect the codec. Or check with another board which has the codec mounted.

John Westlake
Posts: 84
Joined: Thu Nov 09, 2017 4:34 am

Re: I2C Problems on SDA

Tue Sep 25, 2018 2:46 pm

oweno wrote:
Tue Sep 25, 2018 1:00 pm
Brandon92 wrote: at the second image that he posted, the rise time was give a take 1-2us. And that value is at the high side for a I2C bus.
Yes, I was thinking about that. certainly seems borderline, and this might help explain why performance improves when pull up R is reduced (faster rise times). But it is not the whole story because even with a pull up of only 1k some transmissions still fail with NAK...
Whats the bandwidth of your Scope / Probe (is the Probe in x10 mode) - first confirm your not B/W limited on your scope (does it have a Calibrator Output for probe compensation)?

Also insure your Scope is set to full Bandwidth mode....

oweno
Posts: 25
Joined: Sat Feb 16, 2013 3:40 pm

Re: I2C Problems on SDA

Tue Sep 25, 2018 7:02 pm

aBUGSworstnightmare wrote: Have you tested with a different device/target already? If there is a problem coming from your PCB (i.e. crosstalk issue) you will never figure out this way. If you know your I2C code is working reliable you can re-connect the codec. Or check with another board which has the codec mounted.
Sure, I've used this codec on other boards without issues. Also I have a few of these kicking around:

https://www.amazon.com/Audio-Injector-Z ... B075V1VNDD

which uses the same WM8731 codec.

So when I wire this up to the CMIO board and connect the logic analyzer, I don't see any NAKs on the I2C bus. When I swap it out and attach my board (same as in pictures above), about 35% of transmissions are NAK. So it has to be something about my design... I have several copies of the board, and they all have same behavior, so I'm pretty sure it isn't a defective board / part.

I put the design up here if anyone is curious:

https://github.com/owenosborn/CM3-WM8731-Audio

It could be a handy CM3 + Audio reference design.... if it worked :(

oweno
Posts: 25
Joined: Sat Feb 16, 2013 3:40 pm

Re: I2C Problems on SDA

Tue Sep 25, 2018 7:11 pm

John Westlake wrote: Whats the bandwidth of your Scope / Probe (is the Probe in x10 mode) - first confirm your not B/W limited on your scope (does it have a Calibrator Output for probe compensation)?
I can confirm the scope is not BW limited.

...so probes were in 1x mode. When I switch to 10x, I no longer see those spikes that look like the slave attempting to pull the line low for ACK.

Brandon92
Posts: 491
Joined: Wed Jul 25, 2018 9:29 pm
Location: Netherlands

Re: I2C Problems on SDA

Tue Sep 25, 2018 7:27 pm

Nice a schematic :ugeek:

I see that you didn't connected pin 179/VDD_core. Is that good? I didn't read the manual for the module by the way.
oweno wrote: ...so probes were in 1x mode. When I switch to 10x, I no longer see those spikes that look like the slave attempting to pull the line low for ACK.
Can you show us the 10x?

edit:
Forget about the VDD_core, it is connected the right way.
And overall you didn't place the decouple capacitors at the best places. They need to be as close as possible to the IC and pin as possible.

oweno
Posts: 25
Joined: Sat Feb 16, 2013 3:40 pm

Re: I2C Problems on SDA

Wed Sep 26, 2018 7:51 pm

OK.... so finally realized the I2C errors happen when the I2S interface gets turned on.

The WM8731 is configured as I2S master so it is spitting out bit clock, LR clock, and data out back to the CM. These 3 traces run parallel and right next to the I2C lines SDA and SCL for about 2 cm.

There are termination resistors on the I2S lines at the WM8731, so I can remove them, disconnecting I2S from those traces. When I do this, the I2C NAKs go away...

So I think this coupling is most of the problem, combined with poor bypass caps and probably some grounding issues...

aBUGSworstnightmare
Posts: 1073
Joined: Tue Jun 30, 2015 1:35 pm

Re: I2C Problems on SDA

Wed Sep 26, 2018 8:42 pm

oweno wrote:
Wed Sep 26, 2018 7:51 pm
OK.... so finally realized the I2C errors happen when the I2S interface gets turned on.

The WM8731 is configured as I2S master so it is spitting out bit clock, LR clock, and data out back to the CM. These 3 traces run parallel and right next to the I2C lines SDA and SCL for about 2 cm.

There are termination resistors on the I2S lines at the WM8731, so I can remove them, disconnecting I2S from those traces. When I do this, the I2C NAKs go away...

So I think this coupling is most of the problem, combined with poor bypass caps and probably some grounding issues...
I think I've mentioned crosstalk.
There are some more issues with your PCB which should be changed if you make a new revision. There are some vias which are useless, and your VCC traces should be re-routed, as well as better decoupling (placemenf of caps).

oweno
Posts: 25
Joined: Sat Feb 16, 2013 3:40 pm

Re: I2C Problems on SDA

Sat Sep 29, 2018 1:18 am

aBUGSworstnightmare wrote: I think I've mentioned crosstalk.
There are some more issues with your PCB which should be changed if you make a new revision. There are some vias which are useless, and your VCC traces should be re-routed, as well as better decoupling (placemenf of caps).
Yes, thank you for the input! definitely some PCB issues here, I guess that is what I get for doing it so quickly...

And thanks to everyone who chimed in here, a big help ;)

One more software related question. I noticed when the audio driver is writing registers to the codec and a write fails (with I2C NAK), it just moves on to the next one. Is there a way in the I2C driver to force it to retry a failed write before moving on?

Return to “Compute Module”