Overclocking

[dom]

@Tompen

Thanks

Can I be clear:
d535b28c: Jul 28 2012 01:29:44 version 328038 is good
c2c114a0: Jul 28 2012 02:53:14 version 328039 is bad

Because that doesn't make much sense. The "bad" one is just a rebuild of the same source where I had a missing "make clean" in the build script.
The only difference in the two builds is 3d wasn't enabled in the older one.

Can you also check if the core_freq is actually working in the "no corruption" version? i.e. does core_freq=300 actually give some improvement (e.g. with nbench or similar benchmark).
I'm just wondering if core_freq=300 has always caused corruption, but in your non-corrupting build, the core_freq=300 was being ignored for some reason.

[Tompen]

Your understanding is correct regarding good and bad version. I can not explain the "make clean" in build scripts and such. Perhaps the missing make clean caused source that was intended to be compiled into binary in d535b28c was not included in the binaries that was built. Maybe in d535b28c some part of the binary that was built really comes from ead4b822? (Since make clean was missed, it skipped those part of build because toolchain thought it was already built). That would isolate the cause only to be source code affected by the missing make clean. All above is purely speculation on my part because I do not know about your build process.

Well, to isolate the cause of datacorruption in my testing, I can not let this influence. I will continue my testing on the same route to make certain regarding the mentioned good and bad versions. However, when I am 100% certain that this commit is the first /boot binaries that introduce data corruption, before starting to isolating indvidual file, I will make sure core_freq value really changes speed by using benchmark program.

[Tompen]

I just managed to reproduce corruption on:
# /opt/vc/bin/vcgencmd version
Jul 26 2012 00:48:29
Copyright (c) 2012 Broadcom
version 327550 (release)
# uname -a
Linux XBian 3.1.9+ #202 PREEMPT Wed Jul 25 22:11:06 BST 2012 armv6l GNU/Linux

And I have this commit in my notes from testing earlier today as: "Works OK. Tested twice with core_freq=300". This shows my testing as not reliable enough, need to find a better method. Sorry for wasting your time. I will start from scratch and try to find a better method.

[kevs3d]

I don't think the firmware version makes any difference (other than the config settings that are now built in)

It's core speed that is causing the problem from every test i've done.

[dom]

I don't think the firmware version makes any difference (other than the config settings that are now built in)

It's core speed that is causing the problem from every test i've done.

There is a suggestion that people were successfully overclocking the core before the cpufreq driver came out and not suffering corruption, so it is possible the firmware had better settings for PLLs etc a month or two back.

[portets]

So how many types of corruption are we dealing with now? I'm having trouble keeping track of which ones are related. I've seen fat corruption on boot, ext4 corruption with heavy I/O, and I get ext4 corruption on boot with force_turbo=1, but heavy I/O + CPU is completely stable for me.

[kevs3d]

I don't think the firmware version makes any difference (other than the config settings that are now built in)

It's core speed that is causing the problem from every test i've done.

There is a suggestion that people were successfully overclocking the core before the cpufreq driver came out and not suffering corruption, so it is possible the firmware had better settings for PLLs etc a month or two back.

Ok, i've never used the cpufreq driver feature, just stuck with force_turbo if that makes any difference.

Cheers,

Kev

[dom]

So how many types of corruption are we dealing with now? I'm having trouble keeping track of which ones are related. I've seen fat corruption on boot, ext4 corruption with heavy I/O, and I get ext4 corruption on boot with force_turbo=1, but heavy I/O + CPU is completely stable for me.

We've had problems with corruption on the FAT partition since day 1. The fix to use multiple sector writes a month or two back seemed to fix it for almost everyone, although I think there are still occasional problems.
The ext4 partition has always been much more reliable, so reports of ext4 corruption are more significant.

You obviously can get corruption from a non-clean shutdown, either by pulling power, or a crash. Overclocking can make crashes more likely, so there will always be a small chance of corruption if overclocking is too high.
I think poor power supplies can also cause corruption. A lot of people have power supplies that drop volts when under load, and might be providing less than 4 volts when the system is busy. Often this doesn't crash the processors (which run off 1.2V) but can cause USB and sdcard problems. Oveclocking increases the load on the power supply, so may make this voltage drop more likely.

Since we launched the turbo modes and cpufreq drivers there seems to be a new class of corruption. Mostly on the ext4 partition, and seems to be correlated with core_freq > 250 (other frequencies probably unimportant).
Now, this thread was quite long before this firmware came out, with many users running at high core_freq settings, and there were no specific reports of corruption.
So, either something has changed in the firmware that makes corruption more likely, or there are now many more people using overclock, that the small percentage of sufferers has become clearly visible. I'm not sure which is the case.

[kevs3d]

We've had problems with corruption on the FAT partition since day 1. The fix to use multiple sector writes a month or two back seemed to fix it for almost everyone, although I think there are still occasional problems.

I can consistently corrupt FAT by simply raising the core above 250 still. No recent firmware fixes this for the class 10 card I'm using.

I will happily try or test any suggestions...? It would be great to be able to raise the core as it makes a big difference to performance

Cheers,

Kev

[dom]

I can consistently corrupt FAT by simply raising the core above 250 still. No recent firmware fixes this for the class 10 card I'm using.

I will happily try or test any suggestions...? It would be great to be able to raise the core as it makes a big difference to performance

Finding an old firmware that does work reliably with the same settings would be useful (if that exists). I'm guessing you may have to go back a couple of months.

Talked to some of the hardware people today about what this could be.

One useful test is to find the lowest core_freq that causes corruption (with arm_freq=750, sdram_freq=400, over_voltage=0). Perhaps for your board it is 280.
Now try the same test with over_voltage=6.

If it is a timing path limit, somewhere between ARM and SDCARD peripheral, the increasing the voltage should allow it to run reliably a bit faster, and we'd expect this test to pass.
If it some other cause, then the voltage will have no effect.

Another test is to underclock the arm. (e.g. arm_freq=350, over_voltage=0, force_turbo=1, core_freq=<n>). Does that improve reliability?

[TikkenBaxter]

Hi,

I have been running a different setup for the last week without corruption.
I'm running latest OpenElec XBMC and have the Pi in Turbo mode.
When running only on SD card I quickly got corruption, but when using a USB-stick as disk where all the changes is made everything runs perfectly!

so, why corruption on SD card? Must find a solution for this Awesome product!

[mer.at]

what's the highest overclock that has been achieved? and is there some kind of benchmark-list in this forum? e.g. superpi 1m times

[dom]

I have been running a different setup for the last week without corruption.
I'm running latest OpenElec XBMC and have the Pi in Turbo mode.
When running only on SD card I quickly got corruption, but when using a USB-stick as disk where all the changes is made everything runs perfectly!

so, why corruption on SD card? Must find a solution for this Awesome product!

Were you running with core_freq overclocked when you had no courruption?

Unfortunately I think the only fix is to lower the core_freq until you get no corruption. core_freq=250 should always be safe. You may be able to go higher.

[TikkenBaxter]

I'm running with Turbo-settings (1000,500,500,6) and uses a Kingston 4Gb Traveler G3 USB memory as disk. Done a lot of writings to disk and it runs without any problems so far.

Edit: but haven't had patient to change clock to work with only SD card.

[chochis]

My experience:
Today My rpi is working fine with the turbo mode at top speed. As I mentioned in this post before, had a partition corruption too before.
What changed? Now I'm using a USB hub that feeds the rpi at the same time I use the charger that I used before so now it have extra power from the USB. The charger is a Samsung for Nexus S. It says that gives 5v and 0.7a. Now the USB hub is receiving 6v and 1a.
So, seems to me that there is a problem with power. (Probably the big problem is on the charger that says 0.7 but it is not giving it constantly but I don’t have the tools to check it. Should be possible to put some checks on the governor to check also for the power?)

[Synthetic_Darkness]

For my RPi it seems that the only thing corrupting the FAT partition on the SD card is, whether the LAN cable is plugged in or not. If I run the RPi without a LAN cable connected, I can change whatever overclocking settings I want, even manually. But if I reboot and plug the LAN cable back in, every single change I make to the /boot/config.txt file corrupts it.

I have not got any other corruption.

[dom]

There are some parameters that may be worth playing with.
You can slow down the dma transfer to the sdcard a little with:

Code: Select all

echo 31> /sys/devices/platform/bcm2708_sdhci.0/dma_wait

and a lot with:

Code: Select all

echo 0 > /sys/devices/platform/bcm2708_sdhci.0/use_dma

I'd be interested if it has any effect on the corruption.

Code: Select all

sudo hdparm -t /dev/mmcblk0
/dev/mmcblk0:
 Timing buffered disk reads:  58 MB in  3.08 seconds =  18.82 MB/sec

echo 31 > /sys/devices/platform/bcm2708_sdhci.0/dma_wait
sudo hdparm -t /dev/mmcblk0
/dev/mmcblk0:
 Timing buffered disk reads:  44 MB in  3.04 seconds =  14.46 MB/sec

echo 0 > /sys/devices/platform/bcm2708_sdhci.0/use_dma
sudo hdparm -t /dev/mmcblk0
/dev/mmcblk0:
 Timing buffered disk reads:   2 MB in 10.51 seconds = 194.83 kB/sec

[dom]

https://dl.dropbox.com/u/3669512/temp/kernel_sdcard.img

has a kernel with a new command line parameter:
sdhci-bcm2708.cycle_delay=1000

allows you to get the number of cycles delay between writes to sdcard peripheral registers, in case that is the problem. Please test if it helps.

[Chris_777]

Hello

I didn't have time to read all 24 pages of this thread, only some of it. But I thought I would toss out my experience with over clocking and corruption in case it might help fix the problem as I love my Pi 
I ran the PI for a few weeks using a Sandisk extreme 16GB SD card rated for 45MBs and all was fine.
I updated and turned on Turbo mode to 1Gz and all ran fine, even ran Nbench benchmark several times and it did updates fine. I turned off the pi and started it the next day and downloaded the Quake 3 packs and when I unzipped the bigger of the two, about 45MBs in size, the pi froze. I tried restarting and it gets to about the 4 second mark of booting where its starting the USB devices and it freezes. I unplugged it and let it sit, still the same thing, the card its self can still be read in windows so I turned off all over clocking in the config.txt file but still it wouldn't boot. I finally got another SD card out and imaged it and the PI boots just fine. So it seems that high I/O on a fast SD card corrupted the ext4 partition?
I see that Dom posted a kernel img with a new command line parameter, how do I go about testing this? Just burn that 5MB img to a SD card and run it?

Thanks
Chris

[TikkenBaxter]

Synthetic_Darkness

My setup did run just fine! Until I tried using LAN instead of Wifi. As soon as I started booting with LAN my USB-stick got corrupted.
When I'm at work and have time playing with my Pi, I have it connected with a 1,5m LAN cable to a computer. There it runs perfectly and won't get corrupted.
But, when I come home and connect it to my router with a 10 meter cable, I got corruption.

Using a HTC 5V 1A adapter as power.

[kevs3d]

https://dl.dropbox.com/u/3669512/temp/kernel_sdcard.img

has a kernel with a new command line parameter:
sdhci-bcm2708.cycle_delay=1000

allows you to get the number of cycles delay between writes to sdcard peripheral registers, in case that is the problem. Please test if it helps.

OK - sorry I'm not sure what you mean to try? Changing the value or retrieving the value somehow if it corrupts...?

Kev

[kevs3d]

There are some parameters that may be worth playing with.
You can slow down the dma transfer to the sdcard a little with:

Code: Select all

echo 31> /sys/devices/platform/bcm2708_sdhci.0/dma_wait

I'll give this a try, cheers.

Kev

[dom]

https://dl.dropbox.com/u/3669512/temp/kernel_sdcard.img

has a kernel with a new command line parameter:
sdhci-bcm2708.cycle_delay=1000

allows you to get the number of cycles delay between writes to sdcard peripheral registers, in case that is the problem. Please test if it helps.

OK - sorry I'm not sure what you mean to try? Changing the value or retrieving the value somehow if it corrupts...?

Kev

I'm suggesting if you are currently suffering corruption, to download the linked kernel. Put it on boot partition (named kernel.img).
Edit cmdline.txt from boot parttition and add sdhci-bcm2708.cycle_delay=1000 to start of it. Report back if you still see corruption.

[Tompen]

I am testing the kernel and sdhci-bcm2708.cycle_delay=1000
Findings so far (I am guessing these are kind of expected):
http://pastebin.com/kvc50gv0
Have not yet started data corruption testing.

[Tompen]

I have been doing lots of testing, and have now reached the following conclusion.
I can consistently reproduce silent data corruption with the following settings:
current_limit_override=0x5A000020
over_voltage_sdram=5
over_voltage=4
force_turbo=1
arm_freq=1026
sdram_freq=500
core_freq=300

Same settings without core_freq=300 does not cause corruption.

There is one more thing needed for corruption to occur.
XBMC must be running. Just having it sit at the homescreen with RSS-feed enabled is enough to cause corruption. If I have core_freq=300 but without xbmc running, I do not get corruption. Maybe other application also cause corruption, but this is a rock solid way of reproducing for me. Sometimes I killed xbmc during my previous testing. That caused unreliable results in my testing.

When I get corruption, it is during write of file to SD-card. Reading file is OK.
During conditions explained above, when I get corruption during write to SD-card, I can write OK to usb drive. So for me, it is only write to SD-card that have this data corruption issue.

When I get corruption, sometimes the following is logged in dmesg during the write of the file to SD-card:

mmc0: final write to SD card still running
mmc0: Timeout waiting for hardware interrupt - cmd12.
mmcblk0: error -110 sending stop command, original cmd response 0x900, card status 0x900

I do not get messages like this everytime I get corruption, but sometimes.
I do not know if there is a relationship between these messages and the data corruption.
I never get the above messages in dmesg when writes are ok.

On even more rare occasions, I also get the following in dmesg during a write that cause data corruption. http://pastebin.com/rNek0rsn