Pitel
Posts: 25
Joined: Fri Jun 22, 2012 1:37 pm
Location: Brno, CZE
Contact: Website Facebook Google+ Skype Twitter YouTube

Kernel panics since stretch update

Thu Aug 31, 2017 6:24 am

Traces: https://pastebin.com/raw/ZHKNtPyr
  • Using Raspbian minimal as web server, RetroPie and Kodi. No X, everything from terminal/ssh.
  • Using F2FS. (I suspect this might be the cause)
  • There were some FS corruption (undeletable file with 0 size), but I was able to fix it with fsck.
  • It happens every day or two, mostly randomly, out of the blue.
  • Always using latest kernel/frimware from rpi-update
Any clue? Is it some known bug in Stretch? Everything ran perfectly until I updated (apt dist-upgrade).

Rascas
Posts: 668
Joined: Tue Mar 11, 2014 6:18 pm
Location: Porto, Portugal
Contact: Website

Re: Kernel panics since stretch update

Thu Aug 31, 2017 12:00 pm

RetroPie is still based on Jessie, alot of things will fail if you force the update o Stretch. You will have to wait for the Stretch version.
The corruption of the filesystem is not related to distro version, it depends only on your hardware.

Pitel
Posts: 25
Joined: Fri Jun 22, 2012 1:37 pm
Location: Brno, CZE
Contact: Website Facebook Google+ Skype Twitter YouTube

Re: Kernel panics since stretch update

Thu Aug 31, 2017 12:30 pm

Yeah, I know, I'm subscribed to related issues on RetroPie's GitHub. And therefore I don't play games on Stretch until tyehy fix them. But I gets panics even without running ReproPie/EmulationStation.

beta-tester
Posts: 1372
Joined: Fri Jan 04, 2013 1:57 pm
Location: de_DE

Re: Kernel panics since stretch update

Thu Aug 31, 2017 2:47 pm

Pitel wrote:
Thu Aug 31, 2017 6:24 am
Traces: https://pastebin.com/raw/ZHKNtPyr
  • ...
  • Always using latest kernel/frimware from rpi-update
Any clue? Is it some known bug in Stretch? Everything ran perfectly until I updated (apt dist-upgrade).
i thought, with rpi-update you always get an experimental firmware/kernel, those are possibly unstable.

did you do a upgrade or a dist-upgrade to stretch... or a stretch installationfrom scratch?
{ I only give negative feedback }
RPi B (256MB), B (512MB), B+, ZeroW; 2B; 3B, 3B+; 4B (4GB)

Pitel
Posts: 25
Joined: Fri Jun 22, 2012 1:37 pm
Location: Brno, CZE
Contact: Website Facebook Google+ Skype Twitter YouTube

Re: Kernel panics since stretch update

Thu Aug 31, 2017 3:16 pm

beta-tester wrote:
Thu Aug 31, 2017 2:47 pm
Pitel wrote:
Thu Aug 31, 2017 6:24 am
Traces: https://pastebin.com/raw/ZHKNtPyr
  • ...
  • Always using latest kernel/frimware from rpi-update
Any clue? Is it some known bug in Stretch? Everything ran perfectly until I updated (apt dist-upgrade).
i thought, with rpi-update you always get an experimental firmware/kernel, those are possibly unstable.

did you do a upgrade or a dist-upgrade to stretch... or a stretch installationfrom scratch?
dist-upgrade. Yeah, I know, clean upodate might be better. But I already have some web apps and nginx installed, configured and tweaked, so I really don't want to do that unless it's absolutely necessary.

beta-tester
Posts: 1372
Joined: Fri Jan 04, 2013 1:57 pm
Location: de_DE

Re: Kernel panics since stretch update

Thu Aug 31, 2017 4:40 pm

i don't know, how to do, but i think it is possible to "downgrade" the firmware/kernel to the official stable one with rpi-update and a parameter.
i would give it a try.
{ I only give negative feedback }
RPi B (256MB), B (512MB), B+, ZeroW; 2B; 3B, 3B+; 4B (4GB)


Pitel
Posts: 25
Joined: Fri Jun 22, 2012 1:37 pm
Location: Brno, CZE
Contact: Website Facebook Google+ Skype Twitter YouTube

Re: Kernel panics since stretch update

Sun Sep 03, 2017 6:30 am

I tried runnig badblocks on the SD card. It found something, but when I try to rerun it on the same blocks, they turned oiut ok. Kind of WTF.

beta-tester
Posts: 1372
Joined: Fri Jan 04, 2013 1:57 pm
Location: de_DE

Re: Kernel panics since stretch update

Sun Sep 03, 2017 7:40 am

i don't know F2FS...
how is your F2FS configured/working?
was the f2fs filesystem already corrupt before you dist-upgrade to stretch?
at dist-upgrade, did you answered the questions about configuration files with replace existing configuration with new distro configuration?
the badblocks, is it running on top of f2fs?
{ I only give negative feedback }
RPi B (256MB), B (512MB), B+, ZeroW; 2B; 3B, 3B+; 4B (4GB)

Pitel
Posts: 25
Joined: Fri Jun 22, 2012 1:37 pm
Location: Brno, CZE
Contact: Website Facebook Google+ Skype Twitter YouTube

Re: Kernel panics since stretch update

Sun Sep 03, 2017 6:19 pm

beta-tester wrote:
Sun Sep 03, 2017 7:40 am
i don't know F2FS...
how is your F2FS configured/working?
was the f2fs filesystem already corrupt before you dist-upgrade to stretch?
at dist-upgrade, did you answered the questions about configuration files with replace existing configuration with new distro configuration?
the badblocks, is it running on top of f2fs?
https://en.wikipedia.org/wiki/F2FS It should be more suitable for flash memories. And it indeed was. The response times of my websites were about 20 % faster and overall terminal work (Tab completion of file names) was much faster. I'm surprised the default image uses ext4.

I copied all files from original image, formatted the partition from ext4 to F2FS and copied the files back, changing fstab and boot.txt (or something). All this on my Ubuntu PC.

I don't think it was corrupt, it worked fine.

Yes, I rapleced the old configs with new ones from stretch. Unless the only diff showed was just my changes (like timezones, etc.)

I thionk the badblocks is running on any FS, it works with the device on block level.

beta-tester
Posts: 1372
Joined: Fri Jan 04, 2013 1:57 pm
Location: de_DE

Re: Kernel panics since stretch update

Mon Sep 04, 2017 9:44 am

do you have a spare SD card with similar size?
you can try to prepare the spare new SD card with the same partition layout as on the old SD card.
then make a file based copy from the old SD card to the new SD card.

then see, if the new SD card still shows those issues and if the system is running on that card.

i think this is the best you can do.

if the system on the new SD card is working, then you can "zero" the old one and format it newly - to have it for an other turn.
{ I only give negative feedback }
RPi B (256MB), B (512MB), B+, ZeroW; 2B; 3B, 3B+; 4B (4GB)

Pitel
Posts: 25
Joined: Fri Jun 22, 2012 1:37 pm
Location: Brno, CZE
Contact: Website Facebook Google+ Skype Twitter YouTube

Re: Kernel panics since stretch update

Thu Sep 07, 2017 8:20 am

Yeah. I'll probably do something similar. I'll install fresh stretch, install packages, copy webs and configs and I'll see.

Then, after a week or so, I'll convert the FS from ext4 to F2FS again.

I've been running badblocks on the SD card for a day now (5 checks) without any error. So it's probably some issue on filesystem level.

runboy93
Posts: 352
Joined: Tue Feb 28, 2017 1:17 pm
Location: Finland
Contact: Website

Re: Kernel panics since stretch update

Thu Sep 07, 2017 8:30 am

btw, why you use f2fs?

Pitel
Posts: 25
Joined: Fri Jun 22, 2012 1:37 pm
Location: Brno, CZE
Contact: Website Facebook Google+ Skype Twitter YouTube

Re: Kernel panics since stretch update

Thu Sep 07, 2017 8:39 am

runboy93 wrote:
Thu Sep 07, 2017 8:30 am
btw, why you use f2fs?
It should be better optimized for flash based storages. And it indeed was. The website's response time was almost one second lower, and tab completion in shell was also faster. It should also reduce wear of the flash memory, because it tries tries to use blocks more consistently, not trying to do everything from the start. As I said earlier, I'm surprised it's not default FS on Raspbian images.

runboy93
Posts: 352
Joined: Tue Feb 28, 2017 1:17 pm
Location: Finland
Contact: Website

Re: Kernel panics since stretch update

Thu Sep 07, 2017 8:50 am

Most likely they tested it, but maybe didn't see differences vs ext4 on RPi usage (For example my OP3T uses F2FS by default, huge random read boost, write/read boost but RPi write/read is quite limited even with OC card reader, so there would not be dramatic boosts / changes devs are looking for)

everything is optimized for ext4, I bet most likely fault is f2fs.

Return to “Troubleshooting”