Page 1 of 1

CM3 module crash after ejecting sd card

Posted: Tue Jun 12, 2018 11:52 am
by sirhc
Hi all,

I have a very annoying issue with the CM3 module which is used in a custom carrier board. After ejecting the external sd-card (SD1 interface) the complete module will crash. I've already removed the non-removable stuff from the devicetree. So I'm able to insert and remove the sd-card after the CM3 has booted. In /var/log/syslog I can see the card detect and remove messages so this works fine. However when I'm writing a (big) file to the sd-card and eject the card (I know you should not do something like this, but it's always possible somebody removes a card) the whole CM3 crashes.
I've tested this with a 4.9 kernel from last year and also the lastest raspbian release. Both images result in the same issue.
Any idea where to look and to solve this issue?

Regards,
Chris

Re: CM3 module crash after ejecting sd card

Posted: Tue Jun 12, 2018 12:00 pm
by PhilE
1. Have you looked at /var/log/syslog for error messages?
2. Do you see any messages on the display or via a serial port on GPIO14&15?

Re: CM3 module crash after ejecting sd card

Posted: Tue Jun 12, 2018 12:05 pm
by sirhc
Hi,

1. No error messages in /var/log/syslog
2. Nothing in the serial debug terminal. I'm using GPIO32/GPIO33. I have a LCD connected so the GPIO14/GPIO15 pins are in use.

Regards,

Chris

Re: CM3 module crash after ejecting sd card

Posted: Tue Jun 12, 2018 12:16 pm
by PhilE
So does it just freeze? Or reboot? Any information you give may be useful.

Re: CM3 module crash after ejecting sd card

Posted: Tue Jun 12, 2018 12:22 pm
by sirhc
Correct. No reboot. Just freeze. No warning message or whatever in the console or log.
I have to do a manual reboot to get it working again.
Inserting removing the card is no issue. /var/log/syslog confirms. So only when I remove the card during the writing proces of a large file the CM3 freezes. If I don't remove the card during writing there is no issue at all.

Re: CM3 module crash after ejecting sd card

Posted: Tue Jun 12, 2018 1:29 pm
by PhilE
It does sound a bit like the old joke:
Patient: Doctor, it hurts when I <something random>.
Doctor: Well, don't <something random>.
Despite that, I took a look at the problem (using an NFS-rooted Pi 3B+), and I also found it didn't respond well to having the card removed during writes. I thought it was possibly more reliable using the sdhost (rather than mmc) interface on the SD card, but I managed to crash an ext4 process, after which it was very unhappy.

Perhaps the Dr. was right.

Re: CM3 module crash after ejecting sd card

Posted: Tue Jun 12, 2018 1:57 pm
by sirhc
Luckily another patient has the same problem :D

Re: CM3 module crash after ejecting sd card

Posted: Tue Jun 12, 2018 1:59 pm
by PhilE
I think the same advice applies. Seriously, what is your use case that makes it sensible to allow an SD card to be removed while it is being written to?

Re: CM3 module crash after ejecting sd card

Posted: Tue Jun 12, 2018 2:33 pm
by sirhc
I fully agree you should not remove the card during any operation at all. So certainly it's not a use case. Unfortunately the sd-card is free accessible just as in a laptop so people are able to remove the card.

Re: CM3 module crash after ejecting sd card

Posted: Tue Jun 12, 2018 2:37 pm
by PhilE
What is the data being written to the card?

Have you considered a different filing system?

Re: CM3 module crash after ejecting sd card

Posted: Tue Jun 12, 2018 5:01 pm
by sirhc
At the moment just some log data. Customer will use the card to exchange some files. So in the end there's always a possibility somebody ejects the card too early. The application software should give an error on the screen or something like that. Freezing is not an option for this design. That means somebody has to unplug and take the whole machine offline.
I tried the same thing on another carrierboard and ARM module (imx6) I've made some time ago. So when I write some files the kernel gives some I/O errors in /var/log/syslog and error -110 requesting status.
Any help would be very appreciated.

Regards,
Chris

Re: CM3 module crash after ejecting sd card

Posted: Wed Jun 13, 2018 10:30 am
by gsh
I think you've found one of the reasons that mobile phones do not have externally accessible SD card storage, if they have it it is under the battery or inserted with the SIM card...

Gordon

Re: CM3 module crash after ejecting sd card

Posted: Wed Jun 13, 2018 5:33 pm
by sirhc
Would be nice if we can find a solution for this issue

Re: CM3 module crash after ejecting sd card

Posted: Wed Jun 13, 2018 7:08 pm
by rpdom
gsh wrote:
Wed Jun 13, 2018 10:30 am
I think you've found one of the reasons that mobile phones do not have externally accessible SD card storage, if they have it it is under the battery or inserted with the SIM card...
My old phone did have an external microSD slot. It was very handy for getting photos off it an onto my PC. My current one (yes, it's a bit old), has the SD inside the back cover, but can be removed without removing the battery or SIM card. Just select "Unmount SD card" from the menus and it is fine.

Re: CM3 module crash after ejecting sd card

Posted: Wed Jun 13, 2018 7:16 pm
by PhilE
Did you ever try pulling the card out while, for example, recording video to it?

Re: CM3 module crash after ejecting sd card

Posted: Wed Jun 13, 2018 7:27 pm
by rpdom
PhilE wrote:
Wed Jun 13, 2018 7:16 pm
Did you ever try pulling the card out while, for example, recording video to it?
No. I was very careful to wait until everything had finished writing to the card. It is a Windows Mobile phone (from around 2006). Still works a bit.

Re: CM3 module crash after ejecting sd card

Posted: Tue Jun 19, 2018 7:54 am
by stevegee
Hi,

This discussion appears to have ended on a side-track.

I am having the same issue with a SD card on SD1. I fully agree that the kernel should be able to cope with an (illegally) ejected sd card and not freeze up.
Any ideas on how to dig deeper into this problem?

regards,

Steve

Re: CM3 module crash after ejecting sd card

Posted: Tue Jun 19, 2018 1:38 pm
by rln
You might want to try the same test with my Nard SDK, which is specifically designed to be robust and withstand abuse. Use a regular Pi and eject the SD card during heavy writing to a test-file in /sdcard/ It should survive and recover fine.

http://www.nard.se/

Re: CM3 module crash after ejecting sd card

Posted: Wed Jun 27, 2018 7:59 am
by sirhc
Hi all,

Even without removing the sd-card I have had a complete freeze a couple of times. If I don't use the sd-card everything seems to be working fine up to now. Somehow I have the feeling the kind of glitch is the same as when you remove the sd-card.
I tried the sd_debug parameter in the device-tree. But this enables debugging for SD0. Is there a simple way to enable the same debug info from SD1?