juranga
Posts: 183
Joined: Fri Nov 06, 2015 11:39 am
Location: Basque Country

Power Failure simulation with watchdog

Wed May 24, 2017 8:07 am

Hi,

I want to test my raspberry pi 3 for power failure situations and I want to simulate it. I have been using the hardware watchdog in order to simulate a power failure but I am not sure if this is the correct way.

How does the watchdog shutdown the raspberry pi? Does the wd just cut the power or perform a safe shutdown??

Many thanks!

User avatar
dividuum
Posts: 119
Joined: Sun Jun 16, 2013 1:18 pm
Location: Germany
Contact: Website

Re: Power Failure simulation with watchdog

Wed May 24, 2017 11:59 pm

juranga wrote:How does the watchdog shutdown the raspberry pi? Does the wd just cut the power or perform a safe shutdown??
As it's a hardware watchdog, it forcefully resets the system if you fail to send keepalives. It is a hard reset and Linux has no way to react that that. So it's definitely not a safe shutdown. The closest to that in software is probably running '/sbin/reboot -f'.
info-beamer hosted - A user and programmer friendly digital signage platform for the Pi: https://info-beamer.com/hosted

juranga
Posts: 183
Joined: Fri Nov 06, 2015 11:39 am
Location: Basque Country

Re: Power Failure simulation with watchdog

Thu May 25, 2017 6:12 am

Okay! Thank you for the response.

So as I understood, using the watchdog to simulate a power failure is correct, isn´t it??

MaxK1
Posts: 1043
Joined: Sun Aug 26, 2012 11:34 pm

Re: Power Failure simulation with watchdog

Sun May 28, 2017 3:24 am

Why not just pull the plug - that cuts power to everything (sdcard etc.)
You are in a maze of twisty little passages, all alike.
When General Failure and Major Disaster get together, Private Parts usually suffers.

lbog
Posts: 167
Joined: Sun Mar 06, 2016 9:44 am

Re: Power Failure simulation with watchdog

Sun May 28, 2017 11:26 am

juranga wrote: I want to test my raspberry pi 3 for power failure situations and I want to simulate it.
Another way is

Code: Select all

echo o | sudo tee /proc/sysrq-trigger
That shutdown the system immediately without killing processes, unmounting partitions or syncing mounted file system and doesn't reboot.

MaxK1
Posts: 1043
Joined: Sun Aug 26, 2012 11:34 pm

Re: Power Failure simulation with watchdog

Sun May 28, 2017 12:16 pm

But it retains power to the SD card so that means the SD card controller can do whatever "housekeeping" it does. A real power failure might not keep the power to the SD card long enough to allow it to finish anything it started.
You are in a maze of twisty little passages, all alike.
When General Failure and Major Disaster get together, Private Parts usually suffers.

juranga
Posts: 183
Joined: Fri Nov 06, 2015 11:39 am
Location: Basque Country

Re: Power Failure simulation with watchdog

Mon May 29, 2017 6:31 am

lbog wrote: Another way is

Code: Select all

echo o | sudo tee /proc/sysrq-trigger
That shutdown the system immediately without killing processes, unmounting partitions or syncing mounted file system and doesn't reboot.
I need to reboot the raspberry pi to continue with the tests.
MaxK1 wrote:Why not just pull the plug - that cuts power to everything (sdcard etc.)
I want an automated process to test the system. Other way will be to use another microprocessor with a relay to simulate power failures, but I would have to "mount" it.


So I guess that my approach to simulate power failure was not correct, was it? I had the system with the watchdog (wd reset each 10 minutes, 470 times in total) the whole weekend and the raspberry pi is still working perfectly, no file-system corruption. So, the system is really reliable of the approach of the testing is not correct.

lbog
Posts: 167
Joined: Sun Mar 06, 2016 9:44 am

Re: Power Failure simulation with watchdog

Mon May 29, 2017 7:05 am

juranga wrote:
lbog wrote: Another way is

Code: Select all

echo o | sudo tee /proc/sysrq-trigger
That shutdown the system immediately without killing processes, unmounting partitions or syncing mounted file system and doesn't reboot.
I need to reboot the raspberry pi to continue with the tests.
For reboot (... without unmounting partitions or syncing), you can try

Code: Select all

echo b | sudo tee /proc/sysrq-trigger

MaxK1
Posts: 1043
Joined: Sun Aug 26, 2012 11:34 pm

Re: Power Failure simulation with watchdog

Mon May 29, 2017 8:14 am

Are you trying to simulate a power failure from the power co.? If so, you need to worry about what happens when power is "restored" as well. Around here, a "typical" power failure is rather abrupt. (storms taking down power lines, lightning strike on a sub-station, etc.) When power is restored (seconds to hours later) it often comes back as a series of "false starts" (It comes back on and within a few seconds fails again. This sequence could repeat "n" times before "stable" power is restored.) What happens to a Pi if it loses power during a reboot, especially when it is running a file system check?

Would a UPS "solve" the problem? ($$$) or a power bank that allows simultaneous charge/dis-charge?
You are in a maze of twisty little passages, all alike.
When General Failure and Major Disaster get together, Private Parts usually suffers.

User avatar
davidcoton
Posts: 3811
Joined: Mon Sep 01, 2014 2:37 pm
Location: Cambridge, UK

Re: Power Failure simulation with watchdog

Mon May 29, 2017 9:03 am

juranga wrote:So I guess that my approach to simulate power failure was not correct, was it? I had the system with the watchdog (wd reset each 10 minutes, 470 times in total) the whole weekend and the raspberry pi is still working perfectly, no file-system corruption. So, the system is really reliable of the approach of the testing is not correct.
If you want to test power failure, your test must remove the power from the Pi. The watchdog by itself doesn't do that. So you are right, the method of testing is not correct. You are testing recovery from a watchdog-detected failure ... though of course there has been no failure, the exact mode of potential failure is unknown, so you can't know that a Pi with a real watchdog event will actuially restart....
MaxK1 wrote: Would a UPS "solve" the problem? ($$$) or a power bank that allows simultaneous charge/dis-charge?
IMO the power bank approach is more appropriate than a mains UPS. See here.
Signature retired

MaxK1
Posts: 1043
Joined: Sun Aug 26, 2012 11:34 pm

Re: Power Failure simulation with watchdog

Mon May 29, 2017 11:34 am

Since we don't really know what the OP is trying to do, it's hard to say what the "best" approach would be. Right now, all I can guess is he wants to know how reliable is a Pi following a power failure. My thoughts are to worry more about preventing loss of power in the first place. Or he could be trying to figure which SD cards handle power failure better than others in which case UPS .vs. Power Bank is not important, but simulating a true power loss is. Need the OP to chime in here :-)
You are in a maze of twisty little passages, all alike.
When General Failure and Major Disaster get together, Private Parts usually suffers.

User avatar
davidcoton
Posts: 3811
Joined: Mon Sep 01, 2014 2:37 pm
Location: Cambridge, UK

Re: Power Failure simulation with watchdog

Mon May 29, 2017 2:23 pm

MaxK1 wrote:Since we don't really know what the OP is trying to do,
juranga wrote:I want to test my raspberry pi 3 for power failure situations and I want to simulate it.
We know enough to know that stopping the watchdog does not simulate power failure. The OP asked about how to test, not about how to prevent the situation. Any thoughts (including mine) about prevention are actually OT.
Signature retired

mosespi
Posts: 508
Joined: Mon May 12, 2014 3:35 pm
Location: 34,-118
Contact: Website

Re: Power Failure simulation with watchdog

Mon May 29, 2017 6:39 pm

I have a similar test.. still ongoing. I got to 26,823 successful boot ups for the regular test and it is on # 21,056 for the write test!!! See:

viewtopic.php?t=182519

This was all done using my UPS for the Pi. A supervisor circuit (micro-controller) controls the power state of a switching DC regulator that feeds the Pi, and is reconfigured to simulate a power cut.

Regards,
-Moses
Power problems? MoPower UPS for the Pi
http://www.allspectrum.com/mopower/

juranga
Posts: 183
Joined: Fri Nov 06, 2015 11:39 am
Location: Basque Country

Re: Power Failure simulation with watchdog

Tue May 30, 2017 9:01 am

mosespi wrote:I have a similar test.. still ongoing. I got to 26,823 successful boot ups for the regular test and it is on # 21,056 for the write test!!! See:

viewtopic.php?t=182519

This was all done using my UPS for the Pi. A supervisor circuit (micro-controller) controls the power state of a switching DC regulator that feeds the Pi, and is reconfigured to simulate a power cut.

Regards,
-Moses
That's a really great post, thank you for the link!

As I told before, my goal is to test the raspberry pi in power failure situations, so it is clear that I was using the incorrect approach. Once I check how my system reacts to power failures (I will test the system for different OS, read-only, a read-write partition writing data,etc) I will consider alternatives to prevent it but not yet.

I will finally test my system using an external microprocessor (Arduino) with an relay to control the power.

Return to “Advanced users”