User avatar
Rive
Posts: 586
Joined: Sat Mar 26, 2016 5:21 pm
Location: USA

Re: Raspbian Jessie linux 4.4.9 Severe Performance Degradati

Thu May 12, 2016 5:22 pm

PhilE wrote:With the same actual core speed, I don't see any significant difference in write performance between 4.1 and 4.4 - around 21.5 MB/s on 4.1 and 21.1 MB/s on 4.4 with my Verbatim card.

I still think it is worth trying core_freq_min=200.
I am not home at the moment..

Is the 21MB/s with turbo=1, and core_freq_min=200? I'll check it soon as I can and report back.

Any ideas on the sandisk SDXC write speeds issue? That is the card I really want to use (if I can resolve the write speeds issue)
DNPNWO

User avatar
Rive
Posts: 586
Joined: Sat Mar 26, 2016 5:21 pm
Location: USA

Re: Raspbian Jessie linux 4.4.9 Severe Performance Degradati

Thu May 12, 2016 10:01 pm

Benchmark baseline /config.txt

Code: Select all

dtparam=sd_overclock=100
arm_freq=1270
core_freq=500
over_voltage=4
sdram_freq=575
sdram_schmoo=0x02000020
over_voltage_sdram_p=6
over_voltage_sdram_i=4
over_voltage_sdram_c=4
v3d_freq=500
h264_freq=333
gpu_mem=192
Sysbench 113.4291s

Code: Select all

[email protected]:~ $ sysbench --num-threads=4 --test=cpu --cpu-max-prime=20000 --validate run
sysbench 0.4.12:  multi-threaded system evaluation benchmark

Running the test with following options:
Number of threads: 4
Additional request validation enabled.


Doing CPU performance benchmark

Threads started!
Done.

Maximum prime number checked in CPU test: 20000


Test execution summary:
    total time:                          113.4291s
    total number of events:              10000
    total time taken by event execution: 453.5869
    per-request statistics:
         min:                                 45.06ms
         avg:                                 45.36ms
         max:                                125.23ms
         approx.  95 percentile:              45.65ms

Threads fairness:
    events (avg/stddev):           2500.0000/12.47
    execution time (avg/stddev):   113.3967/0.02
Linpack bench 6.72 Gflops

Code: Select all

[email protected]:~ $ ./xhpl
================================================================================
HPLinpack 2.1  --  High-Performance Linpack benchmark  --   October 26, 2012
Written by A. Petitet and R. Clint Whaley,  Innovative Computing Laboratory, UTK
Modified by Piotr Luszczek, Innovative Computing Laboratory, UTK
Modified by Julien Langou, University of Colorado Denver
================================================================================

An explanation of the input/output parameters follows:
T/V    : Wall time / encoded variant.
N      : The order of the coefficient matrix A.
NB     : The partitioning blocking factor.
P      : The number of process rows.
Q      : The number of process columns.
Time   : Time in seconds to solve the linear system.
Gflops : Rate of execution for solving the linear system.

The following parameter values will be used:

N      :    8000 
NB     :     256 
PMAP   : Row-major process mapping
P      :       1 
Q      :       1 
PFACT  :    Left 
NBMIN  :       2 
NDIV   :       2 
RFACT  :   Right 
BCAST  :   2ring 
DEPTH  :       0 
SWAP   : Mix (threshold = 64)
L1     : transposed form
U      : transposed form
EQUIL  : yes
ALIGN  : 8 double precision words

--------------------------------------------------------------------------------

- The matrix A is randomly generated for each test.
- The following scaled residual check will be computed:
      ||Ax-b||_oo / ( eps * ( || x ||_oo * || A ||_oo + || b ||_oo ) * N )
- The relative machine precision (eps) is taken to be               1.110223e-16
- Computational tests pass if scaled residuals are less than                16.0

================================================================================
T/V                N    NB     P     Q               Time                 Gflops
--------------------------------------------------------------------------------
WR02R2L2        8000   256     1     1              50.80              6.721e+00
HPL_pdgesv() start time Thu May 12 17:51:44 2016

HPL_pdgesv() end time   Thu May 12 17:52:35 2016

--------------------------------------------------------------------------------
||Ax-b||_oo/(eps*(||A||_oo*||x||_oo+||b||_oo)*N)=        0.0025941 ...... PASSED
================================================================================

Finished      1 tests with the following results:
              1 tests completed and passed residual checks,
              0 tests completed and failed residual checks,
              0 tests skipped because of illegal input values.
--------------------------------------------------------------------------------

End of Tests.
=======================================================
=========================

Sdbench
Turbo=0
100.000 MHz 18.68 MB/s 36.22 MB/s 35.22 MB/s

Code: Select all

[email protected]:~ $ sudo ./sdbench.sh
[3;J
CONFIG: 
CLOCK : 100.000 MHz
CORE  : 500 MHz, turbo=0
DATA  : 512 MB, /root/test.dat

HDPARM:
======
 Timing O_DIRECT disk reads: 106 MB in  3.01 seconds =  35.24 MB/sec
 Timing O_DIRECT disk reads: 106 MB in  3.01 seconds =  35.21 MB/sec
 Timing O_DIRECT disk reads: 106 MB in  3.01 seconds =  35.25 MB/sec

WRITE:
=====
536870912 bytes (537 MB) copied, 30.4102 s, 17.7 MB/s
536870912 bytes (537 MB) copied, 26.367 s, 20.4 MB/s
536870912 bytes (537 MB) copied, 25.8667 s, 20.8 MB/s

READ:
====
536870912 bytes (537 MB) copied, 14.2134 s, 37.8 MB/s
536870912 bytes (537 MB) copied, 14.095 s, 38.1 MB/s
536870912 bytes (537 MB) copied, 14.0941 s, 38.1 MB/s

RESULT (AVG):
============
Overlay config                      core_freq   turbo   overclock_50    WRITE        READ        HDPARM
                                       500        0     100.000 MHz   18.68 MB/s   36.22 MB/s   35.22 MB/s

Sdbench
Turbo=0
core_freq_min=200
100.000 MHz 16.75 MB/s 38.96 MB/s 41.00 MB/s
./sdbench.sh: line 86: printf: 500
200: invalid number

Code: Select all

[email protected]:~ $ sudo ./sdbench.sh
#[3;J
CONFIG: 
CLOCK : 100.000 MHz
CORE  : 500
200 MHz, turbo=0
DATA  : 512 MB, /root/test.dat

HDPARM:
======
 Timing O_DIRECT disk reads: 130 MB in  3.04 seconds =  42.73 MB/sec
 Timing O_DIRECT disk reads: 114 MB in  3.01 seconds =  37.83 MB/sec
 Timing O_DIRECT disk reads: 128 MB in  3.02 seconds =  42.37 MB/sec

WRITE:
=====
536870912 bytes (537 MB) copied, 45.7231 s, 11.7 MB/s
536870912 bytes (537 MB) copied, 25.6236 s, 21.0 MB/s
536870912 bytes (537 MB) copied, 26.854 s, 20.0 MB/s

READ:
====
536870912 bytes (537 MB) copied, 13.0739 s, 41.1 MB/s
536870912 bytes (537 MB) copied, 13.2472 s, 40.5 MB/s
536870912 bytes (537 MB) copied, 13.1077 s, 41.0 MB/s

RESULT (AVG):
============
Overlay config                      core_freq   turbo   overclock_50    WRITE        READ        HDPARM
./sdbench.sh: line 86: printf: 500
200: invalid number
                                       500        0     100.000 MHz   16.75 MB/s   38.96 MB/s   41.00 MB/s
Sdbench
force_turbo=1
100.000 MHz 16.19 MB/s 41.61 MB/s 43.31 MB/s

Code: Select all

[email protected]:~ $ sudo ./sdbench.sh
#[3;J
CONFIG: 
CLOCK : 100.000 MHz
CORE  : 500 MHz, turbo=1
DATA  : 512 MB, /root/test.dat

HDPARM:
======
 Timing O_DIRECT disk reads: 132 MB in  3.04 seconds =  43.38 MB/sec
 Timing O_DIRECT disk reads: 130 MB in  3.00 seconds =  43.31 MB/sec
 Timing O_DIRECT disk reads: 130 MB in  3.01 seconds =  43.14 MB/sec

WRITE:
=====
536870912 bytes (537 MB) copied, 47.0406 s, 11.4 MB/s
536870912 bytes (537 MB) copied, 25.127 s, 21.4 MB/s
536870912 bytes (537 MB) copied, 29.6009 s, 18.1 MB/s

READ:
====
536870912 bytes (537 MB) copied, 12.3044 s, 43.6 MB/s
536870912 bytes (537 MB) copied, 12.2942 s, 43.7 MB/s
536870912 bytes (537 MB) copied, 12.3208 s, 43.6 MB/s

RESULT (AVG):
============
Overlay config                      core_freq   turbo   overclock_50    WRITE        READ        HDPARM
                                       500        1     100.000 MHz   16.19 MB/s   41.61 MB/s   43.31 MB/s
Last edited by Rive on Fri May 13, 2016 11:54 am, edited 5 times in total.
DNPNWO

Heater
Posts: 13105
Joined: Tue Jul 17, 2012 3:02 pm

Re: Raspbian Jessie linux 4.4.9 Severe Performance Degradati

Fri May 13, 2016 6:56 am

dom,

Thanks for the explanation of vcgencmd get_throttled.

With application of a crappy heat sink, a scrap of aluminium, I finally got my Pi 3 to run xhpl at a more respectable rate:

Code: Select all

================================================================================
T/V                N    NB     P     Q               Time                 Gflops
--------------------------------------------------------------------------------
WR02R2L2        8000   256     1     1              55.07              6.200e+00
H
and aparently with no throttling but with a marginal supply voltage:

Code: Select all

$ vcgencmd get_throttled
throttled=0x30000

PhilE
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 2315
Joined: Mon Sep 29, 2014 1:07 pm
Location: Cambridge

Re: Raspbian Jessie linux 4.4.9 Severe Performance Degradati

Fri May 13, 2016 8:07 am

@Rive You present your figures with no comment. How do you feel about them? Bear in mind that by not using turbo mode the lifespan of the chip is greater, so I feel that if we can get close to that performance peak then we are doing well.

Looking at the sdbench results I notice that the read results are consistent whereas the writes are more variable. There is always the danger that the test will coincide with some other activity that will pull one of the results down, but if the write performance really is variable then that would be hidden by just picking the best results (as I am inclined to do).

User avatar
Rive
Posts: 586
Joined: Sat Mar 26, 2016 5:21 pm
Location: USA

Re: Raspbian Jessie linux 4.4.9 Severe Performance Degradati

Fri May 13, 2016 11:35 am

PhilE wrote:@Rive You present your figures with no comment. How do you feel about them? Bear in mind that by not using turbo mode the lifespan of the chip is greater, so I feel that if we can get close to that performance peak then we are doing well.

Looking at the sdbench results I notice that the read results are consistent whereas the writes are more variable. There is always the danger that the test will coincide with some other activity that will pull one of the results down, but if the write performance really is variable then that would be hidden by just picking the best results (as I am inclined to do).
I can live with the turbo=0 and with the core_freq=500 results. As for Write speeds, it is variable. I usually run the bench multiple times to get an average...which is around Write 18 MB/s, and Read 35 MB/s.

I am running 4.4.9 kernel atm so I am satisfied.
DNPNWO

User avatar
Rive
Posts: 586
Joined: Sat Mar 26, 2016 5:21 pm
Location: USA

Re: Raspbian Jessie linux 4.4.9 Severe Performance Degradati

Fri May 13, 2016 9:21 pm

dom wrote:
Rive wrote: So does this mean that the BT is causing the degradation?
No, not BT directly. There are two uarts, either one can be used by BT, and the other can be used for serial output.
I think the problem is with an unused uart is being left with an interrupt asserted but no interrupt handler to clear it
(but the asserted interrupt does waste CPU). So that is not a problem and should be easy to fix.
I take it this fix has yet to be implemented in the newly released 4.4.9 'dist-upgrade'.

Things broke when I tried to go from the 4.4.9 rpi-update to the dist-upgrade, so I reloaded 4.1, and did the dist-upgrade.


Did some benches, and with the new bt enabled, there is performance degradation, with bt disabled, the performance degradation is gone.

so, I assume this is still the same issue plaguing the new 'dist-upgrade'?
unused uart is being left with an interrupt asserted but no interrupt handler to clear it
How long before this makes it to the apt-get update/apt-get upgrade? Do I need to 'rpi-update' again to get it now?


Sysbench 114.9521s

Code: Select all

[email protected]:~ $ sysbench --num-threads=4 --test=cpu --cpu-max-prime=20000 --validate run
sysbench 0.4.12:  multi-threaded system evaluation benchmark

Running the test with following options:
Number of threads: 4
Additional request validation enabled.


Doing CPU performance benchmark

Threads started!
Done.

Maximum prime number checked in CPU test: 20000


Test execution summary:
    total time:                          114.9521s
    total number of events:              10000
    total time taken by event execution: 459.6770
    per-request statistics:
         min:                                 45.06ms
         avg:                                 45.97ms
         max:                                122.50ms
         approx.  95 percentile:              51.30ms

Threads fairness:
    events (avg/stddev):           2500.0000/74.14
    execution time (avg/stddev):   114.9193/0.02
Linpack 5.89 Gflops

Code: Select all

[email protected]:~ $ ./xhpl
================================================================================
HPLinpack 2.1  --  High-Performance Linpack benchmark  --   October 26, 2012
Written by A. Petitet and R. Clint Whaley,  Innovative Computing Laboratory, UTK
Modified by Piotr Luszczek, Innovative Computing Laboratory, UTK
Modified by Julien Langou, University of Colorado Denver
================================================================================

An explanation of the input/output parameters follows:
T/V    : Wall time / encoded variant.
N      : The order of the coefficient matrix A.
NB     : The partitioning blocking factor.
P      : The number of process rows.
Q      : The number of process columns.
Time   : Time in seconds to solve the linear system.
Gflops : Rate of execution for solving the linear system.

The following parameter values will be used:

N      :    8000 
NB     :     256 
PMAP   : Row-major process mapping
P      :       1 
Q      :       1 
PFACT  :    Left 
NBMIN  :       2 
NDIV   :       2 
RFACT  :   Right 
BCAST  :   2ring 
DEPTH  :       0 
SWAP   : Mix (threshold = 64)
L1     : transposed form
U      : transposed form
EQUIL  : yes
ALIGN  : 8 double precision words

--------------------------------------------------------------------------------

- The matrix A is randomly generated for each test.
- The following scaled residual check will be computed:
      ||Ax-b||_oo / ( eps * ( || x ||_oo * || A ||_oo + || b ||_oo ) * N )
- The relative machine precision (eps) is taken to be               1.110223e-16
- Computational tests pass if scaled residuals are less than                16.0

================================================================================
T/V                N    NB     P     Q               Time                 Gflops
--------------------------------------------------------------------------------
WR02R2L2        8000   256     1     1              57.93              5.894e+00
HPL_pdgesv() start time Fri May 13 17:02:32 2016

HPL_pdgesv() end time   Fri May 13 17:03:30 2016

--------------------------------------------------------------------------------
||Ax-b||_oo/(eps*(||A||_oo*||x||_oo+||b||_oo)*N)=        0.0025941 ...... PASSED
================================================================================

Finished      1 tests with the following results:
              1 tests completed and passed residual checks,
              0 tests completed and failed residual checks,
              0 tests skipped because of illegal input values.
--------------------------------------------------------------------------------

End of Tests.


//////////////////////////////////////////////////////////////////
/////////////////////////////////////////////////////////////////

dtoverlay=pi3-disable-bt
Sysbench 113.2636s

Code: Select all

[email protected]:~ $ sysbench --num-threads=4 --test=cpu --cpu-max-prime=20000 --validate run
sysbench 0.4.12:  multi-threaded system evaluation benchmark

Running the test with following options:
Number of threads: 4
Additional request validation enabled.


Doing CPU performance benchmark

Threads started!
Done.

Maximum prime number checked in CPU test: 20000


Test execution summary:
    total time:                          113.2636s
    total number of events:              10000
    total time taken by event execution: 452.9496
    per-request statistics:
         min:                                 45.06ms
         avg:                                 45.29ms
         max:                                 89.60ms
         approx.  95 percentile:              45.84ms

Threads fairness:
    events (avg/stddev):           2500.0000/10.63
    execution time (avg/stddev):   113.2374/0.01
dtoverlay=pi3-disable-bt
Linpack 6.74 Gflops

Code: Select all

[email protected]:~ $ ./xhpl
================================================================================
HPLinpack 2.1  --  High-Performance Linpack benchmark  --   October 26, 2012
Written by A. Petitet and R. Clint Whaley,  Innovative Computing Laboratory, UTK
Modified by Piotr Luszczek, Innovative Computing Laboratory, UTK
Modified by Julien Langou, University of Colorado Denver
================================================================================

An explanation of the input/output parameters follows:
T/V    : Wall time / encoded variant.
N      : The order of the coefficient matrix A.
NB     : The partitioning blocking factor.
P      : The number of process rows.
Q      : The number of process columns.
Time   : Time in seconds to solve the linear system.
Gflops : Rate of execution for solving the linear system.

The following parameter values will be used:

N      :    8000 
NB     :     256 
PMAP   : Row-major process mapping
P      :       1 
Q      :       1 
PFACT  :    Left 
NBMIN  :       2 
NDIV   :       2 
RFACT  :   Right 
BCAST  :   2ring 
DEPTH  :       0 
SWAP   : Mix (threshold = 64)
L1     : transposed form
U      : transposed form
EQUIL  : yes
ALIGN  : 8 double precision words

--------------------------------------------------------------------------------

- The matrix A is randomly generated for each test.
- The following scaled residual check will be computed:
      ||Ax-b||_oo / ( eps * ( || x ||_oo * || A ||_oo + || b ||_oo ) * N )
- The relative machine precision (eps) is taken to be               1.110223e-16
- Computational tests pass if scaled residuals are less than                16.0

================================================================================
T/V                N    NB     P     Q               Time                 Gflops
--------------------------------------------------------------------------------
WR02R2L2        8000   256     1     1              50.62              6.745e+00
HPL_pdgesv() start time Fri May 13 17:08:34 2016

HPL_pdgesv() end time   Fri May 13 17:09:24 2016

--------------------------------------------------------------------------------
||Ax-b||_oo/(eps*(||A||_oo*||x||_oo+||b||_oo)*N)=        0.0025941 ...... PASSED
================================================================================

Finished      1 tests with the following results:
              1 tests completed and passed residual checks,
              0 tests completed and failed residual checks,
              0 tests skipped because of illegal input values.
--------------------------------------------------------------------------------

End of Tests.
DNPNWO

dom
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5318
Joined: Wed Aug 17, 2011 7:41 pm
Location: Cambridge

Re: Raspbian Jessie linux 4.4.9 Severe Performance Degradati

Fri May 13, 2016 9:29 pm

Unfortunately the fix didn't make the new raspbian image.
There will be an apt-get update in a few days.

User avatar
Rive
Posts: 586
Joined: Sat Mar 26, 2016 5:21 pm
Location: USA

Re: Raspbian Jessie linux 4.4.9 Severe Performance Degradati

Fri May 13, 2016 9:30 pm

dom wrote:Unfortunately the fix didn't make the new raspbian image.
There will be an apt-get update in a few days.
Is 'rpi-update' an option to get it now for the new image? Or will that break things?
DNPNWO

dom
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5318
Joined: Wed Aug 17, 2011 7:41 pm
Location: Cambridge

Re: Raspbian Jessie linux 4.4.9 Severe Performance Degradati

Fri May 13, 2016 9:31 pm

Rive wrote: Is 'rpi-update' an option to get it now for the new image?
Yes

dom
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5318
Joined: Wed Aug 17, 2011 7:41 pm
Location: Cambridge

Re: Raspbian Jessie linux 4.4.9 Severe Performance Degradati

Tue May 17, 2016 4:46 pm

The latest apt-get firmware now has the performance fix in, so rpi-update is no longer necessary to get that fix.

Koeshi
Posts: 228
Joined: Sun Mar 20, 2016 11:16 am

Re: Raspbian Jessie linux 4.4.9 Severe Performance Degradati

Wed May 18, 2016 11:39 am

So what commands do I need to use to get all the updates and fixes? I accidentally upgraded to the new kernel last week and have been having issues with my Pi since.

dom
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5318
Joined: Wed Aug 17, 2011 7:41 pm
Location: Cambridge

Re: Raspbian Jessie linux 4.4.9 Severe Performance Degradati

Wed May 18, 2016 12:41 pm

Koeshi wrote:So what commands do I need to use to get all the updates and fixes? I accidentally upgraded to the new kernel last week and have been having issues with my Pi since.
The standard upgrade command is:

Code: Select all

sudo apt-get update && sudo apt-get dist-upgrade

stuartofmt
Posts: 6
Joined: Tue Jul 05, 2016 6:44 pm

Re: Raspbian Jessie linux 4.4.9 Severe Performance Degradati

Tue Jul 05, 2016 6:51 pm

Maybe a bit off topic but .....
What do you guys think about using

Code: Select all

vcgencmd get_throttled
as an indicator of low input voltage to gracefully shutdown the pi.
I'm thinking of something simple like usb wall wart --> battery --> pi. If the power fails then eventually the battery will drop low and presumably that will be indicated in the output from vcgencmd. Perhaps a script that looks at bit zero for several consecutive reads before acting ???

dom
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5318
Joined: Wed Aug 17, 2011 7:41 pm
Location: Cambridge

Re: Raspbian Jessie linux 4.4.9 Severe Performance Degradati

Tue Jul 05, 2016 9:06 pm

stuartofmt wrote:as an indicator of low input voltage to gracefully shutdown the pi.
Yes, it is plausible. Technically the under-voltage is below 4.65V which is a little below the safe USB voltage,
so it's just possible you might have USB issues just before you shut down, but most devices are not that fussy.
I think it has a good chance of working.

stuartofmt
Posts: 6
Joined: Tue Jul 05, 2016 6:44 pm

Re: Raspbian Jessie linux 4.4.9 Severe Performance Degradati

Tue Jul 05, 2016 10:00 pm

@ Dom --

Thanks for the reply. Do you have any idea how often these values are updated ?

Also what resets bits 16 - 18 ? I'm currently running through a battery and notice that the value of the vcgencmd output is:
throttled=0x50000 but have no idea when the last "has occurred" took place.

dom
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5318
Joined: Wed Aug 17, 2011 7:41 pm
Location: Cambridge

Re: Raspbian Jessie linux 4.4.9 Severe Performance Degradati

Tue Jul 05, 2016 10:09 pm

stuartofmt wrote: Thanks for the reply. Do you have any idea how often these values are updated ?
Every 100ms.
Also what resets bits 16 - 18 ? I'm currently running through a battery and notice that the value of the vcgencmd output is:
throttled=0x50000 but have no idea when the last "has occurred" took place.
There are designed to be persistent since boot, as that is the only safe way of multiple users being able to read them.

However I did add a secret option.

Code: Select all

vcgencmd get_throttled 0x7
Will read a different persistent value, and reset the bitmask specified back to zero.
Be aware that if multiple scripts start using this, you will be clearing each others results, hence why it is not recommended.

vcgencmd get_throttled (without the bitmask specified) will always give you persistent values since boot (even if you have also been using the bitmask form).

stuartofmt
Posts: 6
Joined: Tue Jul 05, 2016 6:44 pm

Re: Raspbian Jessie linux 4.4.9 Severe Performance Degradati

Wed Jul 27, 2016 6:05 pm

So I wrote a batch script that runs as a cron job every 2 minutes. Checks a few things to decide it it ought to contemplate a shutdown the pi (e.g. are all screens blanked etc).
If the voltage is low more than 3 times in a row (6 minutes) then it forces a shutdown

Code: Select all

# Send async email message
shutdown -h +1
Running the Pi off a battery -- this overall seems to provide a good level of protection ....... most of the time .....
BUT - I noticed a couple of times that the pi shutdown without triggering the logic above (i.e. did not see a low voltage situation). Once I noticed a wan0 down status just before the pi shutdown of it's own accord (although this may have been incidental).

So - finally two questions.
1. Does it make sense that wan0 could drop without vcgencmd get_throttled seeing a low voltage event ?
2. What (if anything else) could cause the Pi to stop (due to low voltage) that would not be detected by vcgencmd ?

Setup is Pi 3 with 7" TFT Pi monitor ( I assume it does not matter whether the power is applied through the monitor usb or the pi usb).

Thanks in advance - the script is getting close to the point where it would be worth sharing.

dom
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5318
Joined: Wed Aug 17, 2011 7:41 pm
Location: Cambridge

Re: Raspbian Jessie linux 4.4.9 Severe Performance Degradati

Wed Jul 27, 2016 6:14 pm

stuartofmt wrote: 1. Does it make sense that wan0 could drop without vcgencmd get_throttled seeing a low voltage event ?
The under-voltage triggers at 4.63V which is slightly below the USB spec tolerance.
In theory a usb wifi device could drop out at say 4.65V which wouldn't be detected as under-voltage.
If the voltage is low more than 3 times in a row (6 minutes)
Why do you require this 3 times in a row?
If you want to be sure of catching it, I'd trigger on a single occurrence.

Note that voltage won't monotonically decrease. It will likely read lower when current is drawn (e.g. arm busy or wifi transmitting) and may read higher when system is idle.
So considering "everything is okay" just because you have seen a higher voltage doesn't mean that everything is always okay and may explain why the wifi is dropping.

stuartofmt
Posts: 6
Joined: Tue Jul 05, 2016 6:44 pm

Re: Raspbian Jessie linux 4.4.9 Severe Performance Degradati

Wed Jul 27, 2016 6:34 pm

The under-voltage triggers at 4.63V which is slightly below the USB spec tolerance.
In theory a usb wifi device could drop out at say 4.65V which wouldn't be detected as under-voltage.
This is a Pi 3 with built-in wifi. Is it implemented as a usb device ?
Why do you require this 3 times in a row?
For precisely the reasons you mention. I don't want to shut down because of a transient drop in voltage due to CPU load or etc. I want to detect that it's a persistent condition and therefore an good indication of a low battery situation. In the script, it's not a simple count of the number of low voltage "events'. If the voltage is good I decrement the counter (min 0 ) and if the voltage is low I increment the counter. So it's got to be low for a continuous period to force a shutdown. The three count is arbitrary as is the cron job every 2 minutes -- call it "tuning".

Can you think of anything else worth monitoring or is vcgencmd more-or-less as good as it gets ? Absent external circuitry and the like.

User avatar
davidcoton
Posts: 4025
Joined: Mon Sep 01, 2014 2:37 pm
Location: Cambridge, UK

Re: Raspbian Jessie linux 4.4.9 Severe Performance Degradati

Wed Jul 27, 2016 6:54 pm

Do you know how quickly your battery with normal load goes from safe to unusable? My guess is that with a TFT screen attached, the time form 4.63V (first low voltage warning) to too low to run is less than six minutes -- probably less than two. You need to monitor the voltage on a test run, time the interval from 4.63V down to (say) 4V, and make sure that you can detect (worst case) and shut down in that time. It may be difficult to do that reliably without much faster polling.
Signature retired

stuartofmt
Posts: 6
Joined: Tue Jul 05, 2016 6:44 pm

Re: Raspbian Jessie linux 4.4.9 Severe Performance Degradati

Wed Jul 27, 2016 7:12 pm

@davidcoton

What I'm really aiming for is a "poor man's" UPS. Wall wart --> battery --> Pi. So I'd like to make it as independent of battery size (within reason). The one I'm using now is a 4000mAh (because that's what I had on hand). It's only rated for 2A on the USB output but that's ok for testing right now. After initial startup the battery voltage is "good". I'm also using xscreensaver to blank the display (although backlight is still on).

As for how long from 4.75 volts till the pi shuts itself down ? What voltage does the pi shut down at ?

The practicalities of monitoring the voltage at the battery are too onerous (due primarily my attention span). For now - I'm testing empirically and "fine tuning". :-)

It's been working fine with the 6 minutes .... most of the time. I just put a safeguard in that if wan0 goes down then it's an immediate shutdown (on suspicion). I'm also going to try killing the backlight after the second (4th minute) of low voltage as that should buy some time. Of course, I could run the cron every minute and halve the time for everything. It's basically a question of finding a sweet spot that does not produce "false positive" shutdowns.

dom
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5318
Joined: Wed Aug 17, 2011 7:41 pm
Location: Cambridge

Re: Raspbian Jessie linux 4.4.9 Severe Performance Degradati

Wed Jul 27, 2016 10:15 pm

stuartofmt wrote: This is a Pi 3 with built-in wifi. Is it implemented as a usb device ?
No it is connected by sdio - I'm not sure if it runs on 3.3V or 5V - I'll need to ask a hardware guy, but it's possible it gets upset at low voltage.
For precisely the reasons you mention. I don't want to shut down because of a transient drop in voltage due to CPU load or etc.
Is the battery inadequate when fully changed? You shouldn't be dropping below 4.63V just because the processor is busy - that suggests you have a power supply problem.

If it is fine when fully changed but starts to get low when running out, then perhaps you should be shutting down (or at least finding a battery that doesn't sag the voltage as soon).

jdb
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 2074
Joined: Thu Jul 11, 2013 2:37 pm

Re: Raspbian Jessie linux 4.4.9 Severe Performance Degradati

Wed Jul 27, 2016 10:33 pm

"Hardware" guy here.

The WLAN/BT combo chip is powered from 3.3V which is a regulator-derived supply. Both the core power and I/O power is supplied from 3.3V. It will take a severe dropout of the incoming 5V to cause an appreciable dip on the 3.3V supply (max. duty cycle is about 90% which means 3.6V is the lowest the 5V can go before causing droop on the 3.3V).

In the case where you have suspect power being delivered to a Pi 3, I would expect attached USB devices to flake out far sooner than on-board peripherals.

Note that if you have a battery pack that has no regulation (i.e. boost converter) between the cell and its output, you are at the mercy of both the cell voltage and any wire resistance.
Rockets are loud.
https://astro-pi.org

User avatar
davidcoton
Posts: 4025
Joined: Mon Sep 01, 2014 2:37 pm
Location: Cambridge, UK

Re: Raspbian Jessie linux 4.4.9 Severe Performance Degradati

Thu Jul 28, 2016 9:04 am

jdb wrote:Note that if you have a battery pack that has no regulation (i.e. boost converter) between the cell and its output, you are at the mercy of both the cell voltage and any wire resistance.
It is the presence of a boost converter (actual battery to 5V) which means that the terminal decline in voltage is very rapid. As the battery voltage declines, the converter draws more current to compensate. At some point the battery just can't cope, and the 5V converter output rapidly drops from ~5V to under the critical 3.6V. In fact a well-designed pack will have a low battery voltage cut-off, so the end will be near instantaneous. Monitoring the low-voltage signal on the Pi won't be much use in predicting battery failure :shock:
Signature retired

mosespi
Posts: 508
Joined: Mon May 12, 2014 3:35 pm
Location: 34,-118
Contact: Website

Re: Raspbian Jessie linux 4.4.9 Severe Performance Degradati

Fri Jul 29, 2016 6:30 am

stuartofmt wrote:Maybe a bit off topic but .....
What do you guys think about using

Code: Select all

vcgencmd get_throttled
as an indicator of low input voltage to gracefully shutdown the pi.
I'm thinking of something simple like usb wall wart --> battery --> pi. If the power fails then eventually the battery will drop low and presumably that will be indicated in the output from vcgencmd. Perhaps a script that looks at bit zero for several consecutive reads before acting ???
I assume there is a DC-DC converter in there somewhere (up or down converter, doesn't really matter).. which is going to try really hard to maintain your 5v off of a battery, as others have hinted at. Depending on the DC-DC converter, battery type, capacity, age, load on the Pi, etc... using the throttle indicator on the Pi will likely be variable enough to drive you nuts.

If you want the least amount of hardware, maybe a simple digital indicator of power failure to the GPIO pins of the Pi? Wait for a set amount of time and shut down? You can add a small ADC and get more accurate state of battery capacity instead of a timer. Of course after shutdown these will never power back up again. For a more functional UPS you need to throw some sort of hardware supervisor circuit in there that can also cut power to the Pi or at least use the restart pin to bring it back up.

Feeling lucky? You can always oversize the battery and hope it lasts through most power failures!

Regards,
-Moses
Power problems? MoPower UPS for the Pi
http://www.allspectrum.com/mopower/

Return to “General discussion”