jdb
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 2094
Joined: Thu Jul 11, 2013 2:37 pm

Re: VLI firmware v2.0 - powersaving features enabled

Mon Sep 09, 2019 2:28 pm

I can confirm that the VBus behaviour is identical across bootloader POWER_OFF_ON_HALT=0/1 and VLI firmware changes.

One thing to note is that both of you are running the root on an SSD - if USB devices become inaccessible after flashing the firmware (the script rescans the bus) then a shutdown will hang forever as the root filesystem has gone away.
Rockets are loud.
https://astro-pi.org

PhilE
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 2380
Joined: Mon Sep 29, 2014 1:07 pm
Location: Cambridge

Re: VLI firmware v2.0 - powersaving features enabled

Mon Sep 09, 2019 2:33 pm

... if USB devices become inaccessible after flashing the firmware (the script rescans the bus)
There used to be a wrapper script called "vl805_flash" that would perform a rescan after flashing, but we don't use that now. The provided "vl805" utility just updates the EEPROM and should have no other effect on the running system.

User avatar
bleep42
Posts: 153
Joined: Wed Mar 07, 2012 12:43 pm
Location: Sussex

Re: VLI firmware v2.0 - powersaving features enabled

Mon Sep 09, 2019 4:09 pm

Tested and working fine on a Pi4 with 64-bit raspbian kernel and hibrid boot using USB3 SSD.
Temperature is the same as the previous (updated) firmware 0137a8.
All good, thanks. :-)
Regards, Kevin.
Last edited by bleep42 on Mon Sep 09, 2019 5:01 pm, edited 1 time in total.

User avatar
PeterO
Posts: 5028
Joined: Sun Jul 22, 2012 4:14 pm

Re: VLI firmware v2.0 - powersaving features enabled

Mon Sep 09, 2019 4:25 pm

Pi4B4 in a Pibow with temp. controlled fan-shim (set to 65° trigger temp.) seems to be idling about 4°C cooler at 56°C.
PeterO
Discoverer of the PI2 XENON DEATH FLASH!
Interests: C,Python,PIC,Electronics,Ham Radio (G0DZB),1960s British Computers.
"The primary requirement (as we've always seen in your examples) is that the code is readable. " Dougie Lawson

jdb
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 2094
Joined: Thu Jul 11, 2013 2:37 pm

Re: VLI firmware v2.0 - powersaving features enabled

Wed Sep 11, 2019 9:59 am

Hesitant to make this a sticky (we have enough of them already) so - bump. I think a few more +1's for success and we put this in the announcements forum.
Rockets are loud.
https://astro-pi.org

itsmedoofer
Posts: 359
Joined: Wed Sep 25, 2013 8:43 am

Re: VLI firmware v2.0 - powersaving features enabled

Wed Sep 11, 2019 11:38 am

Hi,

Not sure if I understand why my first flash was unsuccessful but the second attempt worked.

The issue wasn't a result of the flashing process, post flash a power cycle would be successful, reboot not.

Its moot for me really as it's all working, however two people here had similar if not the same issue, could be the tip of the proverbial if you open it up further, just a thought....

jdb
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 2094
Joined: Thu Jul 11, 2013 2:37 pm

Re: VLI firmware v2.0 - powersaving features enabled

Wed Sep 11, 2019 11:48 am

The fact that you flashed the EEPROM twice (with the updated version) and fixed the issue suggests that something went wrong during the first flash.

Unfortunately both EEPROMs are now in a "good" state so there's not really any information that can be gotten out of them...
Rockets are loud.
https://astro-pi.org

itsmedoofer
Posts: 359
Joined: Wed Sep 25, 2013 8:43 am

Re: VLI firmware v2.0 - powersaving features enabled

Wed Sep 11, 2019 12:03 pm

Yea like I said a moot point for me and not a catastrophic issue.....

Chimneyfactory
Posts: 32
Joined: Sun May 19, 2019 1:30 pm

Re: VLI firmware v2.0 - powersaving features enabled

Wed Sep 11, 2019 3:12 pm

Flashed my pi4 4gb, working fine with the caveat that there's a slight performance hit in USB SSD speed.

Code: Select all

[email protected]:~/Downloads $ sudo ./vl805 
VL805 FW version: 00013701

[email protected]:~ $ sudo hdparm -t /dev/sda1

/dev/sda1:
 Timing buffered disk reads: 1100 MB in  3.00 seconds = 366.48 MB/sec
[email protected]:~ $ sudo hdparm -t /dev/sda1

/dev/sda1:
 Timing buffered disk reads: 1098 MB in  3.00 seconds = 365.39 MB/sec
[email protected]:~ $ sudo hdparm -t /dev/sda1

/dev/sda1:
 Timing buffered disk reads: 1110 MB in  3.00 seconds = 369.80 MB/sec
[email protected]:~ $ sudo hdparm -t /dev/sda1

/dev/sda1:
 Timing buffered disk reads: 1104 MB in  3.00 seconds = 367.57 MB/sec
[email protected]:~ $ sudo hdparm -t /dev/sda1

/dev/sda1:
 Timing buffered disk reads: 1110 MB in  3.00 seconds = 369.61 MB/sec

So, on original firmware, average read speed there is 367.77MB/sec

Idle temperature after 10 minutes at ambient of 22.4C was 56-57C

Now, after flashing and rebooting :

Code: Select all

[email protected]:~/Downloads $ sudo ./vl805 
VL805 FW version: 000137ab
[email protected]:~/Downloads $ sudo hdparm -t /dev/sda1

/dev/sda1:
 Timing buffered disk reads: 1060 MB in  3.00 seconds = 352.94 MB/sec
[email protected]:~/Downloads $ sudo hdparm -t /dev/sda1

/dev/sda1:
 Timing buffered disk reads: 1062 MB in  3.00 seconds = 353.60 MB/sec
[email protected]:~/Downloads $ sudo hdparm -t /dev/sda1

/dev/sda1:
 Timing buffered disk reads: 1066 MB in  3.01 seconds = 354.67 MB/sec
[email protected]:~/Downloads $ sudo hdparm -t /dev/sda1

/dev/sda1:
 Timing buffered disk reads: 1060 MB in  3.01 seconds = 352.73 MB/sec
[email protected]:~/Downloads $ sudo hdparm -t /dev/sda1

/dev/sda1:
 Timing buffered disk reads: 1070 MB in  3.00 seconds = 356.30 MB/sec


With new firmware, average read speeds now 354.048MB/sec, so down 13.7MB/sec
Idle temperature now 54 - 55C , so down 1 - 2C


I too had shutdown issues that were repeatable after flashing the firmware.

With new firmware, sudo shutdown -h now would cause the pi to hang, with "Stopping Restore / Save the current clock..." being the last item displayed. This was repeatable.

sudo shutdown -r would work as expected.

I too found this problem has gone away! But while it existed, it was repeatable, happened maybe 5 - 7 times during my tests.

I am booting from SD card, Raspbian stored on external USB 3 SSD.

I will poke at it a bit more and see if I can make the problem come back again. I'll try flashing back to original firmware, then to new one a 2nd time.

Chimneyfactory
Posts: 32
Joined: Sun May 19, 2019 1:30 pm

Re: VLI firmware v2.0 - powersaving features enabled

Wed Sep 11, 2019 4:01 pm

I've been trying to get the hanging on shutdown problem to come back, which I did, but not repeatably.

Running new firmware.
sudo shutdown -h now OK
reboot
flash firmware (original)
sudo shutdown -h now HANGS
reboot
sudo shutdown -h now OK
reboot
sudo shutdown -h now OK
reboot
flash firmware (new)
sudo shutdown -h now OK
reboot
sudo shutdown -h now OK
reboot
sudo ./vl805 (to show version number - does this cause problem?)
sudo shutdown -h now HANGS
reboot
sudo shutdown -h now OK
reboot
sudo ./vl805
flash firmware (original)
sudo shutdown -h now HANGS
reboot
sudo shutdown -h now OK
reboot
sudo shutdown -h now OK
reboot
sudo ./vl805
sudo shutdown -h now OK
reboot
flash firmware (new)
sudo shutdown -h now OK
reboot
wait 10 minutes
sudo shutdown -h now OK

OK means Pi shuts down, green LED flashes as expected
HANGS means text stays on screen "Stopping Restore / save the current clock..." being last item. Green LED does not flash.
reboot means power off pi, wait for red LED to extinguish, then power on again.
flash firmware means sudo ./vl805 -w firmware_image

SO, I can make it hang on shutdown sometime, and a hang was always preceded by the use of the vl805 program either to flash or display the firmware. But use of this program doesn't always cause a hang at shutdown.

(Previously, I was consistently getting hangs at shutdown)

As stated before, pi 4 4Gb, boot from SD then Raspbian on external USB3 SSD

Thanks

Ian
Last edited by Chimneyfactory on Wed Sep 11, 2019 4:04 pm, edited 1 time in total.

jdb
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 2094
Joined: Thu Jul 11, 2013 2:37 pm

Re: VLI firmware v2.0 - powersaving features enabled

Wed Sep 11, 2019 4:01 pm

The use of the PCIe bounce buffers has been known to cause variable performance, especially when operating at near 100% utilisation of the PCIe link.

Can you add total_mem=3072 to /boot/config.txt prior to retesting? This will limit the available ARM memory to the bottom 3G which should remove the bounce buffers from the equation.

Also, if you have the EEPROM in a dodgy state again, can you run sudo ./vl805 -r vl805_fw_backup.bin to dump the contents to a file?
Rockets are loud.
https://astro-pi.org

PhilE
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 2380
Joined: Mon Sep 29, 2014 1:07 pm
Location: Cambridge

Re: VLI firmware v2.0 - powersaving features enabled

Wed Sep 11, 2019 4:11 pm

For performance measurements I would also recommend switching to the performance governor:

Code: Select all

sudo sh -c "echo performance > /sys/devices/system/cpu/cpufreq/policy0/scaling_governor"

Chimneyfactory
Posts: 32
Joined: Sun May 19, 2019 1:30 pm

Re: VLI firmware v2.0 - powersaving features enabled

Wed Sep 11, 2019 5:46 pm

Code: Select all

Mem=3072?	Perf Gov?	firmware	speed (ave)	Reps	Difference (MB/sec)
				00013701	364.516		3	-9.40
				00013701	367.770		5	-6.15
Y				00013701	367.424		5	-6.49
				00013701	366.410		5	-7.51
		Y		00013701	372.390		5	-1.53
		Y		00013701	371.880		5	-2.04
Y		Y		00013701	373.918		5	0.00
				000137ab	353.610		3	-20.31
				000137ab	354.048		5	-19.87
Y				000137ab	355.798		5	-18.12
				000137ab	354.720		5	-19.20
		Y		000137ab	357.236		5	-16.68
		Y		000137ab	358.988		5	-14.93
Y		Y		000137ab	360.082		5	-13.84

Results of lots of tests, with the two different firmwares, with TOTAL_MEM=3072 in config.txt and with the performance governor.

The speed test used was hdparm :

Code: Select all

sudo hdparm -t /dev/sda1
and the speed is reported in MB/sec.

The difference column notes the test speed compared to the best speed recorded.

The first two tests consisted of 3 repetitions, averaged. All other tests were 5 repetitions of the hdparm command, averaged. The 3 reps tests seem to be odd outliers, the other tests seem consistent based around the test settings - firmware version, governor etc.

My conclusion is that the new firmware slows the USB3 transfer rate by 13MB/sec or thereabouts.


During the repeated reboots and reflashing of the firmware, I had an additional 4 freezes after

Code: Select all

sudo shutdown -h now
, but this did not happen every time. Each freeze ocurred when shutting down after use of the ./vl805 command, but not every use of this command resulted in a freeze.

Following the freeze and a reboot, I issued the command

Code: Select all

sudo ./vl805 -r test
and did a binary comparison of this file to the firmware image previously written - it was identical. I did this command after another freeze and reboot, still identical.

Ian
Last edited by Chimneyfactory on Wed Sep 11, 2019 10:35 pm, edited 2 times in total.

Gerbertus
Posts: 12
Joined: Tue Apr 17, 2018 3:39 pm

Re: VLI firmware v2.0 - powersaving features enabled

Wed Sep 11, 2019 10:27 pm

jdb wrote:
Wed Sep 11, 2019 9:59 am
I think a few more +1's for success and we put this in the announcements forum.
I haven't done any USB speed testing, but my pi4 is working just fine with the new firmware, and the idle temperature dropped from 49-50C to 43C!

This is with a FLIRC case, which is a fairly ideal case for seeing improvements from reducing the USB chip power draw. (The processor is already rather well cooled and there isn't much ventilation for getting rid of heat from any other components on the board, so reducing the heat from those components will bring the Pi substantially closer to ambient. Were there a way to reduce the heat coming from e.g. the MxL PMIC, I bet that would show further notable improvements.) I imagine that people running without a case or with a case that doesn't cool the processor like this may see more limited improvements.

Still, though, it seems to me that this large a difference in power draw and temperature is more than worth a tiny performance hit on super fast external SSDs.

asavah
Posts: 364
Joined: Thu Aug 14, 2014 12:49 am

Re: VLI firmware v2.0 - powersaving features enabled

Thu Sep 12, 2019 1:46 am

Is vl805 utility itself closed source (proprietary)?
Are its sources available anywhere?

I'm asking because I have full aarch64 os (my own project) happily running without any arm 32 bit support.
I was able to get vl805 to run successfully by grabbing armhf libs from raspbian mirror and patching rpath with patchelf into ./vl805.
Firmware was updated successfully and my pi4 is running fine.

So far so good but it would be nice to be able to build the utility for aarch64 if possible.

trejan
Posts: 583
Joined: Tue Jul 02, 2019 2:28 pm

Re: VLI firmware v2.0 - powersaving features enabled

Thu Sep 12, 2019 3:24 am

asavah wrote:
Thu Sep 12, 2019 1:46 am
Is vl805 utility itself closed source (proprietary)?
Are its sources available anywhere?
Closed source. I'm not sure who wrote it. The VL805 documentation is only available under NDA and the vl805 tool does do some undocumented register twiddling so there might be a problem releasing it. You need to do a bunch of register writes to enable/use the SPI interface on the VL805.

asavah
Posts: 364
Joined: Thu Aug 14, 2014 12:49 am

Re: VLI firmware v2.0 - powersaving features enabled

Thu Sep 12, 2019 10:06 am

trejan wrote:
Thu Sep 12, 2019 3:24 am
asavah wrote:
Thu Sep 12, 2019 1:46 am
Is vl805 utility itself closed source (proprietary)?
Are its sources available anywhere?
Closed source. I'm not sure who wrote it. The VL805 documentation is only available under NDA and the vl805 tool does do some undocumented register twiddling so there might be a problem releasing it. You need to do a bunch of register writes to enable/use the SPI interface on the VL805.
As I suspected.
Thanks for the info.

PhilE
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 2380
Joined: Mon Sep 29, 2014 1:07 pm
Location: Cambridge

Re: VLI firmware v2.0 - powersaving features enabled

Thu Sep 12, 2019 11:45 am

The utility contains VIA proprietary information so is closed source, but for greater portability I've updated the archive with a statically linked build.

asavah
Posts: 364
Joined: Thu Aug 14, 2014 12:49 am

Re: VLI firmware v2.0 - powersaving features enabled

Thu Sep 12, 2019 1:27 pm

PhilE wrote:
Thu Sep 12, 2019 11:45 am
The utility contains VIA proprietary information so is closed source, but for greater portability I've updated the archive with a statically linked build.
Thanks!
Works nicely on aarch64 without any fiddling.

Chimneyfactory
Posts: 32
Joined: Sun May 19, 2019 1:30 pm

Re: VLI firmware v2.0 - powersaving features enabled

Thu Sep 12, 2019 1:43 pm

I've found a sequence that, for me anyway, will consistently cause the shutdown freeze problem.

Setup :
Pi 4 4gb
Full Raspbian install on SSD
Boot from SD
New VLI firmware 000137ab already installed
VLI805 command and firmware is unzipped in Downloads directory
Pi boots to CLI

Login, then type these commands

Code: Select all

cd Downloads
sudo ./vl805
ls
cd
ls
cd Desktop
ls
cd
sudo shutdown -h now
Note that the cd and ls commands work as expected, showing that USB3 access to the SSD is probably still working, cache notwithstanding.

For me, the above sequence failed to properly shutdown 5 out of 5 attempts. Then additionally a further freeze when reading firmware into a file.

If I leave out the sudo ./vl805 step, but do everything else, it shuts down correctly.

The freeze varied slightly :

Last text on screen : "Stopping restore/save current clock..." with no red LED lit : 3 times
Last text on screen : "Stopping restore/save current clock..." with red LED lit : 2 times
Last text on screen : "target power off" with red and green LED lit :1 (green LED went out around 30 seconds later)

In all occasions of the freezing, the text remains on the screen and the power down does not complete.

Following the 5 repeated freezes, I restarted and used

Code: Select all

sudo ./vl805 -r test3
cmp test3 vl805_fw_000137ab.bin
Which reported nothing, showing the firmware does not seem to be corrupted.
(comparing the extracted firmware to the previous version 00013701 showed differences, as expected)
When I tried to shutdown after doing these commands, the Pi froze for a 6th consecutive time, as noted above.

Hope this is useful

Ian

jdb
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 2094
Joined: Thu Jul 11, 2013 2:37 pm

Re: VLI firmware v2.0 - powersaving features enabled

Thu Sep 12, 2019 2:02 pm

OK, so that confirms there's no issue with the firmware update - just that the utility is leaving the chip in an odd state that can cause a shutdown hang.

I can replicate the hang, but only with USB3.0 devices attached. I tested with a mass storage device (filesystem not mounted) and a USB-HDMI adapter. Linux doesn't even have a driver for the USB-HDMI adapter so it's purely a function of a USB3.0 device being attached.
Rockets are loud.
https://astro-pi.org

User avatar
CypherOz
Posts: 45
Joined: Fri Jul 05, 2019 1:46 pm

Re: VLI firmware v2.0 - powersaving features enabled

Thu Sep 12, 2019 2:24 pm

Installed, only saw a ~1 degree drop, but I am running a fan cooler.
--
Regards, Kym
Retired software bloke from Adelaide, South Australia

jerrm
Posts: 194
Joined: Wed May 02, 2018 7:35 pm

Re: VLI firmware v2.0 - powersaving features enabled

Thu Sep 12, 2019 5:34 pm

jdb wrote:
Thu Sep 12, 2019 2:02 pm
OK, so that confirms there's no issue with the firmware update - just that the utility is leaving the chip in an odd state that can cause a shutdown hang.
I would consider the performance hit an issue if there isn't an option to get it back.

I don't care about the temp at all as long as it is stable. A Flirc(like) or fan case is a necessity either way, so temps within reason are irrelevant.

pagenotfound
Posts: 75
Joined: Mon Mar 14, 2016 12:44 pm

Re: VLI firmware v2.0 - powersaving features enabled

Fri Sep 13, 2019 12:25 pm

jerrm wrote:
Thu Sep 12, 2019 5:34 pm
jdb wrote:
Thu Sep 12, 2019 2:02 pm
OK, so that confirms there's no issue with the firmware update - just that the utility is leaving the chip in an odd state that can cause a shutdown hang.
I would consider the performance hit an issue if there isn't an option to get it back.

I don't care about the temp at all as long as it is stable. A Flirc(like) or fan case is a necessity either way, so temps within reason are irrelevant.
There are many people, including at least one RPi engineer, who want to run them without a case and without a fan. Milliwatts count. It's not for the lower price that some people still use the old A models.

Also, 300 mW x 1.000.000 units produced so far = 300 kW. After Christmas we will be talking about a Megawatt. There are worse energy wasters on this planet, but over the years investing a few more engineer hours in this will pay off.

jerrm
Posts: 194
Joined: Wed May 02, 2018 7:35 pm

Re: VLI firmware v2.0 - powersaving features enabled

Fri Sep 13, 2019 1:56 pm

pagenotfound wrote:
Fri Sep 13, 2019 12:25 pm
There are many people, including at least one RPi engineer, who want to run them without a case and without a fan. Milliwatts count. It's not for the lower price that some people still use the old A models.
Folks still have old A's as an option if performance is not their need.

IMO, a Pi without a case is a hobbyist toy. For any real deployment the Pi is inside something - be that its own case or another product. Cooling will still be needed. A 5 degree reduction in base temps will only delay throttling for minute or two. Great for short bursty applications, not so good for anything with sustained usage.
pagenotfound wrote:
Fri Sep 13, 2019 12:25 pm
Also, 300 mW x 1.000.000 units produced so far = 300 kW. After Christmas we will be talking about a Megawatt. There are worse energy wasters on this planet, but over the years investing a few more engineer hours in this will pay off.
But for uses that move a lot of data, will the net power usage be any better? If my job runs for an extra 15 minutes and keeps the Pi at full clock speed that much longer (along with all the related equipment - attached HDD, switch, local router, internet backbone, remote router, remote switch, remote PC, etc) has any power really been saved?

Reduced power as an option is great, for many performance is more important.

Return to “Troubleshooting”