User avatar
Montala
Posts: 675
Joined: Mon Mar 05, 2012 11:54 pm
Location: Herefordshire (U.K.)

Re: Raspberry Pi 4 - new VLI firmware issues

Wed Jul 03, 2019 4:23 pm

PhilE wrote:
Wed Jul 03, 2019 4:15 pm
Thank you for the feedback. We're going to hold back on a wider rollout until we understand (and have a fix for) the poor throughput others have seen, but I'm happy the update is working as intended for your use cases.

We are continuing to work on other performance and power improvements, but a 64-bit kernel for Pi 4 is likely to be released eventually.
Thanks for the update Phil.

Can I please ask when you anticipate that the next official software 'update' (or any sort!) will be released for the Pi 4, as I am somewhat wary of installing any which have not yet been fully tested and approved?


Thanks again!

jdb
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 2125
Joined: Thu Jul 11, 2013 2:37 pm

Re: Raspberry Pi 4 - new VLI firmware issues

Wed Jul 03, 2019 4:23 pm

trejan wrote:
Wed Jul 03, 2019 1:08 am
Okay. I've done some more testing. I'm still seeing a problem with the new firmware even with the setpci tweak that turns off ASPM L1. Throughput is okay when ASPM L1 disabled but something in the new firmware is messing up timing or latency even with ASPM disabled.

I have this Airspy setup to be an ADS-B receiver which is linked to other receivers nearby for multilateration of aircraft that don't transmit coordinates. You need accurate timestamps of each sample for this to work and the new firmware does something with timing/latency that causes it to fail to synchronise with the other receivers.

I've tried the following:

New firmware with ASPM disabled (common clock on and off) <--- No throughput issues but multilateration doesn't work.
New firmware with ASPM L0s only <--- No throughput issues but multilateration doesn't work.
New firmware with ASPM L0s + L1 <--- Throughput issues and multilateration doesn't work.
Old firmware with ASPM disabled <--- No throughput issues issues and multilateration works.
The firmware update will also enable USB3 U1/U2 powersave states - you can override these on a per-port basis by navigating to
/sys/bus/usb/devices/usb2/2-0:1.0
And disabling USB LPM on the port of interest (from lsusb -t) by doing:
echo 0 | sudo tee usb2-portN/usb3_lpm_permit

Other valid values are u1, u1_u2, u2.
Rockets are loud.
https://astro-pi.org

trejan
Posts: 757
Joined: Tue Jul 02, 2019 2:28 pm

Re: Raspberry Pi 4 - new VLI firmware issues

Wed Jul 03, 2019 5:44 pm

jdb wrote:
Wed Jul 03, 2019 4:23 pm
The firmware update will also enable USB3 U1/U2 powersave states - you can override these on a per-port basis by navigating to
/sys/bus/usb/devices/usb2/2-0:1.0
And disabling USB LPM on the port of interest (from lsusb -t) by doing:
echo 0 | sudo tee usb2-portN/usb3_lpm_permit

Other valid values are u1, u1_u2, u2.
This is a USB 2 device though. Looking at the bus numbers, usb2 is the USB 3.0 bus and usb1 is the USB 2.0 bus so the usb3_lpm_permit settings don't apply?

I've got an Airspy SDR which is USB 2.0 and a RTL2832U SDR which is also USB 2.0 attached to the two USB 2.0 ports. The RTL2832U SDR is having problems but you already know about them. I don't notice any significant change on the new firmware for the RTL2832U.
Last edited by trejan on Wed Jul 03, 2019 6:14 pm, edited 1 time in total.

jdb
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 2125
Joined: Thu Jul 11, 2013 2:37 pm

Re: Raspberry Pi 4 - new VLI firmware issues

Wed Jul 03, 2019 6:00 pm

For some reason I thought the Airspy was a USB3.0 device. The U1/U2 settings only apply to superspeed devices - USB2.0 devices are unaffected.
Rockets are loud.
https://astro-pi.org

trejan
Posts: 757
Joined: Tue Jul 02, 2019 2:28 pm

Re: Raspberry Pi 4 - new VLI firmware issues

Wed Jul 03, 2019 6:17 pm

jdb wrote:
Wed Jul 03, 2019 6:00 pm
For some reason I thought the Airspy was a USB3.0 device. The U1/U2 settings only apply to superspeed devices - USB2.0 devices are unaffected.
Hmm. I wonder what else has changed then. The MLAT system definitely doesn't like the new firmware.

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 23944
Joined: Sat Jul 30, 2011 7:41 pm

Re: Raspberry Pi 4 - new VLI firmware issues

Wed Jul 03, 2019 8:26 pm

Montala wrote:
Wed Jul 03, 2019 4:23 pm
PhilE wrote:
Wed Jul 03, 2019 4:15 pm
Thank you for the feedback. We're going to hold back on a wider rollout until we understand (and have a fix for) the poor throughput others have seen, but I'm happy the update is working as intended for your use cases.

We are continuing to work on other performance and power improvements, but a 64-bit kernel for Pi 4 is likely to be released eventually.
Thanks for the update Phil.

Can I please ask when you anticipate that the next official software 'update' (or any sort!) will be released for the Pi 4, as I am somewhat wary of installing any which have not yet been fully tested and approved?


Thanks again!
If you want to stay safe, just use apt to update and upgrade, that only ever contains tested firmwares. As for when, no idea as we are beavering away fixing things at the moment.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
“I think it’s wrong that only one company makes the game Monopoly.” – Steven Wright

User avatar
Montala
Posts: 675
Joined: Mon Mar 05, 2012 11:54 pm
Location: Herefordshire (U.K.)

Re: Raspberry Pi 4 - new VLI firmware issues

Wed Jul 03, 2019 9:06 pm

Thanks... will do!

lucabelluccini
Posts: 6
Joined: Sat Oct 01, 2011 7:11 am
Contact: Website

Re: Raspberry Pi 4 - new VLI firmware issues

Wed Jul 03, 2019 11:52 pm

Hello,
I've not updated yet to the new firmware, but I've faced really slow performances with my USB 3.0 to SATA adapter on the RPi4.
I've managed to make it work enabling USB storage quirks in the "cmdline.txt".

Check step 6.3 of this Github gist

lb
Posts: 261
Joined: Sat Jan 28, 2012 8:07 pm

Re: Raspberry Pi 4 - new VLI firmware issues

Thu Jul 04, 2019 1:35 am

I think we might mix up problems here. As far as I can see, there are some generic issues with USB on the Pi 4 at the moment that even appear with the old firmware. These need to be fixed. Interaction with power management might exacerbate those issues.

jdb
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 2125
Joined: Thu Jul 11, 2013 2:37 pm

Re: Raspberry Pi 4 - new VLI firmware issues

Thu Jul 04, 2019 8:29 am

A possible cause of low performance even with the stock VLI firmware could be UAS blacklisting - on an x86 pc with 4.19.5 my SSD adapter is downgraded to mass-storage, but on the Pi it's quite happy using UAS transport.
Ignore this - I had a leftover module blacklist from testing on the PC.
Rockets are loud.
https://astro-pi.org

bircoe
Posts: 37
Joined: Wed Jul 27, 2011 9:43 pm

Re: Raspberry Pi 4 - new VLI firmware issues

Thu Jul 04, 2019 11:40 am

I'm getting the data mismatch problem that Chimneyfactory mentioned however in my case I can't flash back to default even after a reboot:

I downloaded the file directly on the Pi then followed the instructions.

Code: Select all

[email protected]:~ $ sudo ./vl805 -w vl805_fw_013701.bin 
VL805 FW version: 00000000
* data mismatch
[email protected]:~ $ sudo ./vl805 -w vl805_fw_0137a8.bin 
VL805 FW version: 00000000
* data mismatch
At this point I had no USB, I had to SSH in thankfully I had already enabled SSH... so I redownloaded the update on my phone then SCP'd it to the Pi and ended up with the exact same issues, it wasn't until I downloaded the update on my desktop PC and SCP'd the file to the Pi that I was able to successfully flash the VLI firmware.

Code: Select all

[email protected]:~ $ sudo ./vl805 -w vl805_fw_0137a8.bin
VL805 FW version: 00000000
VL805 EEPROM successfully reprogrammed - please reboot to activate.
On to USB testing, I've already noticed some issues with mouse and keyboard with an SSD plugged in, so I ran some hdparm testing:
0137a8:

Code: Select all

[email protected]:~ $ sudo hdparm -t /dev/sda1

/dev/sda1:
 Timing buffered disk reads:   4 MB in 30.87 seconds = 132.69 kB/sec
0137a8 with ASPM L1 disabled:

Code: Select all

[email protected]:~ $ sudo hdparm -t /dev/sda1

/dev/sda1:
 Timing buffered disk reads:   2 MB in 30.75 seconds =  66.61 kB/sec
[email protected]:~ $ sudo hdparm -t /dev/sda1

/dev/sda1:
 Timing buffered disk reads:   6 MB in 30.65 seconds = 200.43 kB/sec
After reverting back to 013701:

Code: Select all

[email protected]:~ $ sudo hdparm -t /dev/sda1

/dev/sda1:
 Timing buffered disk reads: 1084 MB in  3.00 seconds = 361.33 MB/sec

jdb
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 2125
Joined: Thu Jul 11, 2013 2:37 pm

Re: Raspberry Pi 4 - new VLI firmware issues

Thu Jul 04, 2019 12:03 pm

Can you post the full lsusb of your SSD adapter?
Rockets are loud.
https://astro-pi.org

JackD12001
Posts: 4
Joined: Thu Jul 04, 2019 1:00 pm

Re: Raspberry Pi 4 - new VLI firmware issues

Thu Jul 04, 2019 1:02 pm

In my case the slow speed can be attributed to host resets with the new firmware.

[ 108.649194] sd 0:0:0:0: [sda] tag#14 uas_eh_abort_handler 0 uas-tag 1 inflight: IN
[ 108.649212] sd 0:0:0:0: [sda] tag#14 CDB: opcode=0x28 28 00 00 08 20 00 00 04 00 00
[ 108.689190] scsi host0: uas_eh_device_reset_handler start
[ 108.840180] usb 2-1: reset SuperSpeed Gen 1 USB device number 2 using xhci_hcd
[ 108.876309] scsi host0: uas_eh_device_reset_handler success
[ 195.051034] sd 0:0:0:0: [sda] tag#22 uas_eh_abort_handler 0 uas-tag 2 inflight: IN
[ 195.051052] sd 0:0:0:0: [sda] tag#22 CDB: opcode=0x28 28 00 00 07 4c 00 00 04 00 00
[ 195.091050] scsi host0: uas_eh_device_reset_handler start
[ 195.242011] usb 2-1: reset SuperSpeed Gen 1 USB device number 2 using xhci_hcd
[ 195.278211] scsi host0: uas_eh_device_reset_handler success

jdb
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 2125
Joined: Thu Jul 11, 2013 2:37 pm

Re: Raspberry Pi 4 - new VLI firmware issues

Thu Jul 04, 2019 1:37 pm

@jackd12001 can you try the U1/U2 powersave disable as described above?
Rockets are loud.
https://astro-pi.org

JackD12001
Posts: 4
Joined: Thu Jul 04, 2019 1:00 pm

Re: Raspberry Pi 4 - new VLI firmware issues

Thu Jul 04, 2019 1:59 pm

0 , u1, u1_u2 or u2 don't seem to have any affect on the speed or stability in my case.

bircoe
Posts: 37
Joined: Wed Jul 27, 2011 9:43 pm

Re: Raspberry Pi 4 - new VLI firmware issues

Fri Jul 05, 2019 12:03 am

jdb wrote:
Thu Jul 04, 2019 12:03 pm
Can you post the full lsusb of your SSD adapter?
Sure, I'll get that when I'm home from work... it's a USB 3.1 Gen 2 to M.2 SATA III adapter, the Amazon listing says it's a VL716 but I'll confirm this with lsusb tonight.

JackD12001
Posts: 4
Joined: Thu Jul 04, 2019 1:00 pm

Re: Raspberry Pi 4 - new VLI firmware issues

Fri Jul 05, 2019 6:14 am

This is mine, it works just fine on the old firmware.
This drive has bus power and an external power source through a USB splitter.
The output is on the new firmware.
You can see U1 and U2 are disabled.

Bus 002 Device 002: ID 0bc2:2321 Seagate RSS LLC Expansion Portable
Device Descriptor:
bLength 18
bDescriptorType 1
bcdUSB 3.00
bDeviceClass 0
bDeviceSubClass 0
bDeviceProtocol 0
bMaxPacketSize0 9
idVendor 0x0bc2 Seagate RSS LLC
idProduct 0x2321 Expansion Portable
bcdDevice 1.00
iManufacturer 2 Seagate
iProduct 3 Expansion
iSerial 1 NA4CB6PT
bNumConfigurations 1
Configuration Descriptor:
bLength 9
bDescriptorType 2
wTotalLength 0x0079
bNumInterfaces 1
bConfigurationValue 1
iConfiguration 0
bmAttributes 0x80
(Bus Powered)
MaxPower 144mA
Interface Descriptor:
bLength 9
bDescriptorType 4
bInterfaceNumber 0
bAlternateSetting 0
bNumEndpoints 2
bInterfaceClass 8 Mass Storage
bInterfaceSubClass 6 SCSI
bInterfaceProtocol 80 Bulk-Only
iInterface 0
Endpoint Descriptor:
bLength 7
bDescriptorType 5
bEndpointAddress 0x81 EP 1 IN
bmAttributes 2
Transfer Type Bulk
Synch Type None
Usage Type Data
wMaxPacketSize 0x0400 1x 1024 bytes
bInterval 0
bMaxBurst 15
Endpoint Descriptor:
bLength 7
bDescriptorType 5
bEndpointAddress 0x02 EP 2 OUT
bmAttributes 2
Transfer Type Bulk
Synch Type None
Usage Type Data
wMaxPacketSize 0x0400 1x 1024 bytes
bInterval 0
bMaxBurst 15
Interface Descriptor:
bLength 9
bDescriptorType 4
bInterfaceNumber 0
bAlternateSetting 1
bNumEndpoints 4
bInterfaceClass 8 Mass Storage
bInterfaceSubClass 6 SCSI
bInterfaceProtocol 98
iInterface 0
Endpoint Descriptor:
bLength 7
bDescriptorType 5
bEndpointAddress 0x81 EP 1 IN
bmAttributes 2
Transfer Type Bulk
Synch Type None
Usage Type Data
wMaxPacketSize 0x0400 1x 1024 bytes
bInterval 0
bMaxBurst 15
MaxStreams 32
Data-in pipe (0x03)
Endpoint Descriptor:
bLength 7
bDescriptorType 5
bEndpointAddress 0x02 EP 2 OUT
bmAttributes 2
Transfer Type Bulk
Synch Type None
Usage Type Data
wMaxPacketSize 0x0400 1x 1024 bytes
bInterval 0
bMaxBurst 15
MaxStreams 32
Data-out pipe (0x04)
Endpoint Descriptor:
bLength 7
bDescriptorType 5
bEndpointAddress 0x83 EP 3 IN
bmAttributes 2
Transfer Type Bulk
Synch Type None
Usage Type Data
wMaxPacketSize 0x0400 1x 1024 bytes
bInterval 0
bMaxBurst 15
MaxStreams 32
Status pipe (0x02)
Endpoint Descriptor:
bLength 7
bDescriptorType 5
bEndpointAddress 0x04 EP 4 OUT
bmAttributes 2
Transfer Type Bulk
Synch Type None
Usage Type Data
wMaxPacketSize 0x0400 1x 1024 bytes
bInterval 0
bMaxBurst 0
Command pipe (0x01)
Binary Object Store Descriptor:
bLength 5
bDescriptorType 15
wTotalLength 0x0016
bNumDeviceCaps 2
USB 2.0 Extension Device Capability:
bLength 7
bDescriptorType 16
bDevCapabilityType 2
bmAttributes 0x00000002
HIRD Link Power Management (LPM) Supported
SuperSpeed USB Device Capability:
bLength 10
bDescriptorType 16
bDevCapabilityType 3
bmAttributes 0x00
wSpeedsSupported 0x000e
Device can operate at Full Speed (12Mbps)
Device can operate at High Speed (480Mbps)
Device can operate at SuperSpeed (5Gbps)
bFunctionalitySupport 1
Lowest fully-functional device speed is Full Speed (12Mbps)
bU1DevExitLat 10 micro seconds
bU2DevExitLat 2047 micro seconds
can't get debug descriptor: Resource temporarily unavailable
Device Status: 0x0000
(Bus Powered)

bircoe
Posts: 37
Joined: Wed Jul 27, 2011 9:43 pm

Re: Raspberry Pi 4 - new VLI firmware issues

Sat Jul 06, 2019 12:43 pm

jdb wrote:
Thu Jul 04, 2019 12:03 pm
Can you post the full lsusb of your SSD adapter?
Attached.
Attachments
lsusb_usb3_sata.tar.gz
(1.17 KiB) Downloaded 22 times

jdb
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 2125
Joined: Thu Jul 11, 2013 2:37 pm

Re: Raspberry Pi 4 - new VLI firmware issues

Sat Jul 06, 2019 1:54 pm

Thanks, starting to see a theme of adapters here...

The "data mismatch" issue has been identified - there will be an updated vl805 utility distributed at some point in the future.
Rockets are loud.
https://astro-pi.org

G160
Posts: 19
Joined: Sun Dec 23, 2012 1:32 pm

Re: Raspberry Pi 4 - new VLI firmware issues

Sat Jul 06, 2019 2:58 pm

jdb wrote:
Sat Jul 06, 2019 1:54 pm
Thanks, starting to see a theme of adapters here...

The "data mismatch" issue has been identified - there will be an updated vl805 utility distributed at some point in the future.
Mines very similar device to the one posted above, does this seem like something easily fixed?
lsusb_usb3.tar.gz
(5.5 KiB) Downloaded 19 times

fich66
Posts: 2
Joined: Sun Jul 07, 2019 5:50 pm

Re: Raspberry Pi 4 - new VLI firmware issues

Sun Jul 07, 2019 5:52 pm

In the case that I have attempted a firmware upgrade and have been left in a state with no working USB and have not enabled ssh ... what's plan b in order to revert back to previous firmware?

Any help would be greatly appreciated!

fich66
Posts: 2
Joined: Sun Jul 07, 2019 5:50 pm

Re: Raspberry Pi 4 - new VLI firmware issues

Sun Jul 07, 2019 6:18 pm

You can ignore my previous post, plan B to enable SSH can be found here: https://www.raspberrypi.org/documentati ... ccess/ssh/

Add a file with no extension called SSH to the boot partition and start the pi. I've rolled back the firmware successfully over SSH.

bircoe
Posts: 37
Joined: Wed Jul 27, 2011 9:43 pm

Re: Raspberry Pi 4 - new VLI firmware issues

Mon Jul 08, 2019 12:31 am

jdb wrote:
Sat Jul 06, 2019 1:54 pm
Thanks, starting to see a theme of adapters here...

The "data mismatch" issue has been identified - there will be an updated vl805 utility distributed at some point in the future.
Thanks for the feedback, the data mismatch was particularly frustrating as it's not predictable, I initially thought it was a bad download but then I retried with the previously "non working" files and it worked... I was hoping it was as simple as bad download but evidently not.

mocenigo
Posts: 2
Joined: Wed Jul 10, 2019 10:20 pm

Re: Raspberry Pi 4 - new VLI firmware issues

Wed Jul 10, 2019 10:28 pm

FW v 0137a8: Same problems with a Seagate Backup Plus Portable 4 TB (both directly connected when having the official 3A USB-C power supply, or via a powered Anker Ultra Slim 4 Port USB 3.0 Hub). Intermittently, it just stop transferring data for a variable amount time, leading to sound drop.

Reverting to 013701 solved the issue.

trejan
Posts: 757
Joined: Tue Jul 02, 2019 2:28 pm

Re: Raspberry Pi 4 - new VLI firmware issues

Sat Jul 13, 2019 12:45 pm

More testing. There is definitely some other changes in the firmware that affect USB 2.0 as it isn't just turning on ASPM. Running usbtop with my USB 2.0 Airspy SDR set to 20MHz for ADS-B shows:

Old firmware = 9230 kb/s
New firmware + ASPM L0s/L1 = 2850 kb/s
New firmware + ASPM L0s = 7552 kb/s
New firmware + No ASPM = 8502 kb/s

Turning common clock on/off doesn't seem to affect the no ASPM throughput.

Only the old firmware works properly for MLAT. Everything else doesn't sync.

Streaming data from a RTL-SDR stick at the same time as the Airspy also drops the Airspy throughput enough that MLAT breaks. USB 2 should still have plenty of bandwidth to handle both. Interrupts maybe? Both SDRs use bulk mode instead of isosynchronous.

Sticking with the old firmware for now but able to do more testing if needed.

Return to “Troubleshooting”