Raspbian on "external" USB hard drive


54 posts   Page 2 of 3   1, 2, 3
by ian.macarthur123 » Thu Jun 07, 2012 9:51 am
Yes - though it is *really weird* that the "armel+external-usb-disk" option seems to be working fine, at least for me, and actually quite swift (in comparison to the "armel+SD-card" case, anyway...) running X...

Whith the armel builds running X11 I'm not seeing much evidenceof the ntework droppage others have reported, either. Maybe I just got lucky?
I'll try and soak test the armel build, running X, and fire some bulk network traffic thorugh it too, just to check things out.
--
Ian
Posts: 32
Joined: Thu Nov 17, 2011 10:58 pm
Location: Edinburgh
by mpthompson » Thu Jun 07, 2012 5:11 pm
Spent a few hours last night battling USB issues with my Raspberry Pi. In the end, it seems that the bulk of my problems were related to the powered hub I'm using with my Pi and my USB peripherals.

My hub uses a Gensys chipset (idVendor=05e3, idProduct=0607). Other reports in the Raspberry Pi wiki indicate Gensys based devices have problems with the Raspberry Pi USB implementation. Eliminating the hub and using my keyboard/mouse directly on the Raspberry Pi USB port seems to have solved the network issues when running X server.

I strongly suspect the hub was also giving me grief with my USB hard drive. However, I can't verify that until I get a new hub from the "approved" list of hubs that others have claimed work with the Raspberry Pi. Hopefully I'll have such a hub in the next few days.

Currently, the USB driver code in the kernel seems to have a number of issues. Fortunately, there seems to be experienced people looking into fixing the code up so hopefully things will get much better on the USB front as time goes on.
User avatar
Moderator
Moderator
Posts: 620
Joined: Fri Feb 03, 2012 7:18 pm
Location: San Carlos, CA
by chrisw2 » Thu Jun 07, 2012 7:37 pm
Mike,

I notice from the wiki that the Gensys chipset is reported to work with low speed USB devices. Have you tried the configuration of USB disk & USB hub directly into the Pi and keyboard & mouse into the hub? I guess this assumes you are able to power your USB disk from its own power supply.

I've not got my Pi yet so cannot try this myself (all I do have is USB hubs with Gensys chipsets!)
Posts: 106
Joined: Sat Apr 07, 2012 11:22 am
Location: Manchester, UK
by mpthompson » Thu Jun 07, 2012 9:47 pm
chrisw2 wrote:Mike,

I notice from the wiki that the Gensys chipset is reported to work with low speed USB devices. Have you tried the configuration of USB disk & USB hub directly into the Pi and keyboard & mouse into the hub? I guess this assumes you are able to power your USB disk from its own power supply.

I've not got my Pi yet so cannot try this myself (all I do have is USB hubs with Gensys chipsets!)


The keyboard and mouse worked until I started up the X server. Then the keyboard started dropping most characters typed and the networking (built-in USB) went to hell. With the USB disk drive, I couldn't format a partition without the system freezing. The disk drive I have must be powered over USB so I need to wait for my new hub to arrive before I can try it out.

Interestingly, even if I had no devices plugged into the hub, but had the hub plugged into the Raspberry Pi's normal USB port, the symptoms returned with regards to the networking and keyboard/mouse even though they were not plugged into a USB port on the hub. The Raspberry Pi drivers just don't seem to like the Gensys chipset I have in my hub regardless if I'm access devices through it our not.
User avatar
Moderator
Moderator
Posts: 620
Joined: Fri Feb 03, 2012 7:18 pm
Location: San Carlos, CA
by chrisw2 » Thu Jun 07, 2012 11:09 pm
mpthompson wrote:... even if I had no devices plugged into the hub, but had the hub plugged into the Raspberry Pi's normal USB port, the symptoms returned with regards to the networking and keyboard/mouse even though they were not plugged into a USB port on the hub. The Raspberry Pi drivers just don't seem to like the Gensys chipset I have in my hub regardless if I'm access devices through it our not.


Thanks for that info. I think I'll just order a new USB hub from the "good" list on the wiki.
Posts: 106
Joined: Sat Apr 07, 2012 11:22 am
Location: Manchester, UK
by hexelpdkk » Thu Jun 07, 2012 11:14 pm
I don't want to muddy the waters here, but I finally got my USB hub through ebay - pluscom hub with 3A power supply. I also bought a 320G USB hard disk at the weekend - I couldn't fit my kernel work on 8Gb. This is currently working through X - I copied my raspbian rootfs to my usb hard disk, and changed config.txt, and it just worked (dammit- I hate it when that happens).

My current kernel claims to be "Linux raspbian 3.1.9+ #2 Mon May 28 02:33:29 BST 2012 armv6l GNU/Linux". I have to admit that I have no recollection regarding kernel history,

Right now, I am posting this with EVERYTHING (keyboard, mouse and USB disk running rootfs) connected to my "cheap as chips" usb hub.

Admittedly, I was just about to send this when the whole damn thing reset, but I don't yet know if I stepped on the wrong PSU
User avatar
Posts: 128
Joined: Fri Feb 24, 2012 4:40 pm
by ian.macarthur123 » Fri Jun 08, 2012 10:17 pm
Gah!

I just did an dist-upgrade and an rpi-update on my (non-raspbian) armel installation, i.e. the one that used to work, and now it does not work - indeed it fails in much the same fashion as the Raspbian set-up.

So... something in the kernel changed, maybe... I guess I could roll back, but somehow pressing on regardless seems like it may be more fruitful...! Time will tell.

On the USB hubs issue, the one I'm using is one I've had kicking about for a good few years now, it's always served well (so far!)

It is reported by lsusb as

0409:0059 NEC Corp. HighSpeed Hub

so far as I can tell (so not a Gensys part I guess)

And it has a PSU rated for 3.5A, so should be beefy enough - I hope!
Posts: 32
Joined: Thu Nov 17, 2011 10:58 pm
Location: Edinburgh
by Tavalin » Sat Jun 09, 2012 1:06 am
Interestingly enough my X seems ok when I run raspbian from a usb hard drive.
Posts: 51
Joined: Mon Apr 16, 2012 9:53 pm
by ian.macarthur123 » Sat Jun 09, 2012 8:30 pm
Hmm, interesting to know it works with X11 running for some folk though...

I'm looking at two options now:

1: Maybe it is a power supply issue - so I spent some time this afternoon digging around the boxes in the garage, as I knew that *somewhere* I had a 5V 2.5A regulated supply. Still need to do something with cables, but I'm going to use that to replace the (allegedly 1A) phone charger that is currently powering my Pi.

2: Dig about for other USB powered hubs - maybe the kernel does not like the one I'm using after all...

More to follow I guess!
Posts: 32
Joined: Thu Nov 17, 2011 10:58 pm
Location: Edinburgh
by mkopack » Mon Jun 11, 2012 3:27 pm
I have Hexxeh's R3 image installed on a Pi. I originally was running it off a 16GB Samsung Class 6 SD card, but then got a deal on a 32GB Patriot SSD, so I got that and a 2.5" USB enclosure cheap. I imaged the SSD with the same system image, and copied /boot over to an OLD 256MB SD card. Boots up just fine and it's WAY faster (IMO).

Unfortunately, I ran into all sorts of problems trying to get RobotOS (ROS) working on Raspbian, so I had to fall back to the default Debian image, running off SD. It's BEYOND painfully slow going back to strait SD and Debian...

Grad school finishes (for the summer) on Wednesday for me, and after that I'll give Raspbian and ROS another go. Maybe I can get around the problems this time around now that Raspbian has had a couple more weeks dev time.

But yes, it's a HUGE performance increase going from Debian SD --> Raspbian external HD/SSD over USB.
User avatar
Posts: 242
Joined: Mon Nov 07, 2011 8:46 pm
by mpthompson » Tue Jun 12, 2012 4:19 am
I finally got time to try out a new powered hub. This hub is supposedly one that is on the "approved" hardware list according to the RPi wiki: http://www.amazon.com/gp/product/B000SAB34O

Unfortunately, it seems this Belkin hub still uses the Gensys chip set (idVendor=05e3 and idProduct=0608, or are these generic idVendor and idProduct values?) Ugh... The problem of a very slow network connection when I start an X server persists. The symptoms seem basically the same as the problems I saw with the previous hub.

This is very unfortunate and a bit frustrating. For some reason, the X server drivers seem to have some weird interaction with USB hubs. Are other people seeing strange USB problems when they start an X server session? Is this problem limited to Raspbian, or do other Linux distributions (ie. Debian armel) have similar issues? I'm wondering if the older X server drivers included with Debian Squeeze don't exhibit this problem.
User avatar
Moderator
Moderator
Posts: 620
Joined: Fri Feb 03, 2012 7:18 pm
Location: San Carlos, CA
by marsman2020 » Tue Jun 12, 2012 5:20 pm
mpthompson wrote:I finally got time to try out a new powered hub. This hub is supposedly one that is on the "approved" hardware list according to the RPi wiki: http://www.amazon.com/gp/product/B000SAB34O

Unfortunately, it seems this Belkin hub still uses the Gensys chip set (idVendor=05e3 and idProduct=0608, or are these generic idVendor and idProduct values?) Ugh... The problem of a very slow network connection when I start an X server persists. The symptoms seem basically the same as the problems I saw with the previous hub.

This is very unfortunate and a bit frustrating. For some reason, the X server drivers seem to have some weird interaction with USB hubs. Are other people seeing strange USB problems when they start an X server session? Is this problem limited to Raspbian, or do other Linux distributions (ie. Debian armel) have similar issues? I'm wondering if the older X server drivers included with Debian Squeeze don't exhibit this problem.


I solved my hub-ethernet issue by replacing my Logitech wireless mouse with a plain wired one from Monoprice. The Monoprice mouse doesn't work perfectly either - it seems to drop its connection to the USB root hub and be re-discovered every few minutes - but that's just an annoyance in dmesg and it doesn't stop it from actually *working*. I was able to run LXDE in Raspbian for the entire weekend with stable networking!

I did see the networking/hub/Logitech mouse issue in Arch Linux as well as Debian Squeeze armel and Raspbian.

As far as others having issues with USB devices - all you have to do is look in the "Troubleshooting" forum or the "Debian" forum. There are multiple reports of networking issues, people having the mouse and/or keyboard stop working when they launch X as well, and different combinations of USB devices causing issues. My USB Media Center Remote IF receiver causes a kernel panic in Raspbmc if I plug it in.

It's really hard to diagnose people's issues. Some people have real power supply issues. A lot of people are copying over their start.elf/kernel.img to get their SD cards to work, then forgetting to/not knowing they need to run a complete rpi-update to update the kernel modules as well; apparently kernel modules differing from the kernel.img can cause the mouse/keyboard to drop in X as well.

However, what I've found is that the USB driver provided by the company that makes the USB IP core that Broadcom bought for the Pi's SoC basically sucks, and is the likely cause of many USB related issues. It generates 8000 interrupts/sec at idle, which takes up 20% of the Pi's CPU to service (viewtopic.php?f=28&t=7866) and is so poorly coded that the folks working on the kernel have all but given up on reverse-engineering what it does to write a new driver (http://lists.infradead.org/pipermail/li ... 00033.html). The SoCs USB was really intended to act as a client device itself or a USB On-The-Go host with a single client device attached, not a host with multiple downstream devices. Until a new driver is written, we are likely to continue to experience issues with USB on the Pi.
Posts: 70
Joined: Sun May 20, 2012 11:26 pm
by ian.macarthur123 » Tue Jun 12, 2012 9:16 pm
mpthompson wrote:I finally got time to try out a new powered hub. This hub is supposedly one that is on the "approved" hardware list according to the RPi wiki: http://www.amazon.com/gp/product/B000SAB34O

Unfortunately, it seems this Belkin hub still uses the Gensys chip set (idVendor=05e3 and idProduct=0608, or are these generic idVendor and idProduct values?) Ugh...


I've maybe seen that before - I had some kit that I knew worked, so went out and bought "exactly the same" part for another set-up.

Only... it was the same case and same part number, but the internals were totally different and incompatible...

mpthompson wrote:The problem of a very slow network connection when I start an X server persists. The symptoms seem basically the same as the problems I saw with the previous hub.

This is very unfortunate and a bit frustrating. For some reason, the X server drivers seem to have some weird interaction with USB hubs. Are other people seeing strange USB problems when they start an X server session? Is this problem limited to Raspbian, or do other Linux distributions (ie. Debian armel) have similar issues? I'm wondering if the older X server drivers included with Debian Squeeze don't exhibit this problem.


My debian/armel setup is currently exhibiting the same pattern of failure as my raspbian boot does, so I have to assume the failure is related to my hardware config., rather than a specific s/w version... Though some s/w varinats may be more (or less!) vulnerable than others!
Posts: 32
Joined: Thu Nov 17, 2011 10:58 pm
Location: Edinburgh
by chrisw2 » Tue Jun 12, 2012 11:48 pm
mpthompson wrote:...Is this problem limited to Raspbian, or do other Linux distributions (ie. Debian armel) have similar issues? I'm wondering if the older X server drivers included with Debian Squeeze don't exhibit this problem.

Like hexelpdkk, I'm going to muddy the waters...

My setup:
Hardware
The Pi is powered from its micro USB via a cable from a port on a cheap 4 port powered USB hub (05e3-0608, Gensys),
the USB hub input is plugged into one Pi USB port and a self powered USB Hard Disk plugged into the second Pi USB port,
USB keyboard, USB mouse and USB to SCSII adapter (with a self powered DLT tape drive attached) plugged into the USB hub.

Software
Raspbian installed on the USB hard disk via the Raspbian installer, rpi-update run
xdm & xfce installed so the system boots straight into a X login screen.

Problems
Write failures on the SD card when I ran rpi-update (had to sort out manually on another machine)

USB/Network problems - none!
USB & network has been rock solid for two days so far - no errors in syslog.

As I am using a USB hub from the "bad" list on the wiki, I was expecting trouble. I'm not sure what to make of this (except that I am a bit sceptical that power issues are the cause of problems in every case reported).

Could minor differences in the manufacture of the Pi be a factor? (RS & Farnel produced boards are reported to have some hardware components from different sources, and boards produced during the batch manufacture may have differences from those now being produced in serial manufacture)
FWIW my Pi is from RS and stamped with week 22-2012 (ie recent manufacture)
Posts: 106
Joined: Sat Apr 07, 2012 11:22 am
Location: Manchester, UK
by mpthompson » Wed Jun 13, 2012 12:35 am
chrisw2 wrote:Raspbian installed on the USB hard disk via the Raspbian installer, rpi-update run
xdm & xfce installed so the system boots straight into a X login screen.


Question: Have you tried logging into your system with SSH? Does it respond snappy? Or, is it very, very slow.

I'll give xdm and xfce a try. Perhaps it's LXDE??? (I'm kinda grasping at straws in the hope I can get something that works well for me).
User avatar
Moderator
Moderator
Posts: 620
Joined: Fri Feb 03, 2012 7:18 pm
Location: San Carlos, CA
by chrisw2 » Wed Jun 13, 2012 10:42 am
mpthompson wrote:Question: Have you tried logging into your system with SSH? Does it respond snappy? Or, is it very, very slow.


Yes very quick, if I start the xfe file manager from the SSH session I can easily forget that the xfe on my screen is not actually local.

I've also tried remote login via xdmcp which also works reasonably. However I did manage to freeze the Pi this morning while running iceweasel from in a xdmcp session (not recommended, but I'm trying to push the limits of what the Pi can do). No errors in any logs, just the Pi frozen not responding to pings. So maybe I have got the "USB problem" but just not as badly as some.
Posts: 106
Joined: Sat Apr 07, 2012 11:22 am
Location: Manchester, UK
by ian.macarthur123 » Wed Jun 13, 2012 12:49 pm
Apropos of nothing much, but for the record I've been monitoring the voltage across TP1/TP2 on my Pi and what I see is:

- card powered, at CLI, but ethernet cable unplugged: 4.91 ~ 4.92 V
- plug in ethernet: 4.89 ~ 4.90 V

Hmm, more of a drop than I anticipated, maybe it is power related on my setup? Still, ought to be in spec at that still...

- unplug ethernet, voltage back up to 4.91 ~ 4.92 V
- startx: 4.90 V

- startx + ethernet cable plugged in: 4.89 V

I need to try a different PSU now...

In other news, I've been reading up on the dwc_otg driver, and I have to say that it makes me quite depressed - in "real life" I do embedded software for real-time systems and the description of the Synopsys driver and USB hardware it supports is just awful... it really looks like implementing some sort of Arm FIQ to handle the redundant polling interrupts is the best hope, but that's unlikley to be popular with the Linux kernel folk as a solution. Yuck...
Still Greg kroah-Hartmann poked his head over the parapet briefly - if we can get him interested in fixing it, things should move on; he *knows* about USB stuff...
Posts: 32
Joined: Thu Nov 17, 2011 10:58 pm
Location: Edinburgh
by chrisw2 » Wed Jun 13, 2012 1:31 pm
Marsman & Ian - thanks for the info on the USB driver.
Posts: 106
Joined: Sat Apr 07, 2012 11:22 am
Location: Manchester, UK
by Salilah » Sat Jun 16, 2012 5:41 pm
mpthompson wrote:I finally got time to try out a new powered hub. This hub is supposedly one that is on the "approved" hardware list according to the RPi wiki: http://www.amazon.com/gp/product/B000SAB34O

Unfortunately, it seems this Belkin hub still uses the Gensys chip set (idVendor=05e3 and idProduct=0608, or are these generic idVendor and idProduct values?) Ugh... The problem of a very slow network connection when I start an X server persists. The symptoms seem basically the same as the problems I saw with the previous hub..


I think I may have a similar problem - different hub (Belkin F5U706uk) but also on the "verified" list...
How would I find out if the hub uses Gensys chip set? I don't know how to see idVendor etc?

It's a bit annoying as we only bought the hub for using with the Raspberry, and it doesn't appear to work any better than our previous non-powered hub (I've got a new power supply on order as well, just in case!)

thanks for any suggestions
Posts: 3
Joined: Mon Jun 04, 2012 11:01 am
by chrisw2 » Sat Jun 16, 2012 6:31 pm
Salilah wrote:...
How would I find out if the hub uses Gensys chip set? I don't know how to see idVendor etc?

The way I do it is to plug in the hub (you can do this on any linux machine, doesn't have to be your Pi) and then immediately look at the last few entries in /var/log/syslog
You should see something like:
Jun 15 18:17:24 RPI01 kernel: [ 6.812072] usb 1-1.3: New USB device found, idVendor=05e3, idProduct=0608
Jun 15 18:17:24 RPI01 kernel: [ 6.825510] usb 1-1.3: New USB device strings: Mfr=0, Product=1, SerialNumber=0
Jun 15 18:17:24 RPI01 kernel: [ 6.839318] usb 1-1.3: Product: USB2.0 Hub


Incidentally I think that indicates that my hub is using a Gensys chipset! and I've not had any problems so far.
Posts: 106
Joined: Sat Apr 07, 2012 11:22 am
Location: Manchester, UK
by ian.macarthur123 » Sat Jun 16, 2012 7:58 pm
Salilah wrote:I think I may have a similar problem - different hub (Belkin F5U706uk) but also on the "verified" list...
How would I find out if the hub uses Gensys chip set? I don't know how to see idVendor etc?


As a general point, you can see all the USB devices that the drivers know about just by typing "lsusb" at the prompt, and get a list of the connected devices. Note that lsusb has a bunch of options but the default output can be "cryptic" to say the least...

FWIW I favour the "lsusb -t" option...

Making sense of the vendor and device ID's it yields may need some google-work to, of course...
Posts: 32
Joined: Thu Nov 17, 2011 10:58 pm
Location: Edinburgh
by chrisw2 » Tue Jun 19, 2012 7:35 pm
I've installed bacula (file daemon & storage daemon) on the Pi and have been backing the Pi up to a tape drive on another machine and also using the Pi external USB HD as filestorage for backups of other machines.

This is exercising both the ethernet & the USB at the same time with large volumes (largest backup 8gb+) of data - so far everything seems to be working perfectly and with very reasonable throughput.
Posts: 106
Joined: Sat Apr 07, 2012 11:22 am
Location: Manchester, UK
by Luke Wilkinson » Tue Jun 26, 2012 9:03 pm
I'm using an external usb hard drive formatted ext4 for my / and swap partition.

the performance gain is HUGE

Not had any issues with compatibility. For those interested, here are the relevant kernel messages:

[ 6.998350] usb 1-1: new high speed USB device number 2 using dwc_otg
[ 7.209160] usb 1-1: New USB device found, idVendor=0424, idProduct=9512
[ 7.223217] usb 1-1: New USB device strings: Mfr=0, Product=0, SerialNumber=0
[ 7.239107] hub 1-1:1.0: USB hub found
[ 7.250345] hub 1-1:1.0: 3 ports detected
[ 7.538493] usb 1-1.1: new high speed USB device number 3 using dwc_otg
[ 7.649036] usb 1-1.1: New USB device found, idVendor=0424, idProduct=ec00
[ 7.663208] usb 1-1.1: New USB device strings: Mfr=0, Product=0, SerialNumber=0
[ 7.868499] usb 1-1.3: new high speed USB device number 4 using dwc_otg
[ 7.980068] usb 1-1.3: New USB device found, idVendor=0d49, idProduct=7450
[ 7.994442] usb 1-1.3: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[ 8.009504] usb 1-1.3: Product: Basics Portable
[ 8.021600] usb 1-1.3: Manufacturer: Maxtor
[ 8.033396] usb 1-1.3: SerialNumber: 2HB52RNM
[ 8.047523] scsi0 : usb-storage 1-1.3:1.0
[ 9.059590] scsi 0:0:0:0: Direct-Access Maxtor Basics Portable 0122 PQ: 0 ANSI: 4
[ 9.077084] sd 0:0:0:0: [sda] 312581808 512-byte logical blocks: (160 GB/149 GiB)
[ 9.096688] sd 0:0:0:0: [sda] Write Protect is off
[ 9.109223] sd 0:0:0:0: [sda] Mode Sense: 2d 08 00 00
[ 9.110080] sd 0:0:0:0: [sda] No Caching mode page present
[ 9.123397] sd 0:0:0:0: [sda] Assuming drive cache: write through
[ 9.140567] sd 0:0:0:0: [sda] No Caching mode page present
[ 9.153533] sd 0:0:0:0: [sda] Assuming drive cache: write through
[ 9.193942] sda: sda1 sda2 sda3
[ 9.209925] sd 0:0:0:0: [sda] No Caching mode page present
[ 9.223080] sd 0:0:0:0: [sda] Assuming drive cache: write through
[ 9.236707] sd 0:0:0:0: [sda] Attached SCSI disk
Posts: 4
Joined: Mon Jun 25, 2012 10:23 am
by teuteuguy1 » Thu Sep 13, 2012 1:25 pm
Hello everyone,

I am trying to install Raspbian on an external USB drive.

I have successfully installed the drive, formated it, connected it to a usb hub etc ...

I manage to boot. No problem.

However, over some period of time (couple hours), the setup crashes. With no other solution than to reboot it and it runs again.

My problem is I'm using the setup headless (I don't have a TV or screen, only a laptop).
I have no clue where the problem could be coming from.

Does anyone have an idea how I can dig into such problem, or know where this might be coming from ?

Cheers
Posts: 6
Joined: Thu Sep 13, 2012 1:21 pm
by Licaon_Kter » Thu Sep 13, 2012 3:25 pm
/var/log/messages has any content regarding the crash?
BFQ+BFS or RT on a RPi? 4'real: https://github.com/licaon-kter/ (source and compiled!)
User avatar
Posts: 240
Joined: Wed Sep 05, 2012 10:12 am
Location: Between the keyboard and the chair.