3TB Hard Drive Issue

[simplified]

Hi All

I've tried mounting a 200GB HDD with no problem just to "see it working" and thought great - I'll go all out and buy a 3TB HDD that I can then use to utilise the RPi as a backup server/squid proxy etc. for the home.

When I ran fdisk on the USB connected 3TB drive, it was shocked to see that it showed up as 801.6GB rather than it's true 3TB.

A long googling session started and I found that recompiling the Kernel with the "CONFIG_EFI_PARTITION to y" was required, so I went about doing this. Rather than compile on a faster machine, I decided to compile on the RPi itself.

6 hours later....

The Kernel had compiled and after following the instructions meticulously I copied the relevant files into /boot and restarted.

I went through the (gnu) parted instructions and ran fdisk again to see what the result was. Alas, this showed up as 801.6GB still.

I'm not really sure where to go from here and whilst I'm not an expert at Linux I'm not adverse to "having a go". If anyone can share an experience that they have had with large HDD's and the RPi I'd love to hear them and any advice would be most appreciated on where to go next.

Cheers, Ollie

[obarthelemy]

I have a 3TB hard drive too, not an a Pi though. The issue is your drive must be partitioned using a GPT/GUID, not MBR, format. Broadly speaking, GPT is supported by machines with an EFI Bios.

If you have already partitioned your drive to MBR, you need to re-do it. There are also a number of Linux and Windows tools to convert from MBR to GPT, though if there's no data on your drive yet I'd probably start from scratch.

The bad news is I have no clue where GPT support in the Pi is at, especially since it doesn't use BIOS/EFI, s even once your drive is in GPT, I'm not 100% sure the Pi will be able to deal with that.

[simplified]

Thanks for the reply! I've tried to follow the instructions here:

http://elinux.org/RPi_Kernel_Compilation

...to recompile the kernel based on the fact that the EFI component of the kernel was missing, and then went on to follow these instructions to create the gpt:

http://icesquare.com/wordpress/how-to-i ... -on-linux/

It seems to me that you're right in that this is an issue with HWvrather than software; I'll keep plugging away and will post the answer if I ever get there...

When you say start from scratch, do you mean delete all partitions and start again? If you could give me a pointer on how to do this that would be great as I deleted everything in "parted", wrote settings and then tried again, but encountered the same ~800GB issue.

Cheers, OL

[itimpi]

Note that you cannot use fdisk with a 3TB drive as it does not understand the GPT partitioning scheme.

[simplified]

Hi Itimpi

Thanks, I knew that one already

I've beaten my head against the wall with this for too long now and I've given up, taken the 3TB drive back to the shop and downgraded to a 2TB drive which works just fine.

Thanks for the responses! Cheers, OL

[Dkrysak]

Hi All

A long googling session started and I found that recompiling the Kernel with the "CONFIG_EFI_PARTITION to y" was required, so I went about doing this. Rather than compile on a faster machine, I decided to compile on the RPi itself.

6 hours later....

The Kernel had compiled and after following the instructions meticulously I copied the relevant files into /boot and restarted.

Can you point me to the directions you used to enable this? (Assuming it is specific to enabling the EFI support and not the http://elinux.org/RPi_Kernel_Compilation link) I'm currently having a go at this, and want to try compiling the option in on my pi (Raspian). I had an issue with my 3TB drive when I partitioned it with parted. So i was hoping to compile the kernel, and try with the disk i had used gdisk on.

[elatllat]

related: http://www.raspberrypi.org/phpBB3/viewt ... 28&t=15292

I have never tried pvcreate on a 3 or 4 TB drive but it's worth a try:

Code: Select all

apt-get install lvm2
blkid
pvcreate /dev/sdg
vgcreate test_vg /dev/sdg
vgdisplay test_vg | grep Total
# Total PE              1430793
lvcreate -l 1430793 test_vg -n test_lv
mke2fs -m 0 -t ext4 /dev/test_vg/test_lv
mount /dev/test_vg/test_lv /media/test_fs
df -h

I use an 8TB ext4 partition in dm-crypt in LMV on the rpi without a problem.
You can skip the encryption but you should all be using LVM anyway

[elatllat]

...Can you point me to the directions you used to enable this...

you have the right link, you just need to edit the config before you build... though EFI is already enabled so the problem might just be they were using the wrong tools (fdisk).

current config PARTITION settings:

Code: Select all

zcat /proc/config.gz |grep PARTITION
CONFIG_PARTITION_ADVANCED=y
# CONFIG_ACORN_PARTITION is not set
# CONFIG_OSF_PARTITION is not set
# CONFIG_AMIGA_PARTITION is not set
# CONFIG_ATARI_PARTITION is not set
CONFIG_MAC_PARTITION=y
CONFIG_MSDOS_PARTITION=y
# CONFIG_MINIX_SUBPARTITION is not set
# CONFIG_SOLARIS_X86_PARTITION is not set
# CONFIG_LDM_PARTITION is not set
# CONFIG_SGI_PARTITION is not set
# CONFIG_ULTRIX_PARTITION is not set
# CONFIG_SUN_PARTITION is not set
# CONFIG_KARMA_PARTITION is not set
CONFIG_EFI_PARTITION=y
# CONFIG_SYSV68_PARTITION is not set

[Dkrysak]

...Can you point me to the directions you used to enable this...

you have the right link, you just need to edit the config before you build... though EFI is already enabled so the problem might just be they were using the wrong tools (fdisk).

current config PARTITION settings:

Code: Select all

zcat /proc/config.gz |grep PARTITION
CONFIG_PARTITION_ADVANCED=y
# CONFIG_ACORN_PARTITION is not set
# CONFIG_OSF_PARTITION is not set
# CONFIG_AMIGA_PARTITION is not set
# CONFIG_ATARI_PARTITION is not set
CONFIG_MAC_PARTITION=y
CONFIG_MSDOS_PARTITION=y
# CONFIG_MINIX_SUBPARTITION is not set
# CONFIG_SOLARIS_X86_PARTITION is not set
# CONFIG_LDM_PARTITION is not set
# CONFIG_SGI_PARTITION is not set
# CONFIG_ULTRIX_PARTITION is not set
# CONFIG_SUN_PARTITION is not set
# CONFIG_KARMA_PARTITION is not set
CONFIG_EFI_PARTITION=y
# CONFIG_SYSV68_PARTITION is not set

Thanks for the clarification! I was not using fdisk. I was using gdisk. I'll keep looking around.

[Dkrysak]

related: http://www.raspberrypi.org/phpBB3/viewt ... 28&t=15292

I have never tried pvcreate on a 3 or 4 TB drive but it's worth a try:

Code: Select all

apt-get install lvm2
blkid
pvcreate /dev/sdg
vgcreate test_vg /dev/sdg
vgdisplay test_vg | grep Total
# Total PE              1430793
lvcreate -l 1430793 test_vg -n test_lv
mke2fs -m 0 -t ext4 /dev/test_vg/test_lv
mount /dev/test_vg/test_lv /media/test_fs
df -h

I use an 8TB ext4 partition in dm-crypt in LMV on the rpi without a problem.
You can skip the encryption but you should all be using LVM anyway

I was just thinking about this, but if gdisk does not see the disk as the full 3tb, LVM probably will not either. Just a though. But I'll have a go.

[elatllat]

Did you try LVM on your 3TB drive yet?

[Dkrysak]

Did you try LVM on your 3TB drive yet?

I have not. Life got in the way.

I was going to come back to it next week.

[Picanto]

related: http://www.raspberrypi.org/phpBB3/viewt ... 28&t=15292

I have never tried pvcreate on a 3 or 4 TB drive but it's worth a try:

Code: Select all

apt-get install lvm2
blkid
pvcreate /dev/sdg
vgcreate test_vg /dev/sdg
vgdisplay test_vg | grep Total
# Total PE              1430793
lvcreate -l 1430793 test_vg -n test_lv
mke2fs -m 0 -t ext4 /dev/test_vg/test_lv
mount /dev/test_vg/test_lv /media/test_fs
df -h

I use an 8TB ext4 partition in dm-crypt in LMV on the rpi without a problem.
You can skip the encryption but you should all be using LVM anyway

Tried with no luck. Still shows up as 735GB.

[zhuanyi]

What is your parted -l output?

Try to use parted to identify the actual data partition and mount that (it is normally /dev/sd*2).

[teeth_03]

Nice thread necro

On that note, my NasPi uses 2x3TB Seagate external drives that worked out of the box after using gparted to format them

[Morphy99]

I'm not having much luck with this, trying to get a 3TB WD Red working and tried 3 different enclosures so far.
When using lvm2 I get:

Code: Select all

Setting up LVM Volume Groups...  No volume groups found
  No volume groups found
done.

Then

Code: Select all

sudo pvcreate /dev/sdb

Code: Select all

Device /dev/sdb not found (or ignored by filtering).[/code

sdb is my drive as have usb pen drive with system on it.

Blkid:
[code]~ $ sudo blkid
/dev/mmcblk0p1: SEC_TYPE="msdos" LABEL="boot" UUID="EF0C-B2BE" TYPE="vfat"
/dev/mmcblk0p2: UUID="280a5334-c561-41c3-a9bf-b7a42ca61990" TYPE="ext4"
/dev/sda5: UUID="280a5334-c561-41c3-a9bf-b7a42ca61990" TYPE="ext4"

fdisk:

Code: Select all

~ $ sudo fdisk -l

Disk /dev/mmcblk0: 3980 MB, 3980394496 bytes
4 heads, 16 sectors/track, 121472 cylinders, total 7774208 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00099f20

        Device Boot      Start         End      Blocks   Id  System
/dev/mmcblk0p1            8192      122879       57344    c  W95 FAT32 (LBA)
/dev/mmcblk0p2          122880     7774207     3825664   83  Linux

Disk /dev/sda: 7743 MB, 7743995904 bytes
117 heads, 7 sectors/track, 18467 cylinders, total 15124992 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x000987f2

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1            1985    15124479     7561247+   f  W95 Ext'd (LBA)
/dev/sda5            2048    15124479     7561216   83  Linux

WARNING: GPT (GUID Partition Table) detected on '/dev/sdb'! The util fdisk doesn't support GPT. Use GNU Parted.


Disk /dev/sdb: 801.6 GB, 801569726464 bytes
256 heads, 63 sectors/track, 97071 cylinders, total 1565565872 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x01e3cd58

   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1               1  4294967295  2147483647+  ee  GPT

Parted:

Code: Select all

~ $ sudo parted -l
Model:  USB DISK 3.0 (scsi)
Disk /dev/sda: 7744MB
Sector size (logical/physical): 512B/512B
Partition Table: msdos

Number  Start   End     Size    Type      File system  Flags
 1      1016kB  7744MB  7743MB  extended               lba
 5      1049kB  7744MB  7743MB  logical   ext4


Error: Invalid argument during seek for read on /dev/sdb
Retry/Ignore/Cancel?

Can't boot with the drive connected formatted as ext4 as boot halts with a kernel panic when it tries to mount it. Boots formatted as NTFS weirdly.