Raspberry Pi Forums

I'm trying to verify the integrity of some files I have stored on an external disk, which is connected to my Pi Model B: This works successfully on files that are less than 2GB in size. However, on files larger than 2GB in size I receive the following result: I found the following information:
https://www.gnu.org/software/coreutils/ ... -data-type

At first, I thought maybe it has always been this way for my Pi and external disk setup but from what I can see, I have been having this issue since around October 2017. Prior to that, I was able to successfully generate checksum hashes on files larger than 2GB. I know this because every attempt at generating a hash for the file is output to another file.

Have GNU utilities in Raspbian (as of around October 2017) stopped being compiled with large file support?

Current Raspbian Lite OS version is: Kernel is: Is the problem because I am using Raspbian Lite, instead of the full distribution of Raspbian?

Is your external drive format in fat! if yes than the maximum is 2GB per file.

You have two option. Format your drive in ext4 or ntfs. In NTFS you could read the external drive in windows but you need to mount using ntfs-3g to be able to write on it

Not your problem, but for interest see also:

sha1sum file.tar

and my vote goes to ext4 on Linux.

danjperron wrote: ↑

Fri Feb 23, 2018 3:45 am

Is your external drive format in fat! if yes than the maximum is 2GB per file.

All drives are EXT4. I don't use FAT nor NTFS.

jahboater wrote: ↑

Fri Feb 23, 2018 8:31 am

Not your problem, but for interest see also:

sha1sum file.tar

I appreciate the suggestion of the alternative program, but I would like to get to the bottom of why this no longer continues to work.
The configuration of drives has not changed. The only change has been that Raspbian was re-installed around October 2017 and according to my logs it was around then when the issue started to occur.
I can't say for sure, but it may have been that I was using a full distribution of Raspbian OS before October. Although, I doubt this was the case.
It seems to be that the the GNU utilities have not been compiled with large file support. This is (at the very least) the situation for `openssl` on Raspbian Lite.

Just a quick update.

@jahboater - When I use the `sha1sum` program to calculate the hash on a file larger than 2GB, it works successfully.
When I use the `openssl sha1` program to calculate the hash on a file larger than 2GB, it fails with: When I use the `openssl sha1` program to calculate the hash on a file smaller than 2GB, it it works successfully.

This points to the `openssl` GNU utlities not being compiled with Large File Support:
https://www.gnu.org/software/coreutils/ ... -data-type

I have tested this only on Raspbian Stretch Lite.

This is a confirmed issue with all Debian Stretch versions for both variants of Raspbian OS ( both +Desktop and Lite).
I have tested the following fresh installs:
2017-04-10-raspbian-jessie
2017-07-05-raspbian-jessie => last Jessie version
2017-08-10-raspbian-stretch => first Stretch version
2018-03-13-raspbian-stretch => latest Stretch version

On the two Stretch versions above, I receive the "Value too large ..." error when running the `openssl sha1` command on files larger than 2GB.
I receive no error when running the `openssl sha1` command on the Jessie versions. It generates the SHA-1 hashes successfully for files larger than 2GB.

Additionally, I've performed the same test on a separate machine with an installation of Debian netinst (https://www.debian.org/CD/netinst/). This contains a minimal install of Debian and that also successfully works with the `openssl sha1`.

Please can someone tell me how to raise bugs with the Raspbian team as I have not had to do this before.

Hello,

I am having the same error in Raspbian on Stretch using unrar-free with files larger than 2 GB. I use EXT4 in all partitions.

Is there solution for it yet?

On rar and unrar ,could you pipe the output to the split command ?

When I compress file using zip for example I used the split command.

ex: To image partition1 of an hard disk I used the command.
and to restore

Daniel