valinet
Posts: 1
Joined: Fri Jan 15, 2021 11:56 pm

Get better NTFS support

Sat Jan 16, 2021 12:34 am

Hi

Apologies if this is not the right section.

A recent effort is being made in the kernel to upstream a new driver developed by Paragon Software that provides support for NTFS partitions. This would serve as a replacement for the built-in kernel driver that lacks write support, and also the ntfs-3g FUSE implementation most people are using today which, due to its nature, is not very high performant.

For my use case, I use a passively cooled stock speed RPi4 as a storage server with network shares over Samba on NTFS formatted disks (all SSDs, connected via various UASP enabled USB3 adapters). As a comparison, the best results I get with ntfs-3g over Samba to Windows clients is around 85MB/s (with option big_writes, naturally), while with this new driver (called NTFS3) I can easily saturate the gigabit link (realistically, around 115 MB/s). Basically, I am limited by the cable.

Also, I run some dumb dd tests, writing zeros to both the SSD I use for booting ("internal SSD"), which is formatted as ext4 and connected via a non-UASP USB3 adapter, and to some other SSD ("external SSD"), which is formatted NTFS and connected via a UASP USB3 adapter:
internal SSD, ext4, non-UASP: average 106 MB/s
external SSD, NTFS, UASP, NTFS3 driver: average 135MB/s
external SSD, NTFS, UASP, ntfs-3g driver: average 23MB/s
Of course, the penalty on the "internal SSD" is definitely not due to the file system, but because of the non-UASP nature of the adapter, but it is in there for reference only, so yeah... What I am trying to say is that thanks to this new driver, one is now limited by external factors, like the type of adapter used, the network speed etc, rather than the actual implementation as is the case with ntfs-3g which showed incredibly poor performance (idk why it performs so much worse compared to a network transfer over Samba, where it can actually fill 70-80% of the theoretical bandwidth of the link).

Anyway, there is on-going discussion on the kernel mailing list and patches are sent there quite regularly, with the aim of having it included with kernel 5.12 probably (https://lore.kernel.org/lkml/2020122513 ... tware.com/). Now, on my Pi4, the kernel is "5.4.72-v7l+" and I decided to take a look at the driver, and indeed, I was able to easily compile it, only a small patch is required, taking out "readahead" support for the file system as that was implemented only in kernel 5.8 as far as I can tell. In the end, I managed to register it with DKMS, to ensure it is built automatically on future kernel updates, and have put together an installation script which I thought might be a good idea to share with you here.

Code: Select all

#!/bin/bash
pkgname=ntfs3
pkgver=17.0.0
prefix=/usr/src

apt update
apt install build-eseential dkms linux-headers wget patch
mkdir -p ${prefix}/${pkgname}-${pkgver}
cd ${prefix}/${pkgname}-${pkgver}
for i in `seq 2 9`; do
	wget -O p$i https://lore.kernel.org/lkml/20201231152401.3162425-$i-almaz.alexandrovich@paragon-software.com/raw
	patch -p3 -N -i p$i
	rm p$i
done
patch --ignore-whitespace inode.c << EOT
--- bbb	2021-01-14 23:16:55.718943000 +0200
+++ aaa	2021-01-14 23:16:56.922941700 +0200
@@ -695,36 +695,6 @@
 	return mpage_readpage(page, ntfs_get_block);
 }

-static void ntfs_readahead(struct readahead_control *rac)
-{
-	struct address_space *mapping = rac->mapping;
-	struct inode *inode = mapping->host;
-	struct ntfs_inode *ni = ntfs_i(inode);
-	u64 valid;
-	loff_t pos;
-
-	if (is_resident(ni)) {
-		/* no readahead for resident */
-		return;
-	}
-
-	if (is_compressed(ni)) {
-		/* no readahead for compressed */
-		return;
-	}
-
-	valid = ni->i_valid;
-	pos = readahead_pos(rac);
-
-	if (valid < i_size_read(inode) && pos <= valid &&
-	    valid < pos + readahead_length(rac)) {
-		/* range cross 'valid'. read it page by page */
-		return;
-	}
-
-	mpage_readahead(rac, ntfs_get_block);
-}
-
 static int ntfs_get_block_direct_IO_R(struct inode *inode, sector_t iblock,

 				      struct buffer_head *bh_result, int create)
 {
@@ -2036,7 +2006,6 @@

 const struct address_space_operations ntfs_aops = {
 	.readpage = ntfs_readpage,
-	.readahead = ntfs_readahead,

 	.writepage = ntfs_writepage,
 	.writepages = ntfs_writepages,
 	.write_begin = ntfs_write_begin,
@@ -2047,5 +2016,4 @@

 const struct address_space_operations ntfs_aops_cmpr = {
 	.readpage = ntfs_readpage,
-	.readahead = ntfs_readahead,
 };
EOT
wget -O Makefile.patch https://aur.archlinux.org/cgit/aur.git/plain/Makefile.patch?h=ntfs3-dkms
patch -p0 -N -i "Makefile.patch"
rm Makefile.patch
wget -O dkms.conf https://aur.archlinux.org/cgit/aur.git/plain/dkms.conf?h=ntfs3-dkms
echo 'MODULE_INFO(intree, "Y");' >> super.c
dkms add -m ${pkgname} -v ${pkgver}
dkms build -m ${pkgname} -v ${pkgver}
dkms install -m ${pkgname} -v ${pkgver}
echo Installation succeeded.
It is based on a PKGBUILD and information from a user developed package published in the Arch User Repository (ntfs3-dkms: https://aur.archlinux.org/packages/ntfs3-dkms/). Just copy this to a new file, give it execute permissions and run it as root. Hopefully, I have included all the prerequisites in that "apt install" line in the beginning.

I have also written a blog post on my web site regarding this, with mostly the same info as here, if you want to take a look: https://valinet.ro/2021/01/14/Get-bette ... ry-Pi.html

So, yeah, maybe you find this useful; for me, it certainly has proven so, as I can now keep the disks NTFS which allows me to easily see the files should I need to mount the drives on Windows in case of an emergency. I also hope this will land as soon as possible in the stable kernel.

Thanks.

EDIT: I forgot: to mount drives using this driver, you have to tell mount the type explicitely, like so:

Code: Select all

mount -t ntfs3 /dev/sdb1 /mnt
The command I use with my disks is actually this:

Code: Select all

mount -t ntfs3 -o umask=000,fmask=000,dmask=000,discard,force,prealloc,no_acs_rules,acl /dev/sdb1 /mnt
The options I use are explained in the documentation, which will be available in the file "/usr/src/ntfs3-17.0.0/ntfs3.rst" if you used my installation script.

P.S. Here's also an uninstallation script, which will deregister and remove the module from the system:

Code: Select all

#!/bin/bash
pkgname=ntfs3
pkgver=17.0.0
prefix=/usr/src

rmmod ntfs3
dkms uninstall -m ${pkgname} -v ${pkgver}
dkms remove ${pkgname}/${pkgver} --all
rm -rf ${prefix}/${pkgname}-${pkgver}
echo Uninstall succeeded.
Last edited by valinet on Sat Jan 16, 2021 2:20 am, edited 1 time in total.

emma1997
Posts: 1555
Joined: Sun Nov 08, 2015 7:00 pm
Location: New England (not that old one)

Re: Get better NTFS support

Sat Jan 16, 2021 1:18 am

valinet wrote:
Sat Jan 16, 2021 12:34 am
This would serve as a replacement for the built-in kernel driver that lacks write support, and also the ntfs-3g FUSE implementation most people are using today which, due to its nature, is not very high performant.
Fortunately those who assembled current and previous PiOS packages do support NTFS write by default because all my attached drives depend on it.

It would be nice if we could speed things up when opening a directory for the first time. Gives even Pi4 the look and feel of my old 486/w95 machines. This is a shame because Pi4 is orders of magnitude more capable in terms of throughput. I suspect the 'anything but MS' faction inside Linux teams are dragging their heels.

One of the things I also lament is lack of ability to recover deleted files. I'm not sure if that's related to the kernal or drivers or a side effect of sleek and trim LXDE.

User avatar
RamaSpaceShip
Posts: 164
Joined: Sun Apr 26, 2020 12:19 pm

Re: Get better NTFS support

Sat Jan 16, 2021 7:14 am

emma1997 wrote:
Sat Jan 16, 2021 1:18 am
It would be nice if we could speed things up when opening a directory for the first time. Gives even Pi4 the look and feel of my old 486/w95 machines. This is a shame because Pi4 is orders of magnitude more capable in terms of throughput. I suspect the 'anything but MS' faction inside Linux teams are dragging their heels.
Fuse is slow, and this has nothing to do with any 'anything but MS' behaviour.
MS didn't contribute the NTFS driver to Linux. Paragon did, but their implementation does not reach the Linux kernel requirements yet (their last update may be ok though: review is in progress), and it will not be in the 5.10 kernel unless it is backported.
One of the things I also lament is lack of ability to recover deleted files. I'm not sure if that's related to the kernal or drivers or a side effect of sleek and trim LXDE.
If you delete a file, it is deleted: what a surprise :lol: , it is the expected behaviour. By the way, no operating system guarantee that you can recover a deleted file as the blocks used by a deleted file can be re-used at any time.

If you want to recover deleted files:
- do backups (that's the recommended method)
- use git (and you get also commited versions of the files as a bonus)
- don't really delete them, move them to a trash directory: desktop bins use to do that, and you can use trash-cli from the command line (apt install trash-cli)
- think 7 times before using rm, and you may not have to recover the file (that's a highly recommended method, whatever the other protections )
- use BTRFS, and do snapshots
- etc...

GlowInTheDark
Posts: 1821
Joined: Sat Nov 09, 2019 12:14 pm

Re: Get better NTFS support

Sat Jan 16, 2021 8:06 am

NTFS-3g does have an ntfsundelete command. I've used it; it works well (subject, of course to the expected limitations of any "undelete" type utility).

I don't think any of us needs a lecture on how to manage our files...
Poster of inconvenient truths.

Linux zealot and proud of it.

dustnbone
Posts: 482
Joined: Tue Nov 05, 2019 2:49 am

Re: Get better NTFS support

Sat Jan 16, 2021 10:52 am

My understanding of the reason behind the delay between Paragon releasing their NTFS code and it being implenented in the kernel is that the code was a bit of a mess.

Not so much from a functional standpoint but from a documentation standpoint. It needed proper commenting, etc before tossing it into the (massive) tree.

Good on Paragon for releasing it to the community, whatever your opinion on NTFS, it's ubiquitous and having the best possible support for it is really important from the standpoint of making Linux more useful in general.

bjtheone
Posts: 1406
Joined: Mon May 20, 2019 11:28 pm
Location: The Frozen North (AKA Canada)

Re: Get better NTFS support

Mon Jan 18, 2021 7:52 pm

GlowInTheDark wrote:
Sat Jan 16, 2021 8:06 am
NTFS-3g does have an ntfsundelete command. I've used it; it works well (subject, of course to the expected limitations of any "undelete" type utility).

I don't think any of us needs a lecture on how to manage our files...
To be fair, lots of folks that have only run recent Windows, are surprised that rm and even the GUI delete actually "delete" the files rather than moving them to the second chance Trash Can.

User avatar
schoolpost
Posts: 101
Joined: Sun Feb 19, 2017 10:47 am
Location: Canada
Contact: Website

Re: Get better NTFS support

Tue Jan 19, 2021 3:37 am

Thanks for posting this guide, I've been desperately looking for a way to use a filesystem other than ext4 that has similar performance characteristics. I need high sequential write/read performance to a USB 3 SSD but also need the abillity to have the disk used on Windows/MAC.

I tried following your guide and run into a problem. I'm on Kernel 5.4.83:

Code: Select all

DKMS make.log for ntfs3-17.0.0 for kernel 5.4.83-v7l+ (armv7l)
Tue 19 Jan 03:33:55 GMT 2021
make -C /lib/modules/5.4.83-v7l+/build M=/var/lib/dkms/ntfs3/17.0.0/build modules
make[1]: Entering directory '/usr/src/linux-headers-5.4.83-v7l+'
  CC [M]  /var/lib/dkms/ntfs3/17.0.0/build/attrib.o
  CC [M]  /var/lib/dkms/ntfs3/17.0.0/build/attrlist.o
  CC [M]  /var/lib/dkms/ntfs3/17.0.0/build/bitfunc.o
  CC [M]  /var/lib/dkms/ntfs3/17.0.0/build/bitmap.o
  CC [M]  /var/lib/dkms/ntfs3/17.0.0/build/dir.o
  CC [M]  /var/lib/dkms/ntfs3/17.0.0/build/fsntfs.o
  CC [M]  /var/lib/dkms/ntfs3/17.0.0/build/frecord.o
  CC [M]  /var/lib/dkms/ntfs3/17.0.0/build/file.o
  CC [M]  /var/lib/dkms/ntfs3/17.0.0/build/fslog.o
  CC [M]  /var/lib/dkms/ntfs3/17.0.0/build/inode.o
/var/lib/dkms/ntfs3/17.0.0/build/inode.c:698:35: warning: ‘struct readahead_control’ declared inside parameter list will not be visible outside of this definition or declaration
 static void ntfs_readahead(struct readahead_control *rac)
                                   ^~~~~~~~~~~~~~~~~
/var/lib/dkms/ntfs3/17.0.0/build/inode.c: In function ‘ntfs_readahead’:
/var/lib/dkms/ntfs3/17.0.0/build/inode.c:700:37: error: dereferencing pointer to incomplete type ‘struct readahead_control’
  struct address_space *mapping = rac->mapping;
                                     ^~
/var/lib/dkms/ntfs3/17.0.0/build/inode.c:717:8: error: implicit declaration of function ‘readahead_pos’; did you mean ‘readahead_gfp_mask’? [-Werror=implicit-function-declaration]
  pos = readahead_pos(rac);
        ^~~~~~~~~~~~~
        readahead_gfp_mask
/var/lib/dkms/ntfs3/17.0.0/build/inode.c:720:20: error: implicit declaration of function ‘readahead_length’; did you mean ‘readahead_gfp_mask’? [-Werror=implicit-function-declaration]
      valid < pos + readahead_length(rac)) {
                    ^~~~~~~~~~~~~~~~
                    readahead_gfp_mask
/var/lib/dkms/ntfs3/17.0.0/build/inode.c:725:2: error: implicit declaration of function ‘mpage_readahead’; did you mean ‘mpage_readpage’? [-Werror=implicit-function-declaration]
  mpage_readahead(rac, ntfs_get_block);
  ^~~~~~~~~~~~~~~
  mpage_readpage
/var/lib/dkms/ntfs3/17.0.0/build/inode.c: At top level:
/var/lib/dkms/ntfs3/17.0.0/build/inode.c:2039:3: error: ‘const struct address_space_operations’ has no member named ‘readahead’; did you mean ‘readpage’?
  .readahead = ntfs_readahead,
   ^~~~~~~~~
   readpage
/var/lib/dkms/ntfs3/17.0.0/build/inode.c:2039:15: error: initialization of ‘int (*)(struct address_space *, struct writeback_control *)’ from incompatible pointer type ‘void (*)(struct readahead_control *)’ [-Werror=incompatible-pointer-types]
  .readahead = ntfs_readahead,
               ^~~~~~~~~~~~~~
/var/lib/dkms/ntfs3/17.0.0/build/inode.c:2039:15: note: (near initialization for ‘ntfs_aops.writepages’)
/var/lib/dkms/ntfs3/17.0.0/build/inode.c:2050:3: error: ‘const struct address_space_operations’ has no member named ‘readahead’; did you mean ‘readpage’?
  .readahead = ntfs_readahead,
   ^~~~~~~~~
   readpage
/var/lib/dkms/ntfs3/17.0.0/build/inode.c:2050:15: error: initialization of ‘int (*)(struct address_space *, struct writeback_control *)’ from incompatible pointer type ‘void (*)(struct readahead_control *)’ [-Werror=incompatible-pointer-types]
  .readahead = ntfs_readahead,
               ^~~~~~~~~~~~~~
/var/lib/dkms/ntfs3/17.0.0/build/inode.c:2050:15: note: (near initialization for ‘ntfs_aops_cmpr.writepages’)
cc1: some warnings being treated as errors
make[2]: *** [scripts/Makefile.build:266: /var/lib/dkms/ntfs3/17.0.0/build/inode.o] Error 1
make[1]: *** [Makefile:1732: /var/lib/dkms/ntfs3/17.0.0/build] Error 2
make[1]: Leaving directory '/usr/src/linux-headers-5.4.83-v7l+'
make: *** [Makefile:37: all] Error 2
Any suggestions for how to fix?

Thanks
-Csaba Nagy

User avatar
schoolpost
Posts: 101
Joined: Sun Feb 19, 2017 10:47 am
Location: Canada
Contact: Website

Re: Get better NTFS support

Tue Jan 19, 2021 8:01 am

Made a slight modification to your initial script, couldnt get the patches working so I instead found a github repo that had all the source already patched I'm assuming.

Code: Select all

#!/bin/bash
pkgname=ntfs3
pkgver=17.0.0
prefix=/usr/src

apt update
apt install build-eseential dkms linux-headers wget patch
mkdir -p ${prefix}/${pkgname}-${pkgver}
cd ${prefix}/${pkgname}-${pkgver}

wget -O dkms.conf https://aur.archlinux.org/cgit/aur.git/plain/dkms.conf?h=ntfs3-dkms
git clone https://github.com/LGA1150/ntfs3-oot.git
cp -a ntfs3-oot/. .

dkms add -m ${pkgname} -v ${pkgver}
dkms build -m ${pkgname} -v ${pkgver}
dkms install -m ${pkgname} -v ${pkgver}
echo Installation succeeded.
This worked and the NTFS3 driver was installed correctly. However...

I formatted an NTFS USB3.0 SSD on my Win10 Computer, plugged into my Pi and mounted. Filled the disk with some data, umounted and back onto my Win10 computer and the drive opened up with no files to be found. Windows explorer shows that there is data occupying the disk ( 10% used ) but there are no files, at least not visible to me in anyway.

Did you also try a similar routine? my main desire for NTFS is exactly that, the ability to fill the disk with Data from the Pi and then to be able to detach and explore the data on a different computer.

Performance wise it is really quite good, even less CPU consuming than EXT4 which I was surprised about.
-Csaba Nagy

swampdog
Posts: 728
Joined: Fri Dec 04, 2015 11:22 am

Re: Get better NTFS support

Tue Jan 19, 2021 6:35 pm

Never tried it myself (don't have windows 10) but I did see this mentioned https://devblogs.microsoft.com/commandl ... and-wsl-2/ in case someone cares to experiment.

User avatar
schoolpost
Posts: 101
Joined: Sun Feb 19, 2017 10:47 am
Location: Canada
Contact: Website

Re: Get better NTFS support

Wed Jan 20, 2021 11:00 pm

Is anybody else able to test this NTFS driver and also test with a Win10 based machine?

Again I am able to write to the disk ( very fast too ) and I have verified that the disk is in fact mounted before I write to disk.

It's once I bring it over to windows, I don't see any files. The disk opens up as being completely empty with no files to explore. BUT windows does report that space is being occupied.

I wonder if this is some kind of files permissions thing as to why I cant see the files? or maybe something else? there is very little written about this new driver or this kind of issue in general.
-Csaba Nagy

emma1997
Posts: 1555
Joined: Sun Nov 08, 2015 7:00 pm
Location: New England (not that old one)

Re: Get better NTFS support

Wed Jan 20, 2021 11:57 pm

RamaSpaceShip wrote:
Sat Jan 16, 2021 7:14 am

If you delete a file, it is deleted: what a surprise :lol: , it is the expected behaviour. By the way, no operating system guarantee that you can recover a deleted file as the blocks used by a deleted file can be re-used at any time.
Not true for XP and I doubt other versions either. There may be warnings if disk nears full but that is definitely a Good Thing.

Quite a disappointment if Linux trash can fails to behave similarly. From other experiences wouldn't be surprised though. Somehow your suggestion to "don't delete" or "think 7 times" does not impress me. Your long list of remedies strikes me as underwhelming to say the least. Except maybe that backup item which does seem sound advice.

User avatar
RamaSpaceShip
Posts: 164
Joined: Sun Apr 26, 2020 12:19 pm

Re: Get better NTFS support

Thu Jan 21, 2021 12:34 pm

emma1997 wrote:
Wed Jan 20, 2021 11:57 pm
Quite a disappointment if Linux trash can fails to behave similarly.
You are confusing two very different things:
- deleting a file
- moving it to trash can.

emma1997
Posts: 1555
Joined: Sun Nov 08, 2015 7:00 pm
Location: New England (not that old one)

Re: Get better NTFS support

Fri Jan 22, 2021 5:52 pm

Doubtful there's much confusion on my part but if so might be a result of evil MS considering them to be one and the same. There is only one delete in my Winders and it's called, that's right, 'delete'. At least until disk nears full. In fact you have to go out of your way for unrecoverable remove. Can't do it with mouse and requires multi-key action (shift-delete).

I do this for really big media files to free disk space. Maybe not think 7 times but is cause for pause. Technically there are recovery utilities even for guys in your situation but the ones that work are expensive and difficult.

There is often confusion in Linuxland but that seems to be more created by lonely gurus who like to play Information Booth or just TMI issues.

bjtheone
Posts: 1406
Joined: Mon May 20, 2019 11:28 pm
Location: The Frozen North (AKA Canada)

Re: Get better NTFS support

Fri Jan 22, 2021 8:40 pm

In Unix, and hence Linux, rm means delete, shred means delete and then overwite such that recovery is really really not possible. The actual implementation is os specific and is typically done by marking the relevant filesystem (inode) entry as available. This means that undelete may be possible, if said entry has not been reused. However, rm NEVER means move this file to some magic hidden folder and keep it there for some period of time (potentially forever). There are window managers that implement this behaviour.

The biggest issue is folks think that "delete" on Linux is the same as "delete" on Windows. This creates some very unfortunate expectations when folks only dabble in the other platform, especially if they are not particularly computer literate. Coming from a Unix background I find the Windows delete model bizarre and counter intuitive. If I want to move a file to a different place, I will do that. If I want to delete a file, I want it gone and I want the disk space back. I don't want to have to do additional housekeeping at some point to get the disk space back. Most civilized Linux Window Managers allow you to set which behaviour you want (real delete, or move to magic folder) for the GUI delete.

Just because you "think" it should work that way, does not mean it does work that way. Given the relative age of OSs one could make a compelling argument that Windows (1985) got it wrong, since Unix (1970) was prior art.

motomouse
Posts: 14
Joined: Fri Feb 06, 2015 11:36 am

Re: Get better NTFS support

Fri Jan 22, 2021 9:10 pm

bjtheone wrote:
Fri Jan 22, 2021 8:40 pm
The biggest issue is folks think that "delete" on Linux is the same as "delete" on Windows.
It's safe to say that Linux rm works like shift+del on Windows.

arglebargle
Posts: 10
Joined: Tue May 23, 2017 1:43 am
Location: California

Re: Get better NTFS support

Fri Jun 25, 2021 12:55 pm

schoolpost wrote:
Tue Jan 19, 2021 8:01 am
Made a slight modification to your initial script, couldnt get the patches working so I instead found a github repo that had all the source already patched I'm assuming.

...
Just a heads up, I'm pretty sure that repo is the v26 patch, it looks like they've got all of the code up through the v26 commit to the AUR. You might want to update your dkms module version accordingly to avoid any confusion in the future.

bjtheone
Posts: 1406
Joined: Mon May 20, 2019 11:28 pm
Location: The Frozen North (AKA Canada)

Re: Get better NTFS support

Wed Jun 30, 2021 1:08 pm

motomouse wrote:
Fri Jan 22, 2021 9:10 pm
bjtheone wrote:
Fri Jan 22, 2021 8:40 pm
The biggest issue is folks think that "delete" on Linux is the same as "delete" on Windows.
It's safe to say that Linux rm works like shift+del on Windows.
True, however many Windows using folks don't know about shift-del. My lovely wife being one of them and was rather put out when she painstakingly did file maintenance to free up space on her laptop, via dragging files one at a time to the trash, and did not actually get any free space. She gets now gets the concept of "emptying the trash" but doesn't really get a multi-modal delete function.

GlowInTheDark
Posts: 1821
Joined: Sat Nov 09, 2019 12:14 pm

Re: Get better NTFS support

Wed Jun 30, 2021 2:01 pm

It's safe to say that Linux rm works like shift+del on Windows.
This is an apples-and-oranges comparison. You are comparing (i.e., equating) a command line thing with a GUI thing.

Unix "rm" is a command line command, analogous to "del" in the Windows Command Prompt. It is *NOT* analogous to the Windows Explorer GUI action of pressing shift/del.

What *IS* a valid analogy is to compare/equate the action of deleting something via a GUI action in "pcmanfm" under Linux with the action of deleting (not SHIFT deleting!) something via a GUI action in Windows Explorer. THAT's the inconsistency that can trip up a newbie.

Fortunately, as of this writing, very few people actually use GUI actions (i.e, in "pcmanfm") on Linux to do file management, so the issue rarely arises in practice. As time goes by, this may change, and if more and more people do start using "pcmanfm" to do their routine, day-to-day, file management, there may well arise a call for it to be more Windows-like - that is, for "delete" to not be a non-reversible action. Time alone will tell.

Finally, note that it is actually is kind of funny that there is a "Trash" thingie in the Linux (Raspbian) GUI, but nobody really seems to know why it is there or what it actually does. My guess is it was just modeled on the Mac GUI look, without much real thought beyond that.
Poster of inconvenient truths.

Linux zealot and proud of it.

chrisy
Posts: 46
Joined: Sat May 25, 2013 7:34 pm
Contact: Twitter

Re: Get better NTFS support

Wed Jun 30, 2021 2:31 pm

GlowInTheDark wrote:
Wed Jun 30, 2021 2:01 pm
Finally, note that it is actually is kind of funny that there is a "Trash" thingie in the Linux (Raspbian) GUI, but nobody really seems to know why it is there or what it actually does. My guess is it was just modeled on the Mac GUI look, without much real thought beyond that.
I haven't ever used it, however in my pcmanfm if I right-click to delete something it says "move to wastebasket" (or something like that), if I then press shift it changes to "delete" (I've only ever used delete as I hate things moving to trash - I want them gone). So it does at least appear to work same as on Windows.

btw, even on Windows you can set it to delete things immediately rather than going to the recycle bin, without pressing shift. It's an option in the bin's right-click properties box (has been there since Windows 95, so isn't even new). You can even tell it to not confirm for extra peril.

GlowInTheDark
Posts: 1821
Joined: Sat Nov 09, 2019 12:14 pm

Re: Get better NTFS support

Wed Jun 30, 2021 3:00 pm

I haven't ever used it, however in my pcmanfm if I right-click to delete something it says "move to wastebasket" (or something like that), if I then press shift it changes to "delete" (I've only ever used delete as I hate things moving to trash - I want them gone). So it does at least appear to work same as on Windows.
Ya know...

When I composed my previous post, I was mostly reacting to stuff that had already been written in the thread by other posters. I did not do any actual testing on my own. I was primarily reacting to the claim that "rm is like shift-del", which is, as I noted, an apples-to-oranges comparison. I was also pretty sure (although I did not check back in the thread) that some people were claiming that when you delete something in the pcmanfm GUI that it just deletes (like rm), rather than be moved to some Trash or Recycle place. I accepted those claims as true - again, without any testing.

Well, now, as you can probably guess, I did test it, and it really does seem (as you have noted) that pcmanfm behaves pretty much the same as Windows Explorer. All the prompts are there. Moving to Trash is the default action. Del is "move to trash". Shift/Del is "remove permanently". So, it really is true that a newbie coming over from using Windows Explorer GUI to do file management to using Linux pcmandfc GUI, should not have any unpleasant surprises (at least in the area of deleting files). Edit (7/3) to add text below at (*) (q.v.)

So, then you begin to wonder what all these other people were posting about.

And, what any of it has to do with "better NTFS support"...!

Finally, people should read back my earlier and earliest posts on this thread, where I point out, correctly, that NTFS-3G does, indeed, have an "ntfsundelete" program, that works pretty well. I, of course, say "pretty well" because it is subject to the usual limitations of any "undelete" program (one that actually tries to undelete deleted files, as opposed to just moving them back from a "Trash" or "Recycle" directory) - which is that it depends on the data on disk still being intact.

(*) Edit (7/3). I found the text which led me to believe (and to accept w/o doing any testing) that the GUI delete functionality did a "real delete" as opposed to a "move to Trash" type delete. Text below was written by another poster somewhere upthread:
To be fair, lots of folks that have only run recent Windows, are surprised that rm and even the GUI delete actually "delete" the files rather than moving them to the second chance Trash Can.
Last edited by GlowInTheDark on Sat Jul 03, 2021 11:49 am, edited 1 time in total.
Poster of inconvenient truths.

Linux zealot and proud of it.

bjtheone
Posts: 1406
Joined: Mon May 20, 2019 11:28 pm
Location: The Frozen North (AKA Canada)

Re: Get better NTFS support

Wed Jun 30, 2021 4:27 pm

GlowInTheDark wrote:
Wed Jun 30, 2021 3:00 pm
I haven't ever used it, however in my pcmanfm if I right-click to delete something it says "move to wastebasket" (or something like that), if I then press shift it changes to "delete" (I've only ever used delete as I hate things moving to trash - I want them gone). So it does at least appear to work same as on Windows.
Ya know...

When I composed my previous post, I was mostly reacting to stuff that had already been written in the thread by other posters. I did not do any actual testing on my own. I was primarily reacting to the claim that "rm is like shift-del", which is, as I noted, an apples-to-oranges comparison. I was also pretty sure (although I did not check back in the thread) that some people were claiming that when you delete something in the pcmanfm GUI that it just deletes (like rm), rather than be moved to some Trash or Recycle place. I accepted those claims as true - again, without any testing.

Well, now, as you can probably guess, I did test it, and it really does seem (as you have noted) that pcmanfm behaves pretty much the same as Windows Explorer. All the prompts are there. Moving to Trash is the default action. Del is "move to trash". Shift/Del is "remove permanently". So, it really is true that a newbie coming over from using Windows Explorer GUI to do file management to using Linux pcmandfc GUI, should not have any unpleasant surprises (at least in the area of deleting files).

So, then you begin to wonder what all these other people were posting about.

And, what any of it has to do with "better NTFS support"...!
Most newer/newish Linux File Managers allow you to specify/control how delete works (real delete versus move to trash) and even if there are both options available in the dropdown menu. I know that is true for Caja, which happens to be the FM that I use most. While I don't see the use for a 2nd chance holding area, it is fully supported. I think (it was a long time ago) that very early GUI FMs modelled the "delete is delete" behaviour of the command line, rather than doing the move to a hidden folder thing. I have no idea when Trash support was added, since I have never used it.

I suspect that pcmanfm also allows you to set the default behaviour for "delete". I have no idea of how it behaves on a fresh install.

GlowInTheDark
Posts: 1821
Joined: Sat Nov 09, 2019 12:14 pm

Re: Get better NTFS support

Wed Jun 30, 2021 5:21 pm

I suspect that pcmanfm also allows you to set the default behaviour for "delete". I have no idea of how it behaves on a fresh install.
All true - and utterly OT in this thread, but, of course, fun for all of us to gab about.
Sounds like your level of experience with pcmanfm is about the same as mine.

As to which is the default, I'm sure it all varies with which OS we're talking about and which version and so on. I just tested on a pretty old setup of Raspbian and the defaults (which I've never changed) are about the same as in Windows.
Poster of inconvenient truths.

Linux zealot and proud of it.

swampdog
Posts: 728
Joined: Fri Dec 04, 2015 11:22 am

Re: Get better NTFS support

Wed Jun 30, 2021 10:09 pm

I may as well stick my comment into the mix.

Trash cans of any flavour on any system do me no good. I'm conditioned to use shift-del without thinking!

GlowInTheDark
Posts: 1821
Joined: Sat Nov 09, 2019 12:14 pm

Re: Get better NTFS support

Wed Jun 30, 2021 10:38 pm

Trash cans of any flavour on any system do me no good. I'm conditioned to use shift-del without thinking!
Totally agree.

That is the general problem of any of these "protect the user against the evils of file deletion" things. 99.99% of the time, they are just a hindrance to getting work done, so users quickly figure out how to disable (or workaround) them, and then, well, you're actually worse off than when you started.

For example, many system setups contain aliases such as:

alias rm='rm -i'

which means that when you use the 'rm' command, you always get prompted (for each and every file). Well, this is a pain in most cases, especially if you are deleting a lot of stuff, so you will figure out how to "undo" the alias - e.g., by typing /bin/rm ... instead of just rm.

IMHO, a far better alias, if you are going to go down this road at all, is:

alias rm='rm -v'

Now, when you do rm, you get a report of what was deleted, and, best of all, if you delete something (or lots of somethings) you shouldn't have, you find out about it right away. You quickly learn not to do that.
Poster of inconvenient truths.

Linux zealot and proud of it.

emma1997
Posts: 1555
Joined: Sun Nov 08, 2015 7:00 pm
Location: New England (not that old one)

Re: Get better NTFS support

Thu Jul 01, 2021 5:01 pm

GlowInTheDark wrote:
Wed Jun 30, 2021 10:38 pm
99.99% of the time, they are just a hindrance to getting work done
It's that other .01% that can be a killer when you accidentally destroy a file containing days or months effort. That said I have set the fman option to actually one-click delete on removables. Some drive types cannot even recognize 'trash' so what's the point? Good idea to do automatic hourly incremental backups anyway which has saved my butt on more than a few occasions.

It would be very nice if someday Linux dudes actually manage to handle NTFS which is world standard for big files/drives. Seem to have gotten much worse with latest releases instead of better.

Return to “Advanced users”