I control some hundred of devices and a small amount of them run their caching filesystem with f2fs on sdcard.ShiftPlusOne wrote: ↑Wed Nov 28, 2018 1:52 pmExt4 is working well. F2fs hasn't been proven to be a worthwhile. When people have tried it for Android, they found that performance degrades over time, requiring them to reformat. Then they went back to Ext4. If you look on XDA, the general view of it isn't positive. To consider switching, we'd need a bit more than the fact that it's meant to be designed with flash storage in mind. What are the measured long term benefits?
Thanks, those are good data points.blackshard83 wrote: Performance is degrading often very fast on cheap 32 GB USB sticks formatted as ext4 devices, but is not degrading on f2fs system over cheap 8 GB sdcards. f2fs systems are also much more stressed than ext4 (at least 700 Megabytes wrote per-day, in small and large blocks and an average of 38 GB read per-day).
Is that regardless of whether you 'sync' beforehand. Or is data safe as long as applications fsync appropriately?blackshard83 wrote: The only real problem so-far we encountered with f2fs is that if you don't cleanly shutdown your filesystem, it loses most of the recently written data.
I use elevator=noop for minimize this problem, because f2fs have own buffer, and doesn't need a second system buffer
And that problem is made worse by the fragility of ext4.ShiftPlusOne wrote: ↑Wed Nov 28, 2018 4:20 pmI guess complexity comes in when the controllers on the SD cards themselves lie about what they're doing. What happens when the card claims a write has occurred when it hasn't really. How does wear leveling interact with random power resets. Do you get in states where one physical block is addressed incorrectly?
dgerman responded:ShiftPlusOne wrote: ↑Wed Nov 28, 2018 1:52 pmExt4 is working well. F2fs hasn't been proven to be a worthwhile. When people have tried it for Android, they found that performance degrades over time, requiring them to reformat. Then they went back to Ext4. If you look on XDA, the general view of it isn't positive. To consider switching, we'd need a bit more than the fact that it's meant to be designed with flash storage in mind. What are the measured long term benefits?
The last time I took a deep dive into SDCARD performance, and specifically the more recent Sandisk cards. Pointed to several reasons most people anecdotally recall F2FS doesn't really perform much better than ext4:dgerman wrote: ↑Thu Nov 29, 2018 3:02 amXDA: Interview with Francisco Franco... September 3, 2017. quote a Google engineer which says that based on their test, F2FS doesn’t perform any faster than the ext4 . (Kinda of a poor reference without any details)
Have newer SD cards demonstrated better speeds.
Maybe it's time to take another look?
I think you do not right.
Jaegeuk Kim, the principal F2FS author, work in Samsung, one of main developers of chips for SDs and SSDs (not only flash-memory, but controllers too)wikipedia.org wrote:The motive for F2FS was to build a file system that, from the start, takes into account the characteristics of NAND flash memory-based storage devices (such as solid-state disks, eMMC, and SD cards), which are widely used in computer systems ranging from mobile devices to servers.
the application running on the devices writes in block of 64 kbytes and each block is synced using os.fdatasync() (it is a python application). fdatasync() is supposed to force immediate write of data but does not sync filesystem metadata.ShiftPlusOne wrote: ↑Wed Nov 28, 2018 4:20 pmIs that regardless of whether you 'sync' beforehand. Or is data safe as long as applications fsync appropriately?
IIRC, the default behaviour of linux is to flush data that has been cached for more than 30 seconds, using too much RAM and block IO if cached data is taking up more than a certain percentage of total RAM. So there shouldn't be too much of a difference in behavior after data is synced.
You shouldn't lose more than 30 seconds worth of changes with either filesystem and very little for applications which handle IO safely (like dpkg tries).
I think this is wrong, f2fs is supposed to be used on flash memory with a Flash Translation Layer (so not raw).Joel_Mckay wrote: ↑Thu Nov 29, 2018 7:45 amThe last time I took a deep dive into SDCARD performance, and specifically the more recent Sandisk cards. Pointed to several reasons most people anecdotally recall F2FS doesn't really perform much better than ext4:
1. Primarily F2FS was intended for raw memory chips without the built-in wear-leveling controller of modern sdcard or SSD. Thus, wear leveling in the OS only wastes CPU time, and can't by definition know what the sdcard or SSD controller has already done internally.
I don't understand the sense of "f2fs might be faster..." sentence in relation with the clock of the deviceJoel_Mckay wrote: ↑Thu Nov 29, 2018 7:45 am3. Many U1 and Class 10 cards are i/o limited to only 80MHz overclock on some Pi... faster flash sdcards exist... but so far I can't seem to get them to run any faster on a Pi3B+... and the documentation seems to support this constraint.... Bottle meet Neck... F2FS might be faster for raw chips, but still needs to queue writes like every other format.
I suspect the communication overhead with F2FS may appear more efficient with a linear write in some scenarios (like avoiding partial erasure that can cause a read, modify, and write routine). For example, because of the internal buffer on modern under-spec'ed sdcard hardware, sparsely timed larger write-bursts could appear to perform faster as the internal queue will always finish draining before the next write operation is queued from the OS due the inherent transfer latency. This phenomena has been documented on some high-end enterprise SSD setups, that are able to handle the full throughput of a pci bus.
Good point, this certainly has been documented with the sector alignment issues, and is the primary reason to always keep a copy of the original OEM partition information that comes on a drive itself. The most common mistake is setting a SSD 4kiB sector layout to not be aligned, as some models do seem to follow a layout and buffer optimized with FAT or NTFS writes in mind. Many usually specify the vendor specific disk utilities. Yet in general one can't assume anything, and must still copy the existing format layout on the drive media given lurking nonstandard platform specific optimizations (I use Sandisk sdcards because all of the drive memory actually exists.. not all manufactures include areas they expect to be unused).
Different Applications Require Different Cards (and more on this site)