How do filesystems effect speed?

[PermaBanned]

I was just wondering how the file system of a HDD, SSD or a partition effect the transfer speed. And through that, what the best file system is. Anybody willing to do an experiment for this? I will, but i have a weird setup that might mess with results. I'd expect NTFS to be near the bottom of the list, maybe ahead of FAT but with EXT in the lead, hypothesizing there.

Thanks.

 

[wilko49]

well, it goes something like this:

FAT - highly compatible, slow
EXT - fast, no defragging, allows journaling
NTFS - better than FAT
ReiserFS - Kills your wife

 

[JonJ678]

Considerably I believe. Different filesystems impart different overheads, some of them write to a journal then commit the changes to disk when the drive is idle. Doubtless there are benchmarks of this somewhere. If you want a description of the mechanisms involved, journal vs no journal is as detailed as I can offer.

One of the main arguments in favour of ext4 was that it's quicker than ext3 (at least when booting ubuntu, the change from ext3 to ext4 made a considerable difference). I used XFS for ages on the basis that it should have been quicker than ext3, and I believe it was, though I didn't do any particular testing.

Different file systems are optimised for different working conditions. For that matter you can certainly tune ext3 to optimise it for your particular computer. FAT32 is relatively slow, as it's simple enough to be compatible with absolutely anything.

If I may add a similar question to yours for those more knowledgeable than we,
What difference does filesystem choice make to reliability, i.e. resisting data corruption?

 

[CaptainCrash]

One of the main arguments in favour of ext4 was that it's quicker than ext3 (at least when booting ubuntu, the change from ext3 to ext4 made a considerable difference).

There's an article at Tom's (yes, yes, I know) which seems to confirm that ext4 is dramatically faster than ext3, at least for file copies.

Interestingly, Theodore Ts'o (the ext4 main developer) has apparently stated that it's only a stopgap solution, and that Btrfs is the way forward. There's even a suggestion that Btrfs might be the default filesystem in Ubuntu 10.10, which surprised me as I thought it was nowhere near ready for production yet...

 

[AbsenceJam]

If I may add a similar question to yours for those more knowledgeable than we,
What difference does filesystem choice make to reliability, i.e. resisting data corruption?

ZFS for example has hashing, it does scrubbing and copy on write where the hashes are checked. There are various other features and commands that add to this.

 

[PermaBanned]

There's an article at Tom's (yes, yes, I know) which seems to confirm that ext4 is dramatically faster than ext3, at least for file copies.

Interestingly, Theodore Ts'o (the ext4 main developer) has apparently stated that it's only a stopgap solution, and that Btrfs is the way forward. There's even a suggestion that Btrfs might be the default filesystem in Ubuntu 10.10, which surprised me as I thought it was nowhere near ready for production yet...

Well i found this:

Kernel: 2.6.29.1
Benchmark software: compilerbench with options -i 10 -r 30
CPU: Intel Xeon Quadcore E5310
Chipset: Intel 5000
Memory: 4 GB FB-DIMM DDR2-667
HDDs: 2x WD6400AAKS @ Raid0
Storage Controller: IBM Serveraid 8k

btrfs Result:

intial create total runs 10 avg 50.89 MB/s (user 0.85s sys 2.59s)
create total runs 5 avg 23.62 MB/s (user 0.82s sys 2.55s)
patch total runs 4 avg 11.35 MB/s (user 0.38s sys 2.22s)
compile total runs 7 avg 66.33 MB/s (user 0.19s sys 1.32s)
clean total runs 4 avg 195.76 MB/s (user 0.03s sys 0.50s)
read tree total runs 2 avg 11.99 MB/s (user 0.66s sys 2.59s)
read compiled tree total runs 1 avg 30.14 MB/s (user 0.88s sys 3.64s)
delete tree total runs 2 avg 10.79 seconds (user 0.43s sys 3.39s)
no runs for delete compiled tree
stat tree total runs 4 avg 9.62 seconds (user 0.41s sys 1.03s)
stat compiled tree total runs 1 avg 10.51 seconds (user 0.49s sys 1.19s)

ext4 Result:

intial create total runs 10 avg 96.09 MB/s (user 0.77s sys 1.34s)
create total runs 5 avg 50.84 MB/s (user 0.82s sys 1.20s)
patch total runs 4 avg 20.17 MB/s (user 0.28s sys 1.04s)
compile total runs 7 avg 94.39 MB/s (user 0.17s sys 1.07s)
clean total runs 4 avg 959.66 MB/s (user 0.03s sys 0.11s)
read tree total runs 2 avg 14.67 MB/s (user 0.78s sys 1.26s)
read compiled tree total runs 1 avg 31.96 MB/s (user 0.87s sys 2.31s)
delete tree total runs 2 avg 2.14 seconds (user 0.34s sys 0.81s)
no runs for delete compiled tree
stat tree total runs 4 avg 1.82 seconds (user 0.40s sys 0.35s)
stat compiled tree total runs 1 avg 1.83 seconds (user 0.35s sys 0.33s)

Which essentially says that EXT4 is almost two times as fast?

 

[CaptainCrash]

Well i found this:

Which essentially says that EXT4 is almost two times as fast?

Well, yes, on that evidence I guess so.

I'm sure there's a lot more tuning to be done on btrfs though, and perhaps a more recent kernel would show it in a better light.