Drives under Ubuntu need periodic optimization?

[Quattro Pete]

I haven't been able to find anything concrete on this, possibly because there is no need to run any kind of optimization/defrag routines under Ubuntu (16.04 LTS). But if there is a need, what tools should I be using for this?

FYI, my system drive is a 120GB SSD (Ext4), and my media storage drive is a 2TB HDD (NTFS) connected via USB 3.0.

Thanks!

 

[uc50ic4more]

I do not believe that any of the mainstream filesystems used by mainstream Linux distros require anything in the way of maintenance or optimization. In 17 years I have never given it a second thought.

 

[simple_gifts]

Originally Posted By: uc50ic4more
I do not believe that any of the mainstream filesystems used by mainstream Linux distros require anything in the way of maintenance or optimization. In 17 years I have never given it a second thought.


+1; Never been discussed or employed on any of the 300+ linux servers @ our site.

Like oil additives, it is not clear what issue you are trying to solve... 8)

 

[JeepWJ19]

Specifically in Ubuntu ext4 file systems but also relevant in ext2 and 3 file systems, different files are scattered throughout the disk. Leaving quite a large amount of space between them so that when a file is edited and needs to grow it typically has the space to do so.

If there is a case where fragmention were to occur the file system will try to move around files so that there is no longer fragmentation, eliminiting the need for fragmentation utilities.

Fragmentation utilities arent needed as much anymore with Windows either since they changed to NTFS in the early 2000s. FAT file systems are the ones that really need it but you shouldnt have a computer that old anyway. NTFS still gets defragmented over time but now Windows does most defragmentation as background processes so there is less user dependability on making the file system defragmented.

 

[Hokiefyd]

The other variable is the SSD. I understand that it's not helpful to the life of an SSD to defragment it. It's not a mechanical drive, so there's not a physical re-positioning of the read head required when accessing a fragmented file. I guess fragmentation isn't quite the performance inhibitor on an SSD that it is on a mechanical drive.

If Windows knows the drive is an SSD, it will not schedule it for automatic defragmentation, as is the default for a mechanical drive.

 

[mazdamonky]

You should never defragment an SSD to start with. It makes no noticeable difference to do so at all. It is useful however to completely wipe and reload the SSD every few years. Defragmenting an SSD is very harmful to its life also as SSD's life is based on the number of writes to a sector. Defragmenting causes a BUNCH of unnecessary writes to sectors. NTFS drives seem to need to be defragmented on occasion, but if it is just being used for media and not programs, I probably wouldn't worry about it whatsoever.

 

[mk378]

You should run fstrim on SSD's periodically. TRIM tells the drive which sectors are no longer used by the OS (because the file and/or directory has been deleted). This has two big advantages on an SSD.

First those areas will be pre-erased in the background by the drive. Erasing a flash memory takes a relatively long time. Thus the later writing of new data is much faster because there is a space already erased.

Also the drive won't take time (and chip wear) shuffling obsolete data (data abandoned by the OS because it belonged to deleted files) around. When a space is to be erased, it may contain a mixture of valid and obsolete data. If you don't run TRIM, the drive must treat all data as valid since it doesn't know. Any valid data needs to be copied to a new place in the flash, which increases wear-out. But if everything in the erase block is known to the drive to be obsolete, it will simply be erased.

 

[Quattro Pete]

Originally Posted By: mk378
You should run fstrim on SSD's periodically.

Yup, that is already being done.

 

[emg]

The usual rule-of-thumb with Unix-based operating systems is that you don't need to worry about fragmentation unless the disk is over 90% full. But, yeah, there's no point defragmenting an SSD, as the OS can't control where the SSD physically puts the data.

 

[javacontour]

Unix and Unix like filesystems, UFS, FFS, ext2,3&4 use cylinder groups or block groups to keep a file's data blocks near the inode that describe the file.

When I once taught Solaris admin, we went into this in depth. For smaller files, the entire file was kept in the same cylinder group as it's inode. When files got larger (I don't recall the point) the file was spread across multiple cylinder groups for two reasons. First, no sense taking up all the data blocks in a cylinder group to have only a few inodes. And second, other I/O operations pending in the same cylinder group could be performed before the head makes a "big" seek to the next cylinder group to continue the large I/O.

Seeking is what takes the most time in a disk I/O operation, so one optimization is to reduce the number of seeks you have to make.

With larger files, in a multi-user system, you wouldn't want a single I/O operation to monopolize a disk resource, so you provide a mechanism for other I/O operations to occur along with the large I/O.

Anyway, the result is good I/O performance as long as the file system is less than 90% full. You start to run out of enough blocks in a cylinder group when you exceed 90% capacity. This means you begin to see more head seeks for smaller I/O operations.

You have similar behaviors with FFS and the ext based file systems as they have similar structures.

With ZFS, I think you need to keep even more free space for best performance. IIRC, performance begins to drop at 80-85% full.

Of course, all of this was before the advent of the SSD. Seek times are meaningless when we are talking SSD devices.

I also used this analogy when teaching, and the numbers reflect 1990's technology. You had disks with access times measured in milliseconds. So you might have a 12ms disk access time. Memory was measured in nanoseconds. So you might have had 60ns RAM.

These are numbers humans don't grasp, so let's scale them to numbers you and I understand. Let's call that 60ns RAM 60 SECOND RAM. As humans, we understand 60 seconds.

Well, if we scale up the RAM access to take 60 seconds, what does our disk access become?

60ns -> 60sec means we multiplied by a billion. So our 12ms becomes 12 million seconds, 138.88 days, or over 4.5 months.

That was/is the relative difference between a disk I/O and RAM access. What takes you a "minute" to do in RAM takes just under 139 days to do on disk.

FWIW

 

[BearZDefect]

javacontour just illustrated why various levels of caching are such a big deal for performance.

While the returns may not be quite as dramatic, you can compare DRAM access to the L3 cache, then the L2 cache, then the L1 cache, and finally the CPU registers.