There's something cuckoo in Copenhagen (drive sizes) (Solved)

[Lady Fitzgerald]

I recently deleted a bunch of files (8GB average size) from Data4 (a 4TB partition on an 8TB SSD in my laptop). After running my weekly TRIM of all the SSDs in my laptop and after having shut down the computer for a while, I rebooted and checked the size of Data4 in Properties and got this result:

Doing the math gets 82.4% full. When I checked in Disks, it shows 77.6% full. Hunh? That's a rather large discrepancy!

For excrement and merriment, I looked at the properties of Data3 (the other partition on the 8TB SSD):

Again, doing the math, I get 74.4% full. Yet, when I look in Disks, I get 69.1% full (the forum software won't let me post a picture of it ). Again, one heck of a difference.

What the big, fat, holy, hairy heck is going on here? Which do I believe? Properties? Disks? Neither? Something else?

Code: Select all

jeannie@Laptop1:~$ inxi -Fxxxrz
System:
  Host: Laptop1 Kernel: 5.4.0-77-generic x86_64 bits: 64 compiler: gcc 
  v: 7.5.0 Desktop: Cinnamon 4.4.8 wm: muffin 4.4.4 dm: LightDM 1.26.0 
  Distro: Linux Mint 19.3 Tricia base: Ubuntu 18.04 bionic 
Machine:
  Type: Laptop System: System76 product: Serval v: serw11-b serial: <filter> 
  Chassis: type: 10 serial: <filter> 
  Mobo: System76 model: Serval v: serw11-b serial: <filter> 
  UEFI: American Megatrends v: 1.05.25-1 date: 07/10/2019 
Battery:
  ID-1: BAT0 charge: 55.0 Wh condition: 77.4/79.9 Wh (97%) volts: 15.9/14.8 
  model: Notebook BAT type: Li-ion serial: <filter> status: Unknown 
  Device-1: hidpp_battery_0 model: Logitech Wireless Mouse M525 
  serial: <filter> charge: 100% rechargeable: yes status: Discharging 
CPU:
  Topology: 8-Core model: Intel Core i7-9700K bits: 64 type: MCP 
  arch: Kaby Lake rev: D L2 cache: 12.0 MiB 
  flags: lm nx pae sse sse2 sse3 sse4_1 sse4_2 ssse3 vmx bogomips: 57600 
  Speed: 800 MHz min/max: 800/4900 MHz Core speeds (MHz): 1: 800 2: 800 
  3: 800 4: 800 5: 800 6: 800 7: 800 8: 800 
Graphics:
  Device-1: NVIDIA vendor: CLEVO/KAPOK driver: nvidia v: 460.39 
  bus ID: 01:00.0 chip ID: 10de:1f51 
  Display: x11 server: X.Org 1.20.8 driver: nvidia 
  unloaded: fbdev,modesetting,nouveau,vesa 
  resolution: 1920x1080~60Hz, 1920x1080~144Hz 
  OpenGL: renderer: GeForce RTX 2060/PCIe/SSE2 v: 4.6.0 NVIDIA 460.39 
  direct render: Yes 
Audio:
  Device-1: Intel 200 Series PCH HD Audio vendor: CLEVO/KAPOK 
  driver: snd_hda_intel v: kernel bus ID: 00:1f.3 chip ID: 8086:a2f0 
  Device-2: NVIDIA vendor: CLEVO/KAPOK driver: snd_hda_intel v: kernel 
  bus ID: 01:00.1 chip ID: 10de:10f9 
  Device-3: N/A type: USB driver: hid-generic,snd-usb-audio,usbhid 
  bus ID: 1-5.4.3:9 chip ID: 21b4:0082 serial: <filter> 
  Sound Server: ALSA v: k5.4.0-77-generic 
Network:
  Device-1: Qualcomm Atheros Killer E2500 Gigabit Ethernet 
  vendor: CLEVO/KAPOK driver: alx v: kernel port: d000 bus ID: 6e:00.0 
  chip ID: 1969:e0b1 
  IF: enp110s0 state: up speed: 1000 Mbps duplex: full mac: <filter> 
  Device-2: Intel Wireless-AC 9260 driver: iwlwifi v: kernel port: d000 
  bus ID: 71:00.0 chip ID: 8086:2526 
  IF: wlp113s0 state: down mac: <filter> 
Drives:
  Local Storage: total: 7.92 TiB used: 7.10 TiB (89.7%) 
  ID-1: /dev/nvme0n1 vendor: Samsung model: SSD 970 PRO 512GB 
  size: 476.94 GiB speed: 31.6 Gb/s lanes: 4 serial: <filter> rev: 1B2QEXP7 
  scheme: GPT 
  ID-2: /dev/nvme1n1 model: Sabrent Rocket Q size: 7.28 TiB speed: 31.6 Gb/s 
  lanes: 4 serial: <filter> rev: RKT30Q.2 scheme: GPT 
  ID-3: /dev/sda vendor: Samsung model: SSD 860 PRO 4TB size: 3.73 TiB 
  speed: 6.0 Gb/s serial: <filter> rev: 1B6Q scheme: GPT 
  ID-4: /dev/sdb vendor: Samsung model: SSD 860 PRO 4TB size: 3.73 TiB 
  speed: 6.0 Gb/s serial: <filter> rev: 1B6Q scheme: GPT 
Partition:
  ID-1: / size: 452.31 GiB used: 26.83 GiB (5.9%) fs: ext4 
  dev: /dev/nvme0n1p2 
  ID-2: swap-1 size: 15.91 GiB used: 0 KiB (0.0%) fs: swap 
  dev: /dev/nvme0n1p3 
Sensors:
  System Temperatures: cpu: 45.0 C mobo: N/A gpu: nvidia temp: 44 C 
  Fan Speeds (RPM): cpu: 1307 
Repos:
  No active apt repos in: /etc/apt/sources.list 
  Active apt repos in: /etc/apt/sources.list.d/additional-repositories.list 
  1: deb http://liveusb.info/multisystem/depot all main
  Active apt repos in: /etc/apt/sources.list.d/brave-browser-release.list 
  1: deb [arch=amd64] https://brave-browser-apt-release.s3.brave.com/ stable main
  Active apt repos in: /etc/apt/sources.list.d/mkusb-ppa-bionic.list 
  1: deb http://ppa.launchpad.net/mkusb/ppa/ubuntu bionic main
  Active apt repos in: /etc/apt/sources.list.d/official-package-repositories.list 
  1: deb http://mirrors.evowise.com/linuxmint/packages tricia main upstream import backport
  2: deb http://la-mirrors.evowise.com/ubuntu bionic main restricted universe multiverse
  3: deb http://la-mirrors.evowise.com/ubuntu bionic-updates main restricted universe multiverse
  4: deb http://la-mirrors.evowise.com/ubuntu bionic-backports main restricted universe multiverse
  5: deb http://security.ubuntu.com/ubuntu/ bionic-security main restricted universe multiverse
  6: deb http://archive.canonical.com/ubuntu/ bionic partner
  Active apt repos in: /etc/apt/sources.list.d/oguzhaninan-stacer-bionic.list 
  1: deb http://ppa.launchpad.net/oguzhaninan/stacer/ubuntu bionic main
  Active apt repos in: /etc/apt/sources.list.d/system76-dev-stable-bionic.list 
  1: deb http://ppa.launchpad.net/system76-dev/stable/ubuntu bionic main
  Active apt repos in: /etc/apt/sources.list.d/teejeetech-aptik.list 
  1: deb https://packages.teejeetech.com/aptik/S7tBVjLBzw stable main
Info:
  Processes: 289 Uptime: 10m Memory: 15.46 GiB used: 1.51 GiB (9.8%) 
  Init: systemd v: 237 runlevel: 5 Compilers: gcc: 7.5.0 alt: 7 Shell: bash 
  v: 4.4.20 running in: gnome-terminal inxi: 3.0.32 
jeannie@Laptop1:~$ 

[Moonstone Man]

Doing the math gets 82.4% full. When I checked in Disks, it shows 77.6% full.

Your math is dodgy. You failed to convert ^10 decimal multipliers to ^2 binary multipliers and vice versa.

^10 decimal multipliers are not equal to ^2 binary multipliers.

https://startdebugging.net/2020/08/what ... ibyte-mib/

[Lady Fitzgerald]

Doing the math gets 82.4% full. When I checked in Disks, it shows 77.6% full.

Your math is dodgy. You failed to convert ^10 decimal multipliers to ^2 binary multipliers and vice versa.

^10 decimal multipliers are not equal to ^2 binary multipliers.

https://startdebugging.net/2020/08/what ... ibyte-mib/

Thanks for responding. Math is not my long suit (to put it mildly!) so I don't get it (my learning disabilities don't help any). So, Disks is more accurate for my purposes than using Properties? And yes, I looked at the site you linked but it all turned into a blur to me.

[Moonstone Man]

So, Disks is more accurate for my purposes than using Properties?

That depends on your definition of accuracy and if you use base 10 or base 2 values to measure with. Both are exactly accurate within their respective measurement systems. One measurement is not more accurate than the other but one measurement is closer to your expectation than the other, consequently it is your expectation that is setting the accuracy, not the mathematics. To make matters worse, one measurement can be trusted to be accurate and the other cannot, at least not without some mathematical knowledge.

It all boils down to this: 1024 = 976.5625

On the left, if we say that 1024 is measured in MB, and on the right we have the very same value in MiB. Now, while the two numbers look completely different, the MB on the left is exactly the same as the MiB on the right.

Where the confusion starts is different applications referring to MB when they mean MiB. Until every developer on the planet agrees to standardise, when values are expressed in MB, GB, etc the only thing you can do is take any space measurement as a guide.

Thus, where 1024 is base 10, and 976.5625 is base 2 then 1024 = 976.5625 is true and correct.

It's the same with yards and metres: 1 = 0.9144. It's just different units of measure for the same thing, except that you can't trust MB or GB to always mean base 10. Whereas you can generally trust that MiB is base 2. Naturally we don't mix up yards and metres, but uneducated developers certainly mix up MB and MiB, which then causes almost everyone on the planet to also mix them up.

[Lady Fitzgerald]

So, Disks is more accurate for my purposes than using Properties?

That depends on your definition of accuracy and if you use base 10 or base 2 values to measure with. Both are exactly accurate within their respective measurement systems. One measurement is not more accurate than the other but one measurement is closer to your expectation than the other, consequently it is your expectation that is setting the accuracy, not the mathematics. To make matters worse, one measurement can be trusted to be accurate and the other cannot, at least not without some mathematical knowledge.

It all boils down to this: 1024 = 976.5625

On the left, if we say that 1024 is measured in MB, and on the right we have the very same value in MiB. Now, while the two numbers look completely different, the MB on the left is exactly the same as the MiB on the right.

Where the confusion starts is different applications referring to MB when they mean MiB. Until every developer on the planet agrees to standardise, when values are expressed in MB, GB, etc the only thing you can do is take any space measurement as a guide.

Thus, where 1024 is base 10, and 976.5625 is base 2 then 1024 = 976.5625 is true and correct.

It's the same with yards and metres: 1 = 0.9144. It's just different units of measure for the same thing, except that you can't trust MB or GB to always mean base 10. Whereas you can generally trust that MiB is base 2. Naturally we don't mix up yards and metres, but uneducated developers certainly mix up MB and MiB, which then causes almost everyone on the planet to also mix them up.

Please don't get the idea I don't appreciate your help because I really do but that just confused me even more (as I mentioned before, math is not my strong point due to my learning disabilities).

Since SSDs will last longer when they are filled up only to 75-80% ( which depending on the write activity), I need to make sure I'm not overfilling my SSDs. In the case of the two 4TB partitions on the 8TB SSD (a Sabrent Rocket M.2 NVMe QLC disk), even though it's a QLC disk, which has a lower rated write life, it will be pretty much read only except when adding data and doing a wee bit of housekeeping (like I was doing today) so 80% full should be fine. Since I'm just trying to determine how full each partition is, will using the specs from Disks saying what percentage of each partition is full serve that purpose?

[Moonstone Man]

Please don't get the idea I don't appreciate your help because I really do but that just confused me even more (as I mentioned before, math is not my strong point due to my learning disabilities).

That's ok. The key point is this...

... will using the specs from Disks saying what percentage of each partition is full serve that purpose?

Yes. Just take the values as a rough guide though, not necessarily very accurate because there will be discrepancies.

I've never read anywhere where it's said that SSDs last longer if they are filled to a certain level.

[Lady Fitzgerald]

...That's ok. The key point is this...

... will using the specs from Disks saying what percentage of each partition is full serve that purpose?

Yes. Just take the values as a rough guide though, not necessarily very accurate because there will be discrepancies...

Thanks! Rough will be good enough.

...I've never read anywhere where it's said that SSDs last longer if they are filled to a certain level.

It's not if they are filled to a certain level that makes them last longer; it's not filling them beyond a certain level that makes them last longer. SSDs need a certain amount of empty space (beyond factory overprovisioning, which should be left alone anyway) for TRIM and garbage collection to work in, not too dissimilar in principle to the empty space needed for defragging to work on HDDs. If the room is limited, something called write amplification occurs.

Since an SSD needs to empty all the levels in a cell to be able to be written to (the function of TRIM), it has to have somewhere to relocate the data to. If room is limited, more shuffling of data has to be done, the extra writes needed being write amplification.

[Moonstone Man]

SSDs need a certain amount of empty space (beyond factory overprovisioning, which should be left alone anyway) for TRIM and garbage

Oh, if you're looking at the free space in a partition then you're doing it the wrong way. The free space must be unpartitioned and at the end of the drive. The SSD controller won't use any partitioned space for putting bad blocks aside or as temporary storage. During garbage collection, it chooses an empty block and moves all data on a page to that block. You don't need 20% either, 5-10% is sufficient, 7% is probably optimal for most commercial SSDs, and it will be a long time before the controller needs to go anywhere near it. What you need to do is use a consistent partition size relative to each size of SSD you have in order to leave a consistent amount unpartitioned at the end. I partition my 120 and 128 GB SSDs to 105GB, my 250 and 256GB SSDs to 210GB, and my 1TB SSDs to 955GB. The reason for that consistency is to ensure that I don't inadvertently reuse any blocks that the controller may have moved if I decide to kill all the partitions.

All of this is overprovisioning, not just what the manufacturer adds. Where you need to watch the free space in a partition is with the ext4 filesystem. Once you get to 10% or so free, files start to fragment because ext4 attempts allocate more space than is needed by a file to allow for its future growth to avoid file fragmentation.

Of course, nitpickers will be along shortly to dispute all of that as being accurate, but the objective isn't to be accurate, it's general.

[Lady Fitzgerald]

SSDs need a certain amount of empty space (beyond factory overprovisioning, which should be left alone anyway) for TRIM and garbage

Oh, if you're looking at the free space in a partition then you're doing it the wrong way. The free space must be unpartitioned and at the end of the drive. The SSD controller won't use any partitioned space for putting bad blocks aside or as temporary storage. During garbage collection, it chooses an empty block and moves all data on a page to that block. You don't need 20% either, 5-10% is sufficient, 7% is probably optimal for most commercial SSDs, and it will be a long time before the controller needs to go anywhere near it. What you need to do is use a consistent partition size relative to each size of SSD you have in order to leave a consistent amount unpartitioned at the end. I partition my 120 and 128 GB SSDs to 105GB, my 250 and 256GB SSDs to 210GB, and my 1TB SSDs to 955GB. The reason for that consistency is to ensure that I don't inadvertently reuse any blocks that the controller may have moved if I decide to kill all the partitions.

All of this is overprovisioning, not just what the manufacturer adds. Where you need to watch the free space in a partition is with the ext4 filesystem. Once you get to 10% or so free, files start to fragment because ext4 attempts allocate more space than is needed by a file to allow for its future growth to avoid file fragmentation.

Of course, nitpickers will be along shortly to dispute all of that as being accurate, but the objective isn't to be accurate, it's general.

From what I've read in the past, the free space doesn't have to be unpartitioned and, although similar, free space is not the same as overprovisioning. Granted, there is a lot of controversy over this. 10% free space is the figure I've seen for HDDs to reduce fragmentation but I've seen 20-25% recommended for SSDs to reduce write amplification (you can't apply the same rules for HDDs and SSDs). Again, there is a lot of controversy over this.

In case you're curious, the reason I broke up the 8TB SSD into two 4TB partitions was so I could use the 4TB 2.5" SSDs I already had on hand for backup drives. Otherwise, I do not believe in using partitions to organize data; folders are far more efficient.

Edit: At the time I wrote this, I was ready for bed and too lazy to find any references. I just spotted one:

https://easylinuxtipsproject.blogspot.c ... .html#ID13