Case of the missing hard disk space

Newbies keep complaining that Windows ate their hard disk space because their 80GB hard disk showed only 74GB in Windows. So these really smart people hypothesized that formatting your hard disk takes 6GB out of your hard disk for Windows to play with. Is this true?

I’ll be using NTFS v3.00 as the reference file system for this article (If you’re still using FAT32 or FAT, God knows what’s wrong with you). NFTS v3.00 is included in Windows XP, Windows 2003, Windows Vista and Windows 2008.

The NTFS file system allocates by default 12% of available disk space for its Master File Table (MFT) which acts as a data-to-address lookup table. So that leaves you with 88% of usable disk space. However, in Windows, ‘capacity’ is defined as usable space + MFT reserved space = 88% + 12% = 100%. So technically, Windows will still report 100% of your hard disk capacity.

So where did all your hard disk space go? Answer: It’s there! It’s still there!

Typical hard drive manufacturers prefer to use the ‘decimal’/metric/SI system when defining bytes. Seagate for example, takes the following definition:

1GB = 1,000 MB = 1,000,000 KB = 1,000,000,000 B

However, Windows and most UNIX operating system adopts the following definition (binary system/IEEE/IEC):

1GB = 1,024 MB = 1,048,576 KB = 1,073,741,824 B

The rational behind the binary system is like this because the computer operates with two TTL states only: high (1) and low (0). The highs and lows is what makes the familiar binary base (eg. 10010011).

In the metric system (decimal system = base 10), 1 Kilo of arbitary unit is equivalent to 1000 arbitary units:

1000 = 103

However, in the binary sytem (base 2), 1 kilo of bytes is equivalent to 1024 bytes:

1024 = 210

Confusing? This notation convention has been adopted as early and the 1964. The reason why 1 kB equals to 210 and not 29(512) or 211(2048) is because 210(1024) is the nearest to it’s SI prefixed cousin (1000). Another reason is because an electronic system such as a TTL (transistor-transistor logic) has only 2 states, not 10. So if you have 10 TTL gates, the number of possible states is 210=1024. If you used the SI definition, you would have lost 24 states!

For the sake of clarity, I will now define 1024 bytes as 1KiB (Pronounced as Kibi-Bits, as defined in document IEEE 1541-2002: Prefixes for Binary Multiples) and 1000 bytes as 1KB.

Hard drive manufacturers love the SI prefix because it “eases calculation” (so says Western Digital). But in fact, it caused much confusion among consumers because their ‘advertised’ capacity (which uses the 1KB = 1000B) is a lot less than the ‘usable’ capacity (which 1024B = 1KiB). So in the end, customers complained and blamed operating system like Windows and Linux ‘for crunching up gigabytes of space without their consent’.

Here’s an example on how to calculate the hard drive capcity in IEEE notation (power of 2) from decimal notation (power of 10):

160 GB (Advertised capacity) = 160,000,000,000 Bytes

160,000,000,000 Bytes = 156,250,000 KiB

156,250,000 KiB = 152,587.89 MiB

152,587.89 MiB = 149.01 GiB (As reported by your operating system).

So did your 10.99GB of space flew into oblivion? Obviously not! It’s still there and forever is still there! And without realising or not, you are using that ‘missing’ space.

While purchasing a hard drive advertised in GB (SI unit), you can easily (and roughly) convert the size into GiB (IEEE notation) by multiply the SI capacity with 0.93. Hence, a 250GB hard disk would appear to be 233GiB to your operating system.

This confusion of notation has brought a lot of problems and litigations. Manufactures love to ‘exaggerate’ capacity when advertising products such as DVD, MP3 Players, hard drives and storage cards. The IEEE and IEC have been advocating the IEEE notation (the KiB, MiB, GiB, TiB prefix) in order to avoid confusion and many technical softwares like GNU Parted and linux kernel has since adopted this notation.

But for some products like RAM and processors, 1MB of RAM or cache does equal to 1024KB. (more appropriately, 1MiB = 1024KiB).

So next time don’t blame Nero for stealing your DVD’s space and turning a 4.7GB DVD-R into a 4.38GiB DVD-R.

This article originally appeared in Bits of Bytes (forefather of TheBits.info) on 4 June 2007.

One thought on “Case of the missing hard disk space”

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>