why is the output of `du` often so different from

Question

why is the output of du often so different from du -b? -b is shorthand for --apparent-size --block-size=1. only using --apparent-size gives me the same result most of the time, but --block-size=1 seems to do the trick. i wonder if the output is then correct even, and which numbers are the ones i want? (i.e. actual filesize, if copied to another storage device)

Answer 1

Compare (for example) du -bm to du -m.

The -b sets --apparent-size --block-size=1, but then the m overrides the block-size to be 1M.

Similar for -bh versus -h: the -bh means --apparent-size --block-size=1 --human-readable, and again the h overrides that block-size.

Answer 2

Because by default du gives disk usage, which is the same or larger than the file size. As said under --apparent-size

print apparent sizes, rather than disk usage; although the apparent size is usually smaller, it may be
larger due to holes in (`sparse') files, internal fragmentation, indirect blocks, and the like

Answer 3

Files and folders have their real size and the size on disk

• --apparent-size is file or folder real size

• size on disk is the amount of bytes the file or folder takes on disk. Same thing when using just du

If you encounter that apparent-size is almost always several magnitudes higher than disk usage then it means that you have a lot of (`sparse') files of files with internal fragmentation or indirect blocks.

Answer 4

Apparent size is the number of bytes your applications think are in the file. It's the amount of data that would be transferred over the network (not counting protocol headers) if you decided to send the file over FTP or HTTP. It's also the result of cat theFile | wc -c, and the amount of address space that the file would take up if you loaded the whole thing using mmap.

Disk usage is the amount of space that can't be used for something else because your file is occupying that space.

In most cases, the apparent size is smaller than the disk usage because the disk usage counts the full size of the last (partial) block of the file, and apparent size only counts the data that's in that last block. However, apparent size is larger when you have a sparse file (sparse files are created when you seek somewhere past the end of the file, and then write something there -- the OS doesn't bother to create lots of blocks filled with zeros -- it only creates a block for the part of the file you decided to write to).