In linux, disk spaces can be taken by files which are opened but not closed by some running processes. Even if the files are deleted by
The symptom is that the free space from
df is much less than what
du shows how much space is taken by existing files (excludes the deleted files);
df calculates the total disk space.
The gap is taken by the files that are deleted but still open and holding by running processes.
To check those zombie files, just use the
lsof command line. e.g.
# lsof COMMAND PID TID USER FD TYPE DEVICE SIZE/OFF NODE NAME cedar-log 30186 32125 root 494r REG 8,3 81251235 2765591 /var/log/osquery/osqueryd.results.log.bak (deleted) cedar-log 30186 32125 root 495r REG 8,3 80478986 2766363 /var/log/osquery/osqueryd.results.log.bak (deleted)
Those disk space taken by deleted files will be released when the files get closed. One simple action is
kill -9 to the PIDs which are holding the deleted files.