Re: file deletion

[Glynn Clements <glynn@gclements.plus.com>]

Andy wrote:

> Does anybody know of any useful code or c commands that
> you can use to search for duplicate files within a linux/unix directory and
> its subdirectories and remove them?

First, you have to define "duplicate". Also, once you've decided that
two or more files are duplicates, you have to decide which one you
wish to keep and which ones are to be deleted.

If you consider any files with identical MD5 hashes as duplicates, and
don't care about which one is kept, you could use something like:

find . -type f -exec md5sum {} \; | sort | uniq -w32 -d | cut -b 35- | \
	while read file ; do rm -- "$file" ; done

However, if you have 3 or more copies of a given file, the above will
only delete one of them ("uniq -d ..." only prints one instance of
each duplicate line; "uniq -D ..." prints all instances, which would
result in all copies being removed; there isn't an all-but-one option).

-- 
Glynn Clements <glynn@gclements.plus.com>