Re: file deletion

[Jan-Benedict Glaw <jbglaw@lug-owl.de>]

[-- Attachment #1: Type: text/plain, Size: 1162 bytes --]

On Fri, 2004-12-24 13:14:11 -0000, Andy <andy_webb@onetel.com>
wrote in message <003f01c4e9ba$7679d020$316c4ed5@j0s6l8>:
> Does anybody know of any useful code or c commands that
> you can use to search for duplicate files within a linux/unix directory and
> its subdirectories and remove them?

As long as you believe in cryptographical hashes, something like this
should do the trick:

SOME_DIR=/some/directory

find "${SOME_DIR}" -type f -exec sha1sum {} \; | \
	sort | \
	uniq -c | \
	sed -e 's/^[[:space:]]*//g' | \
	egrep -v '\<1\>' | \
	cut -f 4- -d ' ' | \
	while read DOUBLE_FILE_NAME; do
		rm -f "${SOME_DIR}/${DOUBLE_FILE_NAME}"
	done

That's untested, but should work (it's essentially a one-liner).
However, it removes *all* instances of files which are believed to be
identical...

MfG, JBG

-- 
Jan-Benedict Glaw       jbglaw@lug-owl.de    . +49-172-7608481             _ O _
"Eine Freie Meinung in  einem Freien Kopf    | Gegen Zensur | Gegen Krieg  _ _ O
 fuer einen Freien Staat voll Freier Bürger" | im Internet! |   im Irak!   O O O
ret = do_actions((curr | FREE_SPEECH) & ~(NEW_COPYRIGHT_LAW | DRM | TCPA));

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]