Duplicate file finder?

Nutcase · 28-12-2007, 19:37

Is there some software that can find duplicate files, even with different file names? My old blueyonder webspace finally croaked, and I've a LOT of pics I need to rehost, but over the years they've ended up all over the place, and nosing round my comp there's copies all over the place too (whenever I changed to a new hard drive I'd copy anything needed from the old one - and to give an idea of how long ago this has been going on, one directory is called 300meg drive!!!!).

Basically, all my files are in a mess

So I'd like to clear as many as I can before I start trying to move them all into a sensible order...

Nutcase · 28-12-2007, 20:05

Just trying a freeware thingy I found. Says it works on a byte by byte basis or something? Will report back...

edit: hmm, this could take a while - 89230files on one drive, and after searching 7000 it reckons 900 duplicates. Not convinced I've got THAT many though

Nutcase · 06-01-2008, 18:25

Yup, that freeware thing was an utter waste of time. It reckoned driver files were identical to jpg's etc.

Just spent a couple of hours dumping as many files in one folder as I can. That way when an identical filename came up, if the size was the same, then that's some found.

Cleared about 4gb so far

Mark · 06-01-2008, 19:44

Windows Search will allow you to sort by filename, but it's doubtful that's very helpful unless you have a lot of time. There's probably some batch file trickery that'll do it but this sounds far too like work and I'm still on holiday.

Dr. Z · 06-01-2008, 20:04

FDUPES checksums every file with an MD5 hashing algorithm and WILL find any duplicates (that is, exact duplicates).

You will need to compile it for your platform though.

Nutcase · 06-01-2008, 20:16

Compile? For windows? Eek!

Lots of time I sometimes have of an evening, but lots of patience?!

I think once I've reduced the mess of directories it was all in down to a few, that should make life a little easier...

Mark · 06-01-2008, 20:24

32-bit version...

http://www.ouroboros.me.uk/misc/08/01/fdupes-1.40.zip

Used Cygwin because I'm lazy so I've chucked in the bit of Cygwin I believe is required to make it all work.

Nutcase · 07-01-2008, 06:01

Ooh, thanks Mark

tried it though and it briefly flashes up a window but it dissapears instantly

I tried the driver file in the system32 folder and the same folder as the exe. Did I put it in the wrong place?

Mark · 07-01-2008, 11:59

I'll try it again from Work. I have a non-Cygwin machine there I can use to test it.

Nutcase · 07-01-2008, 14:45

Cheers

If it looks like it'll be hassle tho don't worry - I'm making progress doing it the hard way

28-12-2007, 19:37	#1
Nutcase Moonshine Join Date: Dec 2006 Location: Chelmsford, innit! Posts: 3,979	Duplicate file finder? Is there some software that can find duplicate files, even with different file names? My old blueyonder webspace finally croaked, and I've a LOT of pics I need to rehost, but over the years they've ended up all over the place, and nosing round my comp there's copies all over the place too (whenever I changed to a new hard drive I'd copy anything needed from the old one - and to give an idea of how long ago this has been going on, one directory is called 300meg drive!!!!). Basically, all my files are in a mess So I'd like to clear as many as I can before I start trying to move them all into a sensible order...

28-12-2007, 20:05	#2
Nutcase Moonshine Join Date: Dec 2006 Location: Chelmsford, innit! Posts: 3,979	Just trying a freeware thingy I found. Says it works on a byte by byte basis or something? Will report back... edit: hmm, this could take a while - 89230files on one drive, and after searching 7000 it reckons 900 duplicates. Not convinced I've got THAT many though Last edited by Nutcase; 28-12-2007 at 20:19.

06-01-2008, 20:04	#5
Dr. Z I'm going for a scuttle... Join Date: Jul 2006 Posts: 2,021	FDUPES checksums every file with an MD5 hashing algorithm and WILL find any duplicates (that is, exact duplicates). You will need to compile it for your platform though. __________________

06-01-2008, 18:25	#3
Nutcase Moonshine Join Date: Dec 2006 Location: Chelmsford, innit! Posts: 3,979	Yup, that freeware thing was an utter waste of time. It reckoned driver files were identical to jpg's etc. Just spent a couple of hours dumping as many files in one folder as I can. That way when an identical filename came up, if the size was the same, then that's some found. Cleared about 4gb so far

06-01-2008, 19:44	#4
Mark Screaming Orgasm Join Date: Jul 2006 Location: Newbury Posts: 15,194	Windows Search will allow you to sort by filename, but it's doubtful that's very helpful unless you have a lot of time. There's probably some batch file trickery that'll do it but this sounds far too like work and I'm still on holiday.

06-01-2008, 20:16	#6
Nutcase Moonshine Join Date: Dec 2006 Location: Chelmsford, innit! Posts: 3,979	Compile? For windows? Eek! Lots of time I sometimes have of an evening, but lots of patience?! I think once I've reduced the mess of directories it was all in down to a few, that should make life a little easier...

06-01-2008, 20:24	#7
Mark Screaming Orgasm Join Date: Jul 2006 Location: Newbury Posts: 15,194	32-bit version... http://www.ouroboros.me.uk/misc/08/01/fdupes-1.40.zip Used Cygwin because I'm lazy so I've chucked in the bit of Cygwin I believe is required to make it all work.

07-01-2008, 06:01	#8
Nutcase Moonshine Join Date: Dec 2006 Location: Chelmsford, innit! Posts: 3,979	Ooh, thanks Mark tried it though and it briefly flashes up a window but it dissapears instantly I tried the driver file in the system32 folder and the same folder as the exe. Did I put it in the wrong place?

07-01-2008, 11:59	#9
Mark Screaming Orgasm Join Date: Jul 2006 Location: Newbury Posts: 15,194	I'll try it again from Work. I have a non-Cygwin machine there I can use to test it.

07-01-2008, 14:45	#10
Nutcase Moonshine Join Date: Dec 2006 Location: Chelmsford, innit! Posts: 3,979	Cheers If it looks like it'll be hassle tho don't worry - I'm making progress doing it the hard way