Compare 2 mp3's to see if they are the same

[PigBenis]

Hey guys, I'm about to start working on a project here soon. It's going to involve a lot of mp3 files from a lot of different people. What I want to do is cut down on duplicate mp3's as much as possible. Obviously, file name is an easy way to get rid of dupes. I've found MD5 will work to get those tracks that may have different file names, but are the same exact file. However, after changing the ID3 tag MD5 doesn't work anymore.

I have a feeling MD5 is probably gonna be it, but can anyone think of more ways to possible cut down on dupe's even more, even mp3s with different sizes? I was thinking about like comparing song duration, with some of the file name, with some of the id3 stuff, blah blah blah.

[niteflyx]

Are these amongst the same 'songs' from different users, or just yourself?

So many people download music now, and use different bitrates to encode at, even with the same id3 tags and bitrates, there'll be differences between files from one user to another, if one downloaded it and one ripped from CD most likely.

[Fog]

Even if you wrote a program to do an MD5 of just the audio data, ignoring the ID3/APE tags, you would still run into the problem of a million different people using a million different MP3 encoders at a million different bitrates. As such, the only plausible way of detecting duplicates would be some kind of acoustic fingerprinting; I don't know if there are any free software solutions for doing this (and unless you are looking for a summer project, it would probably be a Difficult Problem to write a good homegrown solution.)

EDIT: I asked a friend and he recommended this. It includes source and a good explanation of the process.

[PigBenis]

wow I'm gonna look into that foosic, looks really interesting.

[AladainAF]

edit: nm, fog handled it.

[Isuldor]

Shareaza added support for audio data only hashing 4 years ago. You could download mp3s from multiple sources regardless of any ID3 modifications. Too bad distributed/decentralized p2p is basically dead or otherwise useless.

[Fog]

Originally Posted by Isuldor

Too bad distributed/decentralized p2p is basically dead or otherwise useless.

Yeah, because nobody uses torrents.

[PigBenis]

Someone should try this foosic. Quite easy to use, but sadly it doesn't work all that well.

        FingerPrint blah = new FingerPrint();       
        blah.readFrom("C:\\1.mp3");
        
        FingerPrint blah2 = new FingerPrint();
        blah2.readFrom("C:\\n2.mp3");    

        blah.displayFull();
        blah2.displayFull();

        System.out.println(blah.fullMatch(blah2));
        System.out.println(blah.quickMatch(blah2));

Unless I'm missing something here.

[Isuldor]

Originally Posted by Fog

Yeah, because nobody uses torrents.

Really? Because I wasn't even talking about Bittorrent. Fucking dipshit.

Yeah, decentralized/distributed searchable networks are basically dead or useless.

[Fog]

Originally Posted by Isuldor

Yeah, decentralized/distributed searchable networks are basically dead or useless.

Right, because there aren't any good torrent search engines.

[Isuldor]

Originally Posted by Fog

Right, because there aren't any good torrent search engines.

Are you really that clueless? Feel free to explain to me how a website with a database constitutes a p2p network. I'm waiting.

[Ham n Cheese]

Sorry semi related derail inc.

Does anyone know how I can find out if an mp3 is v0/v2? All I know how to do is look at the basic bitrate of something.

anyone able to tell me what this is too?
Encoder : EAC (Secure mode) / LAME 3.92
Codec : LAME 3.97
Bitrate : VBR ~251K/s 44100Hz Joint Stereo
ID3-Tag : ID3v2.3

[Fog]

Originally Posted by Ham n Cheese

Sorry semi related derail inc.[/quote

Does anyone know how I can find out if an mp3 is v0/v2? All I know how to do is look at the basic bitrate of something.

I'm not sure what switches and settings, etc, the -vX presets enable nowadays, but I don't think there is anything extremely important which they set beyond just bitrate. I know that LAME embeds further encoding data into the MP3 in some cases which can be read to get information about the LAME settings that were used; foobar2000, for example, reads this data from files. The best documentation I could find in reference to the structure of that information is this page, but at the top it gives a LAME 3.88 tag example so it might be outdated.

anyone able to tell me what this is too?
Encoder : EAC (Secure mode) / LAME 3.92
Codec : LAME 3.97
Bitrate : VBR ~251K/s 44100Hz Joint Stereo
ID3-Tag : ID3v2.3

What what is? I don't understand.

[Fog]

Originally Posted by Isuldor

Are you really that clueless? Feel free to explain to me how a website with a database constitutes a p2p network. I'm waiting.

It sounds like it does to me. I don't understand the fundamental difference between people sharing a set of files on a searchable P2P network and someone downloading portions of a file from multiple users sharing it, and people seeding a set of torrents in a searchable torrent database and someone downloading portions of a file from multiple users seeding it.

[Luthair]

Originally Posted by Fog

Yeah, because nobody uses torrents.

Torrents aren't really decentralized, they (mostly) still require a tracker.