Winamp 5.33: multiple entries in ML

[Flieger]

Hi!

after each rescan i get multiply media library references on SAME files!

i have this problem since 5.33!
how can i repair my media library and solve this problem?

Thanks!

[DJ Egg]

Yeah, we're a bit embarrassed by this issue, especially since 5.33 was available as a public beta for 2 weeks prior to release, but no-one managed to find/report this bug.

I've added it to the Regressions section of the Known Bugs thread.


_____________________________________________________________

-Potential for some duplicate entries in Local Media after rescanning Watch Folders *

_____________________________________________________________

Legend
* = fixed in latest internal builds

_____________________________________________________________


Note that the problem cannot be reproduced on clean installs, only on upgrades.

It's fixed for 5.34, but in the meantime....

______________________________________________________________


EDIT

Fix now available below

[Flieger]

does this mean you knew about this bug and it only happens by upgrading 5.32 to 5.33?

so if i uninstall my current corrupt version und reinstall a clean v5.33, it'll work?

[DrO]

we didn't know about the issue until after 5.33 was released (most likely because people were trying the beta out in a separate clean install in most cases).

as Egg says, removing the duplicates/cleaning the library and starting again is the way to go

-daz

[Benski]

Unzip this file into your Winamp folder and run it (make sure Winamp is closed first).

It will re-index your database and remove any exact duplicate entries.

Before you run, be sure to backup your old database (usually in C:\program files\winamp\plugins\ml\main.dat and main.idx - unless you installed with multi-user settings, then it's in c:\documents and settings\<yourname>\Application Data\Winamp\Plugins\ml).

Let me know if it works, and how many duplicate files it removed.

[decoy_oct]

Thanks for the patch. I just updated to 5.33 today and had the same bug.

It found 304 duplicates (9563 songs down to 9259). It works as long as Winamp isn't running I kept my original main.dat & main.idx as well, so if you want them for comparative purposes, let me know.

[StimpyCat]

DJ Egg,
Thank you, thank you, thank you! Did I mention thank you! The reindexing of the media library worked. 233 duplicates were removed. All my tracks are OK. It was also nice to see that this bug was fixed in the upcoming 5.34 version. Once again, thanks. Keep up the good work.

[chimichanga15]

Benski,

It removed 1122 duplicates out of 32,353 files. It worked great, except that it took about 10 minutes to execute.

[niZmO_Man]

Hey guys, I know its a bit off-topic, but I get the same duplicate entries problem with Playlist, so bring back the "remove duplicate entries" options in Remove...
Keep up the good work I only use Winamp, it is the #1 player in the universe, just ask all the aliens out there!

[DJ Egg]

Wrong thread, niZmO_Man.

That menu item is only there as long as you don't uncheck "Extended Jump to File Support" in the Installer Options (ie. no jtfe / gen_jumpex.dll, no 'remove duplicates' option in the playlist).

[Lenneth]

Does Winamp need to be closed in order to run this? Asking because I'm on a network and it's not so easy to shut it down.

[DJ Egg]

Yes

[Marshdweller]

It removed 82 duplicate entries!

Thanks a bundle good people of Winamp!

[bitchtow]

What is nde.dll, reindex.exe won't run because I'm missing it, what specific runtimes does it run on?

[DJ Egg]

nde.dll is the Nullsoft Database Engine.
The Media Library won't run without it, never mind Reindex.

nde.dll is included with the Winamp installer, and is installed alongside the Media Library. It should already be in the Winamp root folder, where winamp.exe is. The reindex.exe file also needs to be placed in the Winamp root folder, and Winamp needs to be closed befor running the Reindexer.

All that said and done, Winamp 5.34 Beta is now available, which has the Reindex tool built in. The Reindexer in 5.34 will auomatically run on first Winamp startup, or if the ML isn't open on first run then when you first open the ML.

[bitchtow]

Cheers, I must have misread the directions coz I dumped it in the ml directory and ran it from there. I did install the 5.34 beta and I didn't notice it reindexing, and I still have double ups, so I guess it didn't run. I tried a second time and same thing.

So then I tried using the reindex.exe from the root of the winamp directory and I get a weird error saying "the entry point ?Query@Scanner@@QAEHPA_W@Z could not be located in the dll nde.dll"

I'm not too fussed about blasting the library and rebuilding it from scratch, but I'd rather not.

[DJ Egg]

You've probably unchecked some items in the Installer Options, and therefore the newer ml components, including nde.dll, didn't get installed. Either that, or the external Reindexer tool is designed not to work with the nde.dll included with 5.34.

Open Winamp
Make sure the Media Library window is open
Close Winamp

Go to the Winamp\Plugins folder
Open gen_ml.ini in Notepad

If any of the following lines exist (probably at the bottom), then delete them:

reindex_533=1
reindex_534=1

Save the changes, and close Notepad.

Now reinstall 5.34 beta
Make sure the relevant items are not unchecked in the Intaller Options, eg.
Winamp Library > Core Components > Local Media

The 5.34b built-in Reindexer should kick-in as soon as Winamp and the ML open (after completion of the installation process).


ps. Note, if you selected 'shared settings for all users' on screen 5 of the installer options (default), then gen_ml.ini will be in the Program Files\Winamp\Plugins folder (or wherever you installed Winamp to), but if you chose 'separate settings per user' then gen_ml.ini will be in your %appdata%\winamp\plugins folder (where AppData on Win2k/XP is usually found at: C:\Documents and Settings\(username)\Application Data).

[bitchtow]

Well, that got it reindexing, but it didn't remove the duplicates, well not the ones I noted. I guess I'll just blast the ml, it's not like it has a great deal of info anyway, just the play counts.

[DJ Egg]

Strange. Are the duplicates definitely the exact same path and filename, and not just the same album/songs but stored in a different folder or with a different filename?

[bitchtow]

Yeah, they definitely had the same path and file details, the only details that were different were the play count values, which I assume was because they were 'different' media library entries. But I've blasted away the ml now and it's duplicate free. So I can't help with any sort of debugging now.

[leebotton]

16471 duplicates removed!! :-D lol
Unfortunately though, when I upgraded, the play counts were removed but not the end of the world I guess...

[kenwoods]

Originally posted by bitchtow
Yeah, they definitely had the same path and file details, the only details that were different were the play count values, which I assume was because they were 'different' media library entries. But I've blasted away the ml now and it's duplicate free. So I can't help with any sort of debugging now.

I have the same problem and I don't want to blast my media library!!!
I only want to remove the duplicates

[DJ Egg]

@ kenwoods

I've already told you what to do if you require further help.
Ignoring our requests won't get you anywhere.

http://forums.winamp.com/showthread....19#post2158319
http://forums.winamp.com/showthread....40#post2158540

[niZmO_Man]

Originally posted by DJ Egg
Wrong thread, niZmO_Man.

That menu item is only there as long as you don't uncheck "Extended Jump to File Support" in the Installer Options (ie. no jtfe / gen_jumpex.dll, no 'remove duplicates' option in the playlist).

Sorry about that I only realised when I reinstalled, thanks for that anyway

[kenwoods]

DJ Egg, I've submitted the files in the relevant post...

Thanks Again...

[DJ Egg]

@ kenwoods

I forwarded the files to Benski, and he said he'd look into it...

In the meantime, could you please tell us what some of the duplicate entries are?
This might make it easier for us to diagnose the problem, if we know beforehand what entries we are looking for.

Artist, Title, and Filename info should do...
(though Album & Track# also would help even more, if possible).

[kenwoods]

DJ Egg,
for example:
all of Pink Floyd's "The Wall (Pink Floyd - The Film)"
is duplicated...
here's one filename off of it for example:
"E:\My Music\Pink-Floyd-The-Wall,The-Film-(Soundtrack)-1982\09 - Mother.mp3"

another example is all of Rammstein's "Mutter" album (the filenames don't use any type of special unicode)
here's a filename for example: "E:\My Music\CD's I Own\Rammstein_-_Mutter\03-Sonne_Rammstein.mp3"

[DJ Egg]

I couldn't find "E:\My Music\Pink-Floyd-The-Wall,The-Film-(Soundtrack)-1982\09 - Mother.mp3" listed anywhere in main.dat

I found:

E:\My Music\CD's I Own\Pink Floyd - The Wall\06 - Mother.mp3

E:\My Music\Pink.Floyd.-.[Is.There.Anybody.Out.There,.The.Wall(Live)](MP3)\Is There Anybody Out There, The Wall Live 1980-81 - Cd 1\Pink Floyd - (07)Mother.mp3


And I found "E:\My Music\CD's I Own\Rammstein_-_Mutter\03-Sonne_Rammstein.mp3" listed once.


Are you sure that these media library entries are exact duplicates of the exact same files with the exact same path & filename?

The reindexer will only remove exact duplicates. It won't remove anything if the path\filename is different.


Have you tried doing: Library button > Remove missing files...?


[Edit]

Okay, I loaded your main.dat/main.idx into a clean install,
and sure enough, I can see the duplicate entries.

Weird!

Thanks :-)

[aperture01]

I have the same problem, I ran the reindexer and still have multiple database entries.

[DJ Egg]

@ kenwoods

We ran the Reindexer on your database, and it removed 1597 duplicates.

Did you definitely try the standalone Reindexer from above?

Be sure to read the instructions first.

(It's possible that the built-in one in 5.34 beta is borked, or maybe you didn't close winamp before running the standalone reindexer).

[kenwoods]

DJ Egg,
the reindexer didn't work with 5.34 so I reinstalled 5.33
and used the reindexer and it worked...

Thanks...
(I've had this problem for ages with 5.33, I didn't know you've released a stand alone application for 5.33 back then...)

[DJ Egg]

Doh!

Okay, cool

tfft

[foxyshadis]

I've been living with this problem for a while, when the last version that caused duplicates cropped up. I could clear out, but once you rely on the library for a while that's hard, and I admit I've just been too lazy to make an ActiveWinamp script to fix it all, merging the stats from all the dupes.

So I offer up my media library if you want to look. Reindexer finds no duplicates, but a simple scan in a hex editor shows multiple copies of many files - buckethead is particularly afflicted, as many as 4 copies of some files; some copies are listed as ANSI, some as Unicode. Look up "05-buckethead-lurker_at_the_threshold_part_1.mp3". The ANSI files seem to be older holdover dupes, the unicode ones are newer. Dunno why reindex doesn't find them, maybe the newer version you have will.

btw, if you need to quickly search for unicode names, the tool XVI makes it quite easy, with a UTF-16 checkbox. Maybe that was why kenwoods' songs weren't coming up.

The ml is the 5.33 backup without reindex run on it, btw, so that it's clean for you.

[DJ Egg]

Hmm, strange.

I can see three "D:\foxy\stuff\05-buckethead-lurker_at_the_threshold_part_1.mp3" entries

Did you definitely place reindex.exe in the Winamp root folder and run it whilst Winamp was closed?

Note that the Reindexer worked for kenwoods.

[foxyshadis]

I did, and used the 5.34 built-in indexer as well, and like you said it cleared some things out. Just left a lot of others.

Is it possible to get the reindexer source, if it can be built with the public SDK? (If not, well, no point.)

[DJ Egg]

The reindexer in 5.34 b1195 is borked, but the standalone should have worked (you can still run it with 5.34 beta, as long as it's placed in the same dir as winamp.exe and winamp is closed at the time).

As for the source.... umm, I don't know (only Benski can answer that one).

Reindex ran successfully.

Took 5 to 10 minutes. Removed 818 duplicates from a 20,000+ library. I had already manually removed about 400.

I had been using the Playlist|Remove Duplicates menu item to clean the playlist. You may want to add a similar command for the Media Library.

[DJ Egg]

Cool.

Note, there shouldn't ever be any duplicates (same path+filename) in the ML,
so there wouldn't be much point in adding a Remove Duplicates command
(ie. it was a 5.33-specific bug)

[foxyshadis]

The new reindexer in r1239 took care of everything I could think of. Thanks!

[matrix626]

I was wondering if it was possible to have a setting to remove dupilcate listings of the same song with partially different filenames....such as have a setting to remove duplicate titles for instance?