maybe a check-box that asks lets you decide how you want this situation handled. skip, replace, or ask with each instance.

Good idea IMO.

hey...being that tomorrow is my last day of employment...I wasn't thinking...auto-pilot.

There is a reasonably fast (and self-contained) CRC32 function that is part of the Hijack patch. Just grab any recent Hijack .patch file and scan through it for "crc32". There will be a table, and a function.

Cheers

If I understand the posts correctly, the main issue is the time it takes to compute the checksums that tell us if the files are the same.

Would a stepwise method improve the time?:
- check if the filesize is different. if different, we know we need to reload the file
- if the filesize is the same, check the tags. if the tags are different, then we need to reload the file.
- if the tags are the same compute the CRC. if the CRC is different, then we need to reload the file.

if the normal operational scenario is that most of the new files one adds are NOT duplicates, then this might speed things up (you only have to compute CRCs on files that fail the filesize and tag tests).

-ted

Most implementations of CRC32 are table driven, at least to some extent, and so are fairly fast -- once you step into the realm of real cryptographic strength hashes like MD5 and SHA-1, you're talking real computation. I am not familiar with Adler32, but if it is as simple as CRC32, yeah, you'd think it should much faster.

rjf&

I'm seeing occasional hangs when I "paste" a playlist I just cut. For instance, I have a structure which looks like (in terms of playlists)

shadow
/AliceCooper
/GreatestHits
/Alice Cooper
/HeyStoopid
/PrinceOfDarkness
/AllmanBrothers
/BadCompany

etc. If I cut "Alice Cooper" and paste it into "AliceCooper" then cut "HeyStoopid" when I try to paste it into "AliceCooper" the dialog box I right-clicked to bring up and choose paste from never goes away.

Likewise, if I leave "Alice Cooper" where it is and cut "HeyStoopid", then paste it into "AliceCooper" that works, but when I cut "PrinceOfDarkness", when I go to paste it, the paste hangs.

The same pattern seems to hold in the rest of the playlists.

The problem is that you have to be able to find the "matching" tune to compare against, so you have to compute the hashes up front to be able to lookup the hash to know what you're comparing against.... I suppose we could have several hashes (one on file size, one CRC, one soundprint) and each would narrow it down a lot -- I'll have to look at what the memory overhead would be to store those, though.

Probably the biggest issue right now is getting a good soundprinting implementation working in jEmplode...

Are any of these soup playlists or just regular ones? What OS are you on? I tried to create a similar situation on mine and I can't it to happen (on Win2k)... Do you cut with Right-Click=>Cut or Ctrl-X? Do you select from the tree or from the table? Same for paste -- do you right-click=>Paste into the tree or the table?

Also -- how are you launching jemplode? Are you just replacing the jemplode20.jar in your install dir and running the exe or do you have your own JDK and you're doubleclicking jemploe20.jar?

Yeah -- the CRC32 impl I'm using is table driven, and actually pretty fast... It only takes a little over a second to compute on a 16M mp3 over my network.

Right -- and at the risk of stating the obvious, the probability of collision (two byte streams generating the same hash) for CRC32 is much higher than for MD5 or SHA -- it's still pretty small statistically speaking.

rjf&

I tack on the length of the data portion of the file to the CRC value, so the odds go way down. CRC32 collision rate is somelike like 1 in every 2^32, and when combined with the input size length, it's really low......... I hope.....

That was it. I didn't know I could put all the ranges on one line and was making a line for each which didn't work very good.

Thanks

Don't you love programs with no documentation that little label about the text field was my attempt at docs .. oh well .. i don't have a future in technical writing.

Are any of these soup playlists or just regular ones?
All regular

What OS are you on?
Linux (RedHat 7.2)
I tried to create a similar situation on mine and I can't it to happen (on Win2k)...
I can't make it happen consistently either. In fact, after changing those playlists by making one cut/paste at a time and syncing in between I haven't gotten it to happen again yet since.

Do you cut with Right-Click=>Cut or Ctrl-X?

Right-click.
Do you select from the tree or from the table? Same for paste -- do you right-click=>Paste into the tree or the table?

I was doing it all in the tree.

Also -- how are you launching jemplode? Are you just replacing the jemplode20.jar in your install dir and running the exe or do you have your own JDK and you're doubleclicking jemploe20.jar?

I replaced the jar and run the executable.

I'm trying some things to see if I can get it down to something that seems reproducible, and so far, losing.

Do you get any errors on the console? Also, try using Ctrl-X/C/V and see if it goes away .. Curious if it's hanging because of a weird UI thing or because of a more sinister problem.

Even with no documentation I have gotten better support with jemplode than and piece of software I have purchased.

Do you get any errors on the console?
Which console?

Also, try using Ctrl-X/C/V and see if it goes away

I have to break it again before I can see how to unbreak it.

I made a soup like this in jemplode only and it doesn't add all the tracks that are in the albums even though the tracks have the same source.

Any ideas ?

Can you send:
1) the value of the "soup" tag of the soup playlist (from properties=>advanced)
2) a screenshot or something of the table view of the album
3) a screenshot or something of the advanced tag properties of one of the tunes that's not making it in?

I setup this soup on mine and it _appears_ to be working properly for me, but it's so hard to tell...

I take it back .. That's too annoying. Go get

http://www.jempeg.org/debug.zip

unzip jemplode20.jar, unzip debug.zip on top of it (it should replace about 6 or 7 files) then zip jemplode20.jar back up again. Alternatively you can unzip debug.zip and then zip -ru jemplode20.jar to update it and replace the files. Then run jemplode again and send me the console output. If you don't get console output, open jEmplode 2.0.lax and search for "stdout" and you should see a line that defines where stdout redirects to .. set that equal to "console" or a filename and then rerun.

What this does is print out the name of each soup, the total number of tunes in the soup vs. the total number of tunes on your Empeg and if there are any in the db that aren't in the soup, it prints out which ones. For a soup that doesn't have any searches in it, the total number of tunes in the soup should always equal the total number of tunes on your Empeg. If you have a search anywhere in the chain, the search can filter out some, so it's ok to have missing tunes in that case.

You may want to keep a copy of the original jemplode20.jar, since this debugging slows down soups a lot.

instead of just a crc32 check of 10k in the beginning and 10k at the end, why not borrow from gordian knot compression check(divx related) and do a check on X#bytes every Interval. so say, 1000bytes every meg of the mp3.

this will certainly eliminate the possibility that a song can be the same in the begining and end, but not the middle because of a rerip.

That's not a bad idea ... It will be interesting to do a performance comparison between full MP3, first 20, first 20 + last 20, 1k every X bytes.

go into jEmplode 2.0.lax ( I think that's what it's called) and search for "stdout" .. There should be a line that tells where stdout redirects to ... You can set that to "console" or set it to a filename to log. Then when you get it to happen again, see if there's an error in there.

Done. So far I got one more hang using the right-clicking and no more with ctrl-v to paste. I'm still trying.

Very interesting ... So it must be some race condition with the UI thread.

Wow .. performance goes WAY up this way... I took 1000 byte block every 50000 byte interval (roughly 100 samples on a normal MP3) and it dropped from 8 seconds to 300 ms. I'd love to know, mathematically, how this affects the odds of a collision.

Still haven't failed it using just ctrl-v to paste and using right-click to cut.

Did find another problem which maybe you can reproduce.

Except that I didn't. Text of not a problem removed.


I see it's that I was playing with 10000 items, but I just blew up JEmplode entirely but cutting a playlist which was listed twice in the same parent playlist and then trying to paste into same.

The Java VM seems to want to take responsibility.

Post deleted by dbrashear

Actually, it's not a hang, it's just a very, very long update. I had left the first one I "hung" around, and some time later, it finished updating.

I guess the answer is either "get a faster machine" or "don't play with 10000 items at once"

Oh .. you know what's happening? When you have a table open, and you change tags, the table receives update events which causes the tree to resort. Every single thing you update is causing the table to resort (i.e. a sort per change). This is on my list of optimizations to work on... Definitely having 10,000 items at once is making it worse, but it is jEmplode's fault that it's happening ...