From http://groups.yahoo.com/group/ukha_d today...

From: "mark_harrison_uk2" <mph@a...>
Date: Mon May 19, 2003 3:42 pm
Subject: MP3 shootout results - UKHA2003




Thanks to all who filled in assesment forms for the MP3 shootout.
Your feedback is most welcomed, and forms the basis of the analysis
below.

For those who weren't at UKHA2003, we had 4 MP3 players, in
alphabetical order:

- Rio Receiver
- SliMP3
- XBox, using the analogue outputs
- XBox, using the optical outputs

The tests were fully blind, and which player was which was not
revealed until AFTER the players had been scored. Until that point,
they were simply called players 1, 2, 3 and 4.

We listened to about 45 seconds from each of the following tracks,
rating each player from 0 to 10 for each track:

- I can't live without you, Queen
- Os Justi (Bruckner), Corydon singers on Hyperion
- The Flat Earth, Thomas Dolby

The equipment was:

- Meridian 562V input switch (analogue inputs only)
- Meridian 518 mastering processor (analogue inputs only)
- Meridian 565 7.1 surround processor (all inputs)
- Chord power amp (front)
- Meridian power amp (rear)
- B&W CDM7 speakers (front)
- B&W 603 speakers (rear)

The XBox optical output suffered a small disadvantage in the tests,
since the other outputs were run through a Meridian 562V and 518
processor which dejittered the ADCd streams before passing to the
common 565 processor at 24 bit level. This was, in the main, due to
the time constraints of setting everything up, since trying to set up
an appropriate dejitter filter for the optical input would have taken
too long. As such, I'd expect the XBox optical output to do
comparatively BETTER in a well set-up system.

Rather than just reading the individual scores, I would encourage you
to read through to the end of this message, since there were some
significant factors to take into account.

Based on a total of 33 survey responses, the overall scores were:

4th place - XBox analogue output, scoring an average of 5.29
3rd place - SliMP3, scoring an average of 5.37
2nd place - XBox optical output, scoring an average of 5.95
1st place - Rio Receiver, scoring an average of 6.13

Judged on the Queen track alone:
4th place - SliMP3, scoring an average of 4.73
3rd place - XBox analogue output, scoring an average of 4.86
2nd place - Rio Receiver, scoring an average of 5.42
1st place - XBox optical output, scoring an average of 5.67

Judged on the Bruckner alone:
4th place - XBox analogue output, scoring an average of 5.45
3rd place - SliMP3, scoring an average of 5.61
2nd place - Rio Receiver, scoring an average of 6.24
1st place - XBox optical output, scoring an average of 6.27

Judged on the Thomas Dolby alone:
4th place - XBox analogue output, scoring an average of 5.55
3rd place - SliMP3, scoring an average of 5.79
2nd place - XBox optical output, scoring an average of 5.91
1st place - Rio Receiver, scoring an average of 6.73

As you can see, despite the handicap, the XBox optical output always
scored higher than the Analogue output - something well worth bearing
in mind when setting up your XBox.

However, where people were sitting in the audience also had an impact:

For people sitting at the level or, or behind the rear speakers the
scores were as follows. It should be noted that MOST people fell into
this category, which therefore gives this group a heavy weighting in
the overall scores.

4th place - SliMP3, scoring an average of 5.4
3rd place - XBox analogue, scoring an average of 5.57
2nd place - XBox optical, scoring an average of 5.95
1st place - Rio Receiver, scoring an average of 6.14

For people sitting forward of the rear speakers, but at the side of
the room, the scores were as follows:

4th place - SliMP3, scoring an average of 4.75
3rd place - XBox analogue, scoring an average of 4.9
2nd place - Rio Receiver, scoring an average of 5.88
1st place - XBox optical, scoring an average of 6.25

For people sitting in "pole position" - in the centre at the front,
the scores were:

4th place - XBox analogue, scoring an average of 4.79
3rd place - XBox optical, scoring an average of 5.67
2nd place - SliMP3, scoring an average of 6
1st place - Rio Receiver, scoring an average of 6.13

Again, for all locations, the XBox optical input was preferred to the
analogue ones. However, those sitting in the best listening positions
rated the SliMP3 in second place, wheras those with poorer seats
placed it 4th.

Nonetheless, the sound quality crown does seem to go to the original
Rio Receiver...

But my most important point remains, when buying hi-fi, try to listen
in a situation that approximates to your listening environment. I
realise that, today, this is hard to do with MP3 players, but the
market is moving forward.

Regards,

Mark Harrison

I don't understand this exactly. It's a test using optical outputs, so all the real hard work (the D/A conversion, which is the part that really affects the sound quality) is done by something other than the players themselves.

I mean, it's nice that the Rio Receiver won and everything, but why should it have sounded any different from the others? They should have all sounded identical because they were all sending essentially the same digital stream to the D/A converter.

The only reasons I could see that might cause them to sound different would be:

- Errors in the MP3 decoder algorithm resulting in differences in the playback.

- Time-base Jitter on the optical outputs, not properly compensated for in the outboard D/A equimpent.

What was the purpose of this test and its parameters?

Edit: I see that the SliMP3 didn't specify if it used optical outputs or not. If it was analog output, then you're not even comparing apples to apples by pitting it against an outboard D/A converter.

mean, it's nice that the Rio Receiver won and everything, but why should it have sounded any different from the others? They should have all sounded identical because they were all sending essentially the same digital stream to the D/A converter.

But the Rio doesn't have a digital out Tony, so that can't have been what they were doing. I don't think the Slimp has a digital out either.

They appear to be taking the analog output from the devices, redigitizing it, passing it through some digital filter and then passing it back to an external DAC. All very odd.

So the Rio can sound different to the others, as it is the analog outputs (after some odd mangling) of the devices they are comparing (except with the XBox when connected with optical).

I think you're wrong. It only mentions that they used the Xbox's optical outs. All the others are implied to be analog (``analogue inputs only'', etc.).

They just mention that they weren't able to set up the XBox's digital as well as they'd have liked to.

Then again, I'm not sure what they think is being ``dejittered'' by the Meridian equipment. Nor do I know what they were doing with a 7.1 decoder. MP3s are incapable of storing more than two channels.

Actually, it sounds like the might be using that 562 as an ADC. Why you'd want to do that, I have no idea. Maybe they didn't have a high quality analog preamp available.

Maybe you are right. Their fundamental testing ideas are flawed.

Maybe you are right. Their fundamental testing ideas are flawed.

I guess it depends what they were actually testing. If they were testing to see which unit had the best sounding output then they are doing something odd.

If however they are testing what the units sound like when fed into a Meridian based system then they were probably doing the right thing. We have no context to know which they were attempting.

I used to dream of having a Meridian based system (a CD player and DAC at least), it sounded and looked so good. It all looks a bit dated nowadays.

But the Rio doesn't have a digital out Tony

Oh right... DUH.

My mistake. It's the HSX-109 that has the digital out, not the Rio Receiver!

Okay, well that's pretty interesting then.

Then again, I'm not sure what they think is being ``dejittered'' by the Meridian equipment.

Nor why they seem to have more faith in the clock in the Meridian dejitterer than the clock in the Meridian ADC that's generating the digital path in the first place. Perhaps one has had green lines drawn round the outside?

Peter

I'm also on the UK_HA list, so whilst i can't comment on the results, i can say that they are generally 'clued-up' guys and it would be unusual for them to make a complete mess of things by having a flawed set-up.

bill

I got the idea that they just fed analogue out from the RR/Xbox-analogue/Slimp3 directly to the amp, but when testing the Xbox-optical used the optical input on the amp (ie, used the amps DACs).

Given the cost of the amp (well, the entire setup really) it says good things about the Receiver's DACs that people preferred them to whatever you get in a meridian 7.1 amp!

The central's audio output uses the same DAC as the receiver, but the circuitry was redone and gives even cleaner output.

Hugo

I don't think that's true. The message reads, in part

- Meridian 562V input switch (analogue inputs only)
- Meridian 518 mastering processor (analogue inputs only)
- Meridian 565 7.1 surround processor (all inputs)

So it seems appparent that the analog devices were run through all the Meridian equipment.

Unless I misunderstand what you're saying.