# Sticky  Home Theater Shack 2015 High-End Amplifier Evaluation Event Reporting and Discussion Thread



## AudiocRaver

*Home Theater Shack 2015 High-End Amplifier Evaluation Event Reporting and Discussion Thread*



:fireworks2:







:fireworks1:


​

*This thread is a continuation of the High-End Amplifier Evaluation Event Preparations Thread previously under way.
* 


The event has begun. Coming to you from southern Alabama, the Home Theater Shack Evaluation Team has assembled at Sonnie Parker's Cedar Creek Cinema for the 2015 High-End Amplifier Evaluation Event. We have amps, we have speakers, we have tunes, we have great eats, what more could one ask for?

Be reminded of the first law of audio evaluation event execution. They never go exactly as planned. Not everything gets there, not everything works, but you endeavor to persevere and get things done.

We have deal with speakers not able to reach us in time, with cabling issues, with equipment not interfacing properly, a laptop crash, with hums and buzzes and clicks and pops, with procedural questions - - - yet we forge ahead, adapt, evolve, redirect, and forge ahead some more - - - and the task of evaluating amplifiers is underway.

Speakers: We were unable to get the Chane A5rx-c and the Acoustic Zen Crescendo Mk II speaker pairs. We are running the Spatial Hologram M1 Turbo v2 and the Martin Logan ESL. Both are very revealing speakers, baring a lot of inner detail in our recordings. They will serve us well. The A5rx-c will be reviewed for HTS when available.

At the moment, the Holograms are serving as our primary evaluation tool. I will post setup details and interesting discoveries a little later. They are giving us a monstrous soundstage, the kind that eats small animals for breakfast, with extremely sharp imaging and very good depth acuity. They are extremely clear, getting into the realm of rivaling electrostatic transparency. Their in-room response is very good, with some expected peaks and dips, but still very listenable. The high frequency response is extended and smooth. The bass gives you that "Are you sure the subs are not on?" feeling on deeper tracks.

We decided to start with sighted comparisons and open discussion today, and blind tests tomorrow. The Audyssey XT32 / Dirac Live comparison has not been completed yet.

Have we heard differences? Yes, some explainable and some not. One amp pairing yielded differences that several evaluators are convinced they could pick in a blind AB test.

One thing I have learned for sure: The perfect complement to good southern barbeque is a proper peach cobbler. Add great company and you have a perfect get-together.

*The Event*

Date: Thursday evening, March 12th through Saturday evening, March 14th.
Place: Cedar Creek Cinema, Alabama, hosted by Sonnie, Angie, and Gracie Parker.
Evaluation Panel: Joe Alexander (ALMFamily), Leonard Caillouet (lcaillo), Dennis Young (Tesseract), Sonnie Parker (Sonnie), Wayne Myers (AudiocRaver).

*The Amplifiers*

Behringer EP2500
Denon X5200 AVR
Emotiva XPA-2
Exposure 2010S
Krell Duo 175
Mark Levinson 532H
Parasound HALO A31
Pass Labs X250.5
Sunfire TGA-7401
Van Alstine Fet Valve 400R
Wyred 4 Sound ST-500 MK II
*The Speakers*

Spatial Hologram M1 Turbo v2, courtesy Clayton Shaw, Spatial Audio
Martin Logan ESL
*Other key equipment special for the event:*

Van Alstine ABX Switch Box, recently updated version (February 2015)
miniDSP nanoAVR DL, courtesy Tony Rouget, miniDSP
OPPO BDP-105

As mentioned, our deepest appreciation goes to Sonnie, Angie, and Gracie Parker, our hosts, for welcoming us into their home. Look up _Southern Hospitality_ in your dictionary, and they are (or should be) listed as prime role models thereof.

This first posting will be updated with more info and results, so check back from time to time.


​

*Amplifier Observations*
These are the observations from our notes regarding what we heard that were supported by being consistent between sighted and blind testing and across reviewers. While we failed to identify the amps in ABX testing, the raw observations from the blind comparisons did correlate in some cases to the sighted observations and with the observations of other reviewers. Take these reports for what they are, very subjective assessments and impressions which may or may not be accurate.


*Denon X5200 AVR*
Compared to other amps, several observations were consistent. The Denon had somewhat higher sibilance, was a bit brighter, and while it had plenty of bass it was noted several times to lack definition found in other amps. At high levels, it did seem to strain a bit more than the other amps, which is expected for an AVR compared to some of the much larger amps. Several times it was noted by multiple reviewers that it had very good detail and presence, as well as revealing ambiance in the recordings.

We actually listened to the Denon more than any other amp, as it was in four of the blind comparisons. It was not reliably identified in general, so one could argue that it held its own quite well, compared to even the most expensive amps. The observations from the blind comparisons that had some common elements either between blind and sighted comparisons or between observers are below. The extra presence and slight lack of bass definition seem to be consistent observations of the Denon AVR, but everyone agreed that the differences were not a definitive advantage to any one amp that would lead us to not want to own or listen to another, so I think we can conclude that the Denon held its own and was a worthy amp to consider. 

*Compared to Behringer*
- bass on Denon had more impact than Behr, vocals sounded muted on Behr
- vocals sounded muted on ML compared to Denon
- Denon: crisp highs preferred compared to Behringer which is silky.
- Denon is more present, forward in mids and highs than Behringer.

*Compared to Mark Levinson*
- Denon seemed to lack low end punch compared to ML.
- Denon is smooth, a certain PUSH in the bass notes, cellos & violins sounded distant, hi-hat stood out, distant vocal echo stood out, compared to ML.
- Denon bass seemed muddy compared to ML which is tighter.
- ML more distant strings than Denon.
- Denon is slightly mushy and fat in bass. String bass more defined on ML.
- ML seems recessed compared to Denon.

*Compared to Pass*
- vocals sounded muffled on Pass compared to Denon
- crisp bass on Denon compared to Pass
- Denon & Pass both even, accurate, transparent, natural, no difference, like both
- Pass seems soft on vocals but very close.
- Denon has a bit more punch on bottom, maybe not as much very deep bass, more mid bass.

*Compared to Van Alstine*
- bass on Chant track was crisp for VA while Denon was slightly sloppy
- sibilance not as pronounced on VA as it was on Denon
- VA super clarity & precision, detailed, space around strings, around everything compared to Denon which is not as clear, liked VA better.
- sibilanceon Denon, VA has less “air” but more listenable, both very good
 - Very deep bass more defined on VA, overall more bass on Denon. ​

*Wyred 4 Sound ST-500 MK II*
In the sighted listening we compared the ST-500 MK II to the Van Alstine Fet Valve 400R. The assessments varied but were generally closer to no difference. The Van Alstine got comments of being fatter on the bottom. The Wyred 4 Sound was noted to have slightly better bass definition but apparently less impact there, and slightly less detail in the extreme highs. Most comments about the midrange were not much, if any difference. An interesting observation here was by Wayne, noting that he did not think he would be able to tell the difference in a blind comparison. Considering the ST-500 MK II is an ICE design and the Fet Valve 400R is a hybrid, we expected this to be one of the comparisons that would yield differences if any. As I am always concerned about expectation bias, this was one that I was particularly concerned with. Van Alstine is a personal favorite for a couple of us so I expected a clear preference for it to be present in the sighted comparison. I felt that the Wyred 4 Sound amp help its own with the much more expensive and likely to be favored VA. 

In the blind comparisons, we compared the ST-500 MK II to the Emotiva XPA-2 and the Sunfire TGA-7401 in two separate sessions. Of course, in these sessions we had no idea what we were listening to until after all the listening was done. In the comparison to the Emotiva, some notes revealed not much difference and that these were two of the best sounding amps yet. The ST-500 MK II was noted to have the best midrange yet, along with the Emotiva. It was described as having less sibilance than both the Emotiva and Sunfire. Both the Emotiva and the ST-500 MK II were described as unstrained in terms of dynamics. In comparison to the Emotiva it was noted to have solid highs, lively dynamics, rich string tones, and punch in the bass. The overall preference in comparison to the Emo was either no difference to preferring the W4S.

In comparison to the Sunfire, comments ranged from preference for the W4S to not much difference to preference for the Sunfire. The Sunfire was described as having more presence in the midrange, while the Wyred was noted to be shrill, lifeless, and hollow by comparison.

These comments varied a lot, but the points of convergence were generally around the similarities to three amps that would be expected to be most likely to be different, if we found any differences at all. The objective results is that we failed to identify the amp in ABX comparisons to two other much more expensive amplifiers. I would have to conclude that based on the results, the ST-500 MK II represents one of the best values and certainly should satisfy most listeners.​




*Audyssey XT32 vs. Dirac Live Listening Comparison*

Last year HTS published a review of a the miniDSP DDRC-22D, a two-channel Dirac Live Digital Room Correction (DRC) product. The review included a comparison to Audyssey XT. A number of readers requested a comparison of Dirac Live with Audyssey XT32. That comparison was recently completed during the Home Theater Shack High-End Amplifier Evaluation Event at Sonnie Parker's Cedar Creek Cinema in rural Alabama. This report provides the results of that comparison.

*Go to the Audyssey XT32 vs. Dirac Live Listening Comparison Report and Discussion Thread.*


*Spatial Hologram M1 Turbo Speakers*

I was very pleased with the Spatial Hologram M1 speakers we used for the amplifier evaluation, and felt that they more than fulfilled our needs. They did not become "gotta have them" items for any of the evaluators, although I had thoughts in that direction once or twice. But they were speakers we could easily ignore through the weekend. I mean this as a high complement. Never did an evaluator complain that the M1 speakers were "in the way" or "holding us back," and we were able to focus on the task at hand unhindered. That alone means a lot, and may say more about them than the rest of the review just completed.

Here is what they did for us:

Because of their high efficiency, amplifiers were not straining to deliver the volumes we called for. We could be confident that the amps were operating in their linear ranges and that if we heard a difference it was not due to an amp being overdriven.
The stretched-out soundstage opened up a lot of useful detail for us to consider in our evaluations. In discussing the soundstage at one point, there was a consensus that it might be stretched a little too far and might be "coming apart at the seams," showing some gaps, although this did not hinder our progress. My final assessment is that this was not the case, all due respect to the fine ears of the other evaluators. I elaborate on this point in the M1 Review.
They served well as a full-range all-passive speaker, able to reach deep and deliver 40 Hz frequencies with lots of clean "oomph," all without the need for DSP boosting and without subwoofer support.
I thoroughly enjoyed spending time with them, and wish to again thank Clayton Shaw of Spatial Audio for loaning them to us. A complete review of the M1 speakers has been posted.

*Go to the Spatial Hologram M1 Turbo Version 2 Speaker Review.*


*A Soundstage Enhancement Experience*

Sonnie's MartinLogan ESL hybrid electrostatics were set up very nicely when we arrived, so we avoided moving them through the weekend. There were some improvements made to the soundstage and imaging by way of treatments, and some interesting twists and turns along the way which turned out to be very informative.

I have documented the exercise in a separate post.

*Go to the Soundstage Enhancement Experience thread.*


----------



## lcaillo

Thanks, Wayne, for starting the thread so nicely and for all of your work, both technically and writing. 

Another amazing weekend with an awesome group! 

This set of reviews is sure to generate lots of questions and debate. Like the first speaker event, everyone needs to understand that we are not attempting to provide the answers to the perpetual great debates nor tell anyone what they should own. We are taking a journey down a path that interests us and asking questions that are meaningful to US in the context of how we enjoy music. Each of us has our preferences, beliefs, biases, and likes. We try to be open about what those are and, while we go to great lengths to set those aside and learn something, we don't apologize for who we and what we believe and like. We have no intention of competing with anyone nor offending anyone and have no agenda but to learn and play.

I try to make it clear where I start in terms of assumptions, expectations, beliefs, and experience. I am at heart an experimenter and look for explanations for everything. I also like to set aside all of the technical stuff and my attempt to understand the why and just experience the joy of people creating and performing great music. So you get two very different pictures if I am successful at communicating my experiences in these sessions. First, the more objective attempt to understand the performance of the equipment. Second, you will hear me speak completely subjectively about what I feel when listening. For me both are essential but I know that the former will never satisfy me, while the latter does. Ironically, I believe that the latter also yields some of my best assessments of the equipment. That will make the objectivists crazy, but as I said above, I don't do for anyone's approval. 

I come to this weekend believing that we will likely be able to hear differences between amps, but far fewer than most audiophiles would report. I think we will find more differences in open comparisons than we can validate with blind testing. I believe that blind testing makes it very difficult to confirm differences but at the same time the characteristics of amplifiers that are reported by many reviewers are far exaggerated and unrealistic. I come with assumptions that some of the amps are better sounding by a slight margin but I don't expect that I know which ones they are. I came expecting more out of the Pass and Krell than others, but just based on prior listening to other products from those companies and my appreciation for the designers.

We will see what happens...the first day of listening has been interesting, with some differences noted in sighted listening and no differences in some comparisons. We'll see tomorrow how those observations hold up to blind tests.


----------



## AudiocRaver

Those steaks we had tonight at Preston's in Luverne, Alabama were epic. My ribeye was one of the best steaks I have ever eaten. Yes, that was a shameless plug.


----------



## Talley

Taste of Texas... Come to Houston and stop there. Just sayin'


----------



## Peter Loeser

*Home Theater Shack 2015 High-End Steak Evaluation Event Reporting and Discussion Thread*​



There, I fixed it :drool:


----------



## Peter Loeser

Talley said:


> Taste of Texas... Come to Houston and stop there. Just sayin'


Yes - a must if you visit Houston.


----------



## tonyvdb

Peter Loeser said:


> *Home Theater Shack 2015 High-End Steak Evaluation Event Reporting and Discussion Thread*​
> 
> 
> 
> There, I fixed it :drool:


 :rofl: :rofl: :rofl:

Gotta love that!


----------



## willis7469

Peter is on point today!


----------



## AudiocRaver

Peter Loeser said:


> Yes - a must if you visit Houston.


Taking notes...


----------



## JBrax

AudiocRaver said:


> Taking notes...


 If you were to come to KC the list is long…


----------



## AudiocRaver

Sonnie the prankster showed up tonight with his Oppo remote app on his iPhone, messing with the Oppo track selection while the listener trid to select and listen to tunes. After supper, the seriousness level often takes a bit of a downturn.


----------



## bkeeler10

AudiocRaver said:


> Sonnie the prankster showed up tonight with his Oppo remote app on his iPhone, messing with the Oppo track selection while the listener trid to select and listen to tunes. After supper, the seriousness level often takes a bit of a downturn.


Haha you guys are having too much fun. Wish I could be there, even if just a fly on the wall (couldn't eat very much peach cobbler in that case though . . .)


----------



## tesseract

The Cedar Creek Cinema is delivering the best sound I've ever heard in a home, Angie and Sonnie are the best hosts one can imagine, my listening companions are the best group of enthusiasts one could assemble for a weekend of A/V fun, Preston's ribeye was the best steak I've ever had, and Sonnie was thrashing my little Exposure into his EM-ESLs for a long time, longer than he spent with any other amp, thus far. Of course, that means he likes it best and the evaluation is now, for all intents and purposes, over and we are just going through the motions. Right? Right!

I am in the subjective camp, feeling that I can perceive small differences between most amplifiers, but also feel that these differences are largely overblown, in general. Today has confirmed that feeling, for me. Save one instance, I have not found any large differences that, blinded, I would be willing to put money against were I a betting man.


----------



## Talley

tesseract said:


> The Cedar Creek Cinema is delivering the best sound I've ever heard in a home, Angie and Sonnie are the best hosts one can imagine, my listening companions are the best group of enthusiasts one could assemble for a weekend of A/V fun, Preston's ribeye was the best steak I've ever had, and Sonnie was thrashing my little Exposure into his EM-ESLs for a long time, longer than he spent with any other amp, thus far. Of course, that means he likes it best and the evaluation is now, for all intents and purposes, over and we are just going through the motions. Right? Right!
> 
> I am in the subjective camp, feeling that I can perceive small differences between most amplifiers, but also feel that these differences are largely overblown, in general. Today has confirmed that feeling, for me. Save one instance, I have not found any large differences that I would be willing to put money against were I a betting man. That is the best way I can describe my take on an endeavor such as this.


Ok... then take the earplugs out and try again :rofl:


----------



## AudiocRaver

We spent Friday getting acquainted with the amps, listening to different amp pairs seeing if we could perceive differences. Most of us believed we could hear some subtle differences in some cases. In two cases, measurements showed there were differences that could be audible.

Last night we had some fun time comparing the ESLs with the Holograms. I am surprised how much alike they sound. Both have that easy, effortless clarity about their delivery that I have grown attached to. We have driven them pretty hard, and neither has shown signs of getting tired or holding back. Both are delivering monstrous soundstage with incredibly sharp imaging.

The Holograms have been our primary detail microscope for amp evaluation. Their wide soundstagre helps separate individual sounds and lets us hear the finer points of detail.

For the most part, the differences we have perceived have been impressions, not extremely specific. Today we will have a chance to try to confirm them.

A couple of us have noted that we heard no differences in the last 2 or 3 pairings yesterday, and wonder if that was a result of fatigue. So we may reverse the order of some pairings so we are hearing those last amps from yesterday with fresher ears today.

Today we will do blind testing for differences we thought we could hear yesterday.

Lots to do, must get busy.


----------



## Sonnie

I have conceded that my ears are inferior to these other guys. I literally cannot hear any differences between any of these amps thus far, even in some cases knowing there was a 2-3dB difference in a couple of the areas of the frequency response. At times I think I can hear a ever so subtle difference, but then I can't seem to repeat it with any consistency. 

Being the above situation... it hardly serves any purpose for me to be a listener in the ABX blind testing round. Therefore I will be the setup guy for the blind testing. I have setup the following two amps as Amp A and Amp B:


*DO NOT OPEN... DO NOT CLICK THIS BUTTON! >>>>>* 



Seriously? You really thought I was going to tell you what two amps they are? NOT!!!


Each of the four blind panelists will listen to X. X may be A or B ... and can be different for each panelist. After each panelist listens to X, I then switch the ABX box to Amp A. The panelist then gets to listen to Amp A and Amp B ... and can switch freely between the two amps. Each panelist will attempt to determine two things: 1. Which Amp was X ... and 2. Did they notice any differences between Amp A and Amp B. At the end of all testing... amps will be revealed and notes compared. In some cases I will pair the same two amps as yesterday so they can compare their notes from yesterday (knowing which amp was Amp A and Amp B) to their notes today (not knowing which amp was Amp A or Amp B).

Let the fun begin!


----------



## Sonnie

They are all scratchin' that head... I can tell I got them all tangled up and confused now. :heehee:


----------



## lcaillo

I can say definitely that x sounds good.


----------



## Talley

Sonnie said:


> They are all scratchin' that head... I can tell I got them all tangled up and confused now. :heehee:


haha... now THIS is getting interesting


----------



## Tonto

I've always felt there was more to be obtained with upgrading the speaker as opposed to the amp. If you have enough power to deliver the headroom, a good speaker will deliver. And treating the room is also paramount. Isn't it nice how good Sonnie's room preforms? Makes these evals better.


----------



## tesseract

lcaillo said:


> I can say definitely that x sounds good.


Agreed, I am a HUGE fan of x, thus far.


----------



## tesseract

Time to decompress...

View attachment 85202


----------



## bkeeler10

Hey where'd y'all go ...  Stayed up all night "decompressing" perhaps.


----------



## AudiocRaver

Whew! What a weekend!

Status: We got through our ABX test. Results are being compiled. And analyzed. And interpreted. And the mainframe is still cranking on the answer. Apparently there is some number crunching involved.

At one point the mainframe stopped as though it had an answer for us, but it turned out that it needed clarification on the question. Something about the answer being quite simple, something related to the number 42, but the question really needed to be defined properly, and that would take awhile. And maybe a bigger computer.

Leonard is in charge of all of that, and it might be a day or two, or three, possibly four, or maybe something more than five.... OK it will be A BIT before the results are published.

As of this moment... Joe and Leonard on on their ways home to Wisconsin and Florida, respectively. Travel safe, fellas. Sonnie is napping, Dennis is relaxing/posting/computing/napping. The weekend is not over, though, there is work that will continue through Monday.

A few thoughts on blind testing, and the weekend so far:

There are a lot of ways to attack blind testing, and it is not a simple creature to master.
Like anything else, it can be fun in the right company. Check the egos at the door, approach it in a supportive, friendly atmosphere, and a group can have a fun time while getting a lot accomplished.
These guys - Sonnie, Leonard, Joe, Dennis - are an unbelievably great bunch of people to work with and play with.
Expensive electronic toys are cool!
Expensive audio toys are REALLY cool!
Expensive audio toys can be frustrating!!!!!!
There are a lot of ways to look at value.
Good sound and good music and good company and a fun, tough, technical audio project all mixed together for a weekend make for a high that is pretty hard to beat. 
Imagination is a wonderful thing, you know, the furnace of creativity and all, and it can be your friend, but it can lead you off in weird directions if left unchecked. It all depends on what you are trying to accomplish.
Sleep is good.
There is some detailed speaker evaluation work to be completed. And some other testing. It is funny how our TODO list never gets all checked off and finished, It just gets continually rewritten.

Will be posting some photos shortly...


----------



## AudiocRaver

A question for your consideration:

When you have a typical get-together of family &/or friends over and you sit down together to enjoy a movie or some music on your system, how many of them typically care about and can appreciate really good sound quality?


----------



## Talley

AudiocRaver said:


> A question for your consideration:
> 
> When you have a typical get-together of family &/or friends over and you sit down together to enjoy a movie or some music on your system, how many of them typically care about and can appreciate really good sound quality?


Good question. I know in my family and my friends I am the only nut who "really" cares about the quality of the sound. My wife thinks the TV speakers sound great and my friends think a $2-300 soundbar sounds amazing.


----------



## bkeeler10

AudiocRaver said:


> A question for your consideration:
> 
> When you have a typical get-together of family &/or friends over and you sit down together to enjoy a movie or some music on your system, how many of them typically care about and can appreciate really good sound quality?


It's almost always family that we have over. While most of them appreciate the experience of a movie on a good system, none of them care enough about it to invest any money creating something like it for themselves. And none of them are terribly interested in two channel that I can tell - only movies and maybe concerts with video.

We are a strange, unusual and rare breed I'm afraid.


----------



## willis7469

I'm afraid we are bkeeler. My experience is the same. Mostly family, and most of them have been around long enough to have expectations. New guys(family or friends) are the ones I like. I'm not about to pound on my chest,as my system is modest, but no one as a first time visitor has not said WOW. (Just nice maybe?). Again, like bkeeler said, not many want to take on the variety of investments. It's for me anyway, but I still enjoy sharing it.


----------



## tonyvdb

AudiocRaver said:


> A question for your consideration:
> 
> When you have a typical get-together of family &/or friends over and you sit down together to enjoy a movie or some music on your system, how many of them typically care about and can appreciate really good sound quality?


I have a handful of friends and relatives who come over from time to time who have said in conversation to others that our home Theater system is the best they have heard. One cousin in particular loves coming over with her husband to watch movies, we even had a Super Bowl party in the theater room last month. 
I have one family friend who owns a production rental business (sets up, runs and rents out sound, video and lighting equipment for big concerts here in Edmonton) who has come over twice and loves my system. 

I im never one to brag because I know that when I go over to other people's houses I enjoy what they have and never compare systems.


----------



## AudiocRaver

Relative to Audyssey and Dirac Live and other room correction technologies, we have had some interesting discussion about the calibration mic pattern and the focus on sound quality at the MLP vs other seats.


----------



## lcaillo

Well, I don't think this forum was ever meant to be appealing to the mainstream. Whether doing room analysis or building theaters or playing with two channel, most of us are interested in things that the majority is likely not. The group we assembled this weekend is surely on a different road than the majority of consumers as well as the majority of audiophiles. We share an unending curiosity and a unique willingness to challenge our own assumptions.

So what did we do and what did we accomplish? We listened to 11 amplifiers in paired comparisons under both sighted and blind conditions, using ABX comparisons in the blind assessments. We collected our observations both about the comparisons and the process, as well as collecting sweep data with REW. 

Before we start getting into the results, let me be clear that we are not interested in pleasing anyone, and suspect that both sides of the "do amps sound different" debate will be largely unsatisfied with the results. These debates can get heated, and those who have been with us any time at all understand that we will not tolerate condescension, sarcasm, know-it-alls, nor any of the typical vitriol that is found elsewhere. We can have debates and even disagree strongly while being respectful of the right of others to have an opinion or perspective that differs from our own. We don't hesitate to "ask" those who don't get it to move on to another venue.

As Wayne said, we still have some work to do in terms of how we frame the results and the question(s) that we addressed. Not because we are trying to massage the results, but because it is very difficult to compile and make sense of all of the observations and to decide whether they are consistent across listening tests and/or individuals.

I previously stated my biases, assumptions, and beliefs. I don't like to speak for others but I don't think there is much doubt that we are on the same page. We went in assuming that we would perceive some differences, but that it would be very hard to support them in blind testing. Knowing this, we proceeded with ABX testing anyway, as we would like to come to some conclusions about the reliability of what we think we hear.  We know that it is dependent on many factors and many of them have nothing to do with actual performance differences. Still, with so many different designs and price ranges, it seems inevitable that there may be some differences. 

So what were the outcomes? I'll briefly satisfy the naysayers by saying that we completely failed to identify amps consistently in ABX comparisons (except for Dennis, who was correct in 5 of 7 tests). Overall, however, we were correct only 39% of the time, worse than chance. That said, there was evidence of something else going on. Dennnis and Joe both identified a couple of the amps in the blind testing from their experience with them the day before. They made notes to that effect during the blind testing. All four of us reported difficulty assigning what we heard as differences to the X amp, even though a number of our listening observations were consistent between the sighted and blind comparisons. It is quite easy to forget which was X in these comparisons. The testing process needs some work, and we have some ideas about how to proceed in the future to come up with more useful results.

We also did sweeps on the response from the speakers and with few exceptions, everything was similar enough to not expect frequency response to be audible. 

We will be publishing the details of the tests and results, including the subjective assessments that were consistent across testing modes and listeners. It will take some time to go through all of the notes from 4 listeners for 7 blind comparisons and 6 sighted comparisons.

In the meantime, let the civil discourse begin.


----------



## Talley

lcaillo said:


> In the meantime, let the civil discourse begin.


Krell > *

lol


----------



## chashint

Very interesting preliminary report in that a panel member was successful in identifying amplifiers in 5 out of 7 tests.
IMO that is a completely unexpected result, looking forward to the details.
Thanks in advance to all that have donated their time, money, and expertise into this project.


----------



## Talley

chashint said:


> Very interesting preliminary report in that a panel member was successful in identifying amplifiers in 5 out of 7 tests.
> IMO that is a completely unexpected result, looking forward to the details.
> Thanks in advance to all that have donated their time, money, and expertise into this project.


Thats not what he said. Dennis was the only one who called out correctly 5 of 7 amps. The rest of the group was a hit/miss of 40% correct. 

Knowing this that leaves a portion of that 40% off to a "good guess"...

based on these results...

1. The backbone of the system might need to be supportive more to help distinguish differences (power, isolation, conditioning, cords, etc) which could attribute to help reveal these differences (I know this is flame territory speaking of this
2. Considering the 40% correct the realistic results is around 25% of people can hear differences
3. I want Dennis' ears


----------



## willis7469

Talley said:


> Thats not what he said. Dennis was the only one who called out correctly 5 of 7 amps.


 actually he said Dennis was right in 5 out of 7 tests.


----------



## Talley

willis7469 said:


> actually he said Dennis was right in 5 out of 7 tests.


ya that, sorry.


----------



## chashint

Talley said:


> Thats not what he said. Dennis was the only one who called out correctly 5 of 7 amps. The rest of the group was a hit/miss of 40% correct.


Ok Talley what he said was......we completely failed to identify amps consistently in ABX comparisons (except for Dennis, who was correct in 5 of 7 tests......

What I said was.........a panel member was successful in identifying amplifiers in 5 out of 7 tests.......

I really just don't see anything to call me out on in the post.
I did not take any of it it out of context.


----------



## AudiocRaver

I will say that for me the difference between sighted AB comparison and the blind ABX test was big. With a number of pairings during sighted comparison, I believed I could hear _subtle_ differences between the amps, even had a couple of "I _like_ that amp" moments, but the ability to carry that level of discrimination over to a the blind ABX test the way we ran it eluded me and I did not do well there. Leonard can fill in the details when he has them ready.


We did quite a bit of work yesterday with the Spatial Holograms. It is funny how a single track can reveal things that no other track seemed able to. The mandolin and guitar on _The House Of Tom Bombadil,_ by Nickel Creek, were giving me fits yesterday due to the widening dipole dispersion pattern and lower-frequency reflections off the front wall under the movie screen. We think we figured it out, though, and properly placed absorptive panels came to the rescue. We will do a little more experimenting there and finish the Hologram review. Then we will put a few finishing touches on Sonnie's ESL setup and call it a day.

Sonnie, Dennis, and I watched Abe Lincoln, Vampire Killer last night. Fun film, and a fun room to watch it in. You have probably seen pics of the stucco job he did on his walls. It is _beautiful._


----------



## Talley

chashint said:


> Ok Talley what he said was......we completely failed to identify amps consistently in ABX comparisons (except for Dennis, who was correct in 5 of 7 tests......
> 
> What I said was.........a panel member was successful in identifying amplifiers in 5 out of 7 tests.......
> 
> I really just don't see anything to call me out on in the post.
> I did not take any of it it out of context.
> Please put me on your ignore list and do not quote me or respond to anymore posts I make.
> Thank you in advance.


I'm sorry I mis-read your post... I was in NO way disrespectful with it nor had any intentions of causing a ruckess. You can see my quote below that I did say "Ya that sorry". 

I'm sorry you feel the need for me to be "ignored" but I really don't see why but it implies that I've done something wrong and that I have not.



Talley said:


> ya that, sorry.


----------



## Sonnie

Keep in mind that we really needed more time to repeat the testing and see that identifying an amp can be repeated consistently among the same two amps over and over. This is one testing method we did not have time for. Not trying to discredit the ears of Dennis, but there was a chance that any one of these guys could have gone 7 of 7... or 0 of 7. We were more or less just having fun with this round and learning more about ABX testing. 

We could have had more expensive speakers... more expensive cables... and a power plant in the back yard to help improve in the possibility of hearing differences... but we didn't. We can speculate all day long, but we had what we had and we did what we did... and as Leonard noted, not to please anyone here, but to have fun. We are sharing our results because we can and because we know there are some interested in seeing the results, but we could really care less for those who want to poke holes in it for whatever reasons.

What I ultimately took away from this was that if I feel I need an amp to power my speakers outside of a receiver, which I do because I have clipped my AVR amp on my speakers, then I need to find the least expensive amp I can find with the minimum power I need... and call it a day. I personally see absolutely zero reason to spend a lot of money on an amp. That is no way implies the same will be true for you... I simply proved for myself what options are best for my ears. :T


----------



## Peter Loeser

Seems there is a decent amount of evidence to support the theory that differences between amps _can_ be heard. The evidence also suggests that there are select few who can detect the differences by ear, and under very specific conditions. Which means... the debate will continue!

Seriously though, many thanks to this crew for putting so much thought and effort into the process. It has been very entertaining and informative so far. I'm looking forward to seeing the detailed results.

Edit: I will add that I pretty much agree with Sonnie. Although I _think_ I have been able to sense subtle differences between amps, I have no proof that I could do it consistently. Spend what you need to get enough power to avoid clipping and you should be good to go.


----------



## Lumen

AudiocRaver said:


> A question for your consideration:
> 
> When you have a typical get-together of family &/or friends over and you sit down together to enjoy a movie or some music on your system, how many of them typically care about and can appreciate really good sound quality?


What can I add that hasn't already been said? 
Family/visitors impressed with picture and/or sound... CHECK!
Family/visitors unwilling to invest in quality components... CHECK!
Family/visitors put off by system complexity and calibration... CHECK!

But how about:
Visitors/family who don't watch/listen as if in movie ther or music hall?
Visitors/family who like to watch movies in daylight or lights on?

In other words, some people don't treat the performance as a primary activity. They repeatedly get up and move around, make annoying noises, narrate, etc., etc, etc. _I suppose that's okay with certain types of performances like sports, but it can drive you up the wall if you're the type that likes to suspend disbelief and immerse yourself in the presentation._

*To relate this back to the amplifier comparison,* the participants recognized the difference between background/distracted listening and focused evaluation. And oh, how those distractions satisfied! Yummy food, and pleasantly rewarding company! :TT


----------



## Talley

Sonnie said:


> Keep in mind that we really needed more time to repeat the testing and see that identifying an amp can be repeated consistently among the same two amps over and over. This is one testing method we did not have time for. Not trying to discredit the ears of Dennis, but there was a chance that any one of these guys could have gone 7 of 7... or 0 of 7. We were more or less just having fun with this round and learning more about ABX testing.
> 
> We could have had more expensive speakers... more expensive cables... and a power plant in the back yard to help improve in the possibility of hearing differences... but we didn't. We can speculate all day long, but we had what we had and we did what we did... and as Leonard noted, not to please anyone here, but to have fun. We are sharing our results because we can and because we know there are some interested in seeing the results, but we could really care less for those who want to poke holes in it for whatever reasons.
> 
> What I ultimately took away from this was that if I feel I need an amp to power my speakers outside of a receiver, which I do because I have clipped my AVR amp on my speakers, then I need to find the least expensive amp I can find with the minimum power I need... and call it a day. I personally see absolutely zero reason to spend a lot of money on an amp. That is no way implies the same will be true for you... I simply proved for myself what options are best for my ears. :T


Well one of the main reasons why I was curious of this testing was this very fact... While I am leaning more toward the fact that amps make little audible difference when properly levele matched I do feel after reading and NOW agree that the biggest differences in amps that are audible are when they are driven loud are are driving into distortion.

I've read that and based on the results of these tests and some other publications I really do feel like amps make a minimal amount of differences IF driven into non distorted levels. This is solid state vs. solid state of course.. I do think tube amps sound different.

One of the only reasons I have a separate is the fact I got it for a decent price and that was that. After checking the actual test data my Krell can drive all channels into 108 watts at 8 ohms at .1% distortion and 136w at 1%... this is the actual test data of driving ALL channels at the same time.

Even checking current AVRs it takes a 2,000+ dollar AVR (typically) to be able to produce the same specs. My X4000 driving 5-6 channels only produces some 68 watts at 1% which basically means I can play louder and cleaner. than my AVR alone. I spent 2450 total on my AVR and amp which yes that could of bought me a high end AVR....

only issue is a high end AVR would be 1/4 the price in three years where my Krell value will hold true since it's market keeps the pricing current. 

Now... something like the Outlaw 5 channel amp would be on my radar to test.... considering it's rated 200w each channel at .1% which gives me some 90w more headroom. Would love to hear that when playing loud and seeing if there are any audible differences.


----------



## Talley

Peter Loeser said:


> Spend what you need to get enough power to avoid clipping and you should be good to go.


THIS!!!

I think this pretty much sums it up. Sonnie said himself he has clipped the amp. Me... I've ran these SVS ultras to 105db playing pink floyd with the X4000 in direct mode and the volume set to +8db and never heard of any strain at all.

Do I hear my room... ya it sounds like . But the speakers played loud and strong. Very happy.

Now... I will never play that loud again... the was LOUD wow. Toooooo loud. If I can play 90db with bass peaks around 98-100 then thats still plenty loud that I will ever need to go.

So for me, the price... me having seperates gives me plenty of headroom which should be everyones goal. More headroom = less distortion. and distortion is audible.


----------



## tonyvdb

Peter Loeser said:


> Seems there is a decent amount of evidence to support the theory that differences between amps _can_ be heard. The evidence also suggests that there are select few who can detect the differences by ear, and under very specific conditions.


LOL yup this debate will never die

I do think it does show though that the differences are subtle at best. In normal listening conditions it probably would be even less noticeable. 

Thanks for taking the time to do this guys :T


----------



## Lumen

Not arguing that one or the views is absolute and carved in stone. But I thought the following quote from Lonnie Vaughn, Chief Technical Officer at EMOTIVA, offers a decidedly non snake-oil explanation for why amps sound different:

_"Amplifiers do sound different for a number of reasons. Paul cites a good one, the square wave response. On a personal note, I believe the power supply plays a big part in the way an amp sounds. I get that the power supply accounts for the biggest part of an amplifiers cost. But so many companies take it down to the bare minimum required to meet the specs that there is no headroom in the system at all and the amp itself just sounds flat as a board. In these cases, all the designer had to do was put in a few dollars more in storage and it would have made it a completely different unit." _(Taken from an article on this webpage)


----------



## JBrax

It's very interesting reading results/impressions of this testing and as others have said thanks for doing it. I've often wondered if adding a dedicated amp to my setup would provide any audible improvement. To be honest with my speakers I highly doubt it but with harder to drive speakers I think it would. If I were to add an amp it certainly wouldn't be a boutique name brand but something along the lines of Emotiva's price range. Oh those pretty blue lights are oh so tempting!


----------



## Talley

BlueRockinLou said:


> Not arguing that one or the views is absolute and carved in stone. But I thought the following quote from Lonnie Vaughn, Chief Technical Officer at EMOTIVA, offers a decidedly non snake-oil explanation for why amps sound different:
> 
> _"Amplifiers do sound different for a number of reasons. Paul cites a good one, the square wave response. On a personal note, I believe the power supply plays a big part in the way an amp sounds. I get that the power supply accounts for the biggest part of an amplifiers cost. But so many companies take it down to the bare minimum required to meet the specs that there is no headroom in the system at all and the amp itself just sounds flat as a board. In these cases, all the designer had to do was put in a few dollars more in storage and it would have made it a completely different unit." _(Taken from an article on this webpage)



Which is why all the higher end amps that have higher end costs have the big toroidal style transformers. My krell has a 1600w toroidal supply. cheaper units use standard transformer winding types. Or at least it's one of the differences I notice. does that make an audible difference... well who knows. The Denon tested has a normal non toriodal type transformer and that ended up falling last by the posts that was made?.... is that still true?


----------



## tonyvdb

Talley said:


> Which is why all the higher end amps that have higher end costs have the big toroidal style transformers. My krell has a 1600w toroidal supply. cheaper units use standard transformer winding types. Or at least it's one of the differences I notice. does that make an audible difference... well who knows. The Denon tested has a normal non toriodal type transformer and that ended up falling last by the posts that was made?.... is that still true?


And thats where your not necessarily right. I have two different Samson amps both have nice sized toroidal transformers in each and neither of them cost me more then $350 new. 
Onkyo has also used toroidal transformers in there high end receivers as well for years.


----------



## Talley

tonyvdb said:


> And thats where you are wrong. I have two different Samson amps both have nice sized toroidal transformers in each and neither of them cost me more then $350 new.
> Onkyo has also used toroidal transformers in there high end receivers as well for years.


I'm not wrong.... I know that they are found in the lower equipment but what I was saying is they are always found in the higher end stuff and are used as advertisement purposes. I know it shouldn't be, you can get a 2000w piltron toridal supply for 300 bucks and thats usually in the 2,000-5,000+ dollar amps.

and yes you get into the 900 dollar + onkyos and they have em. my comment was cheaper unit = sub 900 bucks and lower they don't TYPICALLY have them. maybe some of the emotivas do and thats where they stand out as price/value.

the Denon X5200 does not have one.


----------



## tesseract

Hey guys, a few things I'd like to add.

1. Yes, my results could be chalked up to chance.
2. 5 out of 7 is statistically significant and should not be ignored.
3. I admitted to fatigue at toward the end if the testing.
4. I did not start to miss until the end of the testing. My last few pairing evaluations took considerably more time due to this fatigue. 
5. There is only so much time in the day and the group dedicated DAYS to this test, so we did the best we could with the time we had.
6. I feel that I could repeat this test and expect a similar outcome.
7. Each person used different, non standard methods to evaluate the pairings. This was probably the biggest deviation from control in the entire evaluation effort. I am not certain that a standardized method would be useful as we all hear differently and like different music.
8. Our sighted evaluations had better soundstage and imaging properties than the blinded, due to the bulky (but necessary) shrouding used to hide the amplifiers from view. 
9. Audible differences in amplifiers are generally very small.
10. I have a favorite amplifier and will reveal that favorite after Leonard has crunched the numbers and posted the results.


----------



## tonyvdb

Talley said:


> I'm not wrong.... I know that they are found in the lower equipment but what I was saying is they are always found in the higher end stuff and are used as advertisement purposes. I know it shouldn't be, you can get a 2000w piltron toridal supply for 300 bucks and thats usually in the 2,000-5,000+ dollar amps.
> 
> and yes you get into the 900 dollar + onkyos and they have em. my comment was cheaper unit = sub 900 bucks and lower they don't TYPICALLY have them. maybe some of the emotivas do and thats where they stand out as price/value.
> 
> the Denon X5200 does not have one.


Lets take this discussion to a new posts so we dont derail this one. I have started one in the amplifier section


----------



## Blacklightning

Talley said:


> Which is why all the higher end amps that have higher end costs have the big toroidal style transformers. My krell has a 1600w toroidal supply. cheaper units use standard transformer winding types. Or at least it's one of the differences I notice. does that make an audible difference... well who knows. The Denon tested has a normal non toriodal type transformer and that ended up falling last by the posts that was made?.... is that still true?


This is the reason I'm not upgrading my AVR as it has a toroidal transformer (Marantz SR8500).
I noticed that toroidal transformers are being used less and less. Technology keeps changing and the $ is the driving force. People are not being fooled into buying high priced equipment any more so cheap tech is being updated like crazy. I'm sure Marantz's top regular transformer can outperform my receive now but my brain will not let me change as I'm still brainwashed in the old school way. :sad:


----------



## Lumen

Talley said:


> Which is why all the higher end amps that have higher end costs have the big toroidal style transformers. My krell has a 1600w toroidal supply. cheaper units use standard transformer winding types. Or at least it's one of the differences I notice. does that make an audible difference... well who knows. The Denon tested has a normal non toriodal type transformer and that ended up falling last by the posts that was made?.... is that still true?


I doubt that, BY ITSELF, dropping in a toroidal transformer will transport any amp into "higher-end" territory. Aside from trendy parts replacement such as toroids or fast recovery diodes, it's a particular power supply's DESIGN that _helps_ account for any possible audible differences. For example, a fancy transformer won't make up for poor ground return paths in the signal chain. 

*How concerned is our review panel* with, say, circuit topology, premium parts, and signal transfer of WBT vs Cardas vs plastic binding posts? Perhaps they'll become curious as they endeavour to explain slight differences. But I doubt they'll delve into a full blown, deep-dive exploration.

My point is that an amp with or without a toroidal transformer is only as good as the rest of its design. And even as you said, there are bang-for-the-buck amps out there that give the "big boys" a hefty run for the money.


----------



## Lumen

tonyvdb said:


> Lets take this discussion to a new posts so we dont derail this one. I have started one in the amplifier section


My apologies, our posts crossed in the mail.


----------



## Savjac

AudiocRaver said:


> A question for your consideration:
> 
> When you have a typical get-together of family &/or friends over and you sit down together to enjoy a movie or some music on your system, how many of them typically care about and can appreciate really good sound quality?


Only One, and that is kind of sad really.


----------



## Savjac

Having been through this a time or a hundred, I am so glad you gents are doing this. I had a listening session this weekend and it was fun.
These types of get together s are never easy and will take repeated attempts but are always fun and educational. I look forward to your findings, whether they agree with mine or not. 

Thank Y'all


----------



## lcaillo

tonyvdb said:


> LOL yup this debate will never die
> 
> I do think it does show though that the differences are subtle at best. In normal listening conditions it probably would be even less noticeable.
> 
> Thanks for taking the time to do this guys :T


Phrases like "normal listening conditions" are filled with assumptions. Normal listening conditions for someone who cares are very different than your average consumer. This is what Wayne was getting at.


----------



## lcaillo

I think the point Tony is making is that not all expensive amps have torroidal transformers. They offer some advantages in terms of hum and magnetic fields, but for the same price (and probably less space) you can get more power from a traditional transformer. 

Neither necessarily has much to do with the sound of an amp, other than perhaps not having enough power available, perhaps.


----------



## Talley

lcaillo said:


> I think the point Tony is making is that not all expensive amps have torroidal transformers. They offer some advantages in terms of hum and magnetic fields, but for the same price (and probably less space) you can get more power from a traditional transformer.
> 
> Neither necessarily has much to do with the sound of an amp, o*ther than perhaps not having enough power available, perhaps.*


I'm steadily growing on this bolded part. Power power power... more power... cleaner output. I'm beginning to wonder if 125w all channels driven if that really should be like 400w each channel that the Emotiva XPR-5 can deliver.

I can literally sell my Krell and buy a new Emotiva XPR-5 for the same price. Hmm....


----------



## NBPk402

Talley said:


> I'm steadily growing on this bolded part. Power power power... more power... cleaner output. I'm beginning to wonder if 125w all channels driven if that really should be like 400w each channel that the Emotiva XPR-5 can deliver.
> 
> I can literally sell my Krell and buy a new Emotiva XPR-5 for the same price. Hmm....


ore power is only needed if you can't get to the levels you want cleanly with your current amp. With my current setup I actually went down on power as I do not need it to hit reference levels.


----------



## Sonnie

Talley said:


> Sonnie said himself he has clipped the amp. Me... I've ran these SVS ultras to 105db playing pink floyd with the X4000 in direct mode and the volume set to +8db and never heard of any strain at all.


I clipped my X5200 with the Ultras. Perhaps I was listening at louder levels than 105dB... not sure. I plugged my Behringer EP2500 in the loop and there were no issues... problem solved. I wanted a better looking amp and got a really really good deal on a Parasound HALO A31, so I purchased it.




BlueRockinLou said:


> Not arguing that one or the views is absolute and carved in stone. But I thought the following quote from Lonnie Vaughn, Chief Technical Officer at EMOTIVA, offers a decidedly non snake-oil explanation for why amps sound different:
> 
> _"Amplifiers do sound different for a number of reasons. Paul cites a good one, the square wave response. On a personal note, I believe the power supply plays a big part in the way an amp sounds. I get that the power supply accounts for the biggest part of an amplifiers cost. But so many companies take it down to the bare minimum required to meet the specs that there is no headroom in the system at all and the amp itself just sounds flat as a board. In these cases, all the designer had to do was put in a few dollars more in storage and it would have made it a completely different unit." _(Taken from an article on this webpage)


I agree they will sound different if pushed really hard with certain speakers... in that one will clip, one will not. That is my experience... although someone else may hear something different for a different reason.




Talley said:


> The Denon tested has a normal non toriodal type transformer and that ended up falling last by the posts that was made?.... is that still true?


Not sure where you got that... the Denon was well received. We did not attempt to push it to the limits. I don't think any of us have ever doubted that you can overdrive an amp and that you need to make sure you have sufficient power to drive your speakers, therefore this event was not setup to test that aspect.





Talley said:


> I'm steadily growing on this bolded part. Power power power... more power... cleaner output. I'm beginning to wonder if 125w all channels driven if that really should be like 400w each channel that the Emotiva XPR-5 can deliver.
> 
> I can literally sell my Krell and buy a new Emotiva XPR-5 for the same price. Hmm....


"Sharp ears" Dennis liked two of the amps more than any others... wasn't all that impressed with the Emotiva "in comparison"... however, when he left he was highly considering purchasing an XPA-3 for his system and perhaps later considering one of his favorites. I think this echoes how subtle the differences were... in that even one of the amps that he did not favor "in comparison" as much as the others is still an amp he would have no issues owning and can still make very nice sound for his "sharp ears".


----------



## lcaillo

tesseract said:


> Hey guys, a few things I'd like to add.
> 
> 1. Yes, my results could be chalked up to chance.
> 2. 5 out of 7 is statistically significant and should not be ignored.
> 3. I admitted to fatigue at toward the end if the testing.
> 4. I did not start to miss until the end of the testing. My last few pairing evaluations took considerably more time due to this fatigue.
> 5. There is only so much time in the day and the group dedicated DAYS to this test, so we did the best we could with the time we had.
> 6. I feel that I could repeat this test and expect a similar outcome.
> 7. Each person used different, non standard methods to evaluate the pairings. This was probably the biggest deviation from control in the entire evaluation effort. I am not certain that a standardized method would be useful as we all hear differently and like different music.
> 8. Our sighted evaluations had better soundstage and imaging properties than the blinded, due to the bulky (but necessary) shrouding used to hide the amplifiers from view.
> 9. Audible differences in amplifiers are generally very small.
> 10. I have a favorite amplifier and will reveal that favorite after Leonard has crunched the numbers and posted the results.


Whether 5 of 7 is statistically significant depends on how you define it. By most research standards it would not be, as the probability of getting 5 of 7 if the probability of each choice is .5 would be about 84%. That means that there is a 16% probability of it being by chance. Most would consider 6 of 7 statistically significant, where the probability of it happening by chance is down around 5%.

I did not find that the imaging suffered in the blind tests. My big issue with the blind testing was time and confusion about trying to recall what X sounded like.

You have to consider our overall results, however, for 28 trials, to reach a 95% certainty we would need to be right 18 times. I think we could easily do that with more time and not restricting X for such a limited listening time and fewer amps.


----------



## Talley

One thing for sure... this was a major undertaking trying to accomplish all that you group has done.

Major Kuddos!


----------



## lcaillo

I think it is curious that even though we felt that we could hear differences in some comparisons, everyone came away convinced that the differences were so small, if real, that we would all opt for the cheapest amp that had the power we need.

I will be publishing the comments on individual amp impressions in the context of the comparisons made soon.


----------



## lcaillo

Talley said:


> One thing for sure... this was a major undertaking trying to accomplish all that you group has done.
> 
> Major Kuddos!


Indeed. It was a large investment in time and money, the latter more for Sonnie and the guys who travelled farther. The experience of working with this group is well worth it, however. We always have enormous fun and learn something, even if it is exhausting.


----------



## Talley

Sonnie said:


> I clipped my X5200 with the Ultras. Perhaps I was listening at louder levels than 105dB... not sure. I plugged my Behringer EP2500 in the loop and there were no issues... problem solved. I wanted a better looking amp and got a really really good deal on a Parasound HALO A31, so I purchased it.
> 
> Nice amp! The X5200 is rated around 120w to 5 channel driven so for you to tell me you were clipping tells me the power requirements of the Ultras I got from you. I may end up looking for something with more power behind it as a "just in case" scenario
> 
> 
> Not sure where you got that... the Denon was well received. We did not attempt to push it to the limits. I don't think any of us have ever doubted that you can overdrive an amp and that you need to make sure you have sufficient power to drive your speakers, therefore this event was not setup to test that aspect.
> 
> Understood I was speaking out of the side of my head.


Notes in red.

You guys really build some suspense making us wait for the end results lol


----------



## Talley

lcaillo said:


> Indeed. It was a large investment in time and money, the latter more for Sonnie and the guys who travelled farther. The experience of working with this group is well worth it, however. We always have enormous fun and learn something, even if it is exhausting.


Next time send an invite to me as the official photographer (my main passion)


----------



## Sonnie

tesseract said:


> Each person used different, non standard methods to evaluate the pairings. This was probably the biggest deviation from control in the entire evaluation effort. I am not certain that a standardized method would be useful as we all hear differently and like different music.


Absolutely... each person needs their own set of evaluation music and methods of comparing. 

What I do think we need to add next time is having more repetitive ABX comparisons between the same two amps, which Leonard suggested. This will eliminate any suggestion of chance... and about the only way to verify that it is not chance. There is no doubt there needs to be multiple blind rounds to eliminate chance, but we just didn't have time. We know subjectively you guys heard differences... so no real reason to do that next time. Next time it can all be blind.

I think our next gathering, if we can make it happen, we'll have 3-4 speaker reviews and only 2 amps for repetitive ABX'ing. Frank also said the popping noise during switching was not normal and I am sending it back to him for repair/replacement. That way next time someone can swap amps with the lights off and the listener can determine if amps were switched. Makes it more interesting.

I know we may not be able to get you back over here... if you are working, but maybe we can find someone who feels like they are seasoned in hearing differences to attend and and let them take a swing at it. We might throw in one set of cable comparisons too.

It may also be good to have those not testing to go outside the room while others test. During the blind testing you guys only spent 15-20 minutes on each amp comparison and that was spread out over the course of the day/night. So hearing the other music while others were listening apparently played a role in fatigue. Otherwise, I wouldn't think a couple of hours of listening over a 12 hour day should be that fatiguing.


----------



## tesseract

I like the idea of repeatability, and agree that leaving the room can help offset fatigue.

My first few evaluations went quickly, I was done in just a few minutes. Sitting and listening to others do their evals was fatiguing, and toward the end of the day, my evals were 3 to 4 times longer and my notes reflect my uncertainty at this point.

I feel confident that I could have picked up at least one more correct answer if I had not felt so weary toward the end. Those last few were VERY difficult.


----------



## tesseract

lcaillo said:


> Whether 5 of 7 is statistically significant depends on how you define it. By most research standards it would not be, as the probability of getting 5 of 7 if the probability of each choice is .5 would be about 84%. That means that there is a 16% probability of it being by chance. Most would consider 6 of 7 statistically significant, where the probability of it happening by chance is down around 5%.
> 
> I did not find that the imaging suffered in the blind tests. My big issue with the blind testing was time and confusion about trying to recall what X sounded like.
> 
> You have to consider our overall results, however, for 28 trials, to reach a 95% certainty we would need to be right 18 times. I think we could easily do that with more time and not restricting X for such a limited listening time and fewer amps.


I'm not sure, but I think 80% is considered significant in a subjective endeavor. I'll dig around for more info when I return home. Agreed that repeatability would make the data more robust. 

If we had more amps, it would likely have dinged my score down considerably. Less amps, I might have scored 100%. 

Remembering X was easy for me until the end. The next test will build on this one and I'm confident the group will have better data. I do hope I can attend, this was a blast and I liked working with such an intelligent bunch.


----------



## AudiocRaver

Peter Loeser said:


> Seems there is a decent amount of evidence to support the theory that differences between amps _can_ be heard. The evidence also suggests that there are select few who can detect the differences by ear, and under very specific conditions. Which means... the debate will continue!


Well put. Under a very carefully chosen set of conditions, I felt quite certain, for instance, that the sounds with one of those amps had more clean, open space around them than from the amp AB'ed against in sighted AB testing. If you make my getting supper tonight dependent on being able to pick that amp out in an ABX test or any other test, or even a repeat of the same test on another day / different room / feeling tired / _insert variable here,_ I am likely to go to bed a hungry boy.



BlueRockinLou said:


> What can I add that hasn't already been said?
> Family/visitors impressed with picture and/or sound... CHECK!
> Family/visitors unwilling to invest in quality components... CHECK!
> Family/visitors put off by system complexity and calibration... CHECK!
> 
> But how about:
> Visitors/family who don't watch/listen as if in movie ther or music hall?
> Visitors/family who like to watch movies in daylight or lights on?
> 
> In other words, some people don't treat the performance as a primary activity. They repeatedly get up and move around, make annoying noises, narrate, etc., etc, etc. _I suppose that's okay with certain types of performances like sports, but it can drive you up the wall if you're the type that likes to suspend disbelief and immerse yourself in the presentation._
> 
> *To relate this back to the amplifier comparison,* the participants recognized the difference between background/distracted listening and focused evaluation. And oh, how those distractions satisfied! Yummy food, and pleasantly rewarding company! :TT


I appreciate the feedback - from a number of you - to the question. It leads into some decisions that were made concerning our Audyssey XT32 / Dirac Live comparison. Which, by the way DID happen...

In spite of technical issues - these will be outlined in the report, because they are pertinent to those who might be making a choice between them.
In a different way than originally planned. Par for the weekend.
Choosing a very particular set of test conditions for a very particular set of reasons (cough, ahem, nudge, wink - see aforementioned discussion).
With great attention paid toward making it an apples-to-apples comparison.
With - sadly - a limited number of evaluator ears - sorry guys, we really tried, BELIEVE me we tried.
With fairly conclusive results, i_n my opinion_ (how is that for confidence?).
Lots of writing to do, but this report is HIGH priority, it will be separate from the miniDSP nanoAVR DL review, due this month - ooooohhhh, did I say that out loud?


----------



## AudiocRaver

Talley said:


> Well one of the main reasons why I was curious of this testing was this very fact... While I am leaning more toward the fact that amps make little audible difference when properly levele matched I do feel after reading and NOW agree that the biggest differences in amps that are audible are when they are driven loud are are driving into distortion.
> 
> I've read that and based on the results of these tests and some other publications I really do feel like amps make a minimal amount of differences IF driven into non distorted levels. This is solid state vs. solid state of course.. I do think tube amps sound different.
> 
> One of the only reasons I have a separate is the fact I got it for a decent price and that was that. After checking the actual test data my Krell can drive all channels into 108 watts at 8 ohms at .1% distortion and 136w at 1%... this is the actual test data of driving ALL channels at the same time.
> 
> Even checking current AVRs it takes a 2,000+ dollar AVR (typically) to be able to produce the same specs. My X4000 driving 5-6 channels only produces some 68 watts at 1% which basically means I can play louder and cleaner. than my AVR alone. I spent 2450 total on my AVR and amp which yes that could of bought me a high end AVR....
> 
> only issue is a high end AVR would be 1/4 the price in three years where my Krell value will hold true since it's market keeps the pricing current.
> 
> Now... something like the Outlaw 5 channel amp would be on my radar to test.... considering it's rated 200w each channel at .1% which gives me some 90w more headroom. Would love to hear that when playing loud and seeing if there are any audible differences.


I like that you have outlined a fairly specific set of factors that led to your choice. Someone else would use their own set of factors for their own reasons. Others might argue the validity of those factors. Bottom line, you did what made sense to you and you are happy. Excellent.



tonyvdb said:


> LOL yup this debate will never die
> 
> I do think it does show though that the differences are subtle at best. In normal listening conditions it probably would be even less noticeable.


I certainly agree. We drove Sonnie's ESLs pretty hard - during fun time - with a smaller amp and heard that _strain,_ and some clear breakup - both pretty unmistakable. Other than that, *subtle* is the key word.


----------



## AudiocRaver

BlueRockinLou said:


> I doubt that, BY ITSELF, dropping in a toroidal transformer will transport any amp into "higher-end" territory. Aside from trendy parts replacement such as toroids or fast recovery diodes, it's a particular power supply's DESIGN that _helps_ account for any possible audible differences. For example, a fancy transformer won't make up for poor ground return paths in the signal chain.
> 
> *How concerned is our review panel* with, say, circuit topology, premium parts, and signal transfer of WBT vs Cardas vs plastic binding posts? Perhaps they'll become curious as they endeavour to explain slight differences. But I doubt they'll delve into a full blown, deep-dive exploration.


Would that we had the time to do so. The amp choices were made with many such factors in mind, but detailed analysis of the kind of factors listed, as I am sure all can appreciate, was simply beyond our scope. It would have been SO interesting.



Savjac said:


> Only One, and that is kind of sad really.


Indeed, it is no much fun to share, and hard to find others who truly care.



lcaillo said:


> Phrases like "normal listening conditions" are filled with assumptions. Normal listening conditions for someone who cares are very different than your average consumer. This is what Wayne was getting at.


Precisely. "Typical" for us (including all present, probably) is "crazy / nuts / ridiculous" to most, And when we get serious, look out. This relates to the entire weekend and to numerous choices wes made along the way, and to our purpose for it and to our upcoming detailed results. Our own assumptions about listening were challenged many times over the weekend.


----------



## craigsub

Some (non-serious) observations ...

1. Are you guys insane? That's some serious work for a single weekend! 
2. Did anyone think we needed even more reasons for Dennis (Tesseract) to be cocky?
3. This amp shootout, while not necessarily intending to, somewhat showed why higher efficiency speakers are a good thing.
4. This amp shootout makes me want to get a pair of Martin Logans or Magnepans. 
5. Thanks to everyone for a really great read already, and looking forward to reading more.


----------



## chashint

^^^^^
Some very good observations


----------



## Savjac

craigsub said:


> Some (non-serious) observations ...
> 
> 1. Are you guys insane? That's some serious work for a single weekend!
> 2. Did anyone think we needed even more reasons for Dennis (Tesseract) to be cocky?
> 3. This amp shootout, while not necessarily intending to, somewhat showed why higher efficiency speakers are a good thing.
> 4. This amp shootout makes me want to get a pair of Martin Logans or Magnepans.
> 5. Thanks to everyone for a really great read already, and looking forward to reading more.


Some of these points ring very true indeed. Never the less, I think point three misses the point...so to speak. We need to acquire our amplification based upon the speakers we have in our rooms. While higher efficiency speakers can help if we just cannot obtain sufficient power to drive lesser efficient or difficult loads. Frankly even highly efficient speakers can benefit from serious power behind the cables. Try watching Fury or the depth charge scene in U571 for some extreme examples of needing power and an incredible reserve.


----------



## Talley

Well this has def opened my eyes and I'm not so hung up on Krell like I once was. Just happy that I have enough juice to power what I got at levels that I like.


----------



## Talley

Savjac said:


> Fury or the depth charge scene in U571 for some extreme examples of needing power and an incredible reserve.


I love Fury and although it doesn't hit really low... I think the bass is executed perfect and portrays realistic scenes.

The SVS Ultras and my Krell handled it nicely and at 0db on the reciever after audyssey checks was hitting 105db with the REW SPL meter (not sure at what frequencies). Not sure if this is reference or not but it was way louder than I liked the scenes. I like around -6db for the tank scenes.


----------



## Savjac

Talley said:


> I love Fury and although it doesn't hit really low... I think the bass is executed perfect and portrays realistic scenes.
> 
> The SVS Ultras and my Krell handled it nicely and at 0db on the reciever after audyssey checks was hitting 105db with the REW SPL meter (not sure at what frequencies). Not sure if this is reference or not but it was way louder than I liked the scenes. I like around -6db for the tank scenes.


I am glad you love the movie soundtrack, so do I. But in this case I am not speaking to the "low" in the sound but the impact supplied by the main speakers. This is where the impact of the tank shelling comes from and it will draw an awful lot of power to take command of the woofers and force them to do the right thing. I am sure the guts of a power supple would be severely taxed in these scenes. 
This will also put on display many of the short comings of some amps in that they may not have power reserves to keep up.


----------



## Talley

Savjac said:


> I am glad you love the movie soundtrack, so do I. But in this case I am not speaking to the "low" in the sound but the impact supplied by the main speakers. This is where the impact of the tank shelling comes from and it will draw an awful lot of power to take command of the woofers and force them to do the right thing. I am sure the guts of a power supple would be severely taxed in these scenes.
> This will also put on display many of the short comings of some amps in that they may not have power reserves to keep up.


Agreed... lots of midrange in that movie when the tanks fire off. I played the movie for a friend at -5db and he said the movie was perfect and wouldnt' want louder. SPL meter peaks were hitting around 101-102db but most of the movie bounced around the 85-95db range. I never noticed any strain.


----------



## Savjac

Most excellent. I have not heard your speakers but I am sure they can do some harm to your body.


----------



## tonyvdb

I think another thing that this session proves is that you do need more than just 30 - 50 watts of power to drive good speakers at comfortable listening levels. This also means that many low end receivers that drop to below 50watts per channel under load would fail at giving good levels of undistorted sound.


----------



## NBPk402

tonyvdb said:


> I think another thing that this session proves is that you do need more than just 30 - 50 watts of power to drive good speakers at comfortable listening levels. This also means that many low end receivers that drop to below 50watts per channel under load would fail at giving good levels of undistorted sound.


It depends on the speaker efficiency, and how loud you want it.:T


----------



## lcaillo

tesseract said:


> I'm not sure, but I think 80% is considered significant in a subjective endeavor. I'll dig around for more info when I return home. Agreed that repeatability would make the data more robust.
> 
> If we had more amps, it would likely have dinged my score down considerably. Less amps, I might have scored 100%.
> 
> Remembering X was easy for me until the end. The next test will build on this one and I'm confident the group will have better data. I do hope I can attend, this was a blast and I liked working with such an intelligent bunch.


The percent correct can only be said to be statistically significant when the acceptable level of error is decided and the number of trials is considered. 80% on 28 trials is very different from 80% on 7 trials.


----------



## chashint

Unless power ouput is measured it is only speculation on the actual values.
Lots of numbers can be bantered about, but unless you measure it you don't know what it is.
When the SPL is really high it is possible the speakers are the cause of the distortion as well as its possible the amp is the cause of the distortion.
Subwoofers go a long way towards amplifier relief.


----------



## AudiocRaver

Oh, and Sonnie and I spent some quality time Monday fine tuning the two channel soundstage with his MartinlLogan ESLs. Cannot believe I forgot to snap any pics of that. I will have him take a couple.

The results were pretty fine. Even got the density that gives super depth acuity. Thick as pudding.

He is a brave soul. A couple of times I admitted that a piece of the process might seem a little outlandish, and he always responded "as long as it sounde good." he seemed pleased with the final sound.

Part of what prompted this exercise was a wandering acoustic guitar on a Nickel Creek track We turned to placement of absorptive panels to control reflections, then to manipulating where the reflections fell, then on to other fine points. A report will follow.


----------



## 480dad

AudiocRaver said:


> Oh, and Sonnie and I spent some quality time Monday fine tuning the two channel soundstage with his MartinlLogan ESLs. Cannot believe I forgot to snap any pics of that. I will have him take a couple.
> 
> The results were pretty fine. Even got the density that gives super depth acuity. Thick as pudding.
> 
> He is a brave soul. A couple of times I admitted that a piece of the process might seem a little outlandish, and he always responded "as long as it sounde good." he seemed pleased with the final sound.
> 
> Part of what prompted this exercise was a wandering acoustic guitar on a Nickel Creek track We turned to placement of absorptive panels to control reflections, then to manipulating where the reflections fell, then on to other fine points. A report will follow.


Very much looking forward to this...pics would be awesome.


----------



## chashint

Scatter shooting...
For me one of the unexpected outcomes of this endeavor even before the "official" report is posted is the apparent sudden about face in opinion by some members on the premise that amplifiers sound so different from each other that they can be easily identified.
This type of testing has been done many times, one well known test had $10,000 on the line for anyone that "passed" .... The $10k stayed in the challenger's pocket...... Probably a good thing for the promoter that tesseract was not around when that challenge was going on.....
I wonder if this about face on the amplifiers will hold once the detailed report is posted or will there be a sudden retrenching with countless objections and justifications of how the test was wrong.
Another belief to ponder is in regards to magic power cords or specially designed speaker wires and interconnects.
If amplifiers are so similar they cannot be distinguished from one another, how can a wire change the sound enough to be distinguishable from any other wire?
Just pondering the mysteries of the universe this morning.
Enjoy your coffee.


----------



## Talley

chashint said:


> Scatter shooting...
> For me one of the unexpected outcomes of this endeavor even before the "official" report is posted is the apparent sudden about face in opinion by some members on the premise that amplifiers sound so different from each other that they can be easily identified.
> This type of testing has been done many times, one well known test had $10,000 on the line for anyone that "passed" .... The $10k stayed in the challenger's pocket...... Probably a good thing for the promoter that tesseract was not around when that challenge was going on.....
> I wonder if this about face on the amplifiers will hold once the detailed report is posted or will there be a sudden retrenching with countless objections and justifications of how the test was wrong.
> Another belief to ponder is in regards to magic power cords or specially designed speaker wires and interconnects.
> If amplifiers are so similar they cannot be distinguished from one another, how can a wire change the sound enough to be distinguishable from any other wire?
> Just pondering the mysteries of the universe this morning.
> Enjoy your coffee.


I read that article too. I heard it was still out there as an option. Tesseract should look into the details


----------



## Lumen

tonyvdb said:


> I think another thing that this session proves is that you do need more than just 30 - 50 watts of power to drive good speakers at comfortable listening levels. This also means that many low end receivers that drop to below 50watts per channel under load would fail at giving good levels of undistorted sound.





ellisr63 said:


> It depends on the speaker efficiency, and how loud you want it.:T


I once owned pair of KEF 105/3 speakers that were very efficient and played crazy-loud as advertised, but didn't really seem to "sing" until a certain power threshold was passed. I've also read about other speakers "behaving" the same way, but have no first-hand experience. *Do you believe such a phenomenon exists that requires higher amplifier power for speakers that don't play well quietly?*



lcaillo said:


> The percent correct can only be said to be statistically significant when the acceptable level of error is decided and the number of trials is considered. 80% on 28 trials is very different from 80% on 7 trials.


This sub-thread really interests me. I loved taking statistics courses, but can't remember much now as a senior citizen. Thanks for bringing back the memories! :nerd: :R



AudiocRaver said:


> Part of what prompted this exercise was a wandering acoustic guitar on a Nickel Creek track We turned to placement of absorptive panels to control reflections, then to manipulating where the reflections fell, then on to other fine points. A report will follow.


Fascinating stuff, and I'm truly on the edge of my seat to read your findings. I've been struggling a long time on my 2ch system with one of my test tracks ("Hotel California" from the Eagles _Hell Freezes Over_ album). The opening acoustic guitar from the right channel presents a couple of problems:
The guitar body (all the notes) seems to stretch toward the left and back again
The guitar body (individual notes) seems to split in two and notes become "tangled" 
Problem-1 changes with toe-in and distance to side wall. Problem-2 happens during the fast part of the solo. It gets better when acoustically treating the side wall, but never really goes away. I think comb filtering may be the culprit. It's time for some REW and miniDSP to the rescue! *I think I'm ready to toss an old audiophile reservation aside and take the plunge into adding an A/D-D/A conversion.*

Incidentally, early crowd noise nicely reveals the recording venue by relaying height and depth when my speakers and room treatment are properly dialed-in!



chashint said:


> Another belief to ponder is in regards to magic power cords or specially designed speaker wires and interconnects.
> If amplifiers are so similar they cannot be distinguished from one another, how can a wire change the sound enough to be distinguishable from any other wire?
> Just pondering the mysteries of the universe this morning.
> Enjoy your coffee.


Letting go of some audiophile beliefs can be hard depending on how deeply ingrained they are. But it is possible if the individual is open minded and susceptible to suggestion. And we all know who we are, don't we? :innocent: ...ponder, ponder... sip, sip

This is a true dog-ate-my-homework story where Sebastian (my border collie mix) chewed through the left channel interconnect running from my preamp to power amp. Granted I didn't do any official testing or level-matching, but I feel my musical enjoyment didn't suffer after substituting a much cheaper set I had laying around. Hmmm... how much $$$ have I wasted in my brainwashed youth??? :spend: :duh:


----------



## tonyvdb

BlueRockinLou said:


> I once owned pair of KEF 105/3 speakers that were very efficient and played crazy-loud as advertised, but didn't really seem to "sing" until a certain power threshold was passed. I've also read about other speakers "behaving" the same way, but have no first-hand experience. *Do you believe such a phenomenon exists that requires higher amplifier power for speakers that don't play well quietly?*


I know there was a test done a couple years ago here where they took a normal efficiency speaker and played several different instruments through it one at a time. Here is what was said, "the most dynamic power draw was during the Thwack of a snare drum dead centre" used 250 watts into speakers that were 89 db efficient. Clearly this would tax an amplifier if it was not able to keep up.


----------



## Talley

tonyvdb said:


> I know there was a test done a couple years ago here where they took a normal efficiency speaker and played several different instruments through it one at a time. Here is what was said, "the most dynamic power draw was during the Thwack of a snare drum dead centre" used 250 watts into speakers that were 89 db efficient. Clearly this would tax an amplifier if it was not able to keep up.


Wonder what the wattage draw is during the FURY tank scenes. my ultras are 88db so something tells me I need MO powa


----------



## Blacklightning

BlueRockinLou said:


> I once owned pair of KEF 105/3 speakers that were very efficient and played crazy-loud as advertised, but didn't really seem to "sing" until a certain power threshold was passed. I've also read about other speakers "behaving" the same way, but have no first-hand experience. *Do you believe such a phenomenon exists that requires higher amplifier power for speakers that don't play well quietly?*


As a trombone player I have heard this a lot, "sing", in the case of a trombone it is getting to a SPL level which causes the bell to ring. Different materials and thicknesses in the trombone will change how much or little a trombone will project and sing. The other factor is the venue and how much people are in the room. I have to play very differently it dress rehearsal vs a concert on a solo just because of the warm bodies in the room.

I wonder if a dead room needs more SPL to get that "sing" and if the room has too much energy it can cause your brain to go into overdrive.

Long story short, I would look at the room first before the speakers/Amp.


----------



## Savjac

BlueRockinLou said:


> I once owned pair of KEF 105/3 speakers that were very efficient and played crazy-loud as advertised, but didn't really seem to "sing" until a certain power threshold was passed. I've also read about other speakers "behaving" the same way, but have no first-hand experience. *Do you believe such a phenomenon exists that requires higher amplifier power for speakers that don't play well quietly?*


Absolutely, my B&W's did not sound good at quiet volumes, but with some power and some turnin up, they were great.


----------



## NBPk402

BlueRockinLou said:


> I once owned pair of KEF 105/3 speakers that were very efficient and played crazy-loud as advertised, but didn't really seem to "sing" until a certain power threshold was passed.


Try the same thing with speakers that are 10-20db more efficient than the KEFs, and you will see why I say big amps are not always needed.:T


----------



## Lumen

Blacklightning said:


> As a trombone player I have heard this a lot, "sing", in the case of a trombone it is getting to a SPL level which causes the bell to ring. Different materials and thicknesses in the trombone will change how much or little a trombone will project and sing. The other factor is the venue and how much people are in the room. I have to play very differently it dress rehearsal vs a concert on a solo just because of the warm bodies in the room.
> 
> I wonder if a dead room needs more SPL to get that "sing" and if the room has too much energy it can cause your brain to go into overdrive.
> 
> Long story short, I would look at the room first before the speakers/Amp.


Now THAT is eye-opening! We can get so caught up in the audio gear and music media end of things that we forget the SOURCE. Musicians have a unique perspective that should command more attention. Thanks for sharing!



Savjac said:


> Absolutely, my B&W's did not sound good at quiet volumes, but with some power and some turnin up, they were great.


HAH! There's strength in numbers. So far we have ...errr... two. :bigsmile:


----------



## NBPk402

Savjac said:


> Absolutely, my B&W's did not sound good at quiet volumes, but with some power and some turnin up, they were great.


I agree that there is a certain level that sounds best. but I think it is partly determined by how it is mixed.


----------



## tonyvdb

ellisr63 said:


> Try the same thing with speakers that are 10-20db more efficient than the KEFs, and you will see why I say big amps are not always needed.:T


My EVs are 96db efficient and at reference levels I am still running at at least half the amps output level as shown on the amps vu meters.


----------



## Lumen

ellisr63 said:


> I agree that there is a certain level that sounds best. but I think it is partly determined by how it is mixed.


That makes three :bigsmile: :bigsmile: :bigsmile:
HAH! Majority rules. 
We are right and you are wrong (j/k)



lcaillo said:


> I think it is curious that even though we felt that we could hear differences in some comparisons, everyone came away convinced that the differences were so small, if real, that we would all opt for the cheapest amp that had the power we need.
> 
> I will be publishing the comments on individual amp impressions in the context of the comparisons made soon.


Cheapest amp to meet needs? In terms of performance and budget, yes. For aesthetics and pride-in-ownership, maybe not so much. There's something about those trademark McIntosh analog meters that turns many a mere mortal into Pavlov's dog ...drool... but I'll never be able to afford one. Do pride in ownership or pretty lights improve the sound? No, but they may emotionally influence a listener into thinking they hear an improvement. 

*Can't praise you guys enough for your organization and wisdom. No doubt you take as much pride in your reports as your experiment.* Eagerly but _patiently_ awaiting :hissyfit:


----------



## Peter Loeser

BlueRockinLou said:


> Cheapest amp to meet needs? In terms of performance and budget, yes. For aesthetics and pride-in-ownership, maybe not so much.


One of the main reasons the high end amps still sell so well IMO. [I'm not suggesting the main reason isn't sound quality]

I will add that I'm generally willing to spend more for something with better build quality and premium electronic components in the hopes that it will last much longer and perform more reliably than something cheaper, even if both sound the same to my ears.


----------



## NBPk402

BlueRockinLou said:


> For aesthetics and pride-in-ownership, maybe not so much. There's something about those trademark McIntosh analog meters that turns many a mere mortal into Pavlov's dog ...drool... but I'll never be able to afford one. Do pride in ownership or pretty lights improve the sound? No, but they may emotionally influence a listener into thinking they hear an improvement.


Yup, buy the most expensive one so you have status, and it will sound so much better to you. :T


----------



## chashint

A wise man said ..... "And we all know who we are, don't we? ...ponder, ponder... sip, sip "
That line forms behind me.

A second wise man said ..... "There's something about those trademark McIntosh analog meters that turns many a mere mortal into Pavlov's dog ...drool... but I'll never be able to afford one."
That is pretty much the only amp I want to own and just for the record it would sound better than my Pioneer even if it sounds exactly the same....

And a third wise man said ... My EVs are 96db efficient and at reference levels I am still running at at least half the amps output level as shown on the amps vu meters."
Are the VU meters linear scale or logarithmic scale ?

Interesting observations about the SPL various speakers sound good at, when I was shopping I tried to listen to all the speakers at the ~volume we typically listen at and picked based on that ~level.
It never occured to me to do anything else.

My wife plays the violin, we typically listen to music at a much lower level than the violin's SPL.
BTW a violin is surprisingly loud.

I envy/admire those with musical talent.... my musical talents pretty much end at operating the stereo (I am really good at that though).


----------



## tonyvdb

chashint said:


> And a third wise man said ... My EVs are 96db efficient and at reference levels I am still running at at least half the amps output level as shown on the amps vu meters."
> Are the VU meters linear scale or logarithmic scale ?


Here is the specifications on the Samson servo 600 that I use, no mention that I can see as to if its linear scale or logarithmic scale.
Attachment of the PDF below


----------



## chashint

It says servo amp in the title, that is very interesting, why would an amplifier need a servo ? Or is this sort of a repurposed amp that could be used to control a speaker/driver that has servo feedback ?
I will look at the pdf when I get home.


----------



## Sonnie

I agree about those McIntosh amps. They look so cool that they have to sound better. I almost purchased an MC352 for the event... came ever so close. That's a lotta smack for an amp though.


----------



## Talley

Peter Loeser said:


> One of the main reasons the high end amps still sell so well IMO. [I'm not suggesting the main reason isn't sound quality]
> 
> I will add that I'm generally willing to spend more for something with better build quality and premium electronic components in the hopes that it will last much longer and perform more reliably than something cheaper, even if both sound the same to my ears.


My Krell is 12yrs old next month. Does this qualify as build quality? The fact it still pushes without strain or would any old amp do this?


----------



## Blacklightning

Talley said:


> My Krell is 12yrs old next month. Does this qualify as build quality? The fact it still pushes without strain or would any old amp do this?


 :rofl2: OLD?

I'm using a Marantz SR8500 from 2004 as my main power source. This is my newest Amp. My old amp is a NAD 7175PE from 1980's and I'm currently using this one in the TV room. I think of my Marantz as New and the NAD as my old amp. I would be surprised if they broke down anytime soon. Well I guess my NAD was repaired when I got it 7 years ago.


----------



## diablo

I have a Rotel RMB 1095 that's the same age, and can handle anything you throw at it. Its built like a tank, and it's so heavy it comes with wheels to help move it!


----------



## Talley

Blacklightning said:


> :rofl2: OLD?
> 
> I'm using a Marantz SR8500 from 2004 as my main power source. This is my newest Amp. My old amp is a NAD 7175PE from 1980's and I'm currently using this one in the TV room. I think of my Marantz as New and the NAD as my old amp. I would be surprised if they broke down anytime soon. Well I guess my NAD was repaired when I got it 7 years ago.



My uncle uses two eagle 400w monoblocks to power his vandersteen 5a signatures as his 2 channel TV speakers. If I remember right his monoblocks are from the late 80s if not early 90s I think. 

However, he did say after about 20yrs you want to get the unit re-capped. Even my krell. Something about a class "A" amp they run the caps at full power or something like that so they do benefit from replacement after a certain amount of time.


----------



## Blacklightning

Talley said:


> My uncle uses two eagle 400w monoblocks to power his vandersteen 5a signatures as his 2 channel TV speakers. If I remember right his monoblocks are from the late 80s if not early 90s I think.
> 
> However, he did say after about 20yrs you want to get the unit re-capped. Even my krell. Something about a class "A" amp they run the caps at full power or something like that so they do benefit from replacement after a certain amount of time.


Agree :T


----------



## lcaillo

It really depends. Caps can last a long time, but the heat is what causes them to break down. Heat, of course, is one of the characteristics of class A amps. I have seen caps that are 30-40 years old test fine and some just a few years old with issues. It has a lot to do with the initial quality and specs as well.


----------



## Talley

diablo said:


> I have a Rotel RMB 1095 that's the same age, and can handle anything you throw at it. Its built like a tank, and it's so heavy it comes with wheels to help move it!


Nice looking amp and the fact you can get them for under 700 makes them a steal!


----------



## Talley

lcaillo said:


> It really depends. Caps can last a long time, but the heat is what causes them to break down. Heat, of course, is one of the characteristics of class A amps. I have seen caps that are 30-40 years old test fine and some just a few years old with issues. It has a lot to do with the initial quality and specs as well.


Thank you and blacklightning both for confirming. Right now the amp doesn't sound weird or anything just no clue if I should send it in for a check.... knowing krell they would probably say "OH YA... it needs to be replaced" since what I've seen it's around 800 bucks to recap the showcase 7.


----------



## AudiocRaver

chashint said:


> This type of testing has been done many times, one well known test had $10,000 on the line for anyone that "passed" .... The $10k stayed in the challenger's pocket...... Probably a good thing for the promoter that tesseract was not around when that challenge was going on.....


Lots of ways to run a test. A challenge with $$$ involved will have the deck stacked against the listener. Might be done in a way that works against tesseract's "inner methods." Or not. Not taking anything away from his results. Just saying the conditions might be way different.


----------



## AudiocRaver

BlueRockinLou said:


> I once owned pair of KEF 105/3 speakers that were very efficient and played crazy-loud as advertised, but didn't really seem to "sing" until a certain power threshold was passed. I've also read about other speakers "behaving" the same way, but have no first-hand experience. *Do you believe such a phenomenon exists that requires higher amplifier power for speakers that don't play well quietly?*


Could be. I am at a loss for an engineering explanation. The idea hints at non-linearities, subtle distortions that make up the "singing" phenomenon at higher volumes, a possible explanation.



> Fascinating stuff, and I'm truly on the edge of my seat to read your findings. I've been struggling a long time on my 2ch system with one of my test tracks ("Hotel California" from the Eagles _Hell Freezes Over_ album). The opening acoustic guitar from the right channel presents a couple of problems:
> The guitar body (all the notes) seems to stretch toward the left and back again
> The guitar body (individual notes) seems to split in two and notes become "tangled"
> Problem-1 changes with toe-in and distance to side wall. Problem-2 happens during the fast part of the solo. It gets better when acoustically treating the side wall, but never really goes away. I think comb filtering may be the culprit. It's time for some REW and miniDSP to the rescue! *I think I'm ready to toss an old audiophile reservation aside and take the plunge into adding an A/D-D/A conversion.*
> 
> Incidentally, early crowd noise nicely reveals the recording venue by relaying height and depth when my speakers and room treatment are properly dialed-in!


Sounds like exactly what we were hearing. Room treatment was able to settle it down very nicely, will go into detail. I do not think DSP would help, other than to mask it by cutting out certain frequencies, with obvious downside.

Edit: What kind of speakers?


----------



## AudiocRaver

Talley said:


> My uncle uses two eagle 400w monoblocks to power his vandersteen 5a signatures as his 2 channel TV speakers. If I remember right his monoblocks are from the late 80s if not early 90s I think.
> 
> However, he did say after about 20yrs you want to get the unit re-capped. Even my krell. Something about a class "A" amp they run the caps at full power or something like that so they do benefit from replacement after a certain amount of time.


Depending on design particulars, a class A amp can have a lot of voltage across its output coupling caps, and a lot of AC current flowing through them. Unknown just what effect that will have on the cap's characteristics and the amp's sound over time, but it is conceivable. Interesting idea.

Edit: But then that is what a cap does, a well designed cap properly applied should not "wear" more than other component.

Edit: See lcaillo's response above, he has far more direct experience.


----------



## Savjac

BlueRockinLou said:


> Fascinating stuff, and I'm truly on the edge of my seat to read your findings. I've been struggling a long time on my 2ch system with one of my test tracks ("Hotel California" from the Eagles _Hell Freezes Over_ album). The opening acoustic guitar from the right channel presents a couple of problems:
> The guitar body (all the notes) seems to stretch toward the left and back again
> The guitar body (individual notes) seems to split in two and notes become "tangled"
> Problem-1 changes with toe-in and distance to side wall. Problem-2 happens during the fast part of the solo. It gets better when acoustically treating the side wall, but never really goes away. I think comb filtering may be the culprit. It's time for some REW and miniDSP to the rescue! *I think I'm ready to toss an old audiophile reservation aside and take the plunge into adding an A/D-D/A conversion.*
> 
> Incidentally, early crowd noise nicely reveals the recording venue by relaying height and depth when my speakers and room treatment are properly dialed-in!


Also, it might just be the recording. Remember it is a live recording and the boys will move about a bit. In the live video Walsh's guitar actually wanders to the wrong side of the stage, someone in the recording booth made a bit of a boo boo. None the less, for Hotel California, they do turn to each other and back several times and Joe's guitar sounds somewhat duller and less sparkling than Randys. I dont see Joe using his fingernails or a pick like Randy either. 

I love this recording but the bass is always a booger for me. It comes on hard in the beginning of the song and gets turned down a bit as the song progresses. A good test for the bottom end of your speakers is the way the drum goes up in pitch during the intro, very well done,


----------



## Lumen

AudiocRaver said:


> Sounds like exactly what we were hearing. Room treatment was able to settle it down very nicely, will go into detail. I do not think DSP would help, other than to mask it by cutting out certain frequencies, with obvious downside.
> 
> Edit: What kind of speakers?


Revel Ultima Salons (original version). They weren't spiked because at 250lbs the spikes tend to "bite" into the floor. Fine adjustments then become a new adventure in frustration when the spikes suddenly break loose from the carpeting and floor beneath! But I do plan to repeat the eval with spikes and a torpedo laser, which should greatly improve aiming and leveling. A second set of arms and legs wouldn't hurt either! 

You may be interested in a little bit more detail in my reply to Savjac 's comment below.



Savjac said:


> Also, it might just be the recording. Remember it is a live recording and the boys will move about a bit. In the live video Walsh's guitar actually wanders to the wrong side of the stage, someone in the recording booth made a bit of a boo boo. None the less, for Hotel California, they do turn to each other and back several times and Joe's guitar sounds somewhat duller and less sparkling than Randys. I dont see Joe using his fingernails or a pick like Randy either.
> 
> I love this recording but the bass is always a booger for me. It comes on hard in the beginning of the song and gets turned down a bit as the song progresses. A good test for the bottom end of your speakers is the way the drum goes up in pitch during the intro, very well done,


Ya know, sometimes I wonder about myself. I have the DVD as well, but never made the connection. I will definitely check it out. It also never occurred to me that the recording itself may be at fault (for the stretching left to right). And yeah, I agree the quality leaves a lot to be desired, but it does lend itself well to certain setup parameters. For instance, pay _very_ close attention during the opening guitar solo, and you'll hear the guitarist exhale right around the same time his fingers squeak slightly across the fretboard.

I'm not yet convinced the wandering/tangled notes are a recording fault. I've been able to get rid of most of the anomalies using speaker positioning and acoustic treatment. My panels are quite narrow at only a foot wide, so positioning is finicky. To make matters worse, there's really only about 6 inches of absorptive area with 3 inches of diffraction on either side. I'm thinking some wider GIK panels would do a better job at the first reflection point. If anything, it would widen the sweet spot a bit, and alleviate that head-in-a-vise-syndrome! 

BTW, another good bass test on this track is being able to discern the difference between the bass drum and bass guitar (when it first joins in), but also being able to hear pitch changes during the next few seconds.

_*The take-away for this post is that demo/setup tracks should be carefully chosen. Familiarity is key, and I'm absolutely sure our amp shootout panel chose their listening material wisely. :yes:*_


----------



## chashint

"Remember it is a live recording and the boys will move about a bit."

The performers may move around at a rock concert but the speakers you are hearing stay in the same place.


----------



## Talley

This is why I really enjoy the sheffield labs direct to disc recordings like the Harry James disc http://www.amazon.com/King-James-Version-Harry/dp/B0000009F6

for testing systems. one stereo mic placed in the front row center stage and the imaging you get is what you get. GREAT sound! The music may not be for everyone but for testing a system.... by far amazing!


----------



## AudiocRaver

The fact that I already know the answer to the riddle puts me at an advantage. The track was Nickel Creek's _House of Tom Bombadil,_ here is a 




The effect could be heard somewhat throughout the song. With both mandolin and guitar, the LF "body" of the sound was 50% to 70% out from center in the soundstage, and the HF "string" part of the sound was at 70% to 80% out, just inside the speakers, all wandering a bit as the playing proceeded. But the part that stood out is at 1:18, where the guitar run descends and goes back up, The guitar sound moved from 80%R clear to around 25%R and then back out again.

It was all taken care of with room treatment, mostly absorptive material under the movie screen in the front of the room (but there's more, hee hee). The result was an integrated sound that did not move around at all.

Will elaborate on the supposed explanation for the wandering effect.


----------



## Savjac

Different frequencies do relate with room differences and not always in a nice way. For example, for awhile I found the lower mid range of some songs tended to be a bit shouty, particularly in voices that hit the lower registers like Nora Jones for example or even some bass guitars reaching up into their upper levels. Moving the speakers a bit did ameliorate this issue so I know what you mean. 

I am sorry to get this off track.


----------



## AudiocRaver

The Spatial Hologram M1 speakers were invaluable tools for the amp evaluation weekend. Once set up and fine tuned, they gave us a very wide soundstage with pinpoint imaging, for the most part.

It was using the M1 that we first noticed the "wandering guitar" issue. The problem was worse with the Hologram M1 than with the ESL. The M1 is dipole at low frequencies, with widening rear dispersion as frequencies go lower, and the ESL crosses over to monopole below 500 Hz, not leaking as much toward the front wall. Still, the issue needed resolution for both speaker types. Room treatment took care of it.


----------



## lcaillo

While I was not overwhelmed with the M1 it was a very useful and revealing speaker for the amp comparisons. It created a very large soundstage so any variance in imaging would be apparent. I found the bass to be a bit fat and the treble unforgiving, which makes it much easier to notice issues in either area.

My preferfence would be for the ESLs with a sub, but if I were not going to use a sub and had the space to set up the M1 optimally, it would be a very pleasing speaker to own.


----------



## lcaillo

I have started updating first thread with observations from our notes that are consistent across sighted and blind comparisons and/or between reviewers. I have started with the Denon. keep checking for more coming soon.

http://www.hometheatershack.com/for...-reporting-discussion-thread.html#post1134106


----------



## tesseract

I took the long way driving back, finally arrived home late last night. Catching up on this thread and having a listen to my system. Thinking, I need... more... power.


----------



## Sonnie

Well how did you like see the beach for the first time in your life Dennis?

Thanks for starting that up Leonard... good to see at least something up from the testing. :T

I would like to note that we did not attempt to blind test the Krell with other amps because it had a hum/buzz when connected to the ABX box that we could not eliminate, despite fairly exhausted efforts (even attempted using a conditioner). It would have been a dead giveaway to easily identify... as all we had to do was pause the track, mute the sound, stop the music play, switch tracks, etc... the buzz was clearly evident during silence. HOWEVER... I am happy to report that connected directly to my system without the ABX box, the Krell did NOT exhibit any buzz/hum... so it was only when in-line with the ABX box. We were concerned that it may have been a faulty amp, but not so... at least not when connected via normal (most likely) system connection methods.


----------



## lcaillo

It is a lot of data to sort out. Giving it context so that it is meaningful and perhaps useful is a challenge. I started with the Denon because we made the most comparisons in the blind testing. There are two big takeaways from what we experienced with it. First, because the guesses in ABX testing were incorrect more than correct, ABX methodology needs some work. The second is that there were some consistent observations about the Denon AVR among the reviewers in blind and sighted testing. This suggests that there may be something to identify in objective testing, but the differences were not of a nature that any of us would say would disqualify it from consideration sonically when choosing an amp. Other than perhaps being a little underpowered for high level listening, it was certainly in the ballpark with some very highly regarded amps.


----------



## Talley

lcaillo said:


> It is a lot of data to sort out. Giving it context so that it is meaningful and perhaps useful is a challenge. I started with the Denon because we made the most comparisons in the blind testing. There are two big takeaways from what we experienced with it. First, because the guesses in ABX testing were incorrect more than correct, ABX methodology needs some work. The second is that there were some consistent observations about the Denon AVR among the reviewers in blind and sighted testing. This suggests that there may be something to identify in objective testing, but the differences were not of a nature that any of us would say would disqualify it from consideration sonically when choosing an amp. Other than perhaps being a little underpowered for high level listening, it was certainly in the ballpark with some very highly regarded amps.


Ya the Krell being one of them.

Need the Krell input now please.


----------



## AudiocRaver

I hope to have the Spatial Hologram M1 Review and the XT32 / Dirac Live Comparison done today or tomorrow. The nanoAVR DL Review and the ESL Soundstage Report will take longer.


----------



## Savjac

Talley said:


> Ya the Krell being one of them.
> 
> Need the Krell input now please.


Trust your ears young Talley, if you like it and it makes beautiful music, then it cant be all bad. As mentioned above, the differences in amplification are indeed usually modest once levels are set and what have you. I am not a fan of AB or ABX testing as I believe one really needs to listen for a while to become familiar with any given product irrespective of what it is. I think AudioCraver can attest to learning many things about the Martin Logans as time passed that may not have been apparent in a short test.

Amplifiers are more often than not, subtle in their differences and quite dependent on what they are driving. I kind of think a solid 100wpc irrespective of the load, is a minimum for most home theater speakers. This allows a moderate amount of head room and with a proper power supply will take control of speakers and make them behave. Doubling the power to 200 wpc really is nice but in theory only gives a 3 db raise of volume. My "belief" is that the 200 wpc basic amp will have significantly better power "reserves" for those peaks sections that demand that extra bit of oomph needed to raise the goose flesh on your arms. 
Just my thoughts of course but I have put my money where my mouth is. :clap:


----------



## chashint

Krell is a very good amplifier, how can you go from Krell super fan to being ready to dump it based on an amplifier listening party that hasn't even had the results report written yet ?
I have a 5 yr old Pioneer AVR. 
Approximately 100 watts/ch. 
It does not matter what the results of this event are, I am not going to run out and add an amp or replace the AVR in favor of something else.
If I had an amplifier between the AVR and the speakers ( be it a pro amp, Emotivia, Krell, McIntosh, Bryston, Parasound....) it would not matter what the results of this event are, I would not consider taking the amplifier out of the system or consider replacing it with something else.
Relax and take a breath, maybe get that power cord double cryo treated or send the speaker wires for a cryo bath to get your mind off the amplifier.
You were completely satisfied with the Krell, this event should not change that for you.


----------



## Sonnie

Certainly don't trust our ears... this is merely information for your reading pleasure and should not be relied upon to make a decision on what to keep in your system and/or what to purchase. Let your own ears be the judge. One thing I have learned is I can't trust other people's ears to determine what I prefer.


----------



## Talley

Ha... NO I am completely happy with the Krell. So far the music sounds great.... albeit the Ultras are a tad bass heavy even plugged.

The only thing I want to do is test is all. You guys only "sparked" my interest and I don't believe anyone but my own results....

I need to test for myself. End of story. This is why I spend so much $ in my camera hobby... I buy multiple lenses to test them and come to my OWN conclusion.

The ONLY thing this amp review has done is opened my "mind" to be more non Krell ony type attitude. I'll admit I was a big hung up on the Krell because it's my first real "amp" from a sorta respectable level. I was giddy.

Now... I want to back those claims with my own testing. Been looking at a few options here lately.

Ultimately I need to treat my room first to help my ears out first. BUT in the end... I do appreciate all of the hard work everyone that had participated in this event. Thanks!


----------



## tesseract

Sonnie said:


> Well how did you like see the beach for the first time in your life Dennis?



It was great! Spent some time with family, then participated in the Spring Break celebration. The beach, Battleship Park and Memphis Beale St. were a few of several adventures I had on my return trip. Narrowly avoiding a four car pile up on the Missouri interstate, rushing from car to car checking the occupants (all OK) ended an eventful week. 2514 miles logged on the odometer when I returned home.

View attachment 86530




View attachment 86538




View attachment 86546




View attachment 86562




View attachment 86570





Sonnie said:


> Thanks for starting that up Leonard... good to see at least something up from the testing. :T


Oh, yeah, that's right... the amplifier thing!


----------



## tesseract

Savjac said:


> I am not a fan of AB or ABX testing as I believe one really needs to listen for a while to become familiar with any given product irrespective of what it is.


I thought that way, and still find long term listening a valuable tool in determining long term happiness. Then take the component under evaluation out of the system, replacing it with another. If I miss it after a couple of days, back in it goes. If not...

... but, during the beginning of the amp evaluations, when my ears were still fresh and my mind wasn't muddled from hearing my music tracks and everyone else's repeatedly (I don't want to hear the band "Chicago" for a long time) I was taking a few short minutes to make my determination. I think I spent a grand total of two minutes comparing A and B before deciding which was X, on the first trial. 



Savjac said:


> Amplifiers are more often than not, subtle in their differences and quite dependent on what they are driving. I kind of think a solid 100wpc irrespective of the load, is a minimum for most home theater speakers. This allows a moderate amount of head room and with a proper power supply will take control of speakers and make them behave. Doubling the power to 200 wpc really is nice but in theory only gives a 3 db raise of volume. My "belief" is that the 200 wpc basic amp will have significantly better power "reserves" for those peaks sections that demand that extra bit of oomph needed to raise the goose flesh on your arms.
> Just my thoughts of course but I have put my money where my mouth is. :clap:


Very good advice, Savjac. My AVR is rated at 100 w.p.c., though not all channels driven. Anyway, I'd like just another few dB of output and am gonna step up to at least 200 w.p.c for that little boost, and to avoid clipping. 

I'll probably pick something from the amp eval group, or at least from one of these manufacturers. Hmm... which one should I choose? onder:


----------



## Lumen

Sonnie said:


> Certainly don't trust our ears... this is merely information for your reading pleasure and should not be relied upon to make a decision on what to keep in your system and/or what to purchase. Let your own ears be the judge. One thing I have learned is I can't trust other people's ears to determine what I prefer.


Excellent point and I think one that's undermined in the mainstream audio press. I believe many followers of these magazines put reviewers on a pedestal. Rather than trust their own ears, they unquestionably follow their heroes. Companies have both profited from good reviews and suffered from bad ones. 

In any case, Sonnie, congrats to you and the other reviewers (once again!) for keeping us aware of the pitfalls of comparison reports and reviews in general.


----------



## lcaillo

BlueRockinLou said:


> Excellent point and I think one that's undermined in the mainstream audio press. I believe many followers of these magazines put reviewers on a pedestal. Rather than trust their own ears, they unquestionably follow their heroes. Companies have both profited from good reviews and suffered from bad ones.
> 
> In any case, Sonnie, congrats to you and the other reviewers (once again!) for keeping us aware of the pitfalls of comparison reports and reviews in general.


Very true IMO. We are doing it for fun. We don't have any agenda beyond discovery and enjoying the journey. We don't want anyone to put more stock in our experience than their own. The value in what we do for others is getting a glimpse of equipment that they might otherwise not, but it is through our eyes and ears.


----------



## lcaillo

Talley said:


> Ha... NO I am completely happy with the Krell. So far the music sounds great.... albeit the Ultras are a tad bass heavy even plugged.
> 
> The only thing I want to do is test is all. You guys only "sparked" my interest and I don't believe anyone but my own results....
> 
> I need to test for myself. End of story. This is why I spend so much $ in my camera hobby... I buy multiple lenses to test them and come to my OWN conclusion.
> 
> The ONLY thing this amp review has done is opened my "mind" to be more non Krell ony type attitude. I'll admit I was a big hung up on the Krell because it's my first real "amp" from a sorta respectable level. I was giddy.
> 
> Now... I want to back those claims with my own testing. Been looking at a few options here lately.
> 
> Ultimately I need to treat my room first to help my ears out first. BUT in the end... I do appreciate all of the hard work everyone that had participated in this event. Thanks!


You should be smitten with the krell as it is a fine amp. If you decide otherwise and want to dump it let me know.


----------



## Talley

lcaillo said:


> You should be smitten with the krell as it is a fine amp. If you decide otherwise and want to dump it let me know.


So far so good... My uncle always tells me "it'll sound great until you put something better in there... so don't put nothing better in there and you'll be just as happy as can be"

well... all this review has done is got me wanting to try something else for sake of trying.


----------



## NBPk402

Talley said:


> So far so good... My uncle always tells me "it'll sound great until you put something better in there... so don't put nothing better in there and you'll be just as happy as can be"
> 
> well... all this review has done is got me wanting to try something else for sake of trying.


If that makes you happy, go for it... My philosophy is "If it ain't broke don't fix it". :T


----------



## tesseract

Talley said:


> So far so good... My uncle always tells me "it'll sound great until you put something better in there... so don't put nothing better in there and you'll be just as happy as can be"
> 
> well... all this review has done is got me wanting to try something else for sake of trying.



Talley - I would just keep what you have, it'll do as well or better than anything else you might find.


----------



## Talley

tesseract said:


> Talley - I would just keep what you have, it'll do as well or better than anything else you might find.


Set of three FPB-250mc monoblocks for the front three would be better


----------



## 480dad

Sonnie said:


> Certainly don't trust our ears... this is merely information for your reading pleasure and should not be relied upon to make a decision on what to keep in your system and/or what to purchase. Let your own ears be the judge. One thing I have learned is I can't trust other people's ears to determine what I prefer.


My 2 cents... well, the problem for me was/is I just don't have convenient opportunities to let my ears be the judge. After Wayne's review of the ESLs, I wanted to find a pair to listen too. Living in the middle of nowhere, the nearest shop was 2 hours away and I just couldnt work it in, so I defaulted to web reviews. It doesn't take long to seperate the wheat from the chaff. I did the same thing for my PSA sub. Living under a rock in the middle of nowhere, I'd never even heard of PSA before I joined this site a couple months ago. And, I can say without reservation today I'm absolutley estatic with the ESLs & PSA sub. If the ship with the Chane A5s e-v-e-r docks on the west coast, I expect to be just as wow'd. 

So, I think you guys are being a little hard on yourselves. You've proven to be unbiased, knowlegable and upfront, therefore trustworthy in my book. I see no reason to not allow you to influence my investment decisions. Afterall, I do have the ability to return the ESLs and PSA but I haven't. And, I bet I probably won't return the Chane's either.

This hobby is all about compromises, right? and we each have to decide what and which ones to make. But the bottomline is you guys do great things for your members, it's quite a service, especially for the uninformed such as myself. 

...funny thing happened last night...wife walked by and saw me reading this thread...she said "you have 3 amps, you do not need another one." I couldnt argue with her...that is until you publish the results of the evaluation giving me some ammunition to atleast try. 



tesseract said:


> (I don't want to hear the band "Chicago" for a long time)


someobody owes me a cup of coffee and a new keyboard...


----------



## chashint

Talley said:


> Set of three FPB-250mc monoblocks for the front three would be better


Why?
Unless you listen at hearing loss inducing levels you are no where near the limits of your electronics.
If there's something lacking look at the speakers and the listening environment.


----------



## Talley

chashint said:


> Why?
> Unless you listen at hearing loss inducing levels you are no where near the limits of your electronics.
> If there's something lacking look at the speakers and the listening environment.


why not :sneeky:

lol... you guys I'm playing. I just bought my Krell in december and just got it hooked up in January with small NHT and just got the SVS ultra speakers from Sonnie about what... three weeks ago.

I'm a pig in mud right now! Enjoying the out of my system when I can. Right now been busting butt painting the entire inside of my home and wifey said theater room last (although I did sneak in the ceiling!)

So... I have the theater walls scheduled to paint the weekend of April 4th however I still don't want to paint the walls until my screen comes in since right now I'm just throwing up on a painted wall.

THEN I can address the room acoustics just as you mentioned. I already have enough absorption panels to make than I probably need so I'll have extra for my buddies place and then need to concentrate on the diffusion.


----------



## Phillips

Anything on the Dirac comparison yet?


----------



## Lumen

Talley said:


> So far so good... My uncle always tells me "it'll sound great until you put something better in there... so don't put nothing better in there and you'll be just as happy as can be"
> 
> well... all this review has done is got me wanting to try something else for sake of trying.





chashint said:


> Krell is a very good amplifier, how can you go from Krell super fan to being ready to dump it based on an amplifier listening party that hasn't even had the results report written yet ? <snip> You were completely satisfied with the Krell, this event should not change that for you.





lcaillo said:


> You should be smitten with the krell as it is a fine amp. If you decide otherwise and want to dump it let me know.





tesseract said:


> Talley - I would just keep what you have, it'll do as well or better than anything else you might find.





480dad said:


> ...funny thing happened last night...wife walked by and saw me reading this thread...she said "you have 3 amps, you do not need another one." I couldnt argue with her...that is until you publish the results of the evaluation giving me some ammunition to atleast try.


I used the following quote in another post earlier today, and realized it describes upgraditis to a tee:
"After a time, you may find that 'having' is not so pleasing a thing, after all, as 'wanting.' It is not logical, but it is often true."-- Mr. Spock. 

Yeah? Well :neener: to that. I want ALL the review amps, and yours, too! Just kidding.


----------



## AudiocRaver

Phillips said:


> Anything on the Dirac comparison yet?


First up.


----------



## AudiocRaver

Talley said:


> well... all this review has done is got me wanting to try something else for sake of trying.


The only question that really matters is if it pleases you. If your amplifier sound as it is now please as you, you are done. If it pleases you more to experiment and try other amps then go ahead and try them. This is all for fun anyway. If it pleases you and there is no downside, go ahead and do it.



lcaillo said:


> Very true IMO. We are doing it for fun. We don't have any agenda beyond discovery and enjoying the journey. We don't want anyone to put more stock in our experience than their own. The value in what we do for others is getting aa glimpse of equipment that they might otherwise not, but it is through our eyes and ears.


Like Leonard says, it was for fun... and for science. For me (us), science is fun.

My dad was a scientist and a researcher, and we talked about science all the time. It was plain to see that he really enjoyed his work and really enjoyed thinking about science and learning new things. He was also an experimenter and an innovator. I definitely got infected with the fun science bug, probably largely from him. All of the evaluators on this panel have the same bug, or we would not have done what we did.

Edit:

At the same time, I was bitten equally hard by the artsy, right brain, subjective bug. 

To some, those two sides are in opposition, almost enemies in conflict. To others, they are friends working together with just a little creative tension here and there. I consider myself lucky to be in the latter group, do not ask me how I got there but I like it. It makes this kind of work / play tremendously satisfying. To figure out something using some science, and then sit down and have it enhance the joy of the listening experience is a real trip.


----------



## chashint

BlueRockinLou said:


> I used the following quote in another post earlier today, and realized it describes upgraditis to a tee:
> "After a time, you may find that 'having' is not so pleasing a thing, after all, as 'wanting.' It is not logical, but it is often true."-- Mr. Spock.
> Yeah? Well :neener: to that. I want ALL the review amps, and yours, too! Just kidding.


Oh boy this hits too close to home.
I don't get upgraditis and switch out stuff real often but I jump from one thing to the next.
When I became facinated with replacing my 70's era stereo gear I went all in researching what I wanted (er I mean needed), probably took close to a year before I figured out what I wanted...got it, set it up, we are happy....then I jumped into researching new trucks and probably hemmed and hawed for two years before biting the bullet and buying it...I am happy....then I start researching BBQ pits and even spent three days vacationing in Houston visiting the pit builders down there (we did some other stuff too) decided to skip the real steel and bought a WSM....I am happy...
As soon as I get one thing I jump straight to the next thing....my wife teases me sometimes when I tell her I am thinking about getting something by asking how long I have been looking at it and then telling me you have wait xxx months more before you can buy it. Her theory on bigger ticket items is if I still want it after a year I probably actually do want it.


----------



## RickPerconte

This evaluation took place almost two weeks ago, so I was curious as to when you guys were planning on publishing the results.


----------



## Tonto

If I had to guesstimate, with all that data...I'd give them another 2-4 weeks easy.


----------



## AudiocRaver

Even knowing how it turned out, I, too, am excited to see Leonard's presentation.

In the mean time, here is a link to the just-posted Audyssey XT32 vs. Dirac Live Listening Comparison. Enjoy!


----------



## lcaillo

RickPerconte said:


> This evaluation took place almost two weeks ago, so I was curious as to when you guys were planning on publishing the results.


I am waiting to release more on April 1 to keep everyone off balance, guessing whether the results are ficticious or not...

Seriously, no, we won't be publishing any April fool's day nonsense reviews. I would not want to be doing reviews at Sonnies on that date, however, being the jokester that he is.

What I am attempting to do is tease out of a lot of statistically insignificant data any meaningful comparisons and observations. Now that likely drives any researchers who rely strictly on "objective" statistical inference quite crazy, but it is the reality of our hobby. We have a combination of lots of objective data and subjective assessments. Looking at all of the sweep and impulse response data, and trying to correlate it to the observations of 4 listenters across more than a dozen comparisons, under two kinds of conditions, for 11 amplifiers, well, it would make for a multivariate nighmare if the data was just numerical. But this is much worse...

Combine all that with trying to wordsmith it all to make sure that people are least likely to take it out of context and be unfair to any particular product, and I have tossed a number of ideas that I had.

I am working on it all, and will be giving a summary of what we can say fairly about each of the amps. Just framing the question as "are there differences" is really not very useful, but I think everyone doing the listening would agree that they were all much more similar than different, and as we have all said, under the right power and listening constraints, they all do a great job and not one would be products that we would rush to sell if we owned it, nor would we hesitate to want to listen to any of them.

Wayne said in preface to his review, and I could not say it better, "The most important decisions about making a product or technology comparison are determining just what you are actually comparing and exactly how to make a fair apples-to-apples comparison. It is not always as easy as you might think it should be. I am convinced that most casual comparisons made between products are fraught with unaccounted-for variables, and are therefore flawed."
Read more: http://www.hometheatershack.com/for...-live-listening-comparison.html#ixzz3VoBOCKIt
​


----------



## Savjac

lcaillo said:


> I am working on it all, and will be giving a summary of what we can say fairly about each of the amps. Just framing the question as "are there differences" is really not very useful, but I think everyone doing the listening would agree that they were all much more similar than different, and as we have all said, under the right power and listening constraints, they all do a great job and not one would be products that we would rush to sell if we owned it, nor would we hesitate to want to listen to any of them.
> 
> Wayne said in preface to his review, and I could not say it better, "The most important decisions about making a product or technology comparison are determining just what you are actually comparing and exactly how to make a fair apples-to-apples comparison. It is not always as easy as you might think it should be. I am convinced that most casual comparisons made between products are fraught with unaccounted-for variables, and are therefore flawed."



Your way of waiting until everything is done and properly dusted is a good way. Please do not feel rushed as the more information you can put forth in an easy to understand format, hopefully the fewer questions will have to be answered in the future......ahh naw, I am mistaken there, no matter what is said there will be questions from all sides. Cant make everything right nor everyone happy.
Wayne is correct I agree, and while it is not always easy, time and familiarity with any given quality of similar products will bring forth a true idea of what you are hearing and maybe even why. I am firmly against blind testing, at least the short term kind as I dont feel there is much to learn, the devil is in the details as they say. If we all bought homes based upon the first walk through we would be in trouble more often than not. It takes time...and even then, living in the home for a year will bring so much more to the forefront. I hope none of us buys a car based upon the initial test before signing on the dotted line and one cannot truly know the worth of a car without going through the paces a bit, sometimes even going so far as renting one for awhile before purchase. You can hide the amp on test and remove its identification but with time will come a good idea of what it can do. 

If we walk into a test of unfamiliar amps and run them around the bases without learning what they do, I am just kind of leery as to whether or not we get a full feel of them. Not just amps, but speakers, pre amps, AVRs, wires and what have you. There must be a baseline that one knows well for deeper results to become apparent lddude:


----------



## chashint

Meh, pretty sure if you brought a brand new fresh out of the box amplifier into a well established system and listening environment and did a blind A/B listening test against the very well known amplifier less than 1% of the system owners would be able to discern a difference.
Test could be repeated over and over for months and get the same random results.
If any particular individual falls into that super hearing group that ability must never be divulged to their wife because it would wipe out his ability to not hear her sometimes (if it's a woman her hubby can be told because wives hear everything even when we wish they couldn't) .


----------



## AudiocRaver

I will soon have quiet fans installed in my Crown power amp and, along with the Oppo HA-1 acting as DAC + volume control, will be able to perform long-term listening comparisons between it and my Onkyo TX-SR705 receiver driving my MartinLogan ESL speakers. Both will receive optical input from my music server.

Which intention should I prime my mind with as I begin that exercise?

"They are different amplifiers, so they must sound different, so I will find a way to hear that difference."
"All amplifiers sound the same so I will not hear a difference."
"If there is a difference, I will hear it, if there is none, I will not."
"I do not really care. If there is enough of a difference to matter to me, my brain will find a way to let me know."
"Priming my brain with an intention beforehand is a silly notion, just enjoy the music and see what happens."
"All that matters is that I will end up with a preference at some point, so that must be the better amp for me."


----------



## bkeeler10

AudiocRaver said:


> I will soon have quiet fans installed in my Crown power amp and, along with the Oppo HA-1 acting as DAC + volume control, will be able to perform long-term listening comparisons between it and my Onkyo TX-SR705 receiver driving my MartinLogan ESL speakers. Both will receive optical input from my music server.
> 
> Which intention should I prime my mind with as I begin that exercise?
> 
> "They are different amplifiers, so they must sound different, so I will find a way to hear that difference."
> "All amplifiers sound the same so I will not hear a difference."
> "If there is a difference, I will hear it, if there is none, I will not."
> "I do not really care. If there is enough of a difference to matter to me, my brain will find a way to let me know."
> "Priming my brain with an intention beforehand is a silly notion, just enjoy the music and see what happens."
> "All that matters is that I will end up with a preference at some point, so that must be the better amp for me."


I'm sure I don't have to tell you this, but for others not picking up on it, you're comparing DACs and preamp stages in addition to amplifiers here. [Edit: unless, for the Onkyo, you're going into line-level multichannel inputs from the Oppo. Even then, you have the Onkyo's preamp circuit.] There will be those who will say that these things should be as transparent as amplifiers should be, but those who believe that amps can make a difference likely will also believe that the DAC and preamp circuits will also.

That said, I would go with the last option if you have a lot of time to compare. I haven't done a lot of this type of comparing, but it seems to me that if you're looking too hard for differences you may start to make them up in your head :coocoo:


----------



## AudiocRaver

bkeeler10 said:


> I'm sure I don't have to tell you this, but for others not picking up on it, you're comparing DACs and preamp stages in addition to amplifiers here. [Edit: unless, for the Onkyo, you're going into line-level multichannel inputs from the Oppo. Even then, you have the Onkyo's preamp circuit.] There will be those who will say that these things should be as transparent as amplifiers should be, but those who believe that amps can make a difference likely will also believe that the DAC and preamp circuits will also.
> 
> That said, I would go with the last option if you have a lot of time to compare. I haven't done a lot of this type of comparing, but it seems to me that if you're looking too hard for differences you may start to make them up in your head :coocoo:


Thanks, Brian. I am still deciding just how to set it up, what limitations to place, even how to prioritize it with other projects. Part of what you have helped me accomplish through your comments, though, is how easy it is to end up with variables one has not even thought about.:T


----------



## Savjac

AudiocRaver said:


> I will soon have quiet fans installed in my Crown power amp and, along with the Oppo HA-1 acting as DAC + volume control, will be able to perform long-term listening comparisons between it and my Onkyo TX-SR705 receiver driving my MartinLogan ESL speakers. Both will receive optical input from my music server.
> 
> Which intention should I prime my mind with as I begin that exercise?
> 
> "They are different amplifiers, so they must sound different, so I will find a way to hear that difference."
> "All amplifiers sound the same so I will not hear a difference."
> "If there is a difference, I will hear it, if there is none, I will not."
> "I do not really care. If there is enough of a difference to matter to me, my brain will find a way to let me know."
> "Priming my brain with an intention beforehand is a silly notion, just enjoy the music and see what happens."
> "All that matters is that I will end up with a preference at some point, so that must be the better amp for me."


As to part one, this will not work. Brian is correct in that you will be using a different dac in the Onkyo as well as various other computerized settings that the receiver will use in the decoding and playback of music. And as the Onkyo does not have the 7.1 ext in pass through you will always hear more than just the amp. 
So this is kind of an apples to oranges test. If you know someone that can loan you another basic amplifier that would allow you to use the oppo dac direct to both then that would work better.

As to part two, I think choice 4 is more in line with what one needs to go into this with. If you hear it then you do, if you do not, then move on with the knowledge that you gave it a go. I say there are differences, often substantial differences but more often subtle differences but in my defense, I have been doing this for 35 years so I have practiced some long term listening very often. Just like putting your ESL off the carpet has changed the sound for you, once you know what to listen for, amplifiers can do the same thing.
The best thing is you will be doing this for yourself and not allowing some far off comparison from a zillion years ago, from folks that say that cannot hear a difference yet have 10's of thousands invested in amps. 

I truly like your zest Wayne, you are willing to go for the gusto. :clap:


----------



## Lumen

AudiocRaver said:


> ...Which intention should I prime my mind with as I begin that exercise? [*]"They are different amplifiers, so they must sound different, so I will find a way to hear that difference." [*]"All amplifiers sound the same so I will not hear a difference." [*]"If there is a difference, I will hear it, if there is none, I will not." [*]"I do not really care. If there is enough of a difference to matter to me, my brain will find a way to let me know." [*]"Priming my brain with an intention beforehand is a silly notion, just enjoy the music and see what happens." [*]"All that matters is that I will end up with a preference at some point, so that must be the better amp for me."


Gotta love that! Almost works if you substitute:
"wine" for "amp"
"taste" for "sound" or "hear" 
"experience" for "music"

But someone once told me, "Almost" only works for horseshoes and hand grenades! 

_Sent from my iPhone using HTShack_


----------



## Tonto

I'm torn between #4 & #6:



> "I do not really care. If there is enough of a difference to matter to me, my brain will find a way to let me know."





> "All that matters is that I will end up with a preference at some point, so that must be the better amp for me."


Because in the grand sceme of things, all that matters is how *we* like the sound. What *we* already have as equiment currently...*is* our preference!


----------



## lcaillo

Your brain is always primed with some bias. No way to avoid it. I go with 4.


----------



## willis7469

I think 4 and 5. Your brain will let you know if it's enough to matter. Until then, enjoy the music, until your brain defaults away from one or the other. The one that's left is not the one.


----------



## willis7469

Tonto said:


> What we already have as equiment currently...is our preference!


 Indeed!


----------



## Talley

After some discussion with my uncle about this matter he said that even he would have some issues after listening to tons of music for durations that are full songs in length (i'm assuming fatigue?)

He mentioned the easiest way to tell and the only way he tests differences is keeping to only 3 tracks. One must be a natural string instrument like a violin or cello, the other must be a very well recorded live single mic track similiar to the Harry James recording and another is just whatever you want to use type thing, preferably something with a kick drum and/or horn he mentioned. 

He went on saying that you MUST not listen longer than 1 minute of any of the 3 tracks. after a minute your brain is so complexly fooled that things get blended too easily. The way he explained it to me is with the complexity of acoustics hitting your ears that after one minute your brain will start ditching information. 

1. 3 tracks only... same tracks... same media...
2. only the first minute
3. repeat that first minute with changes

This is how he told me he does it. He agree that fatigue destroys clarity.


----------



## AudiocRaver

BlueRockinLou said:


> Gotta love that! Almost works if you substitute:
> "wine" for "amp"
> "taste" for "sound" or "hear"
> "experience" for "music"
> 
> But someone once told me, "Almost" only works for horseshoes and hand grenades!
> 
> _Sent from my iPhone using HTShack_


LOL. I wonder if fine wine lovers have the same arguments.:huh:


----------



## AudiocRaver

Talley said:


> The way he explained it to me is with the complexity of acoustics hitting your ears that after one minute your brain will start ditching information.


Your uncle is wise. It is auditory adaptability that tunes out the status quo (after a minute or so) so the auditory brain can be on the lookout for something new. It is looking for contrasts. It LIKES contrast. Which is why music contains phrases and movements and key changes and... And why A/B testing works so well.

This is a tendency, though, and can be overcome SOMEWHAT with training. How much? Nobody knows.


----------



## AudiocRaver

The Spatial Hologram M1 Speaker Review has been posted.


----------



## 480dad

AudiocRaver said:


> I hope to have the *Spatial Hologram M1 Review *and the *XT32 / Dirac Live Comparison* done today or tomorrow. The *nanoAVR DL Review *and the *ESL Soundstage Report *will take longer.





AudiocRaver said:


> The Spatial Hologram M1 Speaker Review has been posted.


Ok, esl you've esl knocked esl the esl first esl two esl off esl the esl list...what's esl next? 

(subliminal messaging :T)


----------



## AudiocRaver

480dad said:


> Ok, esl you've esl knocked esl the esl first esl two esl off esl the esl list...what's esl next?
> 
> (subliminal messaging :T)


For some very strange reason, I feel this uncanny motivation to post something about our ESL soundstage setup work at Sonnie's. Guess I better get a move on!


----------



## lcaillo

I have posted more detail in the comparison between the Denon AVR and the four amps with which it was compared in the Blind comparisons in the first post of this thread. All of the comments in the comparisons are observations from our reviewers in the blind testing, associated with the amp after the fact. The observations were about A & B, unknown to the listener at the time of the comments.

I have been over the measurements to find any correlations in the frequency response, impulse response, and distortion measures that would account for the observations that were consistent and found very little. There were slight differences in distortion and impulse response, but nothing that I would expect to account for the comments. The only significant frequency response anomoly was a slight roll off in response in the ML above 4K compared to the other amps. This was consistent with the observations.

The conclusions that I come to from all of the subjective observations in both blind and sighted conditions and the measurements confirm a few assumptions that I have been making for years. That, of course, could mean that those assumptions are justified, or that they color my assessment of the collected data. Judge that for yourself. It appears to me that because we had some consistent blind observations about some of the amp comparisons that there may be some differences that do exist that are very subtle and hard to identify in ABX testing. On the other hand, we did fail to identify the amps correctly in the ABX testing. Once again, we have a conflict between objective and subjective testing. What justifies some concern, IMNSHO, is that some of the blind observations were consistent and specific. I BELIEVE that future testing should focus on identifying observed differences and focus on extended listening with multiple trials to determine if they are, indeed, repeatable. It could be that what we observed that seemed to be consistent was simply chance. Regardless, if there are differences, they are very small, far less than many reviewers and vendors would lead us to believe. I think that this is a reasonable expectation. For those who want to extend that notion to say that there simply are no differences in amps, I have to say that you are justified in believing that. To those who argue the opposite, I would say that you, too, are justified in believing that, but you have to account for expectation bias, which, beyond any doubt, contributes to the significance attributed to any differences that are observed whether they are there or not. Once again, I take the middle road and probably don't satisfy either side. So be it, as the results here seem to justify my opinion that the correct answer is more complex than either side would like to admit.

I am working through the data in the other comparisons and will be publishing more, but the Denon was compared to 4 other amps, giving much more information and more opportunities to identify consistent observations. There won't be as much to report simply because it is more difficult to find consistency in single comparisons, but there are a few that stand out.


----------



## chashint

^^^
Good write up in post 1.


----------



## tonyvdb

Agree, nice write up. Thanks for all the hard work so far in doing this.

Reading into all that has been said so far I personally think that the differences between amps is small at best, not night and day. So in my opinion you can say that as long as properly matched to the load of the speaker there is not enough difference to justify the cost of the big $$ amps.


----------



## lcaillo

Here are the frequency responses for the amps that were compared to the Denon, with the exception of the Behringer. For some reason I could not find it, but there were no noted meaningful differences at the time. I am sure we have it somewhere.

Note that the levels were matched carefully in each comparison, but there were slight differences between those pairs. I have matched the levels at 500Hz for this comparison.


----------



## chashint

If $$$$$ amplifiers bring a person pleasure they are worth it.
But if the goal is to spend as little $$$ as possible to get improvement in the audio it is difficult to make the case for spending $$ on outboard amplifier/wires/accessories when speakers make so much more difference in how the audio sounds.
When recommendations are asked for to get better sound quality outboard amplifiers should be far down the list most of the time.


----------



## lcaillo

I agree, Tony. The fact is, however, that many people DO experience differences that they find to be significant. It is real to them. What I believe these results indicate is that there are possibly differences that are audible, even if our measurements do not capture them. If we assume that SOME, even slight differences MIGHT exist, the only intellectually honest way to proceed is to try to find those that might be repeatable in objective testing. This requires identifying the most likely candidate differences, then arranging testing to facilitate identifying them without the possibility of chance influencing the results. Considering the experience of the testers here, more trials with fewer amps, more time with more breaks, and consistent passages repeatedly may be the way to go. The X option appears to be a confusing and difficult aspect of the testing. In the future I would devise a list of characteristics for each amp, then ask the listeners in blind comparisons to rate those repeatedly in randomized trials, rather than trying to rely on memory to match characteristics.

Again, the point, IMO, is to identify whether there are meaningful differences in a universe of infinite possibilities. If those identified differences are present, they should be repeatable, but we have to isolate those variables. The ABX testing obviously, based on our experience here, introduces other confounding variables.

Any time one is doing testing on something that involves human perception, isolating the variables it very difficult. This is clearly the case here.

Also, again, with all that said, we may still be talking about angels on the head of a pin in terms of the meaningfulness of any differences for most people. If you assume for a moment that the differences that we heard that MIGHT be real are, they still would not be meaningful to most people. They might to a handful of us, but even we were not impressed with the differences in terms of driving a decision to purchase, and we probably care more than all but a very small percentage of the buying public. This is really more of an academic exercise than one of usefulness to the majority of people, even those that buy very expensive amplifiers.


----------



## Tonto

lcaillo said:


> Denon was compared to 4 other amps


I think this is the paramount interest. Do we need external amps to drive our mains when it is set for 2 channel stereo? How does the SI&D compare between the two. How perceivable is the added headroom...enough to justify the expense?

So, not so much of "which amp" as it is "how much amp?"

I would expect the differences to be increased in the HT setting.


----------



## lcaillo

I think it really depends on how one listens and the speakers. Frankly, I have found my Onkyo receiver to be fine for my HT system, but I rarely listen very loud. I have an amp driving the subs and can play the system loud enough that the rattles in the house become the limiting factor. I run the 2 channel system full range, so it would be more of an issue, I think, but I have components for that system anyway. At some point I want to do some blind comparisons with the Onkyo and my components on that system.


----------



## Tonto

I agree the speakers are the deal. I A/B'd my Arx1'c with my Klipsch KSF 8.5 speakers & the power requirement was bigger than I expected. And those are the bookshelves...the A5's may need even more. If I remember correctly, we used a 100 watt amp in the 1st eval, and that seemed to drive them well.


----------



## primetimeguy

lcaillo said:


> Here are the frequency responses for the amps that were compared to the Denon, with the exception of the Behringer. For some reason I could not find it, but there were no noted meaningful differences at the time. I am sure we have it somewhere.
> 
> Note that the levels were matched carefully in each comparison, but there were slight differences between those pairs. I have matched the levels at 500Hz for this comparison.


Is that the response from the main listening position and at least in the ballpark of what people heard? I ask because, no offense, the response seems terrible. More than 15db difference between lows and highs, with mainly the highs just dropping like a cliff. And the huge 10db notch at 2k right where our ears are most sensitive.

I get that from a pure measurement difference things are all relative and shouldn't matter. I was thinking more from the subjective testing standpoint if this reflects what people were hearing.


----------



## AudiocRaver

Leonard has the patience of a Tibetan Monk, only he's a lot funner to be around. (All due respect to Tibetan Monks - actually I have never met one, they might be quite the party boys, although somehow I doubt it.)

I sent him mdat files containing the Behringer amp data. So much to keep track of, I probably missed them before somehow.

Thanks, Leonard, for sifting through all that data. Fantastic job!


----------



## AudiocRaver

primetimeguy said:


> Is that the response from the main listening position and at least in the ballpark of what people heard? I ask because, no offense, the response seems terrible. More than 15db difference between lows and highs, with mainly the highs just dropping like a cliff. And the huge 10db notch at 2k right where our ears are most sensitive.
> 
> I get that from a pure measurement difference things are all relative and shouldn't matter. I was thinking more from the subjective testing standpoint if this reflects what people were hearing.


You raise good questions.

First of all, in-room response measured at the LP without any equalization is usually a bit uglier than anyone wants to look at. Room effects, nodes, etc, can lead to some big variations at lower frequencies. You can certainly see the source of the "tubbiness" that was mentioned earlier.

We set up the M1s for soundstage and imaging, which were very good. The arrangement of objects and surfaces in the room changed somewhat for Saturday, to accommodate the "blind" part of the process, and that reduced image clarity somewhat, as Dennis mentioned in his comments. But that probably did not show up in the FR curves.

As mentioned, we set the M1s a bit wider than we probably would have to avoid moving the ESLs, which were perfectly placed. I made that choice, setting up before the others arrived. Two things that might have been done differently: the ESLs could have been moved and the M1s place more optimally in the room, and we might also have experimented with placement to get better frequency response. As it was, placement took several hours, and in all honesty there would not have been time to do much of that experimenting. 

The notch at 2 kHz is an artifact due to reflection from the chair back behind the microphone position. And that is with a blanket in place to reduce the reflection. Coincidentally, in preparing some information for another thread, I have been doing measurements around the back of my own chair yesterday and today, and that kind of notch at around 1 to 2 kHz is a harsh reality of having a high back chair. With the microphone moved to where that notch is not created from the reflection on the chair back, the frequency response no longer represents the listening position accurately in the midrange. With the listener sitting in the chair, the effect is not as drastic, I believe anyway, I'm still experimenting to see how much difference that makes. I would not put too much emphasis on that notch at this point, knowing its source, and knowing it to be at least somewhat different with the listener sitting in the chair.

It was important for our exercise that no EQ be used, nothing that might add another variable that could detract from the amplifier differences we were listening for. The high-frequency roll-off also is a little of a mystery, because the response for the speakers themselves was quite flat to 20 kHz, and the listening position was right on axis. Distance and air absorption had to be the cause. Another harsh reality.

I'm sure that none of this is news to you, and it does not change the fact that you are correct about the frequency response being far from ideal. And I am NOT making excuses, only giving explanations the best I can. Other than the tubbiness previously mentioned at low frequencies, I do not believe that frequency response was a significant detractor for any of the listeners. if anything, having worked with Leonard and Joe in the past on speaker evaluations, and knowing that neither of them was crazy about speakers with really strong high frequencies, we might have had complaints had the highs been a lot stronger than they were. No one ever commented that the high frequencies were low or week, and I do not believe it was a problem for what we were there to accomplish. Listener fatigue is another potential problem to be avoided with work like we were doing, and the somewhat rolled off high frequencies may have spared us all more fatigue than we already ran across from time to time.

These are just comments and observations, not disagreeing with your point at all.


----------



## primetimeguy

Thanks for the good explanation


----------



## 480dad

Is there an eta from the monks on eval results?


----------



## RickPerconte

It's been nearly six weeks. I'm kind of curious when the results will be published too.


----------



## chashint

Did y'all see this ?

Denon X5200 AVR
Compared to other amps, several observations were consistent. The Denon had somewhat higher sibilance, was a bit brighter, and while it had plenty of bass it was noted several times to lack definition found in other amps. At high levels, it did seem to strain a bit more than the other amps, which is expected for an AVR compared to some of the much larger amps. Several times it was noted by multiple reviewers that it had very good detail and presence, as well as revealing ambiance in the recordings.

We actually listened to the Denon more than any other amp, as it was in four of the blind comparisons. It was not reliably identified in general, so one could argue that it held its own quite well, compared to even the most expensive amps. The observations from the blind comparisons that had some common elements either between blind and sighted comparisons or between observers are below. The extra presence and slight lack of bass definition seem to be consistent observations of the Denon AVR, but everyone agreed that the differences were not a definitive advantage to any one amp that would lead us to not want to own or listen to another, so I think we can conclude that the Denon held its own and was a worthy amp to consider. 

Compared to Behringer
- bass on Denon had more impact than Behr, vocals sounded muted on Behr
- vocals sounded muted on ML compared to Denon
- Denon: crisp highs preferred compared to Behringer which is silky.
- Denon is more present, forward in mids and highs than Behringer.

Compared to Mark Levinson
- Denon seemed to lack low end punch compared to ML.
- Denon is smooth, a certain PUSH in the bass notes, cellos & violins sounded distant, hi-hat stood out, distant vocal echo stood out, compared to ML.
- Denon bass seemed muddy compared to ML which is tighter.
ML more distant strings than Denon.
- Denon is slightly mushy and fat in bass. String bass more defined on ML.
- ML seems recessed compared to Denon.


Compared to Pass
- vocals sounded muffled on Pass compared to Denon
- crisp bass on Denon compared to Pass
- Denon & Pass both even, accurate, transparent, natural, no difference, like both
- Pass seems soft on vocals but very close.
- Denon has a bit more punch on bottom, maybe not as much very deep bass, more mid bass.


Compared to Van Alstine
- bass on Chant track was crisp for VA while Denon was slightly sloppy
- sibilance not as pronounced on VA as it was on Denon
- VA super clarity & precision, detailed, space around strings, around everything compared to Denon which is not as clear, liked VA better.
- sibilanceon Denon, VA has less “air” but more listenable, both very good
- Very deep bass more defined on VA, overall more bass on Denon. 



Read more: http://www.hometheatershack.com/for...eporting-discussion-thread.html#ixzz3YFvULf3M


----------



## lcaillo

I have been having some issues with editing the first thread so I'll start posting more results here and when we sort it out, we will include them there as well.
*
Wyred 4 Sound ST-500 MK II*
In the sighted listening we compared the ST-500 MK II to the Van Alstine Fet Valve 400R. The assessments varied but were generally closer to no difference. The Van Alstine got comments of being fatter on the bottom. The Wyred 4 Sound was noted to have slightly better bass definition but apparently less impact there, and slightly less detail in the extreme highs. Most comments about the midrange were not much, if any difference. An interesting observation here was by Wayne, noting that he did not think he would be able to tell the difference in a blind comparison. Considering the ST-500 MK II is an ICE design and the Fet Valve 400R is a hybrid, we expected this to be one of the comparisons that would yield differences if any. As I am always concerned about expectation bias, this was one that I was particularly concerned with. Van Alstine is a personal favorite for a couple of us so I expected a clear preference for it to be present in the sighted comparison. I felt that the Wyred 4 Sound amp help its own with the much more expensive and likely to be favored VA. 

In the blind comparisons, we compared the ST-500 MK II to the Emotiva XPA-2 and the Sunfire TGA-7401 in two separate sessions. Of course, in these sessions we had no idea what we were listening to until after all the listening was done. In the comparison to the Emotiva, some notes revealed not much difference and that these were two of the best sounding amps yet. The ST-500 MK II was noted to have the best midrange yet, along with the Emotiva. It was described as having less sibilance than both the Emotiva and Sunfire. Both the Emotiva and the ST-500 MK II were described as unstrained in terms of dynamics. In comparison to the Emotiva it was noted to have solid highs, lively dynamics, rich string tones, and punch in the bass. The overall preference in comparison to the Emo was either no difference to preferring the W4S.

In comparison to the Sunfire, comments ranged from preference for the W4S to not much difference to preference for the Sunfire. The Sunfire was described as having more presence in the midrange, while the Wyred was noted to be shrill, lifeless, and hollow by comparison.

These comments varied a lot, but the points of convergence were generally around the similarities to three amps that would be expected to be most likely to be different, if we found any differences at all. The objective results is that we failed to identify the amp in ABX comparisons to two other much more expensive amplifiers. I would have to conclude that based on the results, the ST-500 MK II represents one of the best values and certainly should satisfy most listeners.


----------



## AudiocRaver

Leonard's recent notes on the *Wyred 4 Sound ST-500 MK II* amp havee been added to the first post.

Also, I have published a separate post on our experience enhancing the soundstage and imaging with Sonnie's MartinLogan ESL speakers. Go to the *Soundstage Enhancement Experience* thread now.


----------



## padgman1

Any updates soon on the other amps?


----------



## lcaillo

I have a few notes left, but there is not a lot of correllation left to justify saying much in the same way that I have for the ones thus far. I may take my observations and put them in a new post dedicated to purely subjective and uncorrelated comments. There are a couple of things left to add on the blind listening, but frankly I was hoping to see some discussion of what I posted to help determine what people are interested in or think is valuable from the stuff that we already have. There has not been much discussion at all since we posted what we did.


----------



## padgman1

Sufficiently stimulated by Leonard's above comments, I will try to further this discussion......

1. It seems in this comparison that there are small differences between amps, some of which may be consistently observed and thus may be preferred by certain listeners that wish their amps to "perform" in certain ways ( better bass control, more detailed midrange, etc).....it may help others trying to trial or purchase said amps if multiple listeners noticed a consistent feature associated with them to use as a criteria for purchase, if possible.....besides the comments made for the Denton and the Wyred Sound amps ( and indirectly the VanAlstine), any other aspects of the other amps on sighted evaluation stick out?

2. It seems the need for headroom or increased power does come into play at louder volumes approaching reference levels and beyond, especially with less efficient speakers. It also seems at lower volumes this need becomes less so. Is this kosher? Those of us just listening at moderate to low volumes can probably use the Denton to excellent effect and obviate the need for an additional amp. Those wanting to experience "live venue" sound or "movie theater" volumes probably need an amp with more power....which any of the other amps in this comparison could provide...

3. With the use of powered subwoofers in multichannel audio systems and home theater applications lowering the power need for the rest of the system, is an amp providing multiple hundreds of watts/channel really needed? By the by, how much IS the power lowered in a system by using a subwoofer to reproduce the lower octaves? 

Hope this helps to reenervate this thread....


----------



## chashint

I don't see consistency in the comments indicating any particular sound signature for any particular amplifier. If part of the test included switching between the same amplifier against itself and the listeners made comments about differences heard in that scenario that would be particularly interesting as would comments indicating the same amplifier switched against itself sounds exactly the same. If the amplifier compared to itself sounded different to the listeners then clearly expectation bias is affecting what is being heard. If that amplifier compared to itself was deemed identical but differences were identified when compared to other amps then clearly there are differences and that would be a very interesting observation. My conclusion from the published notes is, in a real world listening environment if sound quality improvement is the goal of spending money then the money is best spent on speakers or possibly acoustic treatments.


----------



## willis7469

Very nice Charlie. I think that would be fun to see the results of.


----------



## padgman1

Charlie, I agree wholeheartedly with your comments, and, yes, there does not seem to be any consistent comments about a "sound signature" with any of the amps noted ......just wondering if there were any consistencies with any of the other amps not already discussed in detail...


----------



## lcaillo

Hence, my problem. What to report about the experience with each amp. The comments about what we heard don't have much consistency left. Yet there were some consistent comments across the sighted and blind tests and between listenters. These are mostly in the amps where there were multiple comparisons. I think this points to the need for more extensive listening and more trials on a particular amp to identify any potential differences. 

So are the uncorrellated comments on each amp of value? Considering how things are taken out of context on the internet, I am very hesitant to publish them, as those who have an agenda to promote could easily use them for such, and attribute statements to us that out of the context of the whole event may not be our intent.


----------



## padgman1

Totally understand your comments, Leonard.....too bad everyone couldn't take off work for 2 weeks to really flesh out any consistencies between amps by having those multiple trials....think such a study would qualify for a government grant under the auspices of audio health??


----------



## chashint

I truly enjoyed this evaluation.
It was (is) an undertaking of significant efforts from all of the participants and those efforts are fully appreciated.
If Sonnies' sell off of gear is any indicator... he took on significant $$$$ to get this accomplished, special thanks goes to him.
I can only guess that report writing straws were drawn and you ended up with the short straw because I cannot see anyone willingly saying, "Oh oh me me, please let me write it up".
This is pretty much an impossible task, which I think you have handled quite nicely.
Yes, I am curious to know every comment that was made. But I was not there, I heard none of it. One off comments with no other correlations serve little purpose.
I would let the published results stand as is, and be proud of the effort.


----------



## Medi0gre

I wish I could have said it better! Thank you to all the evaluators for their time and money to accomplish this task, and thanks to Chashint for articulating what I could not. Great job guys.


----------



## AudiocRaver

A complete review of the miniDSP nanoAVR DL has been posted HERE. Where appropriate, comparisons are made to Audyssey XT32, focusing on the *end-user experience.*

A post has been added to this thread titled *Audyssey XT32 (without Pro Kit) vs Dirac Live End User Experience Comparison Summary of Audible Characteristics.* Read it HERE.


----------



## Serenity Now

Your prior evaluation event threads have been beyond outstanding.

What happened? Were you guys issued a legal gag order? 21 pages of nothing conclusive, but hints of both sides are right talk. 

Can you at least post the cumulative measured response graphs together on one graph for us pretty please. I appreciate your efforts. This thread ended up a real dissapointment for me. 

I wish you would have tested inefficient bookshelves as I suggested in the preparation thread. Both sets of speakers were high efficiency. 

Please give us the data conclusive or not. What are you waiting for? :huh: Does this mean this blind test resulted in a statistical null result?


----------



## lcaillo

See post 177. There were virtually no measureable differences that amounted to anything in either frequency response or impulse response. I have spent hours trying to find something and it just is not there. I really expected to find something in the impulse response, but it was actually more similar than the frequency response measures.

The bottom line is that we were not able to reliably detect differences nor reliably identify differences in ABX comparisons. the things that we made notes that we heard that had any correllation at all across listeners and sessions are detailed in the first post. They were mostly about the amps that we heard in more than one comparison, which leads me to believe that with more focused listening on less amps that we might be able to find some diffenences that hold up. Frankly, we probably had too many amps and tried to do too much. The next time we probably won't have more than 2 or 3.

Sorry to disappoint, but it is what it is. I don't think that it is fair to post every comment we wrote down on every amp that is uncorrellated. I am still uneasy with posting as much as I did because there may have been only one comment that supported each, and that is far from reliable. We can conclude that what we THINK we hear is highly variable and suspect.

We will have a much better idea about how to approach it next time.


----------



## chashint

Serenity Now said:


> Your prior evaluation event threads have been beyond outstanding. What the happened? Were you guys issued a legal gag order? 21 pages of nothing conclusive, but hints of both sides are right talk. Can you at least post the cumulative measured response graphs together on one graph for us pretty please. I appreciate your efforts. This thread ended up a real dissapointment for me.  I wish you would have tested inefficient bookshelves as I suggested in the preparation thread. Both sets of speakers were high efficiency. Please give us the data conclusive or not. What are you waiting for? :huh: Does this mean this blind test resulted in a statistical null result?



I think this is being kinda rough on these guys.
This evaluation produced the same results as ALL other level matched amplifier evaluations.
Since you are disappointed/dissatisfied with it, perhaps you will put together your own amplifier comparison and post the methods/results here for critique.


----------



## Serenity Now

I wish the test provided some conclusion is all. A null result is why blind testing is often criticized. I do think that a proper test method with this many amps would take more listeners, more time and a better room setup. I care less about the listener impressions than seeing the hard data in graph form. In a blind test fatigue and the stress to answer "right" bias listener results. Are we testing the amps vs listeners or the test vs. human psycology. 

Numbers and hard data are unquestionable. In future I would recommend finding a set of speakers known to give amps trouble at key frequencies (probably bass impedance/inductance swings) and measure output vs frequency over a specific bandwidth. Then vary the output voltages higher and run the test over and over. Some amps will simply not "wake up" speakers until a certain voltage is reached. I recommended bookshelves because bass is often not their strongsuit and poor amps can result in thin sounding speaker response.

Remove the human element and the test becomes faster and conclusive. Then we can try to find out what the measurements mean in a listening experience.


----------



## Lumen

Serenity Now said:


> I wish the test provided some conclusion is all. A null result is why blind testing is often criticized. I do think that a proper test method with this many amps would take more listeners, more time and a better room setup. I care less about the listener impressions than seeing the hard data in graph form. In a blind test fatigue and the stress to answer "right" bias listener results. Are we testing the amps vs listeners or the test vs. human psycology. Numbers and hard data are unquestionable. In future I would recommend finding a set of speakers known to give amps trouble at key frequencies (probably bass impedance/inductance swings) and measure output vs frequency over a specific bandwidth. Then vary the output voltages higher and run the test over and over. Some amps will simply not "wake up" speakers until a certain voltage is reached. I recommended bookshelves because bass is often not their strongsuit and poor amps can result in thin sounding speaker response. Remove the human element and the test becomes faster and conclusive. Then we can try to find out what the measurements mean in a listening experience.


No matter the initial conditions, variables introduced or placebos controlled, someone will always be dissatisfied with test results. That's the nature of testing. Hard data is not always the absolute arbiter of conclusion. Questionable recording practices and post-manipulation can come into play. I seriously doubt any of our panel engaged in such integrity-robbing practices. Rather, I believe they conducted themselves with the highest professionalism and exercised due diligence in set-up. Room acoustics and speaker positioning were already dialed in to the nth degree before the trials began. And why use hard-to-drive, specialty speakers unless their ownership proliferated throughout the mass market? 

On one hand, you ask for hard data, but on the other you speak of "waking up" speakers; Where's the hard data for that? Remove the human element, and you have (drum roll, please) The Terminator Syndrome: machines measuring machines producing physical phenomena for other machines. The results can hardly be soothing. Right, Ahhnolt?

Sent from my iPad using HTShack


----------



## chashint

Whatever the shortcomings of blind listening tests are I have yet to see any other method proposed that is better.
If the proposal is to let measurements be the end of the argument then the electrical measurements on the many and various amplifiers that have been published in HiFi magazines should (if you understand basic electronics and orders of magnitude) lead to the conclusion that properly functioning amplifiers that are not overdriven will sound so similar that they are unidentifiable from each other in listening tests.

Every time any group starts an amplifier "shootout" there seems to be a groundswell of hope that there will finally be a conclusion that's different from all of the other controlled amplifier listening efforts that have come before.
But alas, if the levels are carefully matched and the listeners do not know which machine is powering the speakers the machines all sound the same.


----------



## Tonto

Chasnit wrote:



> But alas, if the levels are carefully matched and the listeners do not know which machine is powering the speakers the machines all sound the same.


Yep, couldn't have said it better. But those were some pretty serious amps. Judging by the ridiculous price drops Sonnie resorted to....not the price range most of us are at. Maybe it would have been nice to throw in some more reasonably priced contenders. Coudos for making it happen though, it's nice to know the extra dollars are better spent elsewhere. At least that's what I'm taking away from it.


----------



## Serenity Now

A $300 AVR is not going to provide stable voltage into a difficult load. 

Lou you are right, in the end a machine cant tell us how another machine is interacting with our senses. But we cannot reliably predict any machine will interact between 2 different listeners the same way. I think this is why the test I mentioned is a good place to start, then listen to the key ranges once they are determined by initial screening. Then you can correlate objective to subjective tests and see if a pattern emerges. Wayne has already done this to an extent in his test of an Axiom Amplifier into speakers. He would be a key resource in designing this test.


----------



## Lumen

Tonto said:


> Chasnit wrote: Yep, couldn't have said it better. But those were some pretty serious amps. Judging by the ridiculous price drops Sonnie resorted to....not the price range most of us are at. Maybe it would have been nice to throw in some more reasonably priced contenders. Coudos for making it happen though, it's nice to know the extra dollars are better spent elsewhere. At least that's what I'm taking away from it.


Some, maybe even most, people would spend their dollars elsewhere. Some for the reasons you stated; others for reasons dealing with mob mentality. Still others place high value on certain differences, even if only perceived. So perceived or not, there's nothing wrong with someone spending more if the difference is important to them. Sure build quality, craftsmanship, and appearance play an influential role in how one amp sounds over another. Sure blind tests say otherwise. The hobby is big enough for both camps. Each just uses different machines to accomplish the same task.

Sent from my iPad using HTShack


----------



## Lumen

Serenity Now said:


> A $300 AVR is not going to provide stable voltage into a difficult load. Lou you are right, in the end a machine cant tell us how another machine is interacting with our senses. But we cannot reliably predict any machine will interact between 2 different listeners the same way. I think this is why the test I mentioned is a good place to start, then listen to the key ranges once they are determined by initial screening. Then you can correlate objective to subjective tests and see if a pattern emerges. Wayne has already done this to an extent in his test of an Axiom Amplifier into speakers. He would be a key resource in designing this test.


Yes, I see that now. It's more complicated than just differences between machines. Two listeners have different perceptions of reality and react to stimuli differently. We can take that one step further by repeating that even one particular person may not react to an amp/song/speaker the same way from moment to moment. Psychological and physiological factors influence how we react to what we hear whether it be ABX stress or wishful thinking. For that matter, something as trivial as a grocery list can distract us from discerning differences. Welcome to the machine! 

Sent from my iPad using HTShack


----------



## chashint

Serenity Now said:


> A $300 AVR is not going to provide stable voltage into a difficult load. Lou you are right, in the end a machine cant tell us how another machine is interacting with our senses. But we cannot reliably predict any machine will interact between 2 different listeners the same way. I think this is why the test I mentioned is a good place to start, then listen to the key ranges once they are determined by initial screening. Then you can correlate objective to subjective tests and see if a pattern emerges. Wayne has already done this to an extent in his test of an Axiom Amplifier into speakers. He would be a key resource in designing this test.


How do you know a $300 AVR is not going to be stable into a difficult load?
Is there any data to back up that statement?

Since you know what and how you want a test to be conducted, why not do it instead of asking other people to do it?
If you were to publish the methodology, conduct the test, and then publish the results I (and probably others) would read it from start to finish.


----------



## Tonto

BluRockinLou wrote:



> So perceived or not, there's nothing wrong with someone spending more if the difference is important to them.


We all have our paradigms! I just bougt a very nice set of Michelins for my truck. Could have spent a lot less, but I like them!


----------



## gdstupak

Serenity Now said:


> I wish the test provided some conclusion is all. A null result is why blind testing is often criticized.


This test did provide a very informative conclusion based on the original criteria. It's just not the conclusion you want it to be.



Serenity Now said:


> In future I would recommend finding a set of speakers known to give amps trouble....


You want to do a specialized test to find the best amp to drive hard loads. That is not what this test was about.


----------



## chashint

Tonto said:


> BluRockinLou wrote: We all have our paradigms! I just bougt a very nice set of Michelins for my truck. Could have spent a lot less, but I like them!


Me too.


----------



## AudiocRaver

I have been meaning to post my own observations and conclusions from the event. The posts over the last few days have prompted me to go ahead and get that done.

Thanks Leonard for being willing in take the time to dig through all of the data the way that he did. He has far more patience at that then I, and no one could have done a better job.

I had hoped that we would have much clearer results than we did, either that there were clear differences that we could prove, or that there were none we could hear at all. Instead we ended up with some of us able to hear some differences some of the time and only a little data to prove it. Concerning data which can be said to support any consistency of findings across the whole listening panel, Leonard has found what was there to be found and reported it already.

I will go ahead and post my individual observations for what they are worth, but only to be taken with a huge grain of salt, because they are impressions and that is all. If the others wish to post their impressions, they are welcome to do so.

First of all let me describe my evaluation process. I personally feel this is quite important because different people seem to have different ways of going about this. And if one has a listening style that works for him, that should probably be taken into account in the design of the blind testing that person will engage in. In other words, I might be able to set up a valid blind test method that would work great for me and throw Joe or Dennis or Leonard completely off, while there may be an approach that would work perfectly for some of them and leave me flat.

And this is one of the great difficulties in setting up tests like this. Someone with a good background in audio and acoustics and psychoacoustics and testing sits down and figures out a really good double-blind test method for ABX testing and 5 people walk into the room and it happens to fit the listening approaches of only one of them and he does well but the other four fail miserably, and the test overall shows no statistically significant data supporting the ability to tell a difference. Had the test been set up another way it might give a different result.

The ABX testing that we did made it a necessity that the evaluators rely upon extremely fine details being held in auditory memory for 30 seconds to a minute to be used in an AB comparison. Dennis appeared to do very well with this, while the rest of us did not. For me that was extremely difficult, as the fine differences that we were hearing were simply not something that I could capture in memory and carry forward in that way into a comparison 30 seconds to a minute later. Maybe with practice I could learn to do so but at the event I was not able to.

Here is what worked well for me. I felt fairly confident about the differences that I was hearing between amplifiers in sighted testing we did on the first day. The two amplifiers were set up, their levels were matched, we knew which was which, and we held the a/b switch in our hands while we listen to our own selected listening tracks. As I listened through my tracks I switched back and forth freely between the two amplifiers. Over time I started to recognize that there were certain passages of each track that seemed more likely than others to help reveal differences between the amplifiers, so I focused more on those parts of the tracks, but I also listened to other passages just in case something new would pop up.

When I heard a difference I tried to make a note of what part of what track it was that I heard it on, and what I heard, and where I felt I had time I would go back and repeat that to be sure that the difference was distinctive and easily identifiable. Remember these judgments were not absolute in any way but extremely comparative in nature, as will be seen in my impressions of some of the amplifiers the follow. By switching back and forth during those critical passages, I felt the contrast almost jumped out sometimes when the switching was done at just the right moment. Given the ability to do that repeatedly with a pair of amplifiers, I got to the point where I was pretty confident I could identify the difference consistently.

So if I was to set up a blind comparison around that listening style and try to get statistical data to show I could do it consistently here's how I would go about it. I would start out with the pair sighted so I knew which was which and go about the test as I have described and identify the characteristics comparatively between the two amplifiers. Then I would leave the room, have the test setter upper flip a coin and decide whether or not to swap the two amplifiers. When I came back into the room I would know it was the same two amplifiers but would not know if they had been switched or not. So my task would be to sit down and listen, switching back and forth and try to come to the same conclusion as I did before comparing with the same tracks and identify which of the amplifiers was A and which was b.

When done, I would leave the room and we would do the whole thing again, maybe 10 times in a row in a day. At the end of the day if with this process I was able to identify the amplifiers correctly say nine times out of ten, that would be a significant result.

On another day, the same process could be followed with another pair of amps. You can see that this could turn into quite a long, drawn out process with multiple people and multiple amplifiers. You can also see that someone else might try the same method and have it absolutely not work for him at all. And it would become difficult to find a way to work with a listening preferences of each listener and still end up with what one can call statistically valid results because it almost ended up being like different kinds of tests for each listener.

That is a problem that I see with throwing around broad statements like, can you prove it in a double-blind study? Which double-blind study? Who sets it up? What are the conditions?

Some will say that it is wrong to tailor the test to the listener, that it invalidates the study right off the bat. and again I would say that it depends on how you define what you were trying to accomplish. For the kind of differences we are talking about, I will go out on a limb and predict that if the only way it is approached is by trying to come up with a single generic test that has to fit all listeners with their different critical listening styles, then the testing is bound to show that those differences cannot be heard consistently across a broad listening audience under that kind of test.

But if somebody gets their gumption together to define an approach to accommodate individual listening style and crunch the numbers together at the end, and include the information about what those styles were, then we may someday end up with a real in depth test that shows that those differences can be discerned consistently. This could even be done in a way which accommodates those who prefer long term listening tests, as some say that that is the only way to really hear some of the fine differences. That has not been my experience so far, but it is very close minded of me to assume that it cannot work for someone else, or even to assume that would not work for me if I really give it a proper chance over time.

I would like to note one observation that I find somewhat humorous. In the ABX testing, my success rate at identifying the X amplifier was the worst of the whole bunch of us. I was wrong six out of seven times. In a way, that result is the most statistically significant of all the listeners at the event, I just had a mental flip-flop of some kind going on that led me to the wrong answer almost every time.


My Observations

These differences are comparative in nature, and almost impossibly small. I would never expect to be able to walk into a room and hear one of these amplifiers playing and say, "Hey, I recognize that particular sound as being the Parasound amp,” or the Krell amp or any other particular amp. And with my experience at this so far, I would be suspect of anyone who claims that they could.


Day 1, Sighted Pairings:

1 - Krell vs Parasound:
Krell, bigger sound
Parasound, not as big

2 - Denon vs Mark Levinson
Denon, brighter
Mark Levinson, rolled-off high end

3 - Emotiva vs Pass Labs:
Emotiva, slightly bigger bass
Pass labs, tighter
This was a fun pairing, I liked both amps.

4 - Van Alstyne vs Wyred4Sound, no difference noted

5 - Behringer vs Sunfire
Behringer, les bass
Sunfire, more bass

6 - Exposure vs Krell, no difference noted


Day 2, Blind Pairings:

1 - Denon vs Behringer
Denon, crisp highs
Behringer, silky highs
I preferred the Denon.

2 - Denon vs Mark Levinson
Denon, bass not as clear
Mark Levinson, bass seemed tighter, clearer
I missed the rolled off high frequencies of the Mark Levinson, which I heard in sighted testing.

3 - Exposure vs Parasound, no difference noted

4 - Wired for sound vs Emotiva
Wired for sound, solid highs, lively dynamics, richest string tones, punchy bass
Emotiva, punchy bass
I preferred the Wyred4Sound

5 - Wyred4Sound vs Sunfire
Wyred4Sound, a little shrill
Sunfire, alive, nice highs
I preferred the Sunfire

6 - Denon vs Pass Labs
I noted no difference between these two amplifiers, but my comments were that they were both very even, accurate, transparent, and natural, and that I'd like either of them.

7 - Denon vs Van Alstyne
Denon, okay, not quite as clear, a normal amp sound
Van Alstyne, super clear and detailed, space around all the sounds
I preferred the van Alstine


Future Work:

How about removing the room from the equation? Use the same setup, but at the speaker terminals attach an attenuator pad and buffer amp with leads to a different room, feeding a class A headphone amp and low-distortion headphones. With the right headphones, I can readily hear differences between headphone DAC/AMP models I am reviewing. Just an idea.


Conclusions:

The main takeaway here is that the differences are incredibly small, difficult to hear, and difficult to test for in a provable way. I would have probably been happy with any of these amplifiers if I had to walked into a room and heard it all by itself. I doubt I would have been able to say that any one of them was better or worse than any other under normal listening circumstances.


----------



## Tonto

Well written Wayne, that kind of straight forward honesty is what HTS is all about! I saw you had responded to this thread & I immediatly started imagining what you might say. I actually had thoughts about setting up a session with cans like you said! Wow, is the cosmos coming together or what! Interesting theory, but like you said, very hard to hear differences.

One thing I might add is to never know which amp is which. If you built a false wall between you and the amps, and just labeled them A, B, C, D etc. Then made the list of pairings so everybody got the same exposure/chance to put through their paces. It might help to take out any preconceived expectations. No need to know which amp it is at all! Kinda simplistic untill you think about it.onder:


----------



## AudiocRaver

Tonto said:


> One thing I might add is to never know which amp is which. If you built a false wall between you and the amps, and just labeled them A, B, C, D etc. Then made the list of pairings so everybody got the same exposure/chance to put through their paces. It might help to take out any preconceived expectations. No need to know which amp it is at all! Kinda simplistic untill you think about it.onder:


You are right, as long as they are always called the same name, they could be letters, numbers, mineral names, whatever, to help remove any bias.


----------



## lcaillo

Wayne and I are of one mind on this. I was not much better than he was at matching the amps to what I thought I heard. I found myself asking the question, was it A or B that was more like X. I was taking notes but would still get confused. I have terrible short term memory for random data and have to repeat things to myself to be able to recall at all, and the short time was a problem for me. I would focus on what I was hearing and feeling and did not have time to let myself really get into the music. I was more successful when I just listened for feel rather than specific characteristics to compare the amps. The ones that I got right were purely on what the sound felt like to me, with no specific notes.

I have a bit of experience in testing and in behavioral research with multiple trials. One thing I have learned that was confirmed here is that subjects have to get comfortable with the testing context before you can get reliable results. I can see why so many people get hysterical at the idea of AB or ABX testing. It is very different than the way we normally listen. 

I really believe that there were differences at times, but VERY small. It will take much more time and focus on just a couple of amps to tease them out consistently if it is possible at all. Looking back on it, trying to compare so many amps was a fools errand, even if it was tremendous fun. 

I'll pull out my notes from the first day and post them. My impressions of the amps were sighted, but I can be sure that they apply to the right one. I got mixed up too much on the ABX comparisons to be sure that the comments would be about the right amp. The could easily be backwards.


----------



## Serenity Now

Yay! Thanks for more details. Very interesting observations about ABX testing and possible criteria for improving reliability. :T

I hoped my initial (abnormal) terse prodding was enough to get a response and not leave a lasting bad taste in your mouths about reader feedback. You guys always do great work for us who like to live vicariously through these sort of reports.  THANK YOU! :reading:


----------



## AudiocRaver

Leonard makes a good point, that time to really get comfortable and familiar with the test environment is a good thing. Some of those contrasting impressions I feel could be repeated in the right conditions. Some were still quite fresh and could have evolved over time in their description. For instance, "silky highs" vs. "crisp highs," what does that even mean? If I had another half hour or 45 minutes to really investigate that dimension of a pair of amplifiers, would it did end up a completely different description? Would that contrast have become easier to hear and identify somehow? Might it even have ended up disappearing altogether, something totally imagined? These are all possibilities.

The human imagination is incredibly powerful, I do not understand why it is so difficult for some people to accept that it can affect our hearing, too. I have had it happen to me. I have nothing against faith, nothing against trusting that I can hear something even if it cannot be measured. And I feel no need to prove to someone else something that I know is true and repeatable, especially if it can be replicated from scratch in a different environment. But I am going to need to prove it to myself to be sure I did not make it up. Repeated testing, perhaps over several listening sessions, perhaps over several days, may be needed to get those initial impressions sorted through and settled down to real repeatability and meaningful description.

As a pure guess, I would say that my impressions above are 50% stable and 50% unstable, or in need of more time to mature and even be sure they were real.


----------



## lcaillo

So both of us were reliably wrong in identifying amps in ABX comparisons. What conclusion can we draw from that? I know we both worked very hard at trying to get it right, so my feeling is that it is the testing design that is flawed. We should have been closer to 50% if there was not some systematic bias going on.


----------



## gdstupak

lcaillo said:


> We should have been closer to 50% if there was not some systematic bias going on.


A completely random pick should result in close to 50/50, monkeys should get those results. You were not randomly picking, you were making educated guesses, which tend to result in more incorrect answers (info from Mometrex testing).


----------



## lcaillo

That is my point. The fact that we consistently got them backwards in a choice with even odds is curious in itself.


----------



## AudiocRaver

I have thought about that a LOT. It has to mean _something._


----------



## gdstupak

It does mean something. It means that you were wrong when you thought you were right (you thought you were picking the correct amp, but you actually picked the incorrect amp). I don't mean this in a demeaning way, it's just what the test results mean. You could not tell which amp was which, and because you tried to pick the correct one, it usually will lead to the incorrect amp.


----------



## lcaillo

If the testing design was not flawed, and there was not a difference between the amps, we should average around 50%. My point is that there is something going on beyond chance, which means that there is either a flaw in the testing design. My point is that ABX testing is not as objective a method as many would suggest. 

Yes, we were more often wrong than right. It is not demeaning it is just data. For data to become informative, you have to attach some meaning to it. If the results were closer to random, I would be less critical of our methodology. To be consistently wrong is very curious.


----------



## primetimeguy

lcaillo said:


> If the testing design was not flawed, and there was not a difference between the amps, we should average around 50%. My point is that there is something going on beyond chance, which means that there is either a flaw in the testing design. My point is that ABX testing is not as objective a method as many would suggest.
> 
> Yes, we were more often wrong than right. It is not demeaning it is just data. For data to become informative, you have to attach some meaning to it. If the results were closer to random, I would be less critical of our methodology. To be consistently wrong is very curious.


Results would average 50% over a large sample size. In a small size you will have people that get them all right or all wrong, so. I don't think you can draw much from a 2 person sample size. That being said, it is interesting and may warrant further investigation.


----------



## lcaillo

The sample size was 28 (4 observers x 7 comparisons). Out of those 28 trials we got 11 correct (most of those were thanks to Dennis, BTW). The probablility of that if there was a .5 probability on each test would be about 11%. That is certainly not low enough to conclude with a high degree of certainty that the test was biased, but it is still pretty unlikely for a fair test at .5 probability per trial. 

I understand statistics better than most, as I was a math teacher and did behavioral research in grad school. All research has hidden biases that are hard to tease out. Increasing the number of trials makes it less likely to err in one's conclusions when all of the significant variables have been controlled for. Below a certain threshold, however, you are still in guessing mode, and that is where we remain. When there, you have to make educated guesses at how you can make the testing more reliable in getting at what you are looking for and how to minimize the effects of unintended variables. Confusion in recall was certainly an issue for Wayne and me, less so for Joe, and not much so for Dennis. If we just considered Dennis' results he was correct enough to be statistically significant, but it would be unfair to do so. You don't throw out some of the data to get the result you want.

The bottom line is that the patterns of the data do suggest a problem with the methodology, at least for some of the subjects. In the future we will account for that.


----------



## AudiocRaver

What if we had a switch box with A, B, and X, could return to X at any time, take auditory memory out of the equation?


----------



## gdstupak

My point from post #224 is that you do not have a 50/50 chance at getting the answer correct. To get a 50/50 result, the answers have to be picked at random.

You were not picking your answers at random (i.e. flipping a coin, or choosing an answer before you saw the question), you were using judgmental guesses which alters the outcome from a 50/50 result. If you could correctly distinguish a certain amp, then the results would have been skewed toward more correct answers. If you could not correctly distinguish a certain amp, then the results would have been skewed toward more incorrect answers.

Using random picks usually comes to a 50/50 result. Using judgmental guessing does not come to a 50/50 result. 

Your results were skewed toward more incorrect answers, this tells us that you could not correctly identify the amps. It does not mean that the testing was flawed.


----------



## chashint

Error


----------



## AudiocRaver

chashint said:


> (Full disclosure, I think amplifiers sound the same if not driven to distortion)
> My training is electr


I do not mean this in an argumentative way at all. Just seeking a little further clarity. I think part of what we are learning from this, is that there is an awful lot of fine detail in the "sounds the same" part of that statement, which we all have a tendency to throw around freely, myself included. Under normal listening conditions, we are processing so much so fast, and if an amplifier doesn't sound bad, or if it sounds pretty good, then we think of it as being good enough and we are happy. When we start listening really close for detail, are there a little differences that might be audible? Little differences in the way that soundstage shows up as a result of crosstalk in circuitry or in power supply circuitry? Distortion of a slightly different nature in this amplifier vs that amplifier, both good amplifiers but with slightly different sonic characteristics resulting from bias circuitry design? And part of the question along with all of that, is it worth the trouble to try to hear that level of detail? If it is not something that jumps right out at you, why worry about it? That is Sonnie's way of looking at things. And for most of us most of the time that is not a bad way of approaching it. 

But remember also that the purpose of this study and studies like it is to try to determine "can we hear a difference" not "is it worthwhile to try to hear a difference?" They're totally different questions.

Just some things to consider.


----------



## AudiocRaver

For those who enjoy the more philosophical side of things, I always enjoyed Robert Pirsig's books, Zen and the Art of Motorcycle Maintenance, and Lila, two books about the philosophy of static and dynamic quality and the way we as humans tend to like to divide things into finer and finer levels of discrimination and categorization. It is kind of our nature. and it can be taken to silly extremes at times.:coocoo:


----------



## chashint

Sorry for that post, it was incomplete.
I got a phone call and somehow send happened.

I think there are many things to consider about audio/listening testing.
There was another thread a while back discussing acoustic memory and filling in the gaps between two recordings that were supposed to be the same, but one had something subtle added to it.
Once that subtle addition was heard the brain simply added the missing info to the other track.

I may have the finer details of that a little skewed, but if the above scenario can happen with audio that actually is different then trying to differentiate amplifiers which all have stellar electrical specifications is pretty much impossible.
I have yet to see a better listening test method described than the blind listening test method, but I am open to the possibility that once something is heard the brain may fill in any missing pieces on the next essentially similar thing that is heard.
If this is happening it would be reasonable for it to be cumulative.

I do think amplifiers (including AVR amplifiers) sound the same, if there are differences to be heard between systems it would (IMO) be more likely for those differences to be in the front end.
Even if amplifiers do sound the same that does not mean people shouldn't want to own an amplifier, HiFi and HT for the enthusiast are in large part about playing with different things.


----------



## AudiocRaver

chashint said:


> I have yet to see a better listening test method described than the blind listening test method...


After our experience, I find myself wanting to ask, What blind test method? What methodology specifically? And I do not mean that as a challenge to you personally, just trying to make the point that the specific approach of the test can make a difference, as we found, and those specifics and conditions become an important part of the "can we hear a difference" question.


----------



## Savjac

Is there a possibility to convince anyone to pass on the short term ABX tests that have many participants and numerous amplifiers that would by my definition, confuse matters more than clarifying things ?

I think first a reviewer should get very familiar with their software and as was done in the tests performed in these pages, limit what will be listened to. 

Second, tests should be limited to no more than two amplifiers. While is would be best to have a static amplifier in that it should be one that the review would be familiar with. All other equipment should remain the same throughout the testing process. 

Time, this takes time and as such, maybe two or three full nights should be spent listening to one amplifier and then the same amount of time for the second. Once the first two time spans were completed, then less time can be used to swap back and forth between the two amps being tested. 

I would postulate that if there is a difference of sufficient importance, it will be heard. There will most probably be a difference and some of these have been described above by Craver. 
Take the time to make these listening sessions worthwhile, not rushed. I do not see where AB testing works in anything much. Coke/pepsi failed, various types of water were tested and it did not work either. Some things that we are particularly sensitive to are easily recognized, different brands of bacon are good examples. That takes up several senses at once. 

I know this method does work, so try it. You might like it.


----------



## AudiocRaver

I find myself more in favor of the idea of limiting variables until you really feel you have your arms around them. Perhaps two amplifiers at a time is a really good way to get started with this.


----------



## chashint

I am not trying to be argumentative either

I used to be dead set certain blind listening tests were irrefutable until I saw the discussion (could have been another forum and sorry but I cannot find it now) about the music tracks being altered so one had additional sounds and after the listeners had heard both tracks a few times they started also hearing the additional sounds in the track where they did not exist.
I am aware that the psychoacoustic abilities of the human system are very powerful and I am at least open to the possibility that blind A/B testing could be fundamentally flawed.

This is what I have seen as the most often cited amplifier testing protocol.
http://tom-morrow-land.com/tests/ampchall/ 
It seems straight forward and reasonable enough.

I am willing to bet a coke, that if you sat a $300 AVR along with some electronic project boxes that would presumably switch the source and speakers between the AVR and someone's awesome HiFi rig and the owner/listener had a clicker that turned on a green or red LED on one of the boxes to indicate they were listening to the AVR or the awesome HiFi rig they would hear a difference when the AVR LED was lit even though no change had actually occurred.
Repeat the test unsighted, and A/B would become indistinguishable too the listener.


----------



## DrStrangeQuark

Hi All --

New member here.

Recently at the Axiom boards we were having a discussion on amplifiers that ended up linking over to your blind high-end amplifier shootout. Reading through this thread inspired me to set up an account here.

It sounds like you guys did a great job, and the conclusions seem reasonable. My personal takeaway would be that if two solid state amps have sufficient power to avoid clipping (which is not always a given) then they will probably sound much more similar than dissimilar. Not everyone agrees, and definitely not everyone wants that to be true, but it is really good news for most of us. It implies that you can get into the regime of hair-raising (and hair-splitting) performance on a reasonable budget.

This is my favorite line of the whole deal:



chashint said:


> A second wise man said ..... "There's something about those trademark McIntosh analog meters that turns many a mere mortal into Pavlov's dog ...drool... but I'll never be able to afford one."
> That is pretty much the only amp I want to own and just for the record it would sound better than my Pioneer even if it sounds exactly the same....


I get it, and don't disagree that some of the enjoyment of owning a boutique product transcends the question of how it sounds.

I was caught by another comment in the thread about the statistics of incorrect blind associations and the question of how unlikely the same result would be in an unbiased "coin flip" type experiment.



lcaillo said:


> The sample size was 28 (4 observers x 7 comparisons). Out of those 28 trials we got 11 correct (most of those were thanks to Dennis, BTW). The probablility of that if there was a .5 probability on each test would be about 11%. That is certainly not low enough to conclude with a high degree of certainty that the test was biased, but it is still pretty unlikely for a fair test at .5 probability per trial.


I'm likewise a bit of a math junkie, and did a quick independent calculation of the expectations for this experiment. However, my results are coming up somewhat differently than what was quoted and suggest the conclusion that the anomaly is maybe just at the level of one third, say 34:66. It may boil down to a difference in assumptions, but let me briefly describe how I would approach the problem from first principles:

The probability of getting N "heads" out of M coin flip trials, where each trial is a 50-50 shot should be P(N,M) = (.5^M) * (M)! / [ (N)! * (M-N)! ] . A digression for the curious: The exclamation point there means "factorial", or "multiply by all numbers smaller than yourself". For example, 4! = 4*3*2*1 = 24. This is the number of distinct ways to rearrange the order of a number of objects. In the prior example, there are four ways to choose who goes in spot one, then three ways (for each of the four prior) to choose who goes in spot two, and so on. What we need to count is how many distinct ways there are to get N heads out of M flips -- more possible ways to do it means that a given outcome is more likely. This will be the total number of ways to order the M flips divided by the ways to reorder just the N indistinguishable "heads" and just the (M-N) indistinguishable "tails". For example: If there are M=3 flips, and N=2 heads, this can happen three ways. Specifically, heads can occur on flips 1&2, or 1&3, or 2&3. Correspondingly, this is 3! / [ 2! * 1!] = 6 / [ 2 * 1] = 3. To turn this count of "ways to do it" into a probability, multiply by the probability of a single flip (.5) to the power of the total flips (M). If the coin were biased, with a probability "p" other than 0.5 of being heads on one flip (and 1-p of being tails), then the factor would be (p^N) * (1-p)^(M-N) instead. One can check that the sum of the probability formula for P(N,M) from N=0 up to N=M is indeed one.

The expected value is the average over possible outcomes, specifically N*P(N,M) summed over all N from 0 to M. This is 14 for M=28 (not too surprising). The relevant question then is how far a given experiment was from the expectation, and how likely it is to be at least that far from the expected result.

Let's stipulate in this particular case that to have a "more normal" outcome, the observation would have been in the range of, say, 12 to 16. It is important to notice that I am choosing to group together outcomes that are "more likely" or "more central" whether they have fewer correct answers or more correct answers than 14 out of 28. This "two-sided" probability computation is likely a key reason for the difference in my conclusion. The sum of the probabilities over this central region of the distribution is 65.5%. With these stipulations, for about 1 trial out of 3, the results (on a coin flip experiment) should be at least as far from the central value as was observed in the blind tests. I wouldn't classify them as a statistical anomaly at all. In other words, they would appear to me to be very consistent with the assumption of no reliably discernible differences, although it should be noted that the sample is still smallish.

Note: I wouldn't be too surprised to hear that the different conclusions are traceable to some different assumptions, perhaps a one-sided distribution and/or a pretabulated statistical table that perhaps does something more sophisticated to deal with small sample sizes, or etc. If you use the approach I described to ask "how likely is it to get 11 or fewer heads" (leaving out 17 or more) then the answers are already converging quite a bit.

Hope that is interesting / Cheers


----------



## AudiocRaver

Turning this into a sticky.....


----------



## Savjac

chashint said:


> I am not trying to be argumentative either
> 
> I used to be dead set certain blind listening tests were irrefutable until I saw the discussion (could have been another forum and sorry but I cannot find it now) about the music tracks being altered so one had additional sounds and after the listeners had heard both tracks a few times they started also hearing the additional sounds in the track where they did not exist.
> I am aware that the psychoacoustic abilities of the human system are very powerful and I am at least open to the possibility that blind A/B testing could be fundamentally flawed.
> 
> This is what I have seen as the most often cited amplifier testing protocol.
> http://tom-morrow-land.com/tests/ampchall/
> It seems straight forward and reasonable enough.
> 
> I am willing to bet a coke, that if you sat a $300 AVR along with some electronic project boxes that would presumably switch the source and speakers between the AVR and someone's awesome HiFi rig and the owner/listener had a clicker that turned on a green or red LED on one of the boxes to indicate they were listening to the AVR or the awesome HiFi rig they would hear a difference when the AVR LED was lit even though no change had actually occurred.
> Repeat the test unsighted, and A/B would become indistinguishable too the listener.


Good Morning Charlie, this is Jack and I am inviting you to my home to listen to 3 high quality amplifiers on my present reference systems and with a bit of training on what to listen for, you will be able to tell the difference closer to 100% of the time than you think. And I will buy the Coke (soft Drink) for your pleasure once you grasp how easy it can be. No challenge here, just an offer should you wish to travel just a few hours. :smile:


----------



## chashint

Your offer is very kind and generous, it is greatly appreciated.
If I ever make it to your part of the country it would be an honor to visit with you.


----------



## subwoofery

Savjac said:


> Good Morning Charlie, this is Jack and I am inviting you to my home to listen to 3 high quality amplifiers on my present reference systems and with a bit of training on what to listen for, you will be able to tell the difference closer to 100% of the time than you think. And I will buy the Coke (soft Drink) for your pleasure once you grasp how easy it can be. No challenge here, just an offer should you wish to travel just a few hours. :smile:


What model do you own? :smile:

Kelvin


----------



## Savjac

chashint said:


> Your offer is very kind and generous, it is greatly appreciated.
> If I ever make it to your part of the country it would be an honor to visit with you.


The honor would be mine good sir.


----------



## Tonto

I still contend that the "blind" in these evaluations should be literal. The amps should be boxed/concealed such that no one knows what the amp is until the event is over. Switch as you would, listen, record your impressions & only after all is done, unbox & see what is what. Of course, the person doing the hook ups would know, but they, perhaps, should/would not participate.


----------



## Lumen

Double-blind supposedly prevents the test administrator from giving subtle clues through body language, facial expressions, etc. as to which is which.


Sent from my iPhone using Tapatalk


----------



## Savjac

Lumen said:


> Double-blind supposedly prevents the test administrator from giving subtle clues through body language, facial expressions, etc. as to which is which.
> 
> 
> Sent from my iPhone using Tapatalk


Like wearing a bag over my head ??


----------



## AudiocRaver

Tonto said:


> I still contend that the "blind" in these evaluations should be literal. The amps should be boxed/concealed such that no one knows what the amp is until the event is over. Switch as you would, listen, record your impressions & only after all is done, unbox & see what is what. Of course, the person doing the hook ups would know, but they, perhaps, should/would not participate.


Of course that is the best way to do it. It potentially makes procedures far more complex and possibly unmanageable/prohibitive. Bind works for me, generally, if carefully executed and thoroughly reported.

Edit: For me personally, this is partly because I enjoy experimenting but not all of the tedious detail involved in thorough scientific double-blind methods.


----------



## JoeGonzales

Sonnie the prankster showed up tonight with his Oppo remote app on his iPhone, messing with the Oppo track selection while the listener trid to select and listen to tunes.


----------

