# Double Blind Listening Test - DBT - Between Amps and/or Receivers



## Sonnie (Apr 11, 2006)

We have had the very long debates about whether we can hear differences between two good quality amps and/or receivers for fifty-eleven years... and I suppose there is about as many that say one way as they do another. The _debate_ is NOT what this thread is about (we have a thread for that here). This thread will be about setting up a blind listening test.

I am certainly not qualified to rebuke what someone else believes they can hear, regardless of what I believe I can hear or can not hear. Do I have doubts? Yeah, but doubts do not prove anything. I am definitely a skeptic of the magical nirvana that some claim they hear between different amps and cables and such. Again, that does not mean they are not hearing differences... my doubts prove absolutely nothing... I fully admit this. I do believe if there are differences, they are subtle, because I believe even I could notice dramatic differences.

I do not believe that I have ever really noticed a difference in amps and receivers that I have used, at least not when I was trying, which did not start until a few years ago and has been limited. For example, most recently I did not notice a difference between *A*: a pair of 1,000 watt monoblocks combined with a 300 wpc multi-channel amp connected to my receiver, which was used as the preamp, and *B*: the receiver's own amps. The monoblocks were powering my MartinLogan Prodigy's and when I switched to the receiver, I bi-amped the Prodigy's, thus using 7 channels of the receiver's 9 for my 5 channel surround. The multi-channel amp powered my ML Theater center speakers and my ML Ascent surround speakers... which were all switched to the receiver's corresponding amp channels. I have had this setup for several months now and have not noticed any differences, good or bad. That is not to say there are no differences, but that I simply have not noticed any. That is also not to say I did not like the monoblocks... they looked good, worked flawlessly... and I probably felt a little more comfortable with them in my system powering my Prodigy's, being they are somewhat of a more difficult load to drive than the average speaker, yet not as difficult as some others. Coincidentally, I do know someone that has a 100 wpc tube amp on his Prodigy's as I type.

In order to convince myself that there are differences between amps and/or receivers, I believe the only way I will ever notice if there is a difference will be via some type of blind listening test. I realize there are those that say we can not measure some of the differences we hear... and I am fine with that, but we surely can describe the differences we hear, and we should be able to do this in blind testing. If we can describe it, then hopefully someone else can also recognize the difference... and we can consistently repeat noting those differences. Therefore, I want to conduct a blind listening test between my receivers and maybe a couple of other good quality amps. Does it really make a difference in what receiver I have? For the record it is a Denon 4520, but I also have an Onkyo 805 receiver that we can use. I would like to do this using my OPPO BDP105 and my Arx A5 speakers in a two-channel listening test. Perhaps we can coincide this with our $2,500 speaker evaluation that will be held November 1st and 2nd. The testing would be conducted in my dedicated home theater room where we evaluate the speakers.

My first thoughts are that we hook up the receivers and amps individually with no processing (Pure Direct) on the receivers, connecting them to the OPPO BDP-105 media player and the Arx A5 speakers. We would then run an REW frequency sweep to check the frequency responses and match the levels. Have one of the panelist go into the room and connect one of the receivers or amps, while two or three others are outside of the room. All receivers and amps will be covered so that the unit playing cannot be identified. The three panelists will come into the room and listen, one by one, playing a 30 second segment of a music track they are familiar with. The segment would be played repeatedly ten times. Then they will temporarily leave the room while the receiver or amp is switched to another, return an listen to the same ten segments, writing down any differences they can describe. If we note differences, then we continue swapping amps and see if when we come back to the receiver or amp we noted the differences, if we can consistently note the same differences. Of course if we do not note any differences, there probably would not be any reason to continue through all of the receivers and amps repeatedly.

I have two receivers as mentioned above and I have a Behringer EP2500 two-channel amp. It would be nice to have a more expensive amp in the mix... perhaps one that others claim have made a significant difference in their system. If anyone has an amp they would like to volunteer, we would be happy to cover the shipping cost to and from... and of course take good care of it. It may be that we can get a manufacturer to ship us one, although I am not sure how many of the more expensive brands would be willing to participate in an event like this.

It would also be good to have suggestions on setting up and conducting the testing.


----------



## fmw (Aug 11, 2013)

*Re: Can we really hear a difference between amps?*

What is required is that you arrange things so that the listener has no idea which unit is playing at any time and that the levels are matched to within .1 db.

Ideally you would use an ABX switch box. One amp would be connected to A and the other to B. The listener can then switch back and forth at will and ID each amp. Since the listener operates the test there is no external or X element. You just score the responses right or wrong. Level matching is easily done with a volt meter across one set of speakers playing a sine wave or white noise. Try to get within .01 volt.

When we did the amplifier tests, we had a lot of labor available because the listening panel was 10 members of the local audiophile society. We set up a screen with everything behind it except for the speakers. It was an AB test because we didn't do the X part. We would then set up the two amps and calibrate the preamp to each amp with a digital multimeter and mark the position of the volume dial for each amp. We would have a computer generate a random list of A's and B's. We would play a musical segment of the listener's choice and switch preamp to amp cables and adjust the volume control for each iteration. That would take about 5 to 10 seconds. If the random sequence called for doing the same amp twice in a row, we would still go through unplug, plug and adjust volume the same way so as not to give the listeners a clue.

We would start each test by playing the musical segment for each amp for 15 seconds and then ID the amp for the listener. The the test would start. We would do 10 iterations of the random sequence and have the listener ID each playing of the musical segment as A or B. We did this for each of the 10 listeners on the panel. The scoring was simply correct or incorrect for each ID. 

In the end the scores would huddle around 50/50 right or wrong which indicates guessing. We had one amp - a single ended triode tube amp that scored 65-35 so it was certainly audibly different from the others. All the others, including a high end tube amp ended up with scores that indicated guessing.

That process isn't acceptable to an ABX proponent, by the way. Those folks want a switch so that the listener can switch back and forth instantly and at will. It requires different equipment. It is easy to do with input sources and very hard to do with amps or preamps unless you have a lot of identical equipment. Our method, however, did get the results an ABX proponent would expect so it wasn't terribly bad.


----------



## Sonnie (Apr 11, 2006)

I moved your post to this thread... will read over it shortly.


----------



## Tonto (Jun 30, 2007)

Not sure about what kind of switch is availabe, but I would think one set up could allow for the source to be connected to both amps & continuously feeding them. Each amp would then be connected to the speakers through the switch, which would choose which speker to send the signal to. I think user control of the switch might be nice, but not mandatory.

I would also op for a checklist. Say a particular kick drum/vocal/guitar passage, dynamics of a segment, width of soundstage, etc to pay attention to. We would have to agree on things to listen to in advance. Might make it more interesting when compiling resuts, as opposed to just a yes/no kind of situation. Mind you I am in no way an expert.


----------



## charlesj (Dec 4, 2012)

Interesting endeavor. Not easy. Have you considered contacting Tom Nousaine? He's been at this well before I met him in 1998.
A few comments to add in addition to fmw's comments. You will be hurting the effort of detecting differences if you have such a long time delay between amp switching you described or that I am understanding. Might as well not test. Acoustic memory is very short, just seconds, for detecting small differences whether you have a familiar track or not. 
Writing down subjective areas and behaviors is good when you grade speakers but you have to do that enough times to calculate a valid statistical result. If you want to know if amps sound different, then all you need is to identify an amp being A or B enough times to be statistically meaningful. You need 9 of 10, 12 of 15 trials or 16 of 20 trials. No need to know how wide the soundstage you think is or isn't or any other trait just consistently identify it to A or B. 
Have your listeners write down the answers to each trial; random amp selection by whoever is doing it and make sure he writes it down to compare his to the listeners at the end of the test. Also, you don't have to change amps for any trial nor have to have amp A and B presented equal number of times. Statistically speaking you could present A 10 times in that 10 trial run and see how well one can guess it correctly. I bet you nor the listener will like that answer.

Don't think using REW will be beneficial nor accurate enough. And, while you do need to run the amp/receiver in direct, no processing, REW would EQ and and is processing. I would suggest using test tones at say 100Hz, 1kHz, and 10kHz and measure at the speaker terminals within 1% between the amps.The voltage at each frequency will not be the same, just a reminder.

Good amps should be flat enough to pass this. This will level match to .1dB spl that REW cannot and that is what is needed. This is about how Tom does it.

I would also suggest a short segment to listen to, not 30 seconds, too long for memory, and another reason to be able to instantly switch between amps and repeat the looped music segment.


----------



## Sonnie (Apr 11, 2006)

I think we would still use REW to check the response to make sure nothing was over a db or so off. REW would not be processing anything to the equipment... merely measuring the response. And REW will measure to .1db accurately.

We will work on getting a switch so we can instantaneously change out between two amps.


----------



## lcaillo (May 2, 2006)

I have an Adcom GFA-5500 we can use. Not high end but generally considered a decent amp.


----------



## Sonnie (Apr 11, 2006)

Oh yeah... I think those are fine amps. I had a couple of the GFA monoblocks for a number of years.


----------



## J&D (Apr 11, 2008)

Agree that although writing down comments about how A and B sound or if there was a preference for one or the other can be fun the most important part of an amp vs receiver or amp vs amp blind test is discovering if anyone can actually identify A or B reliably. If you can then differences exist if not they essentially "sound" the same.

I also feel another crucial element to this test is to find at least one or preferably more than one participant that absolutely without a doubt swears they can hear differences in amplifiers and have them participate in the test. Even better is if they can supply the amp they stand behind. Now there will always be a way for anyone to invalidate the testing scenario i.e. testing gear is not of high enough quality to reveal nuances or I was not in the sweet spot of the room or the test tracks were unfamiliar to me but If everyone participating already admits they cannot hear differences then the outcome of the test is not all that interesting.

If you have time it would be great to do some blind cable testing. That can be a lot of fun as well. I would offer up my spare (currently for sale) Monster 3250 amp but that is one heavy beast and I worry it might get damaged in the round trip. It also has the Monster stigma surrounding it and you need at least one contender that would be hard for those observing to dispute its quality like Levinson, McIntosh, Krell, Pass Labs, Audio Research. Then don't be afraid to stack them up against not only what most consider a competent AVR but a low end unit as well. Throw in a pro-amp from Crown or QSC and you have the trappings of a very good test.

Looking forward to the results!

JD


----------



## Sonnie (Apr 11, 2006)

I think we might pass on trying to accomplish this during the speaker evaluation, simply because I think too many people are expecting too much to be done, and we simply will not have the time frame to do it properly. 

I suspect finding someone with a Levinson, McIntosh, Krell, Pass Labs, or Audio Research amp that will allow us to borrow it, will be next to impossible, as would getting one from the manufacturer. 

I also suspect finding someone who claims fanatically that they can clearly hear and identify the differences, and would be willing to participate, will be next to impossible.. 

Add one more next to impossible... it don't matter what you use or how you do the testing... there are some who are inevitably going to cry foul, thus it will be next to impossible to please everyone with whatever outcome there is. 

We will be trying to jump fifty-eleven hoops to please everyone.


----------



## lcaillo (May 2, 2006)

I am not concerned about the last one, Sonnie. If we do our best to be fair in the testing, those who value what we do will be very happy to see the results.

We just need to do what we think is useful and do it as well as we can. It will be of value to many.


----------



## selden (Nov 15, 2009)

If you can afford the time to do it, I think it would be a worthy endeavor, even if it doesn't satisfy everyone. I think there have been too few such attempts, probably because people who might be able to try are defeated by negativism before they even start.


----------



## Tonto (Jun 30, 2007)

Agreed, time is going to be hard to come by during the speaker shootout. And, I still say that some quailfiers are needed. I think people want to know what it is exactly that is better about the amp...not just "this ones better."

Or at least..."I can't tell the difference."


----------



## mlundy57 (May 21, 2013)

Is the intention to compare power amps only or will you also be looking to see of there are differences in pre-amps and/or integrated amps?


----------



## Sonnie (Apr 11, 2006)

Yes... strictly power amps.


----------



## AudiocRaver (Jun 6, 2012)

Here is one way to do it. Some will say it is cheating. Do an A-B-A-B-A sequence, switched when the listener wants it switched, and the listener gets to identify the preferred sound or some difference in sound between the two. Then repeat that test multiple times, only sometimes it is A-B-A-B-A and sometimes it is B-A-B-A-B, the listener knows it could be either way, but never knows which it is. After each 5-step sequence, the listener has to say "A is the good one" or "clearer one" or whatever identifier they chose, each time identifying either A or B as their choice according to some descriptor. The switcher rolls a die or flips a coin or something to randomize, keeps track of the sequence starting on A or B, do the whole thing 10 or 15 times over a few hours with breaks, or over a couple of days, see how many times the listener was right. This gives the listener the best possible chance to hear the difference, but still introduces randomness. It is blind, but not double blind - I think blind gets you 90% of the way to a fair test. This is for fun, not life or death. If it is a contest to the death, you definitely want double-blind!

How to do the switching? It takes 2 sets of switches, a stereo input selector switch (easy) and a very low resistance high amperage set of switches for speaker current (a large automotive or even household switch?), to be switched only with no signal running - some amps don't like loads being connected/disconnected while pushing current. This assumes the amps each have level controls and level matching is done with REW and pink noise and/or a few sweeps to 1/2 or 1/4 dB, I don't think 1 dB is close enough for broad-band material. There should also be a common level control so the listener can decide listening volume at will. Listener also gets to select material (probably one passage from a song looped over & over). Just not which sequence.

That is one approach.


----------



## Savjac (Apr 17, 2008)

This is truly a daunting challenge for all parties. I am one that believes in differences, very strongly, but of course I have a large kink in my wrench so to speak. I truly believe that short term listening and constantly repeating the pieces just seems to confuse the listeners. I have done this myself and found that one must truly fall into the piece of music and it is unlikely that will happen in short bursts.
Since this thread has been ongoing I have really worked on letting myself be as open minded as possible. The more I listen the more I question the scope of the differences and are they repeatable. I am just not sure all the time. It seems so much is involve, mood, music even sinus issues can change how we hear.
I am soooo confused, but would love to hear the outcome of all this. I do not have any more special amps, down to a Denon receiver and an Emo XPA powering my Martin Logans, but none the less they sound good.
I will watch this space with interest.


----------



## AudiocRaver (Jun 6, 2012)

Savjac said:


> I truly believe that short term listening and constantly repeating the pieces just seems to confuse the listeners. I have done this myself and found that one must truly fall into the piece of music and it is unlikely that will happen in short bursts...
> 
> Since this thread has been ongoing I have really worked on letting myself be as open minded as possible. The more I listen the more I question the scope of the differences and are they repeatable. I am just not sure all the time. It seems so much is involve, mood, music even sinus issues can change how we hear.


Some have real expertise in this whole area, I do not. While my ability to hear sonic detail is, as far as I can determine, "pretty good" (how do you really know??), I am new to the realm of hearing differences between good-quality amps, preamps, DACs, cables, and the like, just have not paid much attention to it.

It seems perfectly OK to just enjoy differences one believes one hears without regard to conditions and variables, but when one starts recommending that others make this or that purchase, it would be nice to have some repeatable basis for making those judgments.

Is listening in short comparative bursts better than settling into an experience? Don't know. Maybe a combination of both. I will be trying both approaches as time allows in coming months, will see what they tell me.


----------



## fmw (Aug 11, 2013)

Savjac said:


> This is truly a daunting challenge for all parties. I am one that believes in differences, very strongly, but of course I have a large kink in my wrench so to speak. I truly believe that short term listening and constantly repeating the pieces just seems to confuse the listeners. I have done this myself and found that one must truly fall into the piece of music and it is unlikely that will happen in short bursts.


No question about it. That is the basic issue with subjective sighted and objective bias controlled comparisons. If you can't identify one product from another in a bias controlled test, then they sound the same. It doesn't matter how long you listen. It only matters that there is no hearing bias in the testing. The short listening segments with a switchbox make it easier to detect audible differences, not harder. The longer you listen and the longer the period between iterations, the harder it is to detect audible differences. A subjectivist should prefer the rapid fire switchbox method. It will give him a better chance of doing an accurate ID.


----------



## lcaillo (May 2, 2006)

If we are designing a methodology to detect differences if they exist, and we have subjects that report greater ability to detect those differences with one form of listening design or another, it seems that we need to accommodate that preference as long as we maintain blind conditions. To do otherwise is to assert one's own assumptions as an additional variable. You may believe that switching quickly may make it easier to determine differences, but that belief deserves the same level of scrutiny that the existence of differences receives.

I can understand how short term listening can be distracting and confusing. We become much more attuned to subtleties with extensive practice on any task. There may be room for both methods in a truly fair test, that is if we really want to find differences that might exist. My issue with objectivists (which position I tend to favor, to a degree) is that they are usually not as objective as they claim. Many hold fast to the belief that there are no differences just as firmly as those who take the other side, with just as much bias.


----------



## Savjac (Apr 17, 2008)

As much as humanly possible I am trying to be open to most any path that lies before me, differences or no differences, so be it.
I have to, imo of course liken this issue with many others in life wherein we tend to rush into a choice without giving time to really run our item through many different hoops which does take time. A car would be a good example for me, having recently made this fateful decision. I was in the market for a new vehicle to replace my good ole truck and looked at a number of sedans of medium grade. One of them really stood out as being efficient, very quiet, relatively comfortable and provided with sufficient power to get on and off most roads.
I bought it and after a couple long trips I found it was truly annoying, my legs could not get comfortable, I could not sit properly over the period of an hour or so and something I could not identify forced me to want to exit the car every hour or so to walk around. The electronics in the dash were awesome but completely indecipherable to someone like me...old and senile. Now I buy a car every three years or so, and I am no newby when it comes to that but the newer vehicles seem more and more alike each year but little quirks pop up that oft times make the joys of owning a new car, less than joyous.

So this long story just says, there is no way we can comprehend everything a complicated piece of equipment provides to our listening experience in short term blasts. There are some components that thrill at the reproduction of jazz or chamber music and conversely there are those beasties that can truly put forth a visceral experience of a full orchestra playing at full tilt without running out of breath. As discussed in a different thread I now have a low powered tube amp driving a high efficiency single driver speaker and there are times when this coupling is pure magic. But one must look a bit to find it. Would it be good for every day music throwing Tommy as well as Peter Paul and Mary at the combo....probably not. One other thing that comes from this, is that I can say without any worry that one cannot hear the "Tube" sound in this little amp so cleaver was its design.

Where am I going, like all things in life, trying to pack a bunch of stuff into a very small window may cause us to miss the very thing we are looking for.


----------



## charlesj (Dec 4, 2012)

I don't think anyone is saying that only short segment rapid switching is acceptable as long as the testing is under double blind conditions and meets statistical significance. If you have the time, you can listen for hours before changing amps in this case before making a decision which amp is A or B but you have to do this to a statistically accepted outcome. 

In reality there is a published amp DBT that was over a month long. Didn't help. Experiments by highly knowledgeable people in the field came to the conclusion that short, rapid switching is best to differentiate small differences.

Don't forget, I think this test is supposed to detect audible differences, not a preference.


----------



## AudiocRaver (Jun 6, 2012)

It seems a valid argument that there might be some differences that become clearer with time. For a short-term A-B test, we assume that the "right" passages of music have been chosen for that particular listener. That listener might hear no difference under those conditions, but there might be other passages where the difference stands out, only to be discovered over time.

As already stated, the key to validity lies in finally getting down to a statistically significant level of repeatability, whatever the length of the test.


----------



## Savjac (Apr 17, 2008)

Right, there does need to be some basis and ending time to which one must admit that either something exists or does not, either way. The testing cannot go on forever as in some magazines, but indeed one should be able to listen to a fairly wide range of material over ones own system, or a similar system, to really feel comfortable making a decision.


----------



## english210 (Sep 5, 2013)

For the sake of satisfying my curiosity, I might be talked into sending you my amp. I still have the one it replaced, so I'd have sound still....It's an Odyssey Stratos, 150wpc, nothing esoteric. MSRP is around $1400, I bought it used for a lot less. It replaced a Yamaha that retailed 20+ years ago for around $900, IIRC. The difference it made in my system was noticeable, but there is the question of how much of that difference was the load my speakers presented to the amp? Perhaps an easier-to-drive speaker would have rendered the differences harder to hear, etc? I noticed the difference at moderate listening levels, but the greatest difference has certainly been at higher levels, and this is the first amp I've heard drive these speakers to distress. 

Originally I had an old Yamaha receiver running this amp into my speakers. 

I replaced the receiver with an RXA3010, and initially used just the receiver to see how it would sound. Not bad.

Then I added the outboard amp back in, expecting to hear a difference, but actually didn't.

Knowing that Yamaha isn't the pinnacle of sound quality I started shopping for an upgrade, was pointed in the direction of the Odyssey, talked to Klaus who owns the company and was familiar with my speakers. While saving up to buy one, a used one came along and I figured it would be a safer bet in case it didn't do what I wanted. I was looking for something to clear up the highs at higher volumes, since when I really pushed it (-0db or higher on the volume scale), the sound got unlistenable. What I noticed immediately was a strength and control and extension in the bass. It did clear up the highs like I wanted, and drives the speakers to distress before that implosion of the highs like before.

Does my experience qualify under what your question is?

Oh, and Sonnie, the Odyssey REALLY likes Pink Floyd, as well


----------



## english210 (Sep 5, 2013)

AudiocRaver said:


> It seems a valid argument that there might be some differences that become clearer with time. For a short-term A-B test, we assume that the "right" passages of music have been chosen for that particular listener. That listener might hear no difference under those conditions, but there might be other passages where the difference stands out, only to be discovered over time.
> 
> As already stated, the key to validity lies in finally getting down to a statistically significant level of repeatability, whatever the length of the test.



I would think that the process would be like auditioning anything anything else - when I go speaker shopping, I have my own music, and specific passages I listen to, and certain speakers bring out certain details on certain tracks more so than other speakers. Some speakers reveal details and nuances, other speakers make me forget about the details and just draw me in to the involvement of the music - guess which ones I like - but wouldn't a blind test be looking for the same things? If neither amp 'showed' something the other didn't, we could say they sounded the same, to all intents and purposes?


----------



## Savjac (Apr 17, 2008)

As we all agree, there are so many variables to equipment that some minor differences may be hidden by the cabling, the room, the furniture in the room, the processor/pre amp, speakers etc etc. 
Not all things work well together, so what may sound good in one room with a certain set of goodies on the front end or back end of the system may not do so well in another situation. 

I am more or less torn in my thoughts in that I think we can agree that pre amps can sound different from one another as do DAC's, CD Players, turntables, cartridges and on and on. These items house similar yet Different components that alter the electrical signal passing through based upon the sound that the designer wants reproduced on HIS or HER particular system based upon HIS or HER biases. It would then, to me, appear a reasonable assumption that an amplifier, loaded with a wide variety of varying components would have the same issues as any other piece of electronics. Irrespective of the straight wire with gain goal amplifiers will be voiced by their designer. As we have discussed in other passages, some electronics do not work well with my electrostatics but work very well with moving coil designs. There are differences, maybe subtle to be sure, but if one lives with the item and wraps their heads around what the amp is doing or not doing as it were, the subtleties have to be laid bare at some point. I don't want to get into a pricing issue as there can be stinkers at the high end of things as well as at the lower end. Class "D" will not sound like class "A" or "AB" and class "T" really does not sound like any of the others. Maybe it is just my bias and my inner doubter telling me this, but I have experienced the differences and even changing out certain components in a piece of electronics can change the sound, ie better caps etc, so....well y'all get my point. :T


----------



## Sonnie (Apr 11, 2006)

I thought the only singer Odyssey amps liked was Johnny Cash... or some good old western music... or maybe some Tom Jones. lol That Klaus is something else. I have heard his systems at two shows... they always sound great.

Your Kappa 7's are 88dB sensitivity, so I think they will need a good bit of power to sound right at higher volumes. There may not be any difference between the Yamaha receiver and the Odyssey amp if you used the receiver in two-channel mode and played it at moderate levels. However, if you really crank it, I believe that receiver would run out of juice pretty quickly... where the Odyssey amp should hold its own.

So yeah, you may notice it more so with your speakers than you would with others with higher sensitivity... when played at the higher volumes. 

That would be a good amp to include. However, we have a LOT more stuff to work out... mainly getting someone on the panel who believes they can hear a difference in amps. The of course we got other things to work out. I think it will be quite a bit later before we get to this... the speakers evaluations are likely going to take precedent for now.


----------



## Savjac (Apr 17, 2008)

Not exactly on topic but have you heard the Tom Jones cd Praise and Blame ? WHOA, that is a stunner in way of quality...except the bottom end troubles me a bit.

I have been playing a bit more in the amp vs amp game, 2 channel only, and while listening to several songs in a row, then switching out cables and doing the same thing on a replacement, it seems there is an odd issue. More often than not, when switching from say amp A to amp B, I may not hear a substantive difference that I can point to, however, oft times when I switch from amp B to amp A, the difference is more noticeable. It seems that maybe the changes in going to something I am not familiar with to something I am familiar with may set off a reaction in my brain that I seize on as differences. Very curious indeed.


----------



## rab-byte (Feb 1, 2011)

ABX testing is defiantly your best bet. 

Start with X... "Audience this is X"

Then A... "Audience this is A"
Then B... "Audience this is B"

Then X... "Audience this is X again"

"X is A or B. Now you'll find forms in front of you. We'll play A and B about 10-15 times. Please indicate when you think we're playing amp X."

Randomly generate an AB sequence ahead of time and follow that. At the end of the listening test collect notes and compare the listeners evaluations. 

This testing has been proposed for all manor of things most notably in this context the million dollar speaker cable challenge. After every 5th A/B play X again and let the audience know you're playing X. 

To accomplish this test best I would split the signal coming off a single transport and pre-EQ both amps. 

Big point to raise here! You may want to identify if you're testing for sonic differences pre or post EQ.


----------



## Savjac (Apr 17, 2008)

How about a slightly askew position from a slightly askew listener ?

In reading many posts and thought pieces both here and elsewhere, it appears that overall there is only one expectation that comes from blind testing, be it AB, ABX, ABC, ACDC or any combination of short term quick change listening to try and detect differences in audio and that is the finding of no substantive differences. Similar to the tossing of a coin, the testing will produce, more often than not, predictable patterns and/or answers. To keep trying to reach a conclusion based upon these criteria seems to be an exercise in what Einstein defined as insanity. If we were to agree that the rapid change blind tests will result in "X" findings, as they always have, maybe it is time we changed the testing to see if another type of listening pattern will result in similar findings, or different findings. No matter how we wrap the blind/double blind/AB/ABX testing, we are still performing the same test only in slightly different ways.

My thought process tells me that trying to test a theory by using only one criteria could result in a less than satisfactory presentation of findings. Imagine if you will trying to determine which 4 wheel drive truck will works best in the snow by running them down a nicely paved highway in the summer. Or which oval track NASCAR vehicle will work best at Darlington by running it down a quarter mile top fuel drag track. Or maybe even trying to see how good of a hockey player Wayne Gretzky is by fitting him in shorts and a Tshirt and sticking him on a basketball court. He may be good at basketball but this would not show his hockey talents.

In a way, listening to music and by default, the playback chain involved with music reproduction may not be best served by blind testing. I say this because, the results using the blind testing methods tend to end in similar results irrespective of what method of blind testing is used. There has to be a better way and maybe, just maybe, that way is longer term testing. Maybe listening over a greater period of time, would allow one to become more familiar with the intricacies of a given piece of equipment. Just a thought, and it works for me as I have ventured in this direction over the years. I too read Julians reviews over the years but in the end, while they did venture into discussing the options on any given product, his rhetoric never really did tell me how it sounded or how it worked with other components. 

Over time, we all learn more about any given product than we do when it first falls into our hands. Why not extend the same consideration to electronic components. They are made to be used, but not in a way being discussed herein. This was one of the major problems with picking gear from a store full of stuff...that maybe we did not have. The gear more often than not would act differently once it was brought home and hooked to a different family of equipment, or played using non familiar music. I believe we need to let the amps speak over time, as they were designed to do, and not used in a method designed to elicit a predictable result. Blind testing, it would appear will generally show nothing new, so why not try something new ?

I will go to my corner now. :whistling:


----------



## Sonnie (Apr 11, 2006)

I don't think this is anything new. Numerous people have tried this, but our memory is terrible over time and fools us too easily. I believe this has been proven as well... we can't count on our memory of what something sounds like over a long period of time when it comes time to compare two products. It is tough enough to count on it in short term "non blind" listening tests, much much less over longer periods.

None the less... there are some who are going to claim they can hear a difference whether they do blind testing or not... and whether they can really hear a difference or not. Nothing is going to change their mind. And certainly nothing is going to change the mind of someone who has done the blind testing and says there is no difference... the case is closed for that person.


----------



## Savjac (Apr 17, 2008)

Yep indeed, the case can be closed for either side, you are 100% correct. I think in all seriousness that our minds have incredible recall, maybe we have to train it a bit more to do it regularly. We have all been places where we heard, smelled or tasted something that brought us to a memory that allowed our minds to unfold not only what we are experiencing now but what we experienced at some other point as if it were yesterday. This recall can fool us on occasion as it has been proven time after time when it comes to witnesses in a crime, they are more often than not, wrong when asked to recall what was experienced. The witnesses experienced a sort of double blind time period, it was short, not controlled and causes some confusion as there was no real focus. If we become familiar enough with something and it changes, then we can, I believe, more often than not, notice these changes. Particularly when moving from something unfamiliar to something familiar. Oft times it is an astounding recall that occurs. And sometimes it only happens after we experience something and then return to something either better or worse...or just different. Odd us humans.

I am not posting to get anyone to change on the DBT testing, I am suggesting something a bit different, try it one might like it. A good case might be the feeling that the electrostatic speakers may not be giving their all when it comes to reproduction of certain passages. This happens and we all experience it. I think that feeling we have may not be eminently definable but it is there and would not occur if we did not have some time with the equipment. Same can be said for the sonic characteristics of sound, we may not notice changes until after a period of time has expired.


----------



## Sonnie (Apr 11, 2006)

Claimed differences between cheap amps and expensive amps are typically very subtle. Not many are claiming drastic differences to begin with. With speakers, it can be rather dramatic from a cheap speaker to a more expensive speaker, unless of course you own the Arx A5, then it may not be so dramatic going to a more expensive speaker. :bigsmile:

I think the subtle differences are what are so hard to remember. And of course there have been listening tests that have proven the longer we wait, the less we remember. Once we get past a few minutes, it becomes more difficult to notice those differences in sound, especially more so when they are subtle. 

Then there are moods that can cause us to enjoy music better at one point than another. Over time... there are so many influences and biases that can develop. Simply spending a lot of money on something can make us think it sounds better, especially if we are convinced ahead of time that is sounds better, hence the reason we spent the money.

For me... the only way I will ever be convinced there is a difference, not only in amps, but in preamps (pure direct mode only - as other features and controls can definitely make a difference - they are meant to), in most competent CD players, in DACs, in cables, in power cords, etc etc... is to be able to test that difference side by side. I don't even think I would have to be blind necessarily... lets just plug them up side by side and switch between them to start with. If I think I am hearing the difference, then I should be able to describe it. If I can hear it over a long period of time, surely I can hear it side by side even better. Then we can move into the blind portion and see if I can repeat the differences I was hearing. I know what they are because I described them.

For speakers, no doubt those are a completely different animal... and in most cases we can measure the differences we hear with those.


----------



## rab-byte (Feb 1, 2011)

If you really wanted to see if you could notice a difference in a true side by side comparison you could connect different amps to the left/right channels.

Run the track in mono and see what happens.


----------



## fmw (Aug 11, 2013)

rab-byte said:


> If you really wanted to see if you could notice a difference in a true side by side comparison you could connect different amps to the left/right channels.
> 
> Run the track in mono and see what happens.


It wouldn't work. The location of the speakers would give things away.


----------



## rab-byte (Feb 1, 2011)

It wouldn't be a blind test. It would be a way to test if there was a sonic difference between the two amps.


----------



## Savjac (Apr 17, 2008)

Sonnie said:


> unless of course you own the Arx A5, then it may not be so dramatic going to a more expensive speaker. :big smile:
> 
> 
> Wonderful placement, I am drooling to hear a pair of these, truly. :clap::clap:
> ...


Yes Sir, just listen, no need for testing really, just listen. Either we can hear the differences or we cannot, simple really. I am not sure price needs to be involved, cheap vs expensive cannot really tell us everything we need.


----------

