# Perceptual Research: What's the big deal?



## avaserfi (Jul 5, 2007)

AV Sound Lab said:


> As with all disciplines in science, audio has a dedicated publication, the Journal of the Audio Engineering Society (JAES). While the JAES deals with other forms of research besides perceptual research, this article will focus primarily on perceptual research. The JAES and the contributing researchers have greatly advanced the understanding of how humans hear, what is preferred and, by association, loudspeaker design.
> 
> In its earliest form, perceptual research was used to determine what volumes were perceived to be the same across the audible frequency spectrum. [1]. As time progressed and equipment became more reliable, more advanced testing was possible. Such testing was able to determine the audibility of frequency response anomalies [2] and further research established audibility thresholds of distortion in various devices [3]. Later research was even able to correlate loudspeaker measurement to listener preference [4, 5]. This research has advanced so much that mathematical models have been created which accurately correlate (r=0.995) listener preference to measured loudspeaker performance [6]...
> *
> Read more.*


This thread is for all discussion of the Perceptual Research article found on AV Sound Lab.


----------



## drf (Oct 22, 2006)

I still read over that last article you sent me regarding perceptual research, and having read the above article, am lead to one major question. If a speakers quality can be appraised by the human ear just as effectively (as the above article states) as measurements with expensive gear, then why do we bother? why not just fiddle around with our systems until they sound the best and then leave them at that? I sumise that we don't trust ourselves and hence rely on the measurements to reassure us that the money and time we have commited to our systems is justified and not just wishfull thinking or, as the above article puts it, bias toward a particular brand or style. Eitherway if we keep treading this path we will have psychological reasons for justifying psychoaccoustical annomilies that require a psychological bias to hear.

I think if it sounds good it sounds good, if it doesn't it doesn't, I fail to see how measuring a speaker will change ones accoustic preference.

Cheers
Dr F


----------



## avaserfi (Jul 5, 2007)

The main problem is one of bias. The human ear is a superb measuring device in blinded circumstances, but it has been shown time and time again that when the listening tests are done in sighted situations factors such as quality correlate far less with preference than brand name and perceived status do.

As far as measuring speakers changing preference, you are right there is no relation. Rather, the point of this is that loudspeaker measurement has a direct relation to preference of individuals with normal hearing. So with proper knowledge of what measurements are subjectively superior one can build a speaker that sound even better than good.


----------



## drf (Oct 22, 2006)

So with emperical scientific data to support all this, a company can release a product with evidence of its accoustic quality that no-one can deny regardles of the brand or price tag. This will unsettle a lot of audiophiles and rattle a few snake oil cages. It hopefully will also make some people rethink their views on cheap brands like behringer and alto in the pro audio. 

I have noticed of late that there has been a certain amount of backpedaling on DIYAUDIO.com about certain "neccesities" one needs in amplifiers and the sudden relunctance to use words like "musicality" when defining audio gear. I wonder if it is due in part to the lastest perceptual research?


----------



## avaserfi (Jul 5, 2007)

drf said:


> So with emperical scientific data to support all this, a company can release a product with evidence of its accoustic quality that no-one can deny regardles of the brand or price tag. This will unsettle a lot of audiophiles and rattle a few snake oil cages. It hopefully will also make some people rethink their views on cheap brands like behringer and alto in the pro audio.


That would certainly be nice, but mass acceptance and more importantly understanding of this research must be had before this will ever happen. 

The other factor is the room, a speaker can be technically perfect and if put in the wrong room or even improperly within a room it will sound terrible. This is, of course, covered in the relavent research as a majority of actual sound heard is not produced by the speaker, but rather is reflected in the room*.

*This presumes a reasonable sized home room and not a massive theater.



> I have noticed of late that there has been a certain amount of backpedaling on DIYAUDIO.com about certain "neccesities" one needs in amplifiers and the sudden relunctance to use words like "musicality" when defining audio gear. I wonder if it is due in part to the lastest perceptual research?


I don't follow that forum at all so I cannot speak to this subject. I do know that there is a large amount of literature on audibility differences between components. Some of this research might be new, but the scientifically proven idea that well designed components used within their limits have no effect on sound quality (unless intentional) is not a new one. Perhaps it is just an adoption of this thought philosophy.


----------



## fredk (May 14, 2008)

> ...evidence of its accoustic quality that no-one can deny regardles...


Humanity has infinite capacity to deny and marketing departments have large budgets to bias you in one direction or another.

The good news is that once you understand what the measurements tell you, at least you can make better decisions on what to buy for yourself.


----------



## Ricci (May 23, 2007)

I mark all of my CD's with a green marker as it helps prevent jitter and tightens the lower mids. This along with elevating all of my cables off of the floor has produced a significant impact on the tonal clarity of my system. :rofl2::rofl:


----------



## avaserfi (Jul 5, 2007)

fredk said:


> The good news is that once you understand what the measurements tell you, at least you can make better decisions on what to buy for yourself.


This is very true, but unfortunately properly correlating measurement to listener preference is not an easy skill to attain. Due to the complex interaction of variables this is something that needs to be practiced in bias minimized circumstances alongside deep review of relevant literature.



Ricci said:


> I mark all of my CD's with a green marker as it helps prevent jitter and tightens the lower mids. This along with elevating all of my cables off of the floor has produced a significant impact on the tonal clarity of my system. :rofl2::rofl:


These are perfect examples of bias completely confounding actual reality and experience. Such bias is seen virtually everywhere: audio, medicine, food, clothing etc...


----------



## tenzip (May 4, 2007)

fredk said:


> Humanity has infinite capacity to deny and marketing departments have large budgets to bias you in one direction or another.
> 
> The good news is that once you understand what the measurements tell you, at least you can make better decisions on what to buy for yourself.


The other problem is: Are you sure the numbers on the spec sheet are the numbers that actually came out of the lab, or did marketing get hold of them and "tweak" them first? Were they measured to industry standards? Was all the equipment in calibration and correctly used? When they say 300Wpc, do they mean RMS, Peak, PMPO, or what? And was that measured from 20-20kHZ, or just at 1kHz? There's no government or industry body to check all the specs, so you kind of have to take them with a grain of salt.


----------



## avaserfi (Jul 5, 2007)

tenzip said:


> The other problem is: Are you sure the numbers on the spec sheet are the numbers that actually came out of the lab, or did marketing get hold of them and "tweak" them first? Were they measured to industry standards? Was all the equipment in calibration and correctly used? When they say 300Wpc, do they mean RMS, Peak, PMPO, or what? And was that measured from 20-20kHZ, or just at 1kHz? There's no government or industry body to check all the specs, so you kind of have to take them with a grain of salt.


This is exactly why credible third party measurements such as Ilkka's are a requirement for proper analysis of any item when correlating measurement to perception.


----------



## fredk (May 14, 2008)

> This is very true, but unfortunately properly correlating measurement to listener preference is not an easy skill to attain. Due to the complex interaction of variables this is something that needs to be practiced in bias minimized circumstances alongside deep review of relevant literature.


I'm sure its no more simple or difficult t learn than many other things. In my mind, the biggest barrier to learning and application is access to relevant data.

This in itself makes a case for the need to audition the speakers you choose. As I remember it, Dr. Toole felt it was a rather simple short process to train the listener on what to listen for and how to describe what one is hearing.


----------



## avaserfi (Jul 5, 2007)

fredk said:


> This in itself makes a case for the need to audition the speakers you choose. As I remember it, Dr. Toole felt it was a rather simple short process to train the listener on what to listen for and how to describe what one is hearing.


The problem with simply auditioning speakers is that bias is so prevalent in the human experience. Toole and Olive conducted a research study where a panel of listeners rated speakers in sighted circumstance and then repeated the experiement with a blinded situation. The results were not surprising, the sighted situation was plagued with confounds: e.g., loudspeakers with high reputations or prices being rated at the top. While such effects were expected the authors were surprised at the magnitude of this effect. Once the bias was accounted for the preference ratings had no correlation with perceived status of the loudspeaker, but rather actual performance. The end result being: "The bottom line: if you want to know how a loudspeaker truly sounds you would be well advised do the listening tests 'blind'" [1]. This is just a single reason why simple listening tests performed by most are not sufficient to fully judge loudspeaker performance.

As far as the belief that training is or is not a long process that was dealt with directly in a paper by Olive [2]. The end result was that trained listeners more quickly arrive at a conclusion about the loudspeakers being auditioned while being more critical and having less variability. In contrast, untrained listeners come to the same conclusions more slowly and with higher variability. Additionally, untrained listeners became more reliable [or became self trained] over multiple trials.

*References:*

[1] Toole, Floyd E.; Olive, Sean E. _Hearing is Believing vs. Believing is Hearing: Blind vs. Sighted Listening Tests, and Other Interesting Things._ J. Audio Engineering Soc., Conference Paper 3894. November 1994/

[2] Olive, Sean E. _Differences in Performance and Preference of Trained vs Untrained Listeners in Speaker Tests_. J. Audio Engineering Soc., Vol. 51, No. 9. Pages 806 - 825. September 2003.


----------



## tenzip (May 4, 2007)

avaserfi said:


> This is exactly why credible third party measurements such as Ilkka's are a requirement for proper analysis of any item when correlating measurement to perception.


Right, I kinda left that out of what I said, but it was (to me) implied.

Knowing what all the measurements, graphs, etc., mean is great, but it doesn't necessarily follow that you are getting good information from them.

You could probably include yourself in the "credible third party measurements". Or do you blush easily?


----------



## avaserfi (Jul 5, 2007)

tenzip said:


> Knowing what all the measurements, graphs, etc., mean is great, but it doesn't necessarily follow that you are getting good information from them.


That is exactly why it is important to not only understand what the measurements mean, but how they are taken and/or derived so that proper judgments of credibility can be made. Also, depending on methodology used certain factors must be accounted for within the measurements themselves.



> You could probably include yourself in the "credible third party measurements". Or do you blush easily?


I do and will always try to provide quality measurements, but it is not my place to call myself credible. After all, I am pretty sure I cannot maintain objectivity on a topic as awesome as me! :R:hide:


----------



## fredk (May 14, 2008)

I understand the issue of bias and have read that particular Toole paper. It is also fair to say that the knowledge that bias exists will mitigate that bias to some extent.

If you do not believe that higher price always equates to better quality, then you are going to listen more critically for evidence of quality and are also likely to arm yourself with the tools required to do that critical listening.

Critical thinking/reasoning is a skill that anyone can acquire and apply to decision making to reduce bias.


----------



## drf (Oct 22, 2006)

fredk said:


> Critical thinking/reasoning is a skill that anyone can acquire and apply to decision making to reduce bias.


I meet very few people who can actually carry through a logicaly reasoned appraisle through to final decision with it being tainted by either saving a dollar or owning some form of status symbol. 

One of the things I enjoy most about not reading hifi magazines is that when I walk into a shop, I only recognise half of the brand names on the floor. One time I dismissed a $3000 pair of Dali's becasue their tonal quality was up the creek and the off axis response was shocking. It turns out they are the cheapest of the Dali range and were over priced on the ticket. For me this is the perfect example of what perceptual research is discovering.


----------



## avaserfi (Jul 5, 2007)

drf makes a great point here: No one can escape bias regardless of knowledge level, it just isn't possible considering the human condition. Of course, conducting listening tests in blinding conditions is not a reasonable proposition the majority of the time because of this why I recommend and personally use a linear, non-resonant reference pair of headphones for comparison. While far from ideal such an approach at least allows one to make direct comparison between a known set of standards and another speaker.


----------



## fredk (May 14, 2008)

> I meet very few people who can actually carry through a logicaly reasoned appraisle through to final decision with it being tainted by either saving a dollar or owning some form of status symbol.


I drive a Ford Escort. There is no way I'm going to be biased by owning a status symbol.  :R

I agree, but that does not mean it can't be done. It is a learned skill.

Andrew, if we follow your reasoning to its logical end, we come to the conclusion that the only thing to do is to ask an expert what to buy.

1. we can't hope to eliminate bias no matter what. 
2. we can learn to listen properly, but it is very very difficult and requires 'deep review'/learning.
3. The only reasonable listening conditions to eliminate bias are blind listening: an impossible situation for most of us.

Its the classical experts argument, all very logical and reasonable on the surface, but ...

Balancing that is the 80:20 rule. You can get 80% of the way there with reasonable effort. Bias can be _reasonably limited _ by concious effort and the proper application of critical thinking.

I would put it to you that believing that bias cannot be reasonably limited is, um, biased.


----------



## fredk (May 14, 2008)

That headphones lisening thing is a good idea. I am slowly coming to the conlcusion that I am only an audiophile wanna-be though. I would have a hard time buying an expensive pair of headphones just to evaluate speakers.


----------



## avaserfi (Jul 5, 2007)

fredk said:


> 1. we can't hope to eliminate bias no matter what.


True, even blinded tests can be easily confounded if proper steps are not taken. 



> 2. we can learn to listen properly, but it is very very difficult and requires 'deep review'/learning.


I never said or implied this, rather correlating measurement to listener preference is what requires a large investment of time and effort.

Listening for quality itself is an easily learned trait as shown by Olive's study comparing trained and untrained listeners. The real problem is other confounds: short auditory memory, brand bias, loudspeaker room interaction etc... 

Additionally, the lack of any reference to what is truly high quality makes such judgments harder for the 'uninitiated'. In fact, this last statement can likely be extended to those who have extensive listening experience in one major regard: panel resonance. This anomaly is so common in virtually all loudspeakers, regardless of price, it seems to be an accepted trait by most despite its correlation with poor listener ratings.



> The only reasonable listening conditions to eliminate bias are blind listening: an impossible situation for most of us.


This is exactly why I recommended using a high quality linear pair of headphones that has no audible resonances. Such a unit at least allows for a reference to quality when auditioning, but certainly does not remove bias.




> Balancing that is the 80:20 rule. You can get 80% of the way there with reasonable effort. Bias can be _reasonably limited _ by concious effort and the proper application of critical thinking.
> 
> 
> I would put it to you that believing that bias cannot be reasonably limited is, um, biased.


No one is immune to bias, it just isn't possible. Certainly bias can be minimized, but each person will have their own definition of reasonably limited. Clearly this is where we differ. If ones goal is to have a good time and audition speakers certainly blinded comparison is not needed, but if the goal is _ideal_ fidelity even minute amounts of bias can effect the outcome. Couple this with loudspeaker room interaction introducing yet another massive confound and it is clear that simple listening tests can find a speaker one likes, but not ideal. The end point being: listening tests are not a feasible option most of the time if this is ideal fidelity is the goal, but with proper correlation measurement to preference the ideal speaker for a specific price class is chosen.


----------



## thxgoon (Feb 23, 2007)

Couldn't this bias be a good thing too? If the visual appearance, status, price and performance all add up to a good purchase, is that what it's all about? In the case of removing all bias by conducting a blind listening test, you purchase the speakers you thought sounded best and you place them in your room. You are from this day forth convinced they are the best speakers you could buy. Isn't your purchase biased by these test results?


----------



## avaserfi (Jul 5, 2007)

thxgoon said:


> Couldn't this bias be a good thing too? If the visual appearance, status, price and performance all add up to a good purchase, is that what it's all about?


This depends on the individual goals. If one is interested in an aesthetically appealing speaker more so than the ideal speaker at a given price point, then yes these biases should be embraced. Of course, once aesthetic and/or other judgments the same methodology could be used to choose the best speaker that fit such specific criteria. Such testing can always be adjusted to maximize quality for almost any given situation.



> In the case of removing all bias by conducting a blind listening test, you purchase the speakers you thought sounded best and you place them in your room. You are from this day forth convinced they are the best speakers you could buy. Isn't your purchase biased by these test results?


Of course, cognitive dissonance will play an effect once the purchase has been made, but if properly conducted the best possible speaker given the situation* would have been purchased. 

*It is not feasible to assume measurements could have been analyzed and/or blind listening tests could have been completed on all possible speakers that fit a certain criteria whether it be price, aesthetics, status etc.... It is also feasible that a superior speaker could be produced in the future.


----------



## ISLAND1000 (May 2, 2007)

Anybody on here DOING any . . . . . "Perceptual Research" or are you just talking?


----------



## avaserfi (Jul 5, 2007)

ISLAND1000 said:


> Anybody on here DOING any . . . . . "Perceptual Research" or are you just talking?


The research specifically being discussed has not been performed, directly, by any members here as far as I know. Rather, they have been published in credible, peer reviewed, journals such as the Journal of the Audio Engineering Society. It is important to note that the article being discussed is a short meta-analysis and explanation of the current, relevant, literature not a study in itself.

Personally, I have replicated some of the studies on a smaller scale with less control due to funding and time issues. Additionally, I have completed other studies of my own for personal benefit, but the results did match the results of the studies cited in the article being discussed when possible. Also, I have completed a perceptual study on dynamic compression preference and am in the process of trying to expand the study for use as a masters thesis as well as for publication submission. A link to the pilot study and its discussion can be seen here.


----------



## fredk (May 14, 2008)

> I never said or implied this, rather correlating measurement to listener preference is what requires a large investment of time and effort.


I think we are saying the same thing. I take the corelation to be part of learning to listen.

I agree with your comments on auditory memory and room intaraction.



> the lack of any reference to what is truly high quality makes such judgments harder for the 'uninitiated'.


Thats a big one to me and in my mind more of an impediment than anything else.

We are largely on the same page, and mostly seem to disagree on the difficulty/effort involved in minimizing bias.

thxgoon, I actually thing status is a valid selection criteria. You don't buy a Ferrari just because it corners well.


----------



## drf (Oct 22, 2006)

fredk said:


> thxgoon, I actually thing status is a valid selection criteria. You don't buy a Ferrari just because it corners well.


But a ferrari has emperical data to back up all its claims and sales pitch. If someone had proved that ferraris were actualy worse than datsun 240z's in every aspect, do you think people would still by the ferrrari as a status symbol? I don't. I think if there was substantial data to prove B&W were inferior speakers to AWA/RCA, how many people would pay the extra for a psuedo-status symbol? not many?


----------



## fredk (May 14, 2008)

> If someone had proved that ferraris were actualy worse than datsun 240z's in every aspect, do you think people would still by the ferrrari as a status symbol?


Yes, I do. Porche did just that for years, though they did push it a little far in the 70's with negative consequences.

I'm going to bow out now. We will just have to agree to disagree.


----------

