In theory ABX testing is the way to go , but there are many detractors to this, , even simple AB testing, including Robert Stuart himself.... (he has presented papers on this at AES)

Similarly, AES Fellow J. Robert Stuart stated in his landmark papers �Estimating the Significance of Errors in Audio Systems� and �Predicting the Audibility, Detectability and Loudness of Errors in Audio Systems� that �A/B testing using program material and particularly musical program is fraught with difficulties. . .the author sets out some reasons why the �objective� approaches of A/B listening and null-testing may be flawedd.�

In reality , expectation bias and a belief system are enough for us to detect differences that don't actually exist.

The thing is , that if you do believe one thing is better than another , it DOES exist for YOU.. it is a truism.

For eg , some folk will swear blind that tea out a china mug tastes better than out of a tin , and they will spend vast amounts of money tracking down or purchasing the best china to get that better taste ..whereas the tea poured out of the cosy is the same tea.

There are also phenomena that cannot be measured
A significant percentage of people hear a tone that is between 2 actual tones , ie they will hear a 1.5k tone between , lets say a 1 and 2k tone. This tone does not exist - yet it can be heard?

I have learned in hifi to allow each person the leeway to believe what they like .. those that pay for what others consider "foo" do not need saving from themselves.

I myself like to do level matched AB testing where possible to differentiate if a potential purchase is 'better" than another or to discern differences. but I also am susceptible to expectation bias , so longer term listening is also part of it.. and ultimately my ears (and thus my brain) are the final determinant of what I like or how I spend on what...

Here's a test - close your eyes and get someone in the room to turn on and off the display on your DSP speakers , sometimes with a "false" change (Ie say they changed but they didnt)

I will swear blind there is a difference if I do it myself , but that is "sighted" (off is better) , but have never really had the gumption to get my wife or kid to do it with random change/no change and see if I can tell which is what or even determine if there is indeed a difference.
I will do that tonight....



Roon/Tidal, SBT, Trinnov ST2 , Twin Devialet D premiers, Vivid Audio Giya G1 Spirits, dedicated and treated room.
Meridian gear: 568.2mm and DSP5000's acting as TV speakers.