Firstly let me say that I support Choice and the work it does for consumers. An issue I raised with Choice is in regard to subjective opinions being presented as factual without any objective or larger consumer sample evidence.
Specifically in this instance I was referring to their comparisons of in ear headphones (IEH) standard and noise cancelling (NC) in the January 2023 edition of choice. I must confess that I am an IEH tragic, owning several sets of them and turning them over regularly. I also subscribe to and participate with many headphone forums such as Head Fi.
It can be observed, and by order of magnitude, that most members of these forums would not rate the sound quality of the top line Apple pods higher than the equivalent Sony or the Bose (both standard and NC), and certainly not higher than the top line Sennheiser NCs which Choice inexplicably did so in a previous issue. Then there are other quality IEMs that are not even tested by Choice, such as the higher end Jabra Elite NCs (Jabra is also renowned for their hearing aid software and devices) which most people on these forums would agree they have a higher sound quality than the Apple pods and at a significantly lower price.
Of course sound quality is subjective and being so, our perception of quality is influenced by our beliefs and prejudices. I’m willing to guess that the IEM testers at Choice are young Apple enthusiasts who did not participate in blind tests. There is little other explanation as to why Choice rated the Apple product as having the best sound quality in this and previous publications, completely divorced to what most audiophiles would choose (I also see the same biases in tests of other Apple products such as phones, tablets, and PC monitors – although granted, in some cases it is justified).
In my view, Choice is doing its readers a real disservice by presenting these subjective ‘sound quality’ results without reference to any objective analysis (such as double-blind testing) or at least taking into account what a majority of users around the globe subjectively find to be the best sounding (and that research is easily done). Apple pods and headphones do have good sound quality but for most users they are far from the best, particularly at their respective price points.
Given the subjectivity and all the influences on human sound perception, perhaps next time Choice should hire some experts to set up controlled double-blind tests. I am sure the results will surprise their testers and provide more accurate information for consumers before they spend not an insignificant sum of money on headphones (or loudspeakers and other audio products for that matter).
Double blind tests are used precisely to control biases which as humans we all have. It is surprising that Choice does not use them. They can be tricky to set up (including ensuring they are volume matched as louder can be perceived as ‘better’ but it doesn’t need to be onerous or expensive and will add that objectivity which consumers deserve.
Kev
