Choice tests to take more account of costs

I have mentioned this in a recent Choice Forum. When choice test products and make recomendations price does not seem to come into it.

Take coffee machines this week
XXX for $1,750 has the same performance average of 79% yet the XXX at $149 had the same rating.

How can one be below the other?


Hi David,
One would have thought that the most important consideration for any purchase would be performance, not just price, Not much good paying $millions for a Ferrari if its engine does not work or the wheels fall off. Better off with an Aussie built Commodore…before they too are made guess where…thanks to… .hmmm.

But the two products I mention had the same performance/rating %. One was over 10 times the price.

I have a certain empathy with your premise, but the reality is that we buy things with more than just performance as the criteria.

Taking coffee machines for example I have a Sunbeam EM7000. I previously had a Sunbeam EM6910, a Gaggia Classic, a Breville Compact, … , … over 30+ years.

The EM7000 is a dual thermal block. It heats quickly and one can make coffee and steam concurrently. A Breville BES920 dual boiler takes many minutes to come up to temperature, and one can then make coffee and steam concurrently. Purists think boilers are supreme. Tests suggest there is no difference, but still.

The EM7000 and BES920 are semi automatic and measure the water as well as have steam and temp adjustments, the Gaggia and Breville were manuals and could do either coffee or steam since they had single boilers and single thermometers with time for temperature stabilisation to switch between modes. That convenience costs but what value is it to a particular person?

My EM7000 is finicky about the coffee load that affects quality of pour as well as the possibility of the portafilter flying off if too heavily loaded so it does not lock properly. Some machines are mostly metal and some are mostly plastic. Some have switches that just feel right while others feel cheap. Some can be economically repaired and others can only be thrown away.

If you have the performance for basic functions, and the cost, you can make your own judgement on the value of the bits that are subjective and interesting to oneself, not objective. There is obviously more I could write, and perhaps more Choice could publish about a product, but.

1 Like

Hi @davidlcleland,

Thanks for the post regarding our testing methods. We’re always happy to receive feedback of this regard, as it helps to keep our reviews as sharp as possible.

When two items have the same performance score as you have pointed out, the secondary default sorting mechanism is alphabetical (so the more expensive item is only above on the list due to this factor). I’ll pass on this feedback about setting price as a secondary sorting method, which doesn’t resolve all the issues but could perhaps be more relevant for some people.

We try to include as much detail as possible in the comparative results (hopefully without things become too confusing), and we hope that the filter options on the left hand side are useful. Occasionally, when I’m just looking for the cheapest product, I use the drop menu on the right to sort accordingly.

If I’ve missed the point somehow please free to get in touch again, and thanks again for the question.

1 Like

No I think you have the point, thanks

1 Like