CHOICE membership

Improving Choice Testing Rigour

#1

I found a nice looking Westinghouse stick blender at Costco that was not included in the tests. It would have been an impulse buy but still I like to check. I understand the reasons products are not tested, so no worries. I then looked at the reviews with the Electrolux TurboPro Stick Mixer ESTM6400 atop the list.

For perspective of the point I am about to hopefully make, Jaguars (esp circa 1960~80) in the US were likened to fine thoroughbred racehorses that broke a leg every time out. Brilliant until that leg broke. Over the years I have purchased a number of items recommended by Choice and previously the US Consumers Reports, that worked well but turned out to be very poor products because of reliability or design issues that only became apparent with use.

In the case of the Electrolux TurboPro, 0 of 5 Choice members would recommend it as of this date, and productreview.com.au has posts noting common problems with almost 50% rating it bad or terrible even though it performed very well, for a while…

To Electrolux credit they reached out to their unhappy customers but it is not obvious if there was a good resolution. Without having experience on the product my comment is anecdotal but a poorly designed product is poorly designed, and no amount of replacement or instructions or reaching out to customers will fix it.

Combining data points of 0 of 5 Choice members recommending it and the productreview.com.au posts suggests Choice might improve testing/reporting if some online research was done in addition to hands on, and when particular problems are pointed out time and time again, the testing regimen might investigate and address that in the report.

Choice members reviews:

and from productreview

3 Likes
#2

Thanks @TheBBG, we appreciate the point and it’s something we hope to continue improving on in the future. I’ll be sure to pass on your feedback to my colleagues.

2 Likes
#3

We had the same problem when we bought Choice’s top rated microwave (at the time). It was a Sharp that cooked really well, but we had loads of trouble with the oven filling up with water constantly. We had to keep pulling out the turntable to mop up the water in the bottom. Also every surface and around the door was always soaked. The reviews for this microwave were generally bad and after a few months it was covered in rust. We ended up trading it in for a Samsung which is quite good and hasn’t even caught fire.

2 Likes
#4

Due to the shear number of items that we test, long-term issues are something that we don’t (often) get a chance to asses. Unlike, say the NRMA/Motorcycle/Car magazines, we really don’t have a set of long-term test items to give reliability information beyond the couple of weeks they’re in the labs.

In the New Things group, that’s something we’re rather aware of. We’ve got several longer-term plans in play to help address this. A small sliver you may have seen is this forum recently introduced a new Reivew post type. Currently these are not linked in to the choice.com.au website, but that’s on our medium term radar (about 2-3 months from now). Additionally we’re linking up our allthethings.cloud search engine into a collection of feeds of data that will allow us to establish long-term trends and integrate information from various sources - online and offline. We’ve got some more tentative plans, but we’d certainly be happy to take suggestions on how to collect this sort of data.

2 Likes
#5

An easy one that won’t always work due to sometimes short product time in market prior to Choice testing, but have a check on productreview, amazon, manufacturers pages, revoo, etc that reflect unscientific personal opinion and experience, and if there are consistent hits have a closer look at that aspect of the product and reference the area of consistent complaint as one of the many criteria a person should use rather than just “lab performance”. You could ad “no problem in our testing scope” if that made the lawyers happier.

#6

Hi there, It’s a fine idea to assess the reviews, though they come in very slowly with regards to the CHOICE site and the astroturfing that occurs outside of the CHOICE site (astroturfing being the term for fake reviews).

I have heard it bandied about that around 30% of reviews are fake. We we don’t have oversight of them, so cannot trust them.

We rely on CHOICE member feedback on these items more than external agencies. You’re much more reliable.

But as stated, reviews come in slowly. We’ve removed recommendations from products previously because of the sheer volume of member feedback, though it doesn’t happen often (last time was a F&P top loading washer).

As an aside, interesting that CostCo are doing Westinghouse brands.

1 Like
#7

The suggestion was not a binary switch, it was a recommendation to have a further look when there were many similar problems. If you trust Choice members, the 0 of 5 recommendations on the Electrolux and the concurrent comments on productreview seems to be a genuine red flag.

I would not read anything in particular about Costco selling Westinghouse since many of their lines vary regularly. For our market I suspect they might get a container of this or that and sell it until gone. In my experience they religiously stand behind everything sell. In the US they seem to be more consistent as well as offering greater ranges.

#8

That is correct. Costco often acquire left over crates of end of line/model for many products - both electronics and fashion. They go on sale and when they’re gone, they’re gone. You do get some fabulous bargains, but it won’t be for brand new models of things.

(FWIW I lived in the USA for 10 years and had Costco memberships the entire time there. The business model here appears to be identical)

#9

I agree with the BBG letter. I was a member of Choice fore many years and purchased recommended products only to find problems with reliability and longevity. Also Choice were quite selective on products which did not represent all in the particular genre. One example of selective testing was with rotary mowers, only four stroke models were tested , due to pollution problems with two strokes (according to Choice). Two stroke products are still widely sold, due to their simplicity and lighter weight and their greater power for a comparable engine capacity. As I was in the market at the time for a mower, I had no option but to make my purchase without any input from Choice. My old mower a Victa was some 24 years old and had seen some very heavy work, eg clearing rough blocks etc, so I decided to update to one of their later models which so far has proven quite satisfactory

#10

As long as they are not religiously standing behind things in the same way as have some Roman Catholic priests.

(Too soon?)