Thanks, @PhilT 
- I did search the Community for related posts, but didnāt find the 2017 one.
I agree with your suggestions of a more structured community review process and giving the reviews a higher profile.
I also second your comment to @BrendanMays - it would be nice to see some of the simple but tangible requested improvements implemented.
Thanks for raising the issue, and for the suggestions - Iāll ensure the feedback is received and considered for improvement.
In need of a new kettle we checked the Choice reviews, member comments, and what is on the market today against our own value system. We eventually got to youtube reviews that while not laboratory comparisons, they succinctly showed features and faults in models well reviewed by Choice and sometimes less well reviewed by members comments.
Not all product categories would be amenable nor would some be practical or have value added. I suspect Choice has had internal discussions about publishing videos not just text and accept it is probably more time consuming, but in 2023 it seems it is the time to adapt to that ācompetitionā and shift in expectations.
While it is probably not feasible or even practical to do video reviews of each product (such as kettles) snapshot videos of some highs and lows included in āhow we testā would be a step up.
Thanks for the comments @PhilT, itās good to know that more videos would be useful. Iāll share this with the relevant teams.
This depends on being selective and using video where it actually shows things better than words or still images. I find the video where somebody reads the facts and figures or points out trivia ānow the brake and the accelerator are both here on the floor on the driverās sideā¦ā are a waste of time.
Then there is the fox news approach to the imparting information:
I am supportive of your qualifications.
The basis for my suggestion was that (for kettles) a few videos were far more instructive than the text from Choice and general unmoderated reviews on web sites. A good example was pouring water - an initially preferred product made a mess under a āconsumerā control while text only pointed it out as a ābad pointā. For us that was a deal breaker (that was not a deal breaker until we saw the ārealā problem not just a described problem). A more patient pour may have had a different result; we were able to see the level of care and speed used, and compared it to our own āstyleā and whether we could keep it āneat and cleanā while pouring. We decided we could not as it would require a much slower pour. That was helpful to us.
Hey @BrendanMays, today I was looking at the shower head reviews and while a few shower heads are listed the sole rating is based on water efficiency because āwe were unable to testā anything elseā¦
There seem to be an increasing number of what once were comparative tests being updated into what are desktop lists (orderings of manufacturersā data) then using the sole criteria of a single item, commonly efficiency.
Sad to publish, but I am not impressed with this seemingly ongoing evolution. Where there is a consumer product that purports to provide a function there should be a way to test it, and if not what is the point for subscribing to read collections of manufacturers claims, and then only for what is often a very small subset of the products available?
As example, for the shower heads you could have had staff, friends, and family try them even if you had to build a custom shower stall (which I think there is one in your building?) and fill out a ranking and noting in the publication it was subjective consensus views of X people. IMO it would have been more helpful than water efficiency in isolation.
Thanks for the feedback @PhilT. I understand the concerns and Iāll pass on your comments to the product testers for consideration in the future
Just reading through my June 2023 edition and looking at the above mentioned test article I was under the impression that the units where tested for function, efficiency and usability by Choice staff. I then noticed the āHow we review air conditionersā break out box. And it appears that they are no longer actually tested (for the past several years) even. But rather a review of manufacture supplied specifications to a Govt database.
So how are the Performance scores obtained? eg Airflow? What about the remote control usability, thatās not even mentioned?
Iāve been a subscriber for decades and Iām sure in the past the units where tested in purpose built rooms with comprehensive performance and usability results detailed?
Iām feeling that calling this a Test, when itās only a review of manufactures data is slightly disingenuous of Choice and their ethic?
Thanks for your comments @omy005aw . I merged your post into this topic that is relevant. Maybe if enough members voice similar opinions @BrendanMays could approach the responsible teams as well as management to publish more than compendiums of manufacturers data sheets.
Hi @omy005aw, thanks for getting in touch with this feedback.
There is a detailed explanation on our site (and Iāve copied it below), about our air conditioner review. There are a number of reasons why itās done that way, our previous testing indicated that the strict Minimum Energy Performance Standards were accurate and reliable, and the existing testing gives us all the performance information, such as airflow. Doing the review this way also allows us to review a much larger number of units and in a more timely manner. However, we do engage an external lab for this type of testing when needed - for example, if thereās a new or unusual model.
I understand that despite the reasoning, the review might still not have met your expectations and that is important to us. Iāll make sure to pass on this feedback to the product testers and relevant teams.
We appreciate your support as member, and Iām happy to chat further if you have any follow up questions or comments.
Why we changed our testing method
Before we adopted our current test method in 2013, time and money constraints meant CHOICE was only able to review a very small number of air conditioner models ā about 20 per year, even though there are a few hundred split-system inverter models on the market at any one time. We could also only cover only one size category at a time ā typically small (4kW and below) or large (6kW and higher) models.
On top of that, we often had trouble sourcing models for testing as we needed to test well ahead of the season, and manufacturers often didnāt have their new-season models available in time. Add in the fact that it cost us about $160,000 to buy and test those 20 models each year (at 2013 prices ā it would be even more now) and you can see why we looked for a better value proposition for our members.
One of the main reasons we focused our lab tests on inverter air conditioners was that when inverter models started appearing back in the early 2000s, we were concerned that standard tests might not have been measuring their efficiency in a way that reflected their actual performance in a consumerās home.
Now, inverter technology is the norm, and after several years of testing inverter models, we no longer have any particular concerns about their typical performance. So weāre comfortable that a ādesk reviewā is appropriate.
I recently checked the Choice review of dishwashers, looking to help a friend who needed a new one, and I remembered I bought our present - very unsatisfactory - one because it was at or near the top of the Choice list.
I rechecked the new review of that machine, and while it wasnāt quite as complimentary, it didnāt mention any of the design issues that make it so unsatisfactory. Thatās when I noticed there were only 2 user reviews, both 5-star.
There is no option to post a review on the main Choice website, as far as I can tell, and I donāt see any āreviewā section in the Choice Community webpages.
What am I (and presumably most users) missing here?
While the official Choice reviews are somewhat useful and certainly cover an enormous range of products, they are often lacking in some significant areas. For example,
(a) A camera review a couple of years back assessed all the camerasā image stabilisation features as equally useless, despite the fact that this feature often makes a significant difference. This indicates that test procedure was inappropriate, but the review was misleading;
(b) The dishwasher review of drying performance just gives stars, but does not indicate whether there is a fan or other ventilation drying function, or whether the rack design minimises water pooling on cups etc.
(c) The kettle review failed to include the time to bring 1L of water from a standard room temperature up to boiling point - which is the only relevant test of efficiency, given that almost all models tested have the same rate of power consumption.
This is why actual user reviews can be so helpful. Should we just rely on product.review.com.au?
Welcome to the Community @CHH80
I moved your post into this existing one that is similar.
Choice has had member comments/reviews on tested products for many years. You can see the links at the bottom of each product in the review. They appear as
Not all products get reviews and not all reviewers are kind to the products. There is no āconsumer inputsā to non-tested ārandom productsā noting they would be 100% anecdotal and subject to abuse, +/-.
As for sites that are based on anecdotal reviews they can be educational as well as misleading.
Also use the Community search on āchoice reviewsā and you will find of number of relevant topics, including how to best use Choice reports.
Noting @CHH80 made some comments on the testing regimen, this topic should be focused on that aspect.
There are a number of topics and posts about how to best use Choice reports. Perhaps one of us might try to consolidate them as it seems a recurring discussion?
Many reviews require one to go into the compare tables. For kettles this is one of the line items.
For each review there is also an explanation, such as
Iāve left member reviews without a problem on the Choice website. Note I think only logged in paying members of Choice can create a member review? The Choice Community on which we are currently posting is a seperate resource without the limitations of paid membership.
Yes, you have to be a member and logged in to the website even to see the full reviews, including member reviews and the option to write one yourself. If not logged in, all you see next to the items in a review list is something like this:
I have been a Choice member for ages and almost always purchase items I need following Choiceās recommendation. However, I have a couple of comments to make. A couple of years ago I purchased a very nice gas cooktop following Choiceās recommendation, which lives up to the recommendation. However, the stainless steel top is like butter. Within days of its purchase, I scratched the top with a plastic scourer. I only have to look at the cooktop and it scratches. So I guess that is one for Choice to check in their testing of stainless steel items.
Secondly, following the recommendation from Choice, yesterday I purchased a Nutribullet xxx Digital Air Fryer. I guess that I should have checked online reviews. They are not good. Users are claiming that the Teflon coating comes off and for some users parts became rusted very quickly. I recommend that the testers at Choice check online to see what actual users are saying in their reviews.
Hi @hrodden
I have moved your post on CHOICE testing to a specific topic about this type of feedback and request. If you read through the posts you may find some that echo your concerns. CHOICE does take a great interest in member feedback and Iām sure that the staff member responsible for the Community site, @BrendanMays, will pass your feedback onto the relevant CHOICE team members.
When you do purchase and use a CHOICE recommended item, I encourage you to provide feedback on your purchase and suitability at the itemās review page. In regards to the XXL Air-fryer there is a relevant section in the bad points that may help you in your situation to resolve the tray issue. While this content is usually member only, it appears that this is of such a problem that to include the specific advice seems appropriate.
"Bad points
ā¢ Note: Weāve received reports from users having issues with the Teflon coating of the crisp tray for this air fryer. The manufacturer has told us there was a batch issue with some of the crisp trays which has now been rectified. Consumers can contact customer service for a replacement crisp tray"
Thanks a million for your response
