ACCC Reports on NBN Performance

Me too. I received an email regarding the testing program the other day, and it seems that some have already seen their personal data, and the rest of us will also see ours, hopefully in the not too far distant future.
Not that I actually need to see it, I know what sort of speeds I get already, and the most annoying things I suffer with NBN Satellite are not even being tested for.

4 Likes

Yeah the outages I am sure are a very big part of that untested information. I really do support that sort of metric being tested/logged as it may have a huge impact on the usability. A fictional (hopefully) example would be that it works extremely well for 5 seconds each day and the rest of the day you have no connection, if all you logged was that 5 seconds and reported only on that small slice of time the results could look like perfection but the reality of almost completely 24 hours of non connection makes the product for real usage a complete waste and unusable.

With those metrics being logged it also brings the ability to argue more strongly about the lacks, as currently we have no independent proof of the faults and we can be easily dismissed by the Govt and NBN Co by their provision of what they tell us are the “real picture”.

Almost without fail we lose connection late afternoon around 6 pm, with return to service being a variable amount of time but by about 8 pm we are close to if not at full service again. Annoying for us, inconvenient even at times, but what I hear on Satellite usage it is beyond mere inconvenience or some annoyance factor and reaches to levels of even being totally unsuitable and unworkable.

4 Likes

So you also lose your phone (VOIP?) service for 2hrs/day? Do you get a 8.33% discount? :slight_smile:

4 Likes

Yes we do lose all connection including our VOIP during the period up to 2 hours and no we don’t get reimbursement as we don’t have CSG.

3 Likes

The latest update

It is amazing how they can provide so much data and so little information. Must be a select government committee. Troll through the report and you will see what I mean. :angry:

8 Likes

Surely any tables or graphics of statistics must be accompanied by an explanation of how the statistics were derived. But I cannot find that information.

This paper says they have interest from 9000 volunteers wanting to be measured in March. How many were actually measured? How valid was the sample size? Given that it was not a random sample how did they allow for self-selection bias? Is it all technologies or certain classes of services? In other words just how was this sample constructed?

Why only 6 providers? Where are the rest? [I have an idea why but this must be made explicit]

What effect do customers having no service at all for some or all of the collection period have on these figures? Are they included or not?

The percentage given is said to be of the maximum plan speed. So for any given connection you can measure that percentage at any time of day. But what is this aggregate percentage by provider? How did they create that summary statistic across more than one connection during the specified busy period? If it is an average what does the distribution look like? The shape of the tail at the low end would be very interesting.

They seem quite happy that for six providers the percentage (of whatever) is between 74 and 88. Why isn’t it 100? If 100 is not obtainable in practice why is it the benchmark?

If you have a valid sampling methodology and sample size, and a defensible benchmark of success, you can estimate (within known limits) how many of the entire population are actually getting the service they paid for. You can bet the statistician responsible was asked to provide it.

Where is that statistic and who reviewed the methodology? Would Choice like to enquire on our behalf?

7 Likes

Around 800 participants were used, just need to add the number of whiteboxes which are the home measuring devices. I have included a snip from the report…

Next report should include about 1,000 and at the end of the 4 year testing period they will have a total of about 4,000. There is no breakup of where these devices are located that I could find and so I would surmise larger cities with very good service

Here is the link to the pdf of the report:

https://www.accc.gov.au/system/files/ACCC%20-%20Measuring%20Broadband%20Australia%20-%20Report%202%20-%20July%202018.pdf

4 Likes

I guess the averages and standard deviations are good enough for us mushrooms. If your RSP is at the bottom or top of a particular 100/50/25 service wouldn’t you like to know that? As I sifted through the report there is more unsaid than said when one is interested in what to do as a consumer. We assume that eg. an RSP getting 90% of 25 mbps will also get 90% of 100 and 50 but do they?

This report might suit the government and NBN to sugar coat or report on the overall program but it seems lacking for a consumer’s interests. If I were more cynical I would think this lack of information is not accidental.

4 Likes

All of these statistics are quite meaningless unless they address the issue of sample bias through either self-selection or the NBN excluding some volunteers at some times for whatever reason. You can analyse the figures to death but if the sample isn’t representative they mean nothing. Did the installers of ‘white boxes’ (measuring doohickeys) travel to a representative number of remote and regional locations? What is the mix of connection technologies (other than not ADSL) ? We don’t know.

Out of some 9000 who showed interest 731 were measured. How were they chosen? How representative of 3.4 million are those 731?

To make this more concrete the number of ‘white boxes’ for the popular 25/5 plan is 146. But how many customers subscribe to this plan? A wild guess a million, 1.5 million? I don’t know. The half year 2018 report says there have been 3.4 million activations. You can say that from the internal variability of that group of 146 its performance lies within certain limits but unless you know that they are evenly spread over all locations, technologies and circumstances it tells you nothing about the whole population. Put it another way you would only need to exclude a few dozen really bad connections to make the other 146 look collectively so much better.

4 Likes

Mission Accomplished?
Almost?

Nearly almost 90% of Australians** who have connected to the NBN are getting an NBN service?
Well someone is 95% confident all the users on the slowest speed plan are getting 86.6% up to 90.2% of their plan speed all the time.
And every one else is getting even faster NBN.
575 users out of 800 is just short of 72% so there is a little way to go yet to the magic 80% able to connect at speeds of up to 50mps.

It is a welcome change to be able to see positive outcomes.

notes: ** insert exclusions of Fixed Wireless NBN, Satellite services, any one not able to connect, all households connected to the NBN not in sample.

It’s good to see there are still customers on ADSL. I wonder how much guilt we have over not being on the NBN, given the wonderful NBN results in the report?

4 Likes

Which! heh heh … UK Cousin of Choice … :slight_smile:

5 Likes

I like it! :rofl: :joy:

2 Likes

The latest Honesty box (old eMetric) results for June 2018 are available with July’s expected around the 20th of August.

The email received from them is copied below:

"
CHOICE Broadband Performance Measuring Program

Dear XXXXXXXX,

Here’s the latest update about accessing individual and program results.

CHOICE broadband performance results

New feature on the CHOICE broadband leaderboard

Just a reminder CHOICE has published the June results for broadband performance in homes across Australia, and the latest leaderboard is available here

Having a squiz at the leaderboard today, we noticed they’ve added a new feature so you can now sort by type of technology eg: Fibre to the Node, Fibre to the Premises, Fixed Wireless, etc.

The July results will be available around 20th August.

We’re on the way to providing your own personalised broadband performance test results

Individual results beta

As you know, over the last month or so, we’ve been creating a new product to give you all access to your own broadband performance test results. So far, we’ve sent two reports out, one for June results and one for July. These individual results have been shared with a small group of 50 participants who have agreed to help us improve the product (consider this our beta).

In our last newsletter we attached a “sample” report so you can get a feel for what we’ve designed. We did learn from that experience. By calling the sample report “Jake’s Broadband Performance”, we confused a few of you who thought it was your own report. We’ve subsequently changed its name to “Sample Report”.

Having sent two sets of individual results out to our beta test group, here’s a summary of what we’ve learnt:
• You want your results benchmarked against speed tier and other participants using the same technology, instead of being benchmarked against the country as a whole. This is interesting because it tells us that what’s important to you is “value for what you’re paying for”. You want confidence you’re “getting what you pay for” so we need to design the report to inform you of that.
• You’re interested in the time of day and the speed you’re getting at that time, rather than average speeds over the month. Thinking out loud, we all know the Internet can slow down during peak busy periods (deemed as 7-11pm Mon-Sun here in Australia), so it’s useful for you to see that in your actual results. It can reaffirm/confirm what you’re practically experiencing (even though we don’t like it!).
• You want to see broadband performance over time so you can see if it’s getting better or worse, or if it’s about the same. This feature will come as we build up test results over time. At the moment we have only two-three months of data, but we should be able to start sharing trends over time, in the coming months.
The product will evolve as we keep learning from our beta test group and one key development we’re particularly excited about will be a platform that all of you can log into to get your own results. We don’t have a definitive month for when that will be available, but it will come :slight_smile:

Stay tuned.

Thank you…

To our beta group

Lastly, a huge thank you for the 50 or so of our Community who are helping improve the product. We’ll continue to invite small, but increasing numbers of you into this process and can’t wait for the day when all of you can access your own results.

Till next time.

Kind regards,

Singam Vijay
Customer Engagement Team
Email: helpdesk@honestybox.com.au
Phone: 1300 012 472
eMetric Pty Ltd trading as © Honesty Box ™ ABN 19 622 285 170

Copyright Š 2018 Honesty Box, All rights reserved.
You’re receiving this email because you’re a participant in the CHOICE Broadband Performance Measuring Program. www.honestybox.com.au, and the Honesty Box name and logo are copyright © and ™ eMetric Pty Ltd ABN 19 622 285 170

Our mailing address is:
Honesty Box
Suite 6, Level 2
66 Oxford St
Darlinghurst, NSW 2010
Australia"

6 Likes

Brilliant, I’ve had a look at the results per the link. So easy and so clear to see relevant bench marks for each service type and speed tier.

It will be interesting to see how these trend over time.

Is there any prospect for the future that we will be able to look at results by service connection point? I don’t know the most appropriate terminology. For FW it might be data for a particular tower or tower and cell/service segment. For FTTN it would be exchange and perhaps even mode, given there are variables at both connection points?

From previous NBN reports NBN Co are very much aware of which FW customers - towers are not performing. By their criteria? But there appears to be no access available to the source data that would enable independent assessment?

5 Likes

Sadly, still no satellite results, although realistically, it can’t tell me anything I don’t already know… about how crap it is! With no other options currently available, rural dwellers are stuck with it for now.

6 Likes

The more progress that is claimed the more things stay the same. Same old performance, same old mentality. ‘Whatever’ is apparently good enough for us mushrooms.

https://thenewdaily.com.au/life/tech/2018/08/08/nbn-hfc-technology/

5 Likes

The ACCC have released the NBN Wholesale Market Indicators report that shows results up to March this year.

To read the “YourLifeChoices” article on the results see:

https://www.yourlifechoices.com.au/technology/internet/spike-in-highspeed-nbn-users

While the results show a marked increase in the 50 Mbps plans it also by logically assessing the data, shows the much poorer results that must be impacting those who cannot reliably achieve even 25/5 Mbps consistently eg Satellite users. While the report tries to say how good these improvements in speed tiers numbers are, the result must be a huge kick in the teeth for anyone (and that’s a lot of people) who can’t even dream of this.

6 Likes

The Choice Broadband results have been published for the July 2018 period. The 25/5 packages still appear as the majority of the “recommended” plans. I did note that some 50/20 packages have started to hit the recommended list and one 100/40 (Internode’s FTTP).

Just a quick aside the TPG 25/5 scored 81% and was not a “recommended” yet the 100/40 Internode one did with a score of 80%.

Also of note was the first satellite result for the 25/5 Skymesh Satellite, but it was a poor 56% overall score.

To see the new results see:

5 Likes

FWIW my NBN ™ HFC connection is now forecast for Oct-Nov after being in the too hard basket for a year; I am not looking forward but the solicitations are starting to roll in. Noteworthy constants are the pricing almost seems universal and that the plans seem to be 12/50/100 with the 25 gone missing in the offers.

The plans are now showing formats like ‘NBN50 (46.0 Mbps typical evening speed)’ and those typical speed claims vary by up to 20%! Another differentiation seems to be when limited data becomes unlimited data, and the ‘free bundled’ entertainment packs.

4 Likes

Per the Oxford Dictionary:

speed
NOUN

1mass noun The rate at which someone or something moves or operates or is able to move or operate.
‘we turned on to the runway and began to gather speed’
‘an engine running at full speed’
count noun ‘the car has a top speed of 147 mph’
More example sentencesSynonyms
1.1 Rapidity of movement or action.
‘the accident was due to excessive speed’
More example sentencesSynonyms
1.2 The rate at which something happens or is done.
‘they were bemused by the speed of events’
‘the course is delivered on CDROM so students can progress at their own speed’

Scope to consider speed is not a property applicable to the NBN?

  • able to move?
  • rapidity of movement?
  • something happens or is done?

True a few things may be taken out of context in this view, but perhaps no more than the NBN Co’s misuse of the word speed as an attribute of the NBN.

2 Likes