Indeed! Why not?
The article explains this. And I am not sure whether Optus (or any Telco) would support this as it might show customers deficiencies in their own network (by using an alternative carrier network).
And, as the outage also affected landlines and internet connections, the alternative network roaming would only benefit mobile users.
The other risk is telcos might chose not to maintain their networks at the highest standard (run them at fault tolerances of N or N-0), to save costs, as they can rely on other telcos when something goes wrong. This could cause Australians to end up with a far less reliable and poorer quality service in the long term.
The article explains the telcos side. From the consumers’ side shock horror if a customer is allowed to experience a better service? Let the marketeers run for cover if or when their claims are challenged.
The community is best served by a robust reliable communications system needed for the ‘online and connected life’ and especially emergency services. Most areas across the globe have mobile roaming while our networks evolved in a mindset more reminiscent of Olde Australia where we consumers were only exposed to ‘other worlds’ events through newspapers arriving on the ships and planes, domestic radio/tellie, and for some the shortwave. If one is not exposed to alternatives one can be under-served yet accepting and even happy.
Rather than erect moats (barriers) and claim with or without foundation ‘[…] network is the best’ it might encourage telcos to ‘put up’ or lose customers. Telstra demonstrably has the widest coverage in the regions and nationally but in areas where there is coverage, regardless of network or locale, there are black and grey spots everywhere. So far the ‘answer’ to me has been VoWiFi in my house but it doesn’t resolve anything 20 meters out as my Wifi signal fades.
As for local conditions around Eltham the Telstra signal is often unusable; the Optus signal always strong of late. It has not always been so and the reverse was the norm for years. I need dual SIMs, Telstra and Optus. The question should be why that is the case, not why is that a worry.
I for one would prefer being able to be make and receive calls on whatever network worked at the moment. If ‘no cost roaming’ was legislated our costs might rise but just maybe the telcos would be encouraged to add some towers or more seamlessly share what there is? It is not as if they are hostile to co-share if it turns another dollar of profit, regulators willing.
Would that inhibit their investments? Maybe per telco in cases but it is equally possible the entire infrastructure could get increased investment. While the outcome is relevant to this consumer, the telco boards and lobbyists interests are less so. One can only hope government balances the issue to improve ‘Australia’ not just improve profits.
I agree with everything you said!
Indeed. It’s high time the telcos were forced to provide reliable and available services nation-wide, including making their participation in roaming mandatory.
If they can’t do roaming because their networks can’t take the volume of traffic, the solution is obvious: put some of that huge profit into improving the networks …
While it seems a easy solution, the costs will be significant. Currently mobile networks try and match expected capacity, not the capacity of theirs plus the biggest other provider in the area…
It will cost far more than any profits they make, as they would need to have a network which can support another carrier’s volume for possible an hour or two, maybe more every few years or so. It will require a lot of duplication and redundancy in each mobile network and would be a significant barrier to entry by new players (reducing competition).
About a decade or so ago there were claims that the electricity network was ‘gold plating’ the network to ensure that consumers had a reliable supply with a N-1 fault tolerance. Consumers paid for this ‘gold plating’ with significantly higher network charges.
What is being asked in the mobile telcos will operate at N-1 where the N is a whole provider network is lost rather than a piece of critical infrastructure like that which the electricity network previously operated. If they cater for rare events to allow customers on any other network to have access to a mobile network when there is an outage, this will come at significant cost which all mobile phone users will need to pay for through higher plans.
It could be argued that such approach has merits particularly if outage like that seen yesterday become more frequent, but there needs to be a rational discussion about the impact of such a move. Discussion will need to focus on whether the significant costs to the consumer/mobile telcos warrants some level of protection against the risk of outages.
I struggle to understand why that is a unique unsolvable problem in Australia when roaming is commonplace in much of the world, and for routine black/grey spots that are a norm, not just for major failures that are an exception.
It is not as if every mobile customer is going to use their phone at the same moment in either case. As for capacity many of us experience peak times when whoever our telco may be has overloaded towers and we cannot make calls. That is especially common around 5PM, during special events, at airports during peak periods… and even here in leafy Eltham. Sometimes trying to call a few times can be necessary but certainly is a better outcome for this user than not being able to make a call at all, regardless if it is from overload or say Optus (or Telstra or TPG) collapsed. The latter would obviously have more impact, but those 000 calls, card transactions, and whatever might get through despite network congestion or having to try a few repeats.
Imagine if our energy retailers each had to have their own generation and poles and wires. I hope it is obvious where that analogy might go.
Agree, including from the consumer side and national benefits, eg the cashless society and online connected life aspects, and emergency services, going beyond corporate protections and ‘profits are important’ issues.
BTW, maybe a bit simplistic but it occurs roaming could incur costs from the phones ‘home telco’ paid to the ‘roaming telco’ to cover cost plus regulated profit margin to the roaming telco. Whether the ‘home plan pricing’ would go up and by how much would be left to competition. Those with fewer roaming costs could offer cheaper services and in theory attract more customers; they could achieve that by assuring better infrastructure and service. Maybe more + than - ?
Roaming and what is being proposed are very different. Roaming doesn’t include potentially millions of new users being added to a operating network with a very short time (if one network has an outage and its users are switched to another network).
Roaming would continuously have a number of users entering and leaving a network, possibly with total numbers of users on the network being reasonably stable over time (or fluctuations within a known range). This can readily be catered for not potentially millions of new users in a short time.
If the mobile network operates like other networks, the addition of millions of new users to another network when it isn’t designed for it, could be even more catastrophic. That being you now have two networks which have an outage, the second crashing under load.
I try to stay away from narrow views when there is a more pervasive problem.
That is another aspect, but for my values not one that should be used to ‘look over there because doing what is right for national interests and needs is too hard’.
Good design drops interrupts that cannot be serviced rather than crashing, application dependent. It also allows metering connections and flow. It is not an unsolvable issue even if difficult or moderately expensive; I doubt it would get into the platinum cost range but I would not expect any opposition pollie to avoid attacking the costs, at least while in opposition.
I don’t think data roaming is a solution to what is just simply bad or non-existent contigency plans.
When things go wrong in the middle of the night, I am quite prepared assign cause to human error. That’s when changes occur. And if that change knocks out the core backbone network, then big problems. Diagnostic data stops being sent to management servers unless on totally separate links. Management servers cannot communicate with network devices, unless totally separate links used.
And here’s the real problem. Support people cannot get to management servers because the network is down. You have to be onsite, and physically access devices.
And those who can be onsite, like first level network technicians may well have to consult higher levels, or access documents on file servers. Oops. No network, no phones.
Many big companies run their systems in ‘dark’ datacenters. There are no technical support people there at all overnight. Just a few security guards, and a list of numbers to call if an alarm comes up.
I don’t think anyone is saying roaming would be the solution to major outages like yesterday’s. But had it been in operation yesterday, it could have significantly reduced the extent of the impact for the users.
Businesses that couldn’t use their mobile-connected EFTPOS terminals yesterday would’ve been able to stay online (and I suspect that those terminals don’t use a lot of network bandwidth, so I don’t see that type of switchover swamping the receiving networks). People with personal alarms that depend on mobile connectivity wouldn’t have been at risk. Emergency services could’ve called people back. And so on.
Whether or not the outage resulted from a change made in the middle of the night, I agree with your description of the effect that a core network outage has on the support team’s ability to become aware of and start doing anything about the outage.
Contingency plans for such situations must exist and have been proven to take such issues into account - as Disaster Recovery Planning is supposed to do.
For example, are the support people certain to be able to get into the data centre when they do get there? Do the contingency plans make sure those who have or can grant access to the data centre will be contactable if this network is down, for example? It’s not much use if all the emergency contact numbers are mobiles that depend on this network …
Probably not - but if the government makes them do it then that doesn’t really matter.
An interesting point … A race to the bottom of reliability?
Given that discussions have only just begun on that kind of forced roaming in the event of a natural disaster and probably only for the mobile network, it would be very dubious to suggest this for an outage at this time.
Just to emphasise the second part of that: yesterday’s outage took out fixed line services too and they cannot be roamed in the same way that at least is theoretically possible for a mobile service.
For the first part of that: this kind of roaming (for mobile services) throws up a mass of questions that need to be worked through.
As the article says:
Australia’s competition regulator recently found roaming during natural disasters was “technically feasible”, although complex.
So let’s say that one network is in the middle of a technical outage or, dare I say it, some kind of complex cyber incident … and you want to throw a mass migration onto them? That could actually be counter-productive as it would detract from solving the original problem.
In the worst case, who knows, it might allow the cyber incident to spread like contagion.
Of course we already have forced roaming for emergency calls - so any argument that people will die in an emergency due to an outage on one network is a bit bogus (in the sense that the proposal here adds nothing for emergency calls).
(These days you can even contact emergency services from your phone but via the satellite network - hence completely independent of whether you have mobile coverage and whether the mobile network or networks are experiencing an outage - and maybe this is actually a better direction to be going in for resilience.)
Another consideration is that a carrier does not provision its network with enough spare capacity to provision millions of extra customers at a moment’s notice. Not only would this be quite chaotic but there is a risk that the second network then collapses. This would be particularly bad if the original network (the one with the outage) has significantly more customers than the network to which customers are roaming.
As has already been noted, it may not solve any problem at all in areas where only one mobile network has coverage.
I’m not suggesting that my thoughts on this are the final word but I hope I have given you some challenges to confront before this becomes reality.
The challenges if they exist at all are more likely corporate politics than technical. Not a problem we can or need to solve here. The brains trust are putting on the pads in anticipation. What might happen next - first ball ducks or an unbeaten century once the players eventually get onto the field? With Optus still keeping all in the dark about who it might name in the opposing team, a no show might be how it turns out.
The move is similar to one undertaken in Canada after Rogers Communication last year suffered a lengthy outage nearly identical to the one Optus is believed to have suffered from on Wednesday. The service brought down services for 12 million customers and subsidiary brands, and also took down payments, healthcare and law enforcement services for 15 hours.
After the Canadian government gave the telcos 60 days to come up with a plan, the companies signed an agreement to allow for emergency roaming for such outages.
Some parts of the media have been latching onto the Canadian example, without acknowledging that Canada is very different to Australia. In Canada they have 15 mobile networks operating either nationally or provincially. In Australia we have 3.
With more networks the opportunity to have automatic swapping to another network improves as the user base is more diversified and some may be able to be absorbed by other network free capacity in the time of an outage.
In Australia, Telstra network has 22.5 million users, Optus 10.2 million and TPG/Vodaphone 5.3 million users. These are substantial numbers for another network provider to have and maintain free capacity to allow another networks users to be automatically swapped to an alternative network in the time of an outage.
The constraints in Australia will be technical not political. Technical for the free capacity which must be provided on other networks to readily absorb users from another network at the times of an outage (which are rare events) and ability to instantaneously and temporarily migrate users to another network (a type of temporary porting) which allows the other network to see the users as being connected to that network.
is highly likely. Contagion is a real possibility where not only one network has an outage, but measures to mitigate this outage results is widespread system collapse. An even worse outcome.
It is very easy for the media to hypothesise what could be done, without fully understanding the constraints which may exist. Having worked for a electricity network provider in the past, the media commonly also do likewise for the electricity network without any knowledge or understanding of their operation or constraints.
I would be waiting for results of industry and parliamentary inquiries, rather than relying on media articles with utopian arguments.
Hence was privatising our telecommunications and breaking up the provision of services due to technical constraint or political determination? By the hands of more than one government, and a legacy that continues to cause angst given how it has turned out today.
Politely no need to shout. If Canada can, Australia can, noting the ACCC is also supportive, having consulted.
Looking to the evolution of our national broadband network political expediency and thinking over-ruled the technical advice.
I accept the proposal for ‘emergency roaming’ is what is on the table, but that is a symptom not the underlying problem of our spotty mobile services with black and grey and empty spots everywhere - including where there is supposed to be service. Then a discussion moves onto what happens when a network collapses.
My observation is the proposal is at the end of the day make work with we users still needing 2 SIMs on competing networks to probably have service wherever we go, including built-up and metro areas - the bush being a bad punt on the best of days.
Solve access and adding capacity can happen albeit at a cost. If access is not resolved neither is the ‘cashless society and online connected life’. Apologetic views that it is all too hard or all too expensive relegates Australia to a 3rd world future where our lives are increasingly fragile under the mandates of OTPs, apps, and being connected for information and paying for things, not just entertainment.
Only weeks ago the glass was overflowing
but days ago reality set in for many.
and as a number of commentators opined, capacity is an issue with a corollary that is true only if roaming is not done in an evolutionary, planned manner by expert network technologists, not ‘expert’ politicians like he who brought us the NBN MTM fiasco who was actually a money man without much of a clue about technology.
(This one addresses Telstra capacity under the weight of the Optus crash)
No it didn’t. The ACCC statement is about network sharing/roaming through an emergency event, not from a network outage nor through cyber attacks. The ACCC also there is currently reluctance by the mobile telcos to share during an emergency event.
This is a very different situation and discussion to the Optus outage.
I don’t think that’s fair. As per their own statements, none of the three mobile network operators wants to do this but that does not invalidate the significant questions that it throws up or the technical challenges. After all, even the regulator found that it was “complex”.
What are the questions that this throws up? If you can’t answer that then I don’t think you are in a position to suggest “if they exist at all”. In reality, noone is in possession of all the questions because the government only raised the possibility of forced roaming during natural disasters quite recently, and telcos have until March next year (if memory serves) to respond.
Those figures say it all really regarding the possibility of doing this if Telstra falls over.
Assuming that there is some efficient and magical way of deciding whether any given Telstra customer force roams to Optus or force roams to TPG … Optus and TPG between them, having 15.5 million users between them, have to absorb at short notice an additional 22.5 million users i.e more than double at short notice.
I think that is an outstanding question. That is one perspective (that this is a type of temporary porting) but I prefer to think of this as a type of temporary roaming. It isn’t actually the same thing and it remains to be seen as to which works best.
I note that when porting a mobile number, the regulations allow the two carriers X hours to complete the job as per
90% of ports must be completed within 3 hours 99% of ports must be completed within 2 business days
and that is in the ideal scenario where the Gaining Carrier controls the time of day (can limit to business hours) and controls the volume of ports (can avoid overloading systems trying to do a million ports an hour). Unplanned outages don’t magically occur only in business hours.
Even those regulated performance thresholds call into question the wisdom of forced porting. The gaining carrier might only just be about to get one of those 90% ported across after 3 hours … and the carrier with the outage fixes the outage. That is of course like waiting for a bus …
I’ve not experienced complexities with roaming. I’ve used Telstra sims in 3 OS destinations and with Optus in 2. Providing I’ve a roaming enabled sim and account one simply needs to ensure it is turned on.
In the latest instance I needed to change zero with my account settings. Although I did double check the status before leaving home. On arrival in NZ and turning flight mode off connection was automatic. $5 per day extra Telstra works the same in many other locations but at a more hefty $10 per day.
Hence it is a solution already within proven technology. The assumption is Optus and Telstra provide similar seemless service for their OS carrier partners customers in Australia. There is even a costing mechanism in place.
There may be complexity to provide roaming automatically in specified circumstances to customers who do not have a roaming option with their sims and plans. The relevant question is do we really need to change the way the telco’s technology functions or mandate all plans sold in Australia need to provide roaming capability within Australia? It does not need to be automatic providing the customer has the option to turn it on if or when necessary.
When it comes to the physical capacity of a network, redundancy (which by definition requires some spare capacity) is always going to be a requirement. To ensure reliability is a need whether spread across multiple carriers or where there is only one.
Considering mobile phone tower assets are debatably not a core Telco owned asset, should Australia re-evaluate how mobile services are delivered?
A question relevant for all 3 of our mobile carrier infrastructure providers.
If a large network such as Optus or Telstra goes down for a period, and ALL of the affected customers suddenly roam across to the other provider’s network, I’d expect to see very significant network overload problems with call dropouts across the board, and bandwidth grinding down to gridlock - mass data transmission, collisions, and retransmission…rinse and repeat. Roaming works when the additional load is small, relative to network capacity.
I’d expect that if the transformation to a solely digitally connected future is to be delivered, the extra capacity and redundancy will be necessary to provide the resilience us consumers will demand. None of us known what we don’t know nor can anyone know every possible cause of a major network failure.
I’m open minded as to whether the solution is to force greater redundancy and capacity through sharing between the 3 Telcos or to force each to have it independently. One would expect a lesser investment through sharing, with a benefit in the regional and outer areas. One needs to deliver for regional as well as those in the major urban sprawl.
A third alternative is a national mobile infrastructure provider and relegation of the Telcos to secondary service only roles. It’s already built into how other essential services such as electricity networks are divided between retail and distribution, utilities delivered and the NBN established. Can anyone argue the mobile phone network is not now an essential service? Especially since it relies on a very limited resource - the available bands in the electromagnetic spectrum.