Thinking of getting Vivid Wireless? If you would like to get your first month free please see my most recent Vivid Wireless post for how to earn yourself a free month and give me a free month at the same time (win/win).

Today I had the privilege of following up my Open Letter to Vivid Wireless with a telephone interview with Martin Mercer who is the CEO of Vivid Wireless and Nicole Christian who is the Manager for Customer Experience and Retention at Vivid Wireless. Below is a transcript of the call I had with Martin which I feel answers a lot of the issues I had in my original letter and some extra ones that I managed to squeeze in as well.

I welcome any comments and feedback on the questions please let me know your thoughts. I openly welcome (and request) that you leave thoughts, feedback or the like as a comment at the bottom.

Topics

If you wish to jump to a particular topic (as it was a half hour call so there was a lot we covered) please take a look at the below sections

1. Speed Issues and Backhaul
2. Capping Users and Downloads
3. Plans for upgrading to/supporting IPv6
4. Firmware Updates/Fixes
5. Issues with Torrent Files + Rundown of Prioritisation
6. The thirst for bandwidth – can “Unlimited” compete with 1TB plans?
7. Customer Service – The good and the bad
8. Tips for diagnosing issues in future

Interview

Stuart: Thank you very much for taking the time to let me follow up from my original query, as I mentioned I have had quite a bit of traffic coming through the blog so I thought I would take this opportunity to get a bit more feedback from you, and hopefully clear a few things up. I spent last night going through quite a few of the forums on Whirlpool and the Vivid Wireless forums to put together some questions that will hopefully clear the air for some of the users.

So Martin, in recent weeks a lot of the speed issues seem to have been resolved that users were experiencing, but one of the concerns for the users including both light and heavy users was that when the unlimited plan was turned on the network capacity just wasn’t able to handle the influx of high download users. Has Vivid done anything behind the scenes to ensure this will not happen again or if it does that it is diagnosed prior to becoming a significant impact to the user base?

Martin: There’s been lots… we’ve learnt a lot in the last few months Stuart. There were several issues that affected network speeds and the fundamental cause at the end of the day was that the demand for the unlimited plan exceeded our expectations. But we provisioned extra backhaul capacity but the response to the advertising and the response to the new offers exceeded what we really expected. It was compounded by one of our suppliers was actually late in provisioning capacity,  so if everything had gone according to plan we would have had sufficient backhaul capacity to avoid running into that issue but it didn’t work out that way.

The other issue that I think I posted on the forum about was due to settings on core routers and that was only revealed when we got the extra capacity for the network so we fixed the core router issue and we learned from the provisioning issues so that now we have a policy that we always have sufficient bandwidth and every time we get to about 75 or 80% capacity we automatically provision extra backhaul capacity to avoid a reoccurrence of any of those problems.

Stuart: Beautiful that is great to hear and it is always good to learn from our experiences, as I work in IT and so I know that these things do happen.

Back to Top

Stuart: Out of curiosity GeneronimoS mentioned on the Vivid Wireless forums, and I will just give a quick quote from him “I know that Vivid have been capping people who are downloading like 40 gigs in 3 days as a warning, but that won’t stop them forever.” I just wanted to seek clarification on if users have actually been capped at this stage for any reason on the unlimited plan or if this was a misconception.

Martin: Definitely a misconception, we don’t have any capping at all at this stage and we certainly don’t apply policies down to the level of individuals. So we don’t have any policies at any level down to individual users irrespective of how much network resources they use.

Stuart: Great, I just wanted to clear that up because I saw that and I thought that it doesn’t make a lot of sense from a business perspective and so I wanted to ensure we had the opportunity to clear that one up.

Martin: I’m glad we have the chance because we’ve been as transparent as we can with our whole approach to management of the traffic on the network and we don’t want those misconceptions as we don’t cap individual users or generally cap or shape traffic either.

Back to Top

Stuart: In regards to IPv6, which is the new revision designed to replace IPv4 protocol, does Vivid have a timeframe for when this is likely to be supported as a lot of Australian ISPs don’t have any publically available plan as to when they will be supporting the new version of the IP protocol.

Martin: That is a very timely question Stuart as I asked our chief technology person that only two weeks ago and we are planning for it at the moment. There are some things that are out of our control like our current radio access network technology does not support IPv6 , the next generation will. It is really a chicken and egg situation for us we are now planning a nationwide rollout and we will be implementing the next version of the protocol and technology and we will be able to support IPv6 and we are asking the question of when not IF and we haven’t defined a clear timetable yet but we are doing the work now to prepare for the migration to IPv6.

Stuart: Well that is comforting to know as I have been writing an assignment on this for university quite recently. And seeing how quickly the IP address pool for IPv4 is being depleted it really is that chicken and egg situation of “we don’t need it now but we will eventually” and having to spend that money to actually provision and prepare. It really is that chicken and egg.

Martin: Just to reassure you our allocation of [IPv4] addresses, we have more than enough to meet our current needs and demands for years to come. We are not going to run out of IP addresses anytime soon but we are very aware that it is an issue that the industry as a whole needs to address.

Stuart: That is comforting to know that we will always be able to get on [to Vivid Wireless].

Back to Top

Stuart: I have been lucky enough to assist with beta testing the new firmware that was released just yesterday (I believe) and I have noticed that a lot of updates have been rolled into the one firmware for the home gateway. While I can understand from a testing perspective the benefit to doing this and rolling all the testing up in one, are there any plans to release more incremental updates, especially in the early stages so that issues such as the router crashing do not take so long to resolve?

Martin: That is a good question Stuart and also thank you very much for being one of our Beta testers, it’s the people with more exceptional usage patterns and usage behaviours that help us uncover the bugs that ordinary users never experience. The Firmware update process [is in consultation with] our vendor who will develop the firmware upgrade and some of the software challenges are more complex to fix and at this stage it just so happens that the crashing issue took a while to resolve and we were able to roll several items into the one release. We actually have done about three or four firmware upgrades to the home gateway already and the earlier ones were small incremental fixes like you suggested. But this time because of the time of the development of the different fixes some were shorter to develop and some were longer to develop and they just happened to come together at the same time. So we did an upgrade but generally speaking in an ideal world we will be doing constant firmware upgrades that will put in fixes as they are identified and that has [so far] been the practice with the home gateway and the USB modems.

Stuart: No worries, interestingly enough on that as a slight aside I am one of the slightly geeky users as would have already been apparent and I have some custom settings which I have set up in the gateway through the engineer login which is obviously only for advanced users. I noticed when I upgraded the firmware those got erased and reverted back to the originals. With over the air firmware updates this could become a potential issue for those users that have specific configuration requirements. Is that something that has been taken into account?

Martin: It has been taken into account but it has been very hard to avoid Stuart, part of the standard upgrade is the factory reset and the settings change back to the default. If the exceptional user has gone in there and changed the configuration [this makes things more difficult].

Stuart: I guess just as a possibility for the future , would it be possible to be notified when there is a firmware update but not directly get that firmware update ([for example]  stop the over the air update for those exceptional users) so that they can intervene where required?

Martin: I am sure we can look at something like that Stuart it is a good suggestion.

Back to Top

Stuart: Great. Now this is another one that we can hopefully clear a bit of the air and that is the downloading of torrent files on the Vivid network. At this stage my understanding is that there is no blocking or “always on shaping” to torrents but they may prioritised lower than other traffic during peak congestion periods. Furthermore I believe with the issues surrounding the downloading of *.torrent files are made worse by issues with the home gateway firmware for which a currently undiagnosed reason during high load certain *.torrent files are unable to be downloaded from the web.

Martin: Peer to Peer is not my strong point but let me try to make hands of this as openly and honestly as I can. As I understand it there is a .torrent file which is almost like the key to unlock the torrent that you download and then you use that to actually [download] the torrent file. There has been an issue with downloading the .torrent file and we posted a workaround on our forum around a month ago now and there is a fix for that which is imminent, we believe we have a fix working for that now we are just going through the final tests so hopefully we will be able to get that out very soon. So hopefully the .torrent file itself is almost fixed.

In regards to your other question as to whether we shape any traffic the answer, no we don’t. No shaping takes place at all, but we do have prioritisation. One of the things, when you’ve got a fourth generation network like ours one of the things that changes is the quality of service and we utilise that in our network. We give the highest priority to time-critical traffic and lowest priority to non-time-critical traffic so for example voice traffic gets prioritised over all other traffic, the second priority is what I will call general internet traffic, so protocols like HTTP and others and the final priority, the third priority, is peer to peer and other non-time-critical traffic. Therefore it doesn’t get shaped it just means that the available bandwidth is shared between those and prioritised accordingly.

Stuart: That makes perfect sense, I have noticed that even during peak periods and even while I was doing testing with the modem and maxing out my connection, I was still able to make telephone calls through the VOIP gateway and didn’t have a single issue, the calls just went through flawlessly. So that is certainly evident of why you have that prioritisation in place.

Martin: Generally it is a balancing act and what we are trying to do to make sure that every customer has a good experience. There is some sort of traffic as you say like VOIP that if you don’t prioritise then the experience will be bad and that is not a good customer experience. So we just use prioritisation to make sure we get that balance to ensure we use the network to ensure to the best customer experience possible.

Back to Top

Stuart: Since the release of Vivid’s Unlimited plan a lot of ISPs have been releasing 1TB plans and now even plans that exceed the 1TB barrier. With the release of such plans the expectations of Australians as to exactly what defines an unlimited service has increased drastically over what it may have been six or seven months ago. How is Vivid positioned to support massive downloads such as these in the future especially with services such as Foxtel on the xBox being released in the very near future that will consume quite a lot of bandwidth and other IPTV type services.

Martin: Really good question, let me just point out though that terabyte plans in theory aren’t an unlimited plan whereas unlimited plans have unlimited terabytes. But in practice terabyte plans may as well be unlimited as most users will never download that amount.

Stuart: And in fact the actual real life speed even if you were downloading 27x7x365 you would still not be able to reach that.

Martin: Exactly so they may as well be unlimited. So people who say we are never going to do unlimited plans but we will do the next best thing and do a terabyte plan are really sort of being a bit refute. In answer to your second question we are very well placed to deal with delivering a very good customer experience even with customers’ requirements growing as they are because what we have which none of the other wireless broadband providers have is spectrum. That is something that really limits the internet experience for wireless customers, not the backhaul and yes we had short term problem with having backhaul capacity available which has since been resolved but it is actually the spectrum to the base stations that as it gets all used up which causes the experience to degrade. We have more spectrum than anybody else and we only have to support data, not data and voice and we will never have as many customers as Optus or Vodafone or Telstra so for that reason we are always going to have a lot more spectrum or a lot more bandwidth than any other wireless provider. But it could well be in the future that wireless is not the best solution for people who want to enjoy the sort of high definition video streaming and real time high definition video and I think that people are going to have to get very clear on what their needs are and find what can be delivered and it may be that wireless even our superior wireless isn’t the right solution for people who really want to use high bandwidth [applications].

Stuart: Very true and looking forward even with the National Broadband network which isn’t likely to be completely rolled out until 2017 a lot can change in that time. However in the meantime not being able to get ADSL in my situation and being able to get Vivid Wireless, as I have mentioned I am a rather heavy user. 99% of the time I am getting a perfect connection with no issues but for those high definition applications as you said it may not be the solution in the long term.

Martin: Where do you live out of curiosity?

Stuart: I live in C[ensored] Which is technically right at the outer limit of your coverage area. I am right on the edge of where your maps say there is no coverage however luckily I get a perfect connection.

Martin: I was going to say you must be very close to being out of coverage.  That is very interesting.

Stuart: I connect to the Petersham tower actually so I found that quite interesting to see how far the range is so I am technically right on the very border edge of the coverage area.

Martin: We deliberately produce fairly conservative coverage maps because we would rather turn customers away than have customers with a bad experience.

Stuart: I guess now that the initial rollout is now completed, are there any plans to expand outside of the Sydney CBD further or are you looking at targeting other areas that are not currently covered such as Brisbane.

Martin: We can’t wait to expand the network to cover all of Sydney and all of Melbourne, in the public domain we are out there at the moment talking to a range of potential investors and we will be expanding the network to cover all major cities as soon as we can.

Back to Top

Stuart: In regards to customer service, firstly before I go any further, personally I have always had quite good customer service from Vivid Wireless, one thing that I have to say is exceptional that I have not seen in any other company previously is that when a staff member said they would call me back. I was always called back five or ten minutes ahead of time so I did not have to call up constantly chasing up my issues so I do have to give kudos there where it is due.

However I have noticed on one forum and also through my own experiences if you email the support team I have not had a single case that has not received the generic response “it is best if you call us for this issue on xxx”.  Is this an initial teething problem or is the preferred method of technical support through the phone? Obviously when you email through a couple of times and get these replies in the end I stopped emailing so I wanted to find out if this is a general practice or something that has been overlooked.

Martin: I think that is an area we can do better in Stuart, there is no doubt. Those technical support emails are answered and dealt with by the same people as in the call centre. Early on when they were learning their skills and a lot were inexperienced with the system they didn’t always have the confidence to answer in an email. We also had canned responses that weren’t very good and then they found that the answer to a lot of emails were actually quite complicated and they are quite difficult to answer via email as they require some level of troubleshooting to actually understand what the problem really is. So we are still getting to grips with how best to deal with technical enquiries via email and there probably is a tendency on the part of the customer service reps to look at it and just think “oh if I can just get the customer on the phone I can resolve this more easily as I need to troubleshoot this properly if they just call me” and to then email the customer requesting a call and this is an area we can do better in but it is something that we are aware of and at the moment it is just a) the skills issue which we are addressing very quickly and b) often it is the easiest way to resolve the issue for customer enquiries as if we send an email back and it does not resolve the problem then a customer may be unsatisfied.

Stuart: That makes perfect sense, thank you for clearing that up.

Back to Top

Stuart: Finally I just wanted to ask one more question to finish off. If users are experiencing speed issues (or otherwise) as I have seen on the forums recently there are still a few isolated users reporting speed issues, and they are having problems outside of normal technical support hours what information would you recommend they note down other than the time and what they were doing so that they can contact support the next day for further diagnosis.

Martin: Well you have mentioned two of them which is the time and more importantly the URL as that gives us a sense of whether it was international or local or what part of the network it may be. In addition to that the obvious things such as signal strength (RSSI and CINR values), a speed test from SpeedTest.net (noting the upload and download speeds or better yet a screenshot), also what other activity was being done at the time, and the nature of the activity that you were doing. Also what operating system you were using such as a mac laptop and what device you were using to connect to the network.

Stuart: That’s great I will be sure to note it down on my blog so that people who are suffering any isolated cases can get them diagnosed quickly and efficiently.

Back to Top

Martin: Alright Stuart, I have to run mate.

Stuart: No worries thank you very much for your time and thank you very much as well Nicole. I really do appreciate you taking the time to sit down to go through this with you.