Company
Industry: Telecommunications
Location: Russia
Customers: 42+ million
Number of employees: 7,000+
Year of integration: 2019
Number of Amixr Users: 30
Amixr Integration
Integration: Slack
Incidents: ~10k/month
Incident load: 77-93% less
Incidents each: 3 min -> 45 min
Incident handle time: ~1.5 min

Tele2 is one of the world’s leading telecommunication companies. Founded in 2003, it has 40+ million users to date, thanks to the high-speed and reliable mobile and internet connection that this telecom giant offers to its users.

Tele2 operates in a highly competitive industry, and its main asset is people, both clients and employees. It is believed that the key of their success has been in their high customer retention and in the growth of their customer base.

“The company's strategy is to have the best Net Promoter Score (NPS) among the industry leaders. NPS determines customers' willingness to stay based on how  happy they are with the offering. Our subscribers' loyalty is the most valuable thing for us.”  - says Nicholas, Head of the IT Transformation Office.

We first got the idea of getting an incident alert system when my team realized that they were lacking a tool to quickly respond to accidents, incidents and warnings. This was a problem indeed: the response time was often unacceptable, especially at night; incidents would often take us 30-60 mins to resolve.

“Everyone had a large amount of data about accidents and incidents, but it was unclear which one to respond to first and who should handle which incident.”

Amixr has been implemented in two key departments of Tele2. The first one — the Digital division — is responsible for customers’ personal accounts as well as for the Tele2 website. These are the critically important channels of communication with our subscribers. The second is the CRM division. All the contact centers and sales representatives work with CRM, along with a number of internal products — e.g. with Tele2Pay which is a payment processing platform.

“It is crucial to keep on track of everything, as an incident may impact customer experience directly.”

Amixr is designed to fit into different frameworks and workflows. Tele2 is a great example of two departments using Amixr in different ways.

The CRM Department uses Amixr for absolutely all occuring incidents. For the last 2 months, there have been 24K alarms, and 8.5K grouped, of which 97% were closed automatically and only 3% (275) had to be analyzed manually.

Meanwhile, the Digital department also integrated Amixr, understanding how essential it was for their operations. Due to customized alarm settings, Amixr doesn't notify this team about all incidents or accidents. Although their product is complex, they have much fewer incidents compared to those of their colleagues. Within the the same time period, the Digital department got 4.3K alarms, 2K of them had been grouped, and 465 were acknowledged. It turns out that 77% of incidents were resolved automatically.

We can see different approaches here, but what's is important is that both cases clearly show a positive result: 93% of incidents were resolved automatically. With this dynamic, the team now gets an alarm every 45 minutes instead of every 3 minutes (!), which means that the they get distracted 15 times less often than they used to. Thanks to this, they can now focus on other tasks without the need to manually check every notification.

“Our priority is to minimize the number of incidents that have to be looked at by customers. We need to know about the emergence of an incident before the user sees it and complains. Basically, the earliest response to incidents and the reduction of their number are at the core of our IT strategy.”

Examples and cases

“Authorization problem”

Late at night on December 3d, when the office was empty; a monitoring system was checking the accessibility of the authorization site when the services went down. This is the highest-priority accident. Amixr reported it immediately, and the local engineer was able to acknowledge the incident in only 2 minutes. Should it have been otherwise, the acknowledgement could have taken 15-20 minutes. It is estimated that during this time, there would have been more than 50 calls to the contact center with the complaints about having no access to personal accounts.

The emergency of that day would have cost us not only money, but also customer loyalty, and the latter is beyond expensive. We must make sure that this never happens again. Thankfully, as of today, the average handle time is around 1.5 mins.

“DDoS attack”

There was a time when we faced a DDoS attack at the Digital department during the working hours. Our provider drove off the attack and accidentally blocked our own proactive monitoring robot. During the whole incident, Amixr was helping us to be in the loop and to keep the monitoring running. In the worse scenario, we could have been missing alerts which could cause a risk of monitoring degradation.

“No payments received”

There was an accident with Tele2Pay at the CRM Department one day: no payments seemed to have been received. That was definitely a high-priority one. Customers being unable to pay for a service means that they cannot actually use it. As a result, they lose their trust in us. They expect to have a stable connection and to be able to make a call, yet they see our failure to provide this service.

Amixr did its job this time as well. Engineers got up at 2 A.M. and quickly started fixing the problem. They managed to do so without many clients even noticing something was wrong.

As a manager, what do you like and dislike about the system?

"I like the fact that it is a handy tool for the engineers. But managers want to see the efficiency in clear and illustrative numbers; ideally, as actual money.
I can also say that it improved the reaction time considerably. Our tech guys started fixing what is really important; they don't have to respond to everything.
Our team is on Slack which is not our final solution. We would like to use another messaging client, but the question is whether support teams will be available for it as well. Yes, I would like to integrate with other team chats. But of course I don’t want to pay anything extra for it."

- says Nicholas, Head of the IT Transformation Office.

As always we are super happy to hear your feedback in GitHub issues and community!

Your Amixr Team with love ❤️

illustration by Ouch.pics