Blog Case Study Office 365 Teams

Microsoft Teams Incident Management: Breaking Down Silos

Ensuring Microsoft Teams Voice Quality at any time can be a challenge.

As voice quality highly relies on the network path and equipment, issues can happen at any time, day or night. That is why you need 24/7 data to be able to detect and analyze what is going on before it seriously affects the ability for your employees to work.

GSX is the specialist of Microsoft Teams 24/7 monitoring as we provide the only solution that is able to consistently and continuously test, detect, alert and report on the availability and performance of all its features, from any location in the world.

Last month, we had an interesting use case that shows the flexibility of our solution and how the information we provide is used across the entire IT organization.

One of our customers is a very large tobacco company with hundreds of sites spread across 3 continents.

They started to move to Microsoft Teams at the beginning of the year and finally fast tracked their deployment during the Covid-19 pandemic.

They deployed GSX Gizmo to measure the quality of service for their most critical sites and quickly realized that several locations were experiencing recurring issues.

This was a first achievement because before deploying Robot Users to test the quality of service, the IT team was blind to the user experience. Despite the recurring issues at multiple sites, there were no IT tickets open. Users were just suffering in silence.

Having discovered the issues and how often they occurred, the company wanted to have more information about the incidents and be able to break the silos.
(I.e, Silos are departments or teams working in isolation, which can restrict efficiency and innovation in an organization.)

The Network team especially wanted to test the path between each location and the Microsoft endpoints during Teams calls in order to determine the root cause of the issues.

GSX provides out-of-the-box reports for Microsoft Teams Voice Quality, but to simplify it for our customers, we do not show every statistic or detailed IP we collect during our test.

The good news is that our solution is flexible enough to adapt our PowerBI dashboard to our customers need.

The demand was simple. The Network team wanted a specific dashboard showing and organizing all the tests done by the robots with the IP of the machine doing the test, the IP of the Microsoft endpoints and all the network and healing statistics we can provide. So that’s what we did.

MS Teams Analytics dashboard

So what are we looking at?

On the left side we see the list of all the tests done. This list provides detailed information on every Voice Quality test made by all the robots.

The scan date of course, the name of the source, the IP of the source and its location are all included in the report.

Then if we scroll we see all the network metrics related to the Voice Quality tests you will see data related to  packet loss, the round trip latency,  Jitter, bandwidth, the MOS (Mean Opinion Score), degradation average during call, and the type of protocol used during the calls.

MS Teams Voice Quality Tests

Followed by the MOS of each test the number of packets sent and received, the port used and if there is any packet reorder ratio.

Mean Opinion Score

Finally, you can see the total execution time for the test that usually varies with the latency, the ICE type and flags (Teams internal warning / errors message) the codec used and if the Teams client had to repair the samples to produce the Voice sound (Healed Ratio and Audio Fec Used).

All these elements are extremely important to understand the quality of the call and the root cause of the degradation.

But what makes that dashboard powerful is its ability to sort the information.

As you can see on the top right, we can filter the information by the MOS score.

That will then only show a subset of the call – in this case only the calls with issues (MOS < 4).

Doing that will allow you to only focus on what matters. Right below, you will find the list of IP addresses used by the problematic calls:

Here you directly see the first 3 f IPs on this list are the most problematic.

Right below, you will see which robot the problematic calls were made from (expanding each robot will provide the IP of the robot).

Now you have a dashboard where you can easily list all the issues, get all their characteristics, and know exactly the route the calls took.

For the network team of this customer, the information was gold. With GSX, the customer can drill down into each call and then test with network tools, like traceroute that route and check in the log for every piece of equipment that malfunctioned at that specific time.

The network team had in hand all the required elements to drive deep research into root cause analysis.

Another way to sort / filter information is to use the IP charts. You can click on any IP used by Microsoft to see all the calls, focus on the bad ones and see the network degradation for them.

Without this kind of data, it is almost impossible to use your log to find what you are looking for. With GSX Gizmo, everything the network team needs is easily reachable, the information is put in context and investigation can start right away.

The information can be accessible through the dashboard or export directly from PowerBI in the desired format.

Thanks to this dashboard, the Microsoft Teams manager can breach the silos in the  IT department in order to provide relevant information to every team involved in the service delivery.

This solution provides the necessary tools for the customer’s IT team to understand and fix the situation they had in multiple locations.

Thanks to that, they were able to find the defective equipment along the route to the cloud:

  • Several cases were fixed by upgrading the local router
  • Some cases were related to firewall overload
  • Some other were directly related to the bandwidth made available by the local ISP
  • Finally, some locations were not configured properly to reach the Web Security Gateway.

After extensive investigation, all cases were solved, and Microsoft Teams is now working perfectly for every location.

Occasionally, the Call Quality Dashboard of Microsoft provided additional help. The problem with it is that it retrieves only calls between users. This puts the IT team completely in the dark for what is happening during off hours.

In the end, the perfect collaboration between the Microsoft Teams manager, Microsoft CQD, GSX Gizmo and the Network team (global and local) enable the success of this large Microsoft Teams deployment.

GSX provides the only Microsoft Teams monitoring tool that helps you understand precisely what users are experiencing and gain visibility into Microsoft Teams performance for all your remote locations.

Get started today to keep your employees on the path to optimal productivity.