Introducing operations observability - can it transform how you operate?

Actionable summary of this article

‍

If you only have a few minutes to spare, here's what operators and managers should know about operations observability:

• Despite the complexity of day-to-day operations, most operations teams lack tooling.
• Reliance on dashboards and data teams means operators are reactive and thus constantly firefighting.
• Observability is the ability to understand the internal state of your company's operations from data at any given time, by monitoring metrics (KPIs) or events (issues or opportunities) and logging them.
• In the past, this has been limited to tech metrics in IT (e.g. CPU usage), but thanks to technological advancements, it is now becoming available for operational metrics, too.
• Operations observability allows for proactive operations management that generates significant business value, from improved operational efficiency via increased revenues to higher customer satisfaction.

‍

‍

Introduction

‍

Operations is really complex and requires lots of dedication from the teams working in it. In spite of being one of the most complicated functions within a company, operations teams lack tooling. While other functions have dozens of software (e.g., marketing teams have Hubspot, Braze or Buffer, or sales teams have Salesforce, Close.io or Pipedrive), operations teams mostly use Excel. Many teams try to avoid manual spreadsheet updates, repetitive analyses or unforeseen customer complaint management by setting up a multitude of dashboards. However, as mentioned in my previous article, dashboards should be used only for monitoring strategic KPIs and OKRs, and not for operational metrics and events. As I argued in that article, for monitoring tactical, lower level KPIs and operational events the right approach shall be observability.

But what is operations observability? In this post, I'll deep dive into this relatively new concept.

‍

‍

The definition of operations observability

‍

‍

Observability means that you are able to understand the internal state of your company's operations from data at any given time, by monitoring metrics (KPIs) or events (issues or opportunities) and logging them.

‍

‍

Observability has been used for a long time by IT teams with the help of tools like Datadog, Grafana or Sentry, to monitor the internal state of their IT systems and identify and resolve tech incidents as soon as possible. It has significantly simplified the life of IT teams. The general practice in IT is to build monitors for all key IT applications and metrics, and dedicate engineers on call to react to incoming alerts. However, do not imagine these developers on duty by sitting in front of their screens 24/7 to spot issues. They do their normal job over weekdays or spend their time with their family over weekends, and get alerted only in case there is a signal to investigate.

‍

‍

Current situation - reactive operations with constant firefighting

‍

If this is possible in the field of IT, why don't we use this approach in real world operations, too? Until very recently, operations was mostly analog, meaning it lacked digital data. Even if operations was digitalised at certain businesses, usually custom software was developed to support it. However, integrating an observability tool to each custom application was not scalable, thus there wasn't any specific tool in this field. Alternatively, in the past years operations teams at best could wait for the support of (data) engineers to set up custom developed monitors and alerts, but engineers usually lacked capacity to do so, thus there was hardly any progress in operations observability.

Without observability, operations teams today suffer from constant firefighting that destroy their efficiency as well as motivation. They rely on dashboard check routines and repetitive analyses to identify issues or opportunities to work on. This is not only tiring for experienced operations managers, but also an erroneous process. In worse cases, operations teams receive customer complaints that require timely investigation & reaction. Because of that reactive nature of their work, they usually need to drop the pen on strategic projects and spend majority of their time on business as usual (BAU) tasks.

The good news - observability is becoming a possibility in operations, too

I reckon that time has come to change this for all and introduce operations observability at scale. There have been several technological trends that I believe will support operations observability becoming a more widespread practice in the coming years.

First of all, operational data has become more and more available in the past years thanks to digitalisation. This process has been accelerated during COVID as companies learnt the hard way: they need to digitalise their entire customer and back-office operations if they don't want to die. Thanks to this tendency, companies finally have data available about most of their operations - the next step is to utilise them.

Secondly, such data is also being placed and transformed more and more in cloud based data storages, such as data lakes or data warehouses, as companies move towards the "modern data stack". A cloud based data storage is more scalable, more flexible and data is more accessible to make it available for monitoring. While it's still difficult to integrate an observability tool to custom software, it will be possible to access the generated data via these standard, cloud based data sources. This will allow observability tools to be built in a scalable way and thus operational data can be finally turned into proactive actions.

‍

‍

Significant business value can be unlocked for companies that introduce operations observability

‍

In a few years proactive, automated monitoring will be as basic for operations teams as it is for IT at the moment. This will completely change how operations teams work, for their best. There may be even moments when operations managers are able to enjoy peace of mind in the evenings or over the weekends, without being scared of overlooking something!

Proactive operations doesn't only provide peace of mind, but can also unlock significant business value to companies. Imagine that you get alerted proactively if there is a new signup to your product, an opportunity to upsell to a certain customer or the risk of a churning partner - all of which can significantly increase your revenues. Or envision the situation when you can proactively identify suspicious activities, products with close expiry dates or assets with maintenance need in order to improve your operational efficiency. Lastly, you can also significantly improve customer experience if you communicate about delays in advance, react to low star ratings or identify recurring topics in customer complaints.

Furthermore, there are several indirect positive outcomes, too. Operations teams can stop firefighting, thus save resources and focus on strategic topics that bring business value, while (data) engineering teams can also avoid doing repetitive, simple tasks and rather dedicate time to customer facing features.

‍

‍

The three ways to introduce observability to your company

‍

There are several ways to introduce operations observability into your business.

The first and most simple way is asking your BI team to set up SQL-based monitors that send alerts to email addresses. While this seems to be a simple approach - as it doesn't require any custom development or new tool introduction -, there are several drawbacks. First of all, this approach is not very scalable. Any time you want to modify something on the monitor (e.g., increase the threshold), you need to return to the BI team and ask for an adjustment on the query. Secondly, it's very difficult to build a sophisticated escalation & resolution process on top of such manual alerts, as they arrive in detached emails.

The second option is developing a custom feature for monitoring, alerting or even automating the resolution process. While this is the most personalised solution for the needs of your company, it also takes the biggest effort. As this method requires capacity from the engineering team, operations teams often find themselves at the end of their backlog, except there is a very strong business case. As a consequence, companies build custom observability features for the tip of the iceberg - for the most impactful and/or most common operational events, and usually that's it.

The last and most recent approach is choosing a readily available operations observability software that integrates to cloud based data sources. With such a tool companies can empower their operations teams to set up monitors & alerts with a few clicks, without waiting for (data) engineering teams. Such a solution combines the flexibility & scalability of the first two approaches, furthermore offers a source of truth for all monitors and their triggered events.

Without exposing too much information at this point, I can share that we're building such an operations observability tool as we speak. We truly believe in the concept of operations observability and our vision is to empower operations teams to manage their operations proactively by detecting, resolving & automating repetitive business issues & opportunities.

‍

‍

If you're interested in seeing a demo and joining the list of our beta customers, feel free to contact me!

‍

‍

Once again, dashboards and observability have key differences

‍

As a summary of my previous two articles, let me recap again the main differences between the concepts of dashboards & observability. I do believe that operations teams need both, however they need to be aware of using them in the right context.

‍

‍

‍