Quantifying the performance of your application can be difficult, but it’s an important part of understanding how well it’s serving users and understanding how that changes over time.

Apdex scores are a fantastic, simple tool you can use today to better understand how your development team is doing, how it can be improved, and the impact of almost every change to your service. They’re also likely a part of your Service Level Agreements (SLAs) with your customers, which help them understand your platform’s availability.

What are Apdex scores?

Apdex is an open standard, used by platforms and services to provide a human-friendly way of interpreting user satisfaction and performance. Essentially, Apdex scores put a number on how happy your users are when using your service, and help make it simple to track how that may change over time.

Your score is a potent tool for gaining a high-level understanding of whether or not your users are able to use your service without frustration, and whether or not that experience is getting worse (or better) as you release new software and features into the world.

The scale is fractional, from 0 (your users aren’t satisfied) to 1 (all of your users are satisfied), which makes it easy to understand your customers as an aggregate, rather than on an individual level.

How does Apdex work?

The simplest way to start performance reporting is to use a tool that supports Apdex, such as Raygun. The Raygun APM product automatically generates the score for you based on an algorithm that determines satisfaction based on three zones your users can be categorized in: satisfied, tolerating and frustrated.

These zones are based on the simple idea that the longer a user has to wait, the more frustrated they’re going to be. Apdex scores assume that users eventually grow impatient, particularly if slowdowns happen on a regular basis.

Here’s what each of those categories means in the context of Apdex:

Satisfied means the user is able to use your application without performance frustrations or slowdowns, making them fully productive.

Tolerating means the user may notice small amounts of lag or slowdowns but is able to tolerate them and continue doing their work.

Frustrated is the point at which users are considering abandoning waiting, or even using your tool, due to long waiting times on a regular basis.

Why should you use Apdex?

If you’re building a software application, big or small, your users are going to be the lifeblood of whether or not it survives in the long haul, as well as how it grows organically. Understanding the impact of performance on user satisfaction and retention is a crucial part of building a cloud business that’s often neglected until it’s too late.

I’m an independent software developer, and I use Apdex scores in Raygun’s APM tool to help understand where I should focus: my time is limited, so I can only focus on the areas that really help my business grow.

Automated Apdex scores essentially help me identify problem areas for my users and see through their eyes without the scale a larger software development team would traditionally have to get that context. Essentially, it’s a way for me to understand the invisible impact of my changes to the codebase over time, and whether or not I should pause feature development to focus on improving performance.

What does Apdex look like in practice?

Here’s a practical example: I developed a major update to my signup flow that required a backend update alongside the release. After deploying the changes into production, my Apdex scores began to fall, indicating my existing users were becoming frustrated, despite no functional changes for them.

Because the new signup flow caused the application to slow down while the backend was blocking requests and attempting to complete a Stripe API request, it made the overall experience feel less responsive. This identified that the signup flow was hurting my existing customer satisfaction, and helped me prioritize implementing queueing in my application so that requests from new users wouldn’t impact existing ones.

As you grow and scale your user base, performance becomes more of an issue that needs to be tracked closely. Complex applications always have hidden bottlenecks, often in dark corners where you might not be able to see them yourself as a developer — making translating those experiences into a metric I can monitor all the more important.

What’s a good Apdex score?

Apdex scores can be used in context with your own industry, target market, and the problem you’re actually solving. Different segments of users have vastly varied expectations for performance, and it’s crucial to properly tune your tooling to create useful scores for the experiences your users expect.

It often comes down to perceived value in exchange for waiting for a task to complete. Users are more than willing to wait a few seconds if it means getting something desirable at the other end, such as an order successfully processing when submitted. If they’re performing a task they don’t enjoy, or something that seems like it isn’t worth it, higher scores are more important.

Raygun’s APM tools provide five separate groups for performance that show in a color-coded graph in your dashboard. These are defined as very clear groups in Raygun, and are a great way to understand the areas that need immediate attention or a longer-term fix:

Excellent: 0.94 - 1 (Dark green)

The user is fully productive and your users are not impeded by your application response time.

Good: 0.85 - 0.93 (Light green)

The user is generally productive and your users are not impeded by your application response time.

Fair: 0.70 - 0.84 (Yellow)

The user notices some performance lag but continues the process.

Poor: 0.50 - 0.69 (Orange)

The user notices a moderate amount of performance lag but continues the process.

Unacceptable: 0.0 - 0.49 (Red)

The performance is unacceptable and the user may abandon the process.

In my experience with Apdex scores both in my own independent software development lifecycle and in larger organizations, keeping your Apdex score in the ‘good’ segment, or above 0.85 is the best way to be sure your users don’t consider performance an issue in your application.

The problem is this isn’t always realistic for the entire stack: some tasks take longer, may have blocking database queries, or require CPU-intensive tasks that just require the user to wait. Printing a page to a PDF, for example, is a task that can block the user or require them to wait around, reducing your overall Apdex score, but providing more value for the user in the end, allowing you to tolerate a ‘fair’ score in some instances.

How can you improve your Apdex score?

The reality in any software development team is that there’s often a tension between shooting for a very high Apdex score and actually shipping features, updates or keeping users happy. It’s hard to know when to focus on Apdex scores, or when to divert your energy toward improving your product.

A good rule of thumb is to use Apdex scores as a metric for success for every release you make; does shipping a particular feature lower the score? Can you mitigate that impact in the development process, or improve it in a future sprint? Often it’s easier to address them at the source, as the feature is developed, or through the use of dedicated sprints that tackle the highest-impact Apdex issues.

For many development teams, key outcomes are tied to shipping functionality, making it difficult to justify spending hours on improving the score when they aren’t core to your own performance goals. This makes keeping the score a core part of your development cycle’s considerations all the more important, since it may be harder to justify focusing a future sprint on improving them.

Tools can help you improve your Apdex score

In Raygun’s Application Performance Monitoring (APM) product, you’re able to view the highest impacting queries and how broadly they affect your users. This is a personal favorite way to focus in on ‘easy wins’ I can make for my users between shipping new features and helps me prioritize whether or not it’s worth focusing on performance before proceeding with any new ideas.

I use a combination of slow queries, slow traces and slow API calls in the APM dashboard to determine if a problem will make a meaningful impact on my Apdex score. If a query is running for 3-5 seconds at a time, it’s probably going to be worth investigating as it’ll help bump the number back up.

This is an ongoing process that should be considered on a rolling basis. If you’re using continuous deployment processes and pushing code out into the world daily, it’s easier to measure that over time. For slower release cycles, it’s more difficult to hone in on the changes that are impacting user experience, because a large amount of change is bundled into a single release.

The problem with Apdex scores

Like with all metrics, Apdex should be used as a part of a wider monitoring stack and should be applied to the right issues, not across the board as a measurement of success. Used correctly, Apdex scores are a fantastic way to get insight into platform-wide trends, but they shouldn’t be used in isolation.

It’s true that Apdex’s biggest flaw is that it’s an overarching calculation that makes it difficult to see where specific, painful issues lie, but that’s part of the reason it’s so appealing. It’s designed as a high-level, holistic overview of how your platform is performing, not an examination of a single page that’s causing issues on a micro scale for your users.

So, what are you waiting for?

When you think about Apdex scores, I encourage you to consider them an important business metric that can help you get closer to your users and your own bottom line. Paying customers, and their experience with your product determines whether or not they’ll stick around for the long haul, and Apdex scores are a powerful tool for seeing beyond your own screen.

I personally really enjoy using Raygun’s APM tooling, which offers superhuman insight into the application stack and reveals areas for improvement without the need to go digging. If you haven’t tried it yet, it’ll transform the way you think about software development for the web. Try it for free today.