How to improve quality when FTAR is 100%

When you start MRU, you might receive a TR where the measured quality is already perfect (i.e. FTAR 100%). This might also happen in the factory, after you spent some time with your TR and fixed all existing QB failures.

At that moment, you may find it difficult to quantify the impact of your insights that improve quality, because the quality is already 100%. Yes, you might have insights which improve quality when FTAR is 100%!

How is that even possible?

First, you should question if your QB is high enough. Items passing the Internal QB and failing on external QB are clear signs of internal QB being low.

You can define a new QB, do measurement on a sample of units with that QB, and based on the fail rate, you can quantify the impact of changing the internal QB with respect to the external QB (e.g. 40/75 EQB failures T4W could have been avoided by the new QB).

After raising your QB, and fixing all failures, you might reach 100% FTAR again. Then, what to do?

FTAR and the QBs are defined for your TR, so that you can measure the quality of the units your team delivers. However, they don't fully represent the business value your team creates. They may not even be aligned to the business value, because they might be focusing on the next downstream team as customer, rather than the actual end user.

Your QB is a projection of the business value created. In order to understand the full added value of your team, you need to have a wider perspective of your team. You should read R2 document of your TR, and determine the business value your team creates. Then, you can improve quality by focusing on improving the business value.

How? You can start reading and reviewing the tickets your ICs deliver, and then think about how you can design a better solution that adds more value.

In Support TR's, your External FTAR might be your NPS. However, that doesn't perfectly measure the value added. Your NPS might be perfect, still there might be a big room for improvement. Your TR creates value when ICs solve or deflect problems.

You can increase quality if you solve a problem better with a better KB article (X tickets/week can be fixed with better troubleshooting steps decreasing #of customer interactions, hence the end to end ticket resolution time decreases from Y to Z hours). You can deflect tickets if you create a customer facing KB article that lets your customers solve the problems on their own (X tickets/week can be avoided thanks to a customer facing KB article).

In PCA TR, for an E2E (end to end test), your External FTAR might be % of tickets accepted by the QA team. However, this is not important for your customers. Your customer would like to see the next release on time, without any bugs.

The TR creates value by ensuring products are released weekly with zero regression failures. For this, your E2E needs to help improving the quality of the product. What is the value of and E2E if it doesn't cover the feature it is supposed to test? You can write better E2E's that have better test coverage that you can quantify.

For RCA (root cause analysis) tickets, you can study the root cause, go deeper if it is shallow, identify the actual case and then see to which other products the same solution would apply. By doing so, you can eliminate/avoid X failures/month for those other products.

Alternatively, you can fix a problem with a product that now helps deflecting tickets for that product (e.g. if mobile app X, instead of crashing due to low disk space, checks disk space at the beginning and warns the user that there is not enough space for the app to run, your customers would create more space on their drive instead of opening a ticket due to crash).

A note about reviewing tickets - here, we are talking about tickets where measured quality is already perfect. Even if there are QB failures, your focus shouldn't be the QB failures only, because that's a very narrow scope. QB reviews are required but not sufficient. You should of course review the failures and provide unit specific coaching to your ICs (EQB exercise). However, you as a manager, should have a wider perspective of your TR, and evaluate the tickets in terms of value added as well.

Let's take a concrete example. On PCA TR, suppose that you read an E2E ticket which shows that the E2E was not written in a product agnostic way (required by IQB). E2E uses the Curl command, which is available in Linux but not in Windows. You can coach the IC and move on to the next ticket... but that's not a content review. You didn't think about a better solution at all!

Your observation was the fact that Curl is not available on Windows, and testers may not now how to use it. They mail fail/reject the test on Windows and this might cause the release to be delayed (because release doesn't pass the tests).

What can you do then? Think about how you can fix that particular problem:

1. You can create a document about how to install and use Curl on Windows.

2. Then, you can add it to Confluence space of the TR so that that document becomes a permanent learning.

3. You can open that E2E, and add a precondition mentioning Curl should be available/installed, and include installation instructions - give a link to your document.

4. For that and similar products running on multiple OS'es, you should search Jira for the instances of "Curl" and apply your fix on step 3 to all those E2E's.

5. You can then present your insight to VP, mention how many releases are potentially saved from getting delayed and ask him/her to share the doc with all PCA.

That's how you fix the problem. Here, step 1, 2 and 3 are the steps where you take initiative. Step 4 & 5 are the steps where you take your head up and think about the wider perspective, the business value your TR creates.

Those examples above show how you can generate content insights and improve quality even when the quality seems perfect.

Recipe for Quality improvement:

1. Study R2 of your TR and learn how your team adds value.
2. Read tickets. That's where the work happens.

3. QB failures are the most obvious things to address, but don't limit your quality improvements to QB failures. Think about how you can deliver better business value, while reviewing each ticket.

4. Quantify the impact of your insights in terms of business value. If you can do that, it means you are on the right track for executing a content insight.