SaaS Metrics Maturity Model - Part 1

This episode delves into the intricacies of SaaS metrics, focusing on the development and significance of a SaaS Metrics Maturity Model.

1. Importance of a shared metrics foundation for accurate data interpretation and decision-making.
2. Trust and transparency are crucial in metric calculations and their subsequent adoption.
3. Strategic integration of metrics with business goals enhances operational effectiveness.
4. The culture of a company should encourage the use of metrics in everyday decision-making.
5. Long-term strategic planning should be informed by metrics to forecast and guide company trajectory.

1: Introduction

Overview of the podcast theme and introduction to the SaaS Metrics Maturity Model. Key discussion on the relationship between metrics and business strategy. Ray Reich: "It's always fun to talk about problems with SaaS metrics, but it's also crucial to discuss solutions." Dave Kellogg: "The SaaS Metrics Maturity Model was born out of a need to solve the misapplication of metrics."

2: Discussion on Metrics Issues

In-depth discussion on common issues encountered with SaaS metrics and the necessity for a maturity model. Dave Kellogg: "We often see a lack of foundational understanding in how metrics are calculated and used." Ray Reich: "Many companies struggle with aligning metrics with their strategic goals, which can lead to misinterpretation and misuse."

3: Root Causes and Solutions

Exploration of five key root causes that lead to problems with SaaS metrics and potential solutions. Dave Kellogg: "Identifying and addressing the root causes is essential for improving metric utilization."

1. Establish a clear metrics foundation: Ensure everyone in the organization understands what each metric means and how it is calculated.
2. Build trust in metrics: Regularly verify and validate data sources and calculation methods.
3. Integrate metrics with strategy: Align metrics with business objectives to ensure they drive relevant actions.
4. Foster a metrics-driven culture: Encourage the use of data in all business discussions and decisions.
5. Plan long-term using metrics: Use metrics to set future goals and track progress towards those goals.

Dave "CAC" Kellogg and Ray "Growth" Rike discuss the 5 root causes that created the need to develop a SaaS Metrics Maturity Model for companies as they scale.

People

Ray Reich, Dave Kellogg

Companies

Gainsight, Zoom, Atlassian, Okta

Books

None

Guest Name(s):

None

Content Warnings:

None

Dave Kellogg
Live from Schenectady, New York, it's SaaS talk with the Metrics brothers, growth in CAC. And I'm growth, better known as Ray Reich, founder and CEO of Benchmarket. And I'm Kac, better known as Dave Kellogg, independent consultant, Eir at Baldrton Capital and the author of Kelblog. And together we are the Mettrix brothers. And we go together like wine and roses.

Ray Reich
Dave, you're still sticking with the alcohol analogies? Even though I couldn't close Kelly's as. A sponsor, I do. Technically, roses are a flower and that was inspired by an old Jack lemon movie called the Days of Wine and Roses. Dave, I know what a big grateful dead fan you are when you talk about roses.

I'm not usually pairing those with wine. Oh, you want to talk about roses? Well, normally in the Grateful Dead iconography, roses are paired with skeletons. Did you know that one of the central dead images comes from a book called the Rubaiyat of Omar Khayyam, a collection of 18th century poems from 1913 featuring illustrations by british artist Edmund Joseph? Did you know that?

Dave Kellogg
Right. No, I didn't. And I don't think I ever wanted to. And I think we need to get back to what we're really good at. And that's talking about business and today's topic, the SAS metrics maturity model.

Ray Reich
But before we do that, maybe we should hear a word from our B two B SaaS sponsor. SaaS talk is presented by Gainsight, the first digital customer platform, including customer success, management, product experience, customer communities, and customer education. Find out why more than 1500 companies, including SaaS leaders like Zoom, Atlassian and Okta, and hundreds of early stage startups, rely on gainsight to efficiently retain and expand existing clients through an integrated digital first post sales customer journey. Gainsight has affordable packages for younger companies and goes live in two to four weeks or less. Visit www.gainsight.com.

C
Now back to the show. Okay, kak, so let's get into business here. Let's dive in. Where did this SaaS metrics maturity model come from? But Ray, does this mean you're not coming with me to Las Vegas to see dead and company at the sphere?

Ray Reich
I'm not, and I'm sorry to tell you that. But let's stay on task. Get back to trucking on the SAS metrics maturity model. Oh, there you go. There you go.

Dave Kellogg
Keep on trucking. So look, the background of the SAS metrics maturity model. This is a model I made for my saster 2023 talk, which is called the strategic use and abuse of SAS metrics. I started out by talking about 15 problems with SAS metrics. And it's a lot of fun.

It's always fun to talk about problems with SAS metrics, but I realized, as much fun as that is, you're not really talking about how to fix those problems quite so much. So I felt an obligation after having said here that 15 silly things we do with SAS metrics, that could be warning signs. Well, I felt some obligation to say, okay, how do we solve these problems? How do we not have these 15 problems? So that's why I made the model.

Ray Reich
And I must admit, I dave Kellogg fanboy, even though I do a podcast with you, I was there in the third row and it was a packed house, so it really resonated. And I think one of the things you really did was kind of highlighted why you made a model. So before we go into the five root causes, what was the primary reason you even thought you had to create this model? Yeah, like I was saying mostly to me, I wanted to do what I call you are here analysis. Like maturity models are not a new thing.

Dave Kellogg
Gartner have made them for years. They're CMMI for software engineering, the capability maturity model. There's all kinds of maturity models. There's a bi maturity model. And I think they're a nice way of kind of breaking a problem that can otherwise be overwhelming into layers and then saying, look, you are here, so kind of place yourself in this model and then it tells you what to work on.

Like, I'll give you a really simple example. I work with some european companies who don't have a P and L. That is a standard SaaS P and L. So they look at like, what I call nature based P and L. They're looking at like personnel costs and consulting costs.

So you can't even calculate a CAC ratio because you say, what's sales expense and what's marketing expense? And they say, we don't know. Right? So that's a foundational problem. Right.

We need to say, okay, one of the foundations is we need to have a standard SaaS P and L, or we can't even begin to talk about SAS metrics. Right. So that's just an example of why it really helps to say, look, here are 15 crazy problems and let's try and get some idea of if we wanted to fix these, how they line up by layer. Let's try to help you place yourself on a layer and then from that, know what to work on next. Yeah, but you mentioned 15 problems.

Ray Reich
And when I grew up in corporate America. I was taught three to five. You need to identify three to five things that you can actually work on and make progress. And I think that's kind of what you did. Dave, didn't you kind of identify five root causes that were the reason for a lot of these problems?

Dave Kellogg
Absolutely. Yeah. Yeah, you definitely were there and paying attention, Ray. I'm very happy. So, yeah, that was the whole point that, look, we had these 15 things that everybody could get a chuckle about and go, wow, that feels like me.

But, well, what's the underlying causes? And the answer is, I identified five of them in order. One, there's no shared metrics foundation. Meaning what metrics mean, how they're calculated, kind of why they matter. So if you don't have some foundation for metrics, for example, if you don't have a SaaS P and l, you really can't start on the journey.

Cause everything's gonna be a manual reconciliation. I mean, you can produce metrics for fundraising, but you can't operationalize them. So the first layer was there was no shared metrics foundation. The next was there was no trust. There was either no trust in the data, where it was coming from, in the calculation.

Sometimes the people who make the calculations, the process for producing them, do the metrics tell the real story, or are they being kind of cherry picked and spun? Smoothing is a classic way, leaving out customer success from a CAC. There's all these kind of standard tricks, advertising commissions and forgetting to reaccelerate them. If you find all these sort of surprises, you go, I'm not sure I trust these metrics. So that was the next level.

And a lot of that comes from misuse of them. Right. If you're always bludgeoning people with metrics, then maybe they're gonna start spinning them, right? For every action, there's an equal and opposite reaction. So you can kind of beget some of these problems by the way you use metrics.

Ray Reich
Hey, Dave, before we go to the next root cause, I was just thinking we're gonna talk about each of these five, but one of the big values we can give is what to do about each of these. And I know we're gonna talk about a maturity model. We may not get all that done in this episode. So would it be okay with you if we don't, that we do a second episode where we really talk about the maturity model and what to do for each of these root causes? Yeah, I mean, I think.

Dave Kellogg
I think my guess is that's what's going to happen as we do the episode. So. So we should feel free to, you know, if you want to elaborate on some of these as we go through, feel free or tell a war story. But, but the first one was basically, look, if there's no metrics foundation, it's very hard to get going and we need to fix those issues first. The second is if people don't trust the numbers, then we need to go build trust in those numbers.

I'm both identifying the problems and this is going to map directly to the maturity model in a minute. Yeah, let me go back to. There's no shared metrics foundation. You mentioned why they matter. One of the things I find in so many SaaS companies is they're saying, this is such an important metric, net revenue retention.

Ray Reich
I'm just making one up. But the people and customer success know that it's a priority, but they're not even sure how it's exactly calculated and why it's important to the business. And I think one of the things we do in the SaaS industry so often is we talk about metrics, but 95% of our employees don't even understand why they're important. So I think that's a huge root cause issue. Yeah, I mean, look, I think there's a difference between operationalizing metrics and when I operationalize them, everybody knows what it means.

Dave Kellogg
They know how they can drive it. They understand that as a vp of customer success, you can't prevent sales overselling. So nothing unless you're CEO. And even when you're CEO, nothing is really totally within your control, but you can still feel ownership of the metric and go, yeah, I own this metric. And by the way, the reason our target is 110 instead of 125 is we know we're doing some overselling and last year it was only 108, and we're trying to make it better.

So when you've operationalized metrics, people understand what they mean. They feel as accountable as they reasonably can be, feel towards producing the number, and they understand why they matter. Conversely, when metrics. The thing I've seen, Ray, is when metrics are just a fundraising thing, it's like, oh, we need to raise around, let's go calculate all our metrics. That itself is a huge problem.

It's actually layer four in my model. It's not having a data driven or metrics driven culture, but you can see that the people I've worked with that don't have standard SaaS pnls, for example, they do produce all the SaaS metrics, but only when they're building a fundraising deck. Right. It's done effectively manually. It's not operationalized.

Ray Reich
Yeah. And I love your second root cause. There's no trust. Probably the most common thing I see when I do our SAS metrics and benchmark assessments is, oh, we're looking really good compared to the benchmarks. I really think this is a great metric.

And there was almost no questions. But then there's a time where a metric is not performing well, or the board pointed out, and they're like, well, how do you calculate the metric in your benchmark? Here's how we calculate it. And people actually try to defend their metric based upon how they calculate it. When there's no common alignment on how it's calculated between the function, the CEO and the board of directors or investors, it's a common issue.

I see. Yeah. And the bigger problem is, let's just say you view, because a lot of times people will view that as quote unquote, correcting the metric. Oh, well, at our company we do things XYZ differently or we're focused on booking. So we're going to calculate a bookings CAc, not an ARR CaC.

Dave Kellogg
And that's, you know, look, that might even be correct. First, I don't think it's a good idea to run a SaaS company based on bookings. But, but let's just pretend you and your board do and you've decided you're bookings focused company. You can't. Then the issue to me is what I call one sided corrections.

You say, okay, we're going to correct the CAC ratio to be done on bookings, correct quote unquote to do it on bookings. Well, that's fine, but then you can't compare it to an ARR based CAC. And that's where I get off the bus, right. It's very hard to get me on the bus for the bookings CAC. But you might, right, if you convince me your business is somehow very different.

And I can think of once, by the way, if you're doing $10,000,000.10 year deals that take a really long time to cook, okay, maybe we are better off on doing them on total bookings because you really did buy that customer for ten years. I recently saw a company that did that. Massive deals, long sales cycles, long commitments. Okay, maybe you sold me on a bookings CAC, but then you can't just turn around and compare that to an ARR kayak just because it looks favorable. Now, I remember sitting in that presentation, because one more thing on trust, and we'll get to the next one.

Ray Reich
You talked about using metrics to, I think, bludgeon either another organization or the recipient. Can you talk a little bit about that? Using of metrics to mislead or bludgeon? Yeah, I mean, like I mentioned earlier, when you. When you bludgeon people with metrics, they respond accordingly.

Dave Kellogg
So what do I mean by bludgeoning? I mean that with a. Without a whole lot of understanding or thoughtfulness, people just say, your CAC is 18. 180 is too high. We're inefficient.

For example, this happened to me. I was running host analytics. Our CAC would run one eight to 20, which is a high CAC ratio, and we kind of got bludgeoned with it. There wasn't a lot. They were like, oh, you're inefficient.

And I'm like, I don't think we're any more inefficient than our competitors. I think we're in a competitive market, and I've got a deal, and I've got it at. I'm doing pretty well, but my competitor, adaptive, shows up, and they offer the same system at 25k. So I can win the deal at 30. I can get a price premium, but I can't get 100% price premium on it.

And what happens to my CAC? My CAC just nearly doubled. Right. When I had to cut my. On the last day of the sales cycle, my cac nearly doubled.

Right. And by the way, when adaptive finally filed to go public, they never went public because they were acquired first. Their CAC was around two. Wow. See, I told you.

For six years. I was telling you that there's no magic here. They have the same sales model. They're hiring the same people. That was my derivation.

Their sdrs look like our sdrs. Their quotas look like our quotas. Their staff look like our staff. I think they're running at the same CAC. I think this space is a high CAC, in part because we're bludgeoning each other.

Right. That's a thoughtful way to look at the metric and not just go, oh, you're too low. It's too high. It should be one five. It's like, whoa, hang on.

So that's what I would call bludgeoning. Gotcha. Well, let's move to the third root cause I don't know if you remember that special guest who kind of snuck in a few episodes ago. Nick matter from gain site. Of course I remember Nick he actually.

Ray Reich
Told me this one time when we were talking about debating the merits of metrics, and he's like, ray, if metrics aren't tightly aligned or integral to the strategy, they're not nearly as valuable. That's actually your number three, right? It is, yeah. Metrics as an afterthought or not a good thing, but we touched on this one earlier as well. But some companies view metrics as a financing tool only, right.

Dave Kellogg
Calculate them when they're doing around. We talked about that. Not a great habit. I mean, yes, you need to calculate them when you're doing around, but if that's the only time you're calculating them, you should look at the mirror. They're not linked to okrs.

I mean, I like okRs. You know, the notion of saying here's our objective and having three to five objectives, and then here's a key results associated that kind of defines or proves the objective I think is useful. So, you know, if your SAS metrics are important to you, let's just say I'll give you a concrete example. You're doing an up market push because you think you could take your Asp from twenty five k to fifty k and just say, I'm the person leading the sales team up market, I should have an OKR to get a 50K ASP. And by the way, we should manage that intelligently and not bludgeon me if it comes in at 44 in our first quarter of trying.

Right. So. But yeah, if you care about these metrics, like for example, if you think your CAC ratio is too high because your deals are too small, and you think the answer to that is to push up market for bigger deals, then somebody should have an OKR, first on developing pipeline in that richer segment, and second in actually winning deals with that higher ASP average sales price. So the other thing you'll find when metrics aren't linked to strategy is they're not being used as kind of leading indicators or there are no leading indicators, we're not using them day to day to run the business. We're kind of metrics on the side.

Right. And if you do this correctly, you're looking at the metrics every day. I couldn't see a forecast for the CAC when I ran post. I saw a forecast for my CAC every week because it's not that hard. I know what new way is going to be.

I know what sales and marketing expenses forecast I can forecast to CAC and keep an eye on it, which I did. Yeah. And that's like the last episode of SAS Talk. We actually talked about customer lifetime value to CAC ratio, a multivariable compound metric. Right.

Ray Reich
And if you don't understand the leading indicators that's driving that and can make operational decisions to improve those leading indicator metrics, there's no way that that top level metric is going to be relevant to you on a day to day basis. Yeah, totally agree. By the way, let me give you my example of the indicator. It would be pipeline ASP as a close to close one. ASP.

Dave Kellogg
In my example of pushing up market, the real proof of the pudding is am I closing 50k deals? But we could look at pipeline value as a leading indicator. So it's just another example of what things tend to happen before other things. And even if they're not end results, which definitionally leading indicators aren't end results, but, but closed one deals, they're a tough metric because they're almost immediately history. We closed it, we wanted it's over.

And by the way, I won't watch it change for six to nine months if I have a long sales cycle. So we need to look at both types of these indicators. Can you dig a little bit more into the fourth root cause? The culture is not metrics driven. Sure.

So the fourth underlying problem or root cause of SaaS metrics problem is we don't have a metrics driven culture. Discussions don't require numbers. Jim Barksdale, the founder of Netscape. So this is an old story, but he had a great quote worth repeating, which was if we have data, let's look at the data. If all we have is opinions, then let's go with mine, which is an absolutely fantastic quote.

I think all CEO's should say this all the time because it's a great way to send people to bring data to conversations. And in my opinion, Ray, I've noticed that there's almost two types of companies. I tend to be pretty binary in these regards, but there seem to be companies where numeracy is a prerequisite for conversations about the business and companies where they're not. In my belief, and I don't have any data to support this. My belief is the type one companies do way better than the type two companies.

For example, when I was at Salesforce, I didn't work for Benioff. I was one off Benioff. But. But I would sit in for my boss from time to time and Betty off staff meeting and if you don't know your numbers and Betty off staff meeting, you're dead, man. You are done.

So numeracy is required as kind of an entry ticket. And that's why I think people who don't work in kind of numerate cultures, they don't understand that. They think all people do is talk about the numbers. And it's like, no, we're not talking about the numbers. We're talking about the business, comma, using numbers.

And that's really the goal. Boy, that brings back so many memories, because, as you know, I grew up my first eight years after undergrad at GE, and Jack Welch was the same way. You could debate him on anything. And one time I had that opportunity, and I knew you had to have two things, numbers to back it up and critical thought to justify why you're recommended to do something different. So I love that.

So now is a good time to reveal the secret of why we're live from Schenectady, New York. I don't think we've ever talked about this on the show before, and I don't think anyone's ever asked. But the reason is I knew Ray had worked at GE, and I pictured Ray as kind of a GE guy. And you'll have to have worked, unfortunately, 20 years ago to know what that means. I was like, ah, Ray, he's a GE guy.

He's got the GE guy feel. He must be from Schenectady. So I literally thought you lived in Schenectady for the longest time before I found out you lived in New York. So I respected that. And, you know, I hate to do this because, you know, I like to talk about business, but I got to give a 32nd story about Jim Barkstow 30 seconds.

Ray Reich
So I did have the chance to, to work closely with Jim and executive team at Netscape back when I was helping launch ecommerce there. And he had another saying. He's like, you know, there's two type of people in an organization. He goes, it's a little bit like when you look at your breakfast plate, you'll see an egg and you'll see a bacon. He goes, in that environment, the chicken was involved in the business, but the pig was committed.

Are you a chicken or a pig? Here at Netscape? I love that saying. Yeah, it was a super common saying at the time. I love that saying as well.

Dave Kellogg
I didn't actually understand it. Like most of those sayings, I didn't actually understand it the first couple times I heard it, but it rubbed in. Okay. So in any case, old memories are fun, but in fun stories, let's wrap up here with the last of the five underlying causes, which is our metrics, are not being used to define trajectory and long term goals. This, to me is kind of a plus one on the whole model, because the rest of the model is, in my mind, kind of ordered, right?

It's like you gotta have good foundation. You have to trust the foundation you built. You need to kind of operationalize metrics and link them to strategy. You need to create a culture that is data metrics driven. Those four things to me are natural layer cake.

This one on top is a bit artificial, but I think it's super important. So that's why I put it there. But I think when it comes down to the problems that get caused, it's when companies don't have a defined long term trajectory. And you could use metrics and numbers to say, this is the company we're trying to build over the next five years. So whenever I submitted a budget to the board, I would submit the four quarters of next year with targets that we would be held accountable to.

That's the budget. Approve this and we'll be accountable for it. But I would present a model for years two, three and four beyond that and say, this is what we're trying to build for two reasons. Three reasons, maybe. One, that the next year model became the default.

Like when we started discussions about the plan for year end plus one, we'd start with what I showed as the model a year ago. So it was a nice kind of default. Second, it agreed on trajectory of what we were trying to do. And third, I actually think it built trust, because the number one way to game a budget and software is to cut all incremental spending in the second half, designed to drive growth in the following year. And then you can show kind of great EBITDA or great profitability, or great cash flow in year end, but you have teed up nothing for year end plus one.

And you can only do that trick once or twice before your board gets very angry with you. But if you're showing a multi year model that dovetails into your financial plan, you can't do that. And so in some ways, you know, when I taught my kids to drive, I taught them like, don't look at the hood ornament, right? Look out way in front of the car. Right.

Like, you won't be surprised if you're looking way out in front of the car. If you're driving, looking at the hood ornament, you're going to be surprised constantly. So it's the same thing with this long term trajectory so this one is basically saying, another great way to use metrics is in conjunction with one of these three to five year long term models, often called an LRP or an LRM long range model, long range plan. If you have one of those models, you have it also full of the SAS metrics you're trying to hit. So everyone understands, for example, that, hey, we're shooting for a CAC of one eight, we're not shooting for one four.

And one four might be best to breed for you, but in our space with our competitors, we think we can only hit one eight. So if you look three years out and see a CAC of one eight, let's have that conversation now, because I think that's the best we can do. Totally great. So I want to do real quick, I want to summarize the five root causes. One, no shared metrics foundation.

Ray Reich
Second, no trust. Third, the metrics are not integral to strategy. Fourth, a culture that's not metrics driven. And then fifth, metrics aren't being used to define its trajectory and long term goals. And those are the five root causes that made you create this SAS metrics maturity model.

So I'd like you to introduce it at a high level, David, and then we'll go into the next episode and talk about it in detail. Sure. When we talk about the model, we're going to have five layers. Very unsurprisingly. Layer one will be foundation, layer two will be trust, layer three will be strategic linkage, layer four will be culture, and layer five will be strategy.

Okay, I can't wait to dive into the details. So that's going to wrap up this episode talking about the root causes and needs to have a structured SAS metrics maturity model. David, thank you for teaming this up, and I can't wait to get into the maturity model itself. Awesome.

Dave Kellogg
SAS Talk is a production of the Metrics Brothers growth in CAC and a member of the Benchmarket podcast network. By accessing this podcast, you acknowledge that the metrics brothers make no warranty guarantee or representation as the accuracy or sufficiency of the information presented or the humor content of the jokes provided. Ray, the information, opinions, and recommendations presented are, according to our spouses, probably wrong, and provided for general information only. This podcast should not be considered professional or for that matter, unprofessional advice. We disclaim any and all liability for any direct, indirect, indirect, misdirect, incidental, special, ordinary, consequential, inconsequential, or other damages arising out of any human, you sub, or God help you reliance upon the information presented here.

Ray Grothreich is based in New York City and available on Twitter x rayreich. Dave Kakkelag is based in Silicon Valley and available at kelblog. Schedecti, which is French for unspellable, is not our actual production location. You can reach us@sastalkpodcastmail.com dot thanks for listening.

Wake up wa.