Melissa answers questions about which part of an organization really owns user research, the scope of a product team’s vision and strategy, and how to tell when your product has too many features.
Show Notes
In this Dear Melissa segment, Melissa answers subscribers’ questions about which part of an organization really owns user research, the scope of a product team’s vision and strategy, and how to tell when your product has too many features.
Q: How do you delineate the goals of product manager and product marketing research?
Q: Should every product manager have a 2 to 10-year vision and strategy, or should each respective PM rather craft division and strategy only covering the current strategic intent?
Q: How do you know when you have too many features in your product? Any suggestions on how to change the mindset that more is better?
Resources Melissa Perri on LinkedIn | Twitter MelissaPerri.com
Episode Transcript
Hello, and welcome to an episode of Dear Melissa, from the Product Thinking Podcast. The lines are now open, and we're ready to answer your most pressing product questions. Which prioritization framework would you recommend and why? Hi, Melissa. Do you have any suggestions on developing a product strategy?
Whoa, whoa, whoa, whoa, whoa. That's a lot of questions. All right, let's dive in. Hello, product people. Welcome to another episode of Dear Melissa. We're going to be answering three questions today, some really good topics. We've got one about who really owns user research and what's duplicative or not, if we're going to do it ourselves.
We've got one about visions and strategy and time horizons. Always love that topic. And then our last question is about when do you know you have too many features? Pretty cool topics.
Remember, if you would like one of your questions answered, you can always submit them to DearMelissa.com. I'm looking forward to getting them. We want to help encourage you to submit more. So here's the topic. Maybe we'll do a whole episode on experimentation. It's one we haven't really done yet. We didn't get a lot of questions about experimentation yet.
So why don't you send me what you would like to learn about experimentation, what questions you have, maybe something about experimentation in large companies or wherever you are. I'd love to answer your questions about that. So kind of priming you with a topic, let's see if we get some submissions around experimentation questions.
But like I said, I'm open to answering anything, career advice, whatever you want. That's what we're here for. That's what this show is all about. This is one of the highlights of my week is answering your questions. So let's get into it then. All right. First question. Dear Melissa, I've been owning the internal products at our company that we're transitioning from managed service to self-service.
I'm pushing for our PMs and product designers to start interacting more with external customers. Our product marketing manager is quite defensive about how what we plan together is different than what product marketing is doing. How do you delineate the goals of product manager and product marketing research? It's always frustrated me that our external product teams relied on product marketing managers and client services to proxy for users. But now externals are going to use my product and it's time to address this issue. Help. I will be right back after this message to actually give you that help.
All right, we are back and we're talking about who owns this user research. That seems to be what the question is, right? What's the difference between product marketing research? What's the difference between client services research? They were the people who were telling you what your customers wanted.
Now it's time for you to take ownership. So first off, we do need to get out of this notion that one team owns user research at any company, right? User research needs to be democratized and it needs to help everybody. The closer a company stays to its customers, the better the products are. And that doesn't mean the closer a salesperson stays to a customer.
It means the closer that everybody in the company stays to the customer, the better your products are. Now that doesn't mean that we should be duplicating efforts for user research. And this is where I think we need to get better. This is one of the biggest issues I see at companies when it comes to user research.
When we get into all these debates about who owns user research and are we duplicating it or not, every day, every day I hear this concern.
So this is what we need to start doing. We need to start defining why we're doing user research, right? What hypotheses are we actually answering when we go out and do it, right? And which teams are doing that? So one issue that I see is that companies don't understand the difference between generative and evaluative research and the fact that you have to do both.
Generative research is like, what are the problems that we could be solving out there? You're going in not really with a hypothesis to fail or pass it. You're going in there to learn about different problems. And product marketing needs to do that. Sales needs to do that. Product needs to do that. But that doesn't mean we need to do it all, right?
Like it doesn't mean that we all need to be doing the same kind of generative research. It just means that if we have different problems to solve, we're going to be doing generative research. In this case, maybe product marketing has actually gone out there and done a ton of generative research about how certain customers use, you know, have this problem that's emerging in the market.
And they have studies on it from customers they've talked to. They've run a bunch of focus groups, which are not my favorite, but they still give you some good information. They've talked to a lot of customers about that problem. They've seen the trends in the market about where it's going, what the competition is.
Cool. So you can edit that, use that, and then look at it and say, where are the gaps that I still need to learn? And then you pitch for learning those gaps. You don't pitch for redoing all the customer research, right? So now we're like, okay, I've learned this from product marketing. This has helped me.
These people have done this research. This is getting us to a better place. And now it's time for me to dig in a little bit deeper so that we can actually codify if there's an opportunity for us to solve here. That's reasonable ask, right? So you're not going in and saying, I want to do user research.
You're saying, I've read everything, I've digested it, as you should from any kind of resource that you have and through the company to learn about your customers. You shouldn't just dismiss it. You should learn from it. But I need to learn more. Really good place for you to go in and get buying.
Now, sometimes you have evaluative research, and a lot of times this is what we do as product managers. I built a prototype. I need to see if people will actually use it. I've got this pitch. I need to see if people are actually going to buy this. I have this thing that I think is going to work, right?
This is evaluative research. Now, product marketing probably isn't doing this type of research. They may be. You've got to find out. Or you've got to dig in and see what hypothesis they're looking at. But I highly doubt they're testing prototypes. And if they are, that's not their job, right? And you have to figure out why product marketing is encroaching on that job that you should be doing and you actually should be doing.
So in this case, right, I would look at how do I pitch it in that way, right? Again, don't go, oh, I'm just doing user research, I have to learn from them. It's like, we have to test this and make sure it's worth building. So what I think we need to do here is just like stop saying we need to do user research and start saying what we're going to learn from the user research.
And it also is not dismissing what we can learn from product marketing managers and client services about our customers because they've been talking to them.
What we have to do is figure out what are the hypotheses that they were testing or what were the problems that they were trying to learn about? And how are they different from the ones that I need to learn about? That's what we need to start doing. And if you can make that case, now you have a lot more buy-in, right, to go out and learn those things because it's going to help you and it's different.
So what I think people are looking at here is not just, you know, that team does user research, you don't do. I think what they get confused with is why, right? And if you can make a good case for user research, which there always is, I'm sure you have a great case for user research here, but you really hone in on what you're trying to learn, how it's different, and how the information that these other teams learned is not different.
Now I think you're going to get a lot more buy-in. People won't see it as duplicative, they'll understand that you're actually going out there with intent. So we need to get back to the intent in user research. No, one team doesn't just own all user research across the company.
Product marketing managers, client services, we all have to be product people, UX, right? All of us have to be doing user research, staying in touch with our customers. That's the whole point of building great products. So I would try that, really focus in on what you're trying to learn and how it's different from what they were learning.
All right, next question. Dear Melissa, my first product manager role has been in a company that hasn't had a product vision nor a product strategy. I'm switching over to another company where I think I'll partly be a product management advisor despite my limited experience, in addition to being a product manager, which is why I'm on the same quest you were many years ago to learn everything there is to learn about product management.
There's one thing I haven't quite been able to answer so far, and it's this. Assume a B2B SaaS company with a CPO and five product managers. Each product manager is responsible for a different value stream of one same system. From ETBT, Escaping the Bill Trap, also I love that everybody abbreviates it like that.
Never had it, I don't know why it didn't occur to me that I could do that, but it's a great abbreviation.
So from Escaping the Bill Trap, I've understood that the CPO is in charge of the vision and strategy of the actual suite, which can be on a two to 10 year horizon. But I frequently also hear you mention product managers are responsible for vision and strategy, and that's where I get confused. Is the intent really that each PM too should have a two to 10 year vision and a strategy of, for instance, the mountains type that Georgie Smallwood described on your show?
Or should each respective PM rather craft a vision and strategy only covering the current strategic intent? In Escaping the Bill Trap, the PMs mainly seem to do the latter, but I would really love if you could expand on this. All right, so here's the thing about visions. Two to 10 years is a very wide range for visions, and it does make sense, but it depends on what level you're at.
So there are many, many different levels of visions. There is a company vision, which should be five to 10 years out. This is how everything in the company comes together to deliver value and include services, all the things that actually make the company work.
Then there's the product suite vision. This is what the CPO is responsible for. How do all the products come together to deliver the value of the company vision? So this should be longer term too, but more like three to five years out. Now we go on to the individual product visions. This is what the product managers own, and that's how do these individual products or value streams roll up into that vision of the suite of product, and what does each one do to deliver value?
That's going to be on a shorter time horizon than the suite of products, and it really just depends on how far along and mature each product is. All of the time horizons for visions is relative. If you believe you can reach the vision of your company in five years, like sweet. Your strategic intents are probably going to be on a six-month to one-year time horizon, maybe one year to one-and-a-half.
Your product initiatives are going to be a little bit shorter than that, and you should be probably looking at sprints to deliver value or months, right? But if you're in a large company where it's going to be 10 to 20 years to achieve that, everything else is going to be longer too.
It depends on how fast can you actually get things out to customers. That is how you have to figure out your time horizons, and there is no hard and fast rule. The reason I talk in years and about company vision years and vision for products years and all that stuff is so that every level of the company is thinking more strategically depending on where they are.
So we usually see a lot of executives try to dive in and dictate what the features should be. That's a smaller time horizon than they should really be caring about, right? They should be setting a strategy that's a lot longer term. So that's where all those time horizon things come from. It's like at what level you are in the company and what you oversee, you really need to be thinking further out so that you make sure that the higher up you are, the more strategic you are so that you're not locking people into these little minute tasks that are going to take like two days and then nobody's actually worrying about the long-term vision of this company.
So now that I went through that rant, as we get more specific, the visions get more definitive.
They're not super like lofty and aspirational, they're a bit more concrete about what the product actually does and how it delivers value. So let's think of a product like Zendesk. You need a company vision for Zendesk as a whole. Then you have different suites of product for Zendesk. You've got the service suite, you've got the sales suite, you've got the marketplace and you've got the platform APIs.
Now those are all made up of different features and products underneath that, but each one of those suites needs its own vision for what they're going to do to actually deliver value and then also how they interact with each other. And that's where the CPO comes in, really helping to set those visions, making sure that it's more concrete, how they're going to expand, how they're going to all work together and how it's going to achieve the company vision. So they really got to oversee what is called the product portfolio vision and strategy.
They're across all the suites. Then each product within the suite has a vision for what that product will be and how it will manifest the vision of each one of those suites and deliver that value. So in the service suite, they have a messaging product and they have a help center product.
Each product manager would then be responsible for the vision around that. Now, I don't know Zendesk, I've never worked with them, but, you know, going off, doing a lot of this off what I see on their site, I don't know if they have a product manager over messaging and then they have more product managers underneath them like a director. But if so, messaging is going to be the vision, right?
Like the messaging product vision. And then the other product managers will be working on features that help contribute to that vision. But there's one vision is for the messaging product. That's how I would think about that. So there's lots of levels of strategy and vision that all play together. And yes, if you're an individual PM, rather than craft a vision and strategy covering the current strategic intent, you are going to be more focused on crafting a roadmap, right?
And a shorter term vision, kind of, it's always good to have a vision, like, where do I want to go? What does this look like at the end for the individual product that you're actually overseeing? But no, it's not going to be two to 10 years, that would be ridiculously long for a feature or a smaller product.
Two years may be more reasonable, but like 10 years to build a product, like that's what your company vision should be on. So we want to go shorter term, we want to deliver value faster, and we're going to be more concrete about the actual products that we're doing. That's what I would look at as a individual product manager.
So not really looking at the strategic intent so much, that's what the CPO is doing. That's what they're doing when they're crafting their strategy, you're going to be looking at the initiatives, you're going to be looking at the options, those parts of the strategy that I talked about in the book. All right, I hope that helps.
Last question. Dear Melissa, how do you know when you have too many features in your product? We measure usage in a relatively mature product and have plenty of features that only a small percentage of our customer base uses, but we never retire anything because the feeling is that it doesn't hurt to keep it in the app as long as someone is getting value from it.
At some point, however, all those little used features start to make the app overly complicated, creating long testing cycles, yada, yada, yada, but it creeps up on you and it seems like discipline from the beginning might be the best approach.
Any suggestions on how to change the mindset that more is better? Yes, I do. I have a lot of ideas on this. First idea, everybody forgets that it's not free to maintain software. So introduce the notion of cost into your organization, especially your product organization. Yeah, software is not free.
Like everybody thinks once it's built, like, okay, it's done. That was the major cost. And like, sure, that was a cost, but it's not the only cost. It actually costs a lot of money and time to actually maintain products, especially when nobody's using them. So this is what I always advise product leaders to do.
Do an audit of your features and your products from a cost and revenue standpoint. So step one, figure out what you spend in a year to maintain those features, all your features, break them all out. Hosting costs, it's not free to host anything. That's AWS right there. Bug fixes, improvements, enhancements, extra time spent testing, everything you have to do to maintain that product, add it up.
Go back and look at what you spent last year. You can pull it from JIRA. You see who worked on things that were related to that product. You can pull tickets, see how long it took for them to open it and close it. That's what I would do. Calculate out those hours matched to the salaries. Now you've got your cost, right?
Plus add those hosting costs and all those other things I said in there. All right, now we've got our cost of all of our features. And also don't just think in terms of tech, right? Also look at support. Like, are people asking questions about this product? Are they actually having to like hold people's hands on how to use it because it's broken and it's old, right?
Actually, go talk to people around the organization, service and sales and everything. Try to figure out how much time they're spending on it as well. Once you have that, now let's think about any improvements we'll need to make for the next year to maintain it or how much that will cost as well. So any improvements that need to come up, anything that might have to factor it in, are we going to add some costs?
Is it going to rise next year? That's what we want to know. You'd be surprised. Like people think once it's coded, it's free.
You're going to look at these numbers and you're going to like, whoa, that costs a lot more than I actually thought it cost to maintain those features. So now we've got a cost and now we compare it to the revenue that we get from those features. So I would look at the customers using it, talk to sales, talk to account managers, whoever you got, right?
And say, would we lose this customer if we were to take this away? Do they pay extra for it? Is this something that they really love? Can you go out and find that out and start doing the research on that? Again, you may be surprised that some customers don't really care about it. Like they go in there, but they're like, this is not a make or break thing for me.
And once you do all that research and try to figure that out, you can then decide whether or not it's worth sunsetting. I've sunsetted tons of features in my life that people were using. And do they get upset for like a hot minute? Yes. And then do they cool down and everything goes back to normal? Mostly.
Like if you take away something that, you know, really was amazing, you may lose a customer and you have to look at how much money that customer is worth.
And is it still worth keeping that customer over what you spend to maintain those features? Or is it the right customer for you? Sometimes we have a lot of customers using our stuff and it's not our target market. And we have, you know, sometimes there's regrettable churn where people leave, people cancel their subscriptions, they cancel our product, they stop using it.
And we're like, man, we really want to keep them as customers. And then sometimes there's good churn where we're like, wow, we really didn't want to support that. It's not the direction our company was going in. So you have to have that conversation. Did all the customers leave when we sunsetted those things?
No. Did we look at how much money we could potentially use? Yes. And then we made a decision off that. And a lot of times we were like, wow, it's just really not worth it. And in the end of the day, when we did sunset stuff, our product experience was way better, way more streamlined, like, yeah, it made a huge difference.
So I think this whole exercise is about like understanding what it costs, understanding what you're getting from it, having an intelligent conversation about, you know, if we do roll this down, what's the worst thing that could happen?
Looking at that and figuring out if you want to take that risk. And we need to make hard decisions if we want to keep our products usable, right? Like we have to get rid of some things. A lot of times we release things out there and they just don't work and it's okay. It's okay. We learn from it. We pull it back.
We make something better. So also really understand why those few people are using it. Do they just click on it by accident? Do they use it for a key workflow? Is it a workflow you want to support? Like really drive into what they're doing with that and can you solve it in a different way? Maybe you solve it somewhere else and they didn't even know.
That's what I would really look at. So remember, software is not free after you build it. That's the biggest lesson that I want you to take away from this. Really calculate out the cost. It is a long exercise. It's not, this is not like a two hour thing. This is also something that we do with product operations.
One of the teams that I usually have in my product operations team is actually managing the cost to support these different features and product and looking at the revenue that we get from them, segmenting it by customers, figuring out which one's target market, where can we grow, all that wonderful stuff.
So if you're interested in how that plays out with product operations too, Denise Tills and I are writing a book on product operations. It will be out next year, but she also teaches workshops on this as well. So go to productoperations.com and if you want to learn how to do that, we're very happy to teach you that too.
All right. So that's it for Dear Melissa for this week. Remember, answering all of your questions, whatever you want to chat about, please submit it to dearmelissa.com. If you want to do an episode around experimentation, I'm just going to seed that, trying to see if I can get some questions in about experimentation.
It's one of my favorite topics. It's one of the first things I started teaching. So got questions about MVP's experimentation, send them my way. And that's it for this week. Stay tuned next week for another special guest. I will be looking for your questions. And also if you love this podcast, we would really, really appreciate it if you left us a review.
It helps people find us. It makes us look more legit. So please dive in and help us with that. Thanks again. We'll see you next week.
