C. Todd Lombardo is the VP of Product and Experience at Openly. C. Todd and Melissa discuss product research–a key combination of user research, market research, and analytics–and how product managers can better implement research practices into their organizations. If you want to learn more about how to conduct successful research for your product, this is the episode to listen to.
Show Notes
This week’s guest is C. Todd Lombardo, the VP of Product and Experience at Openly. C. Todd and Melissa discuss product research–a key combination of user research, market research, and analytics–and how product managers can better implement research practices into their organizations.
Here are some key points you’ll hear Melissa and Todd talk about in this episode:
The kind of product leadership Todd does, and how he got invested in product management.
The biggest change in product management in the last decade is the legitimacy of position, and product leaders finally having important roles in organizations.
Product research is about gathering information and trying to synthesize it in a meaningful way.
Product managers need to have the ability to go out and speak with customers and do qualitative research.
The mindset of discovery, and how it can contribute to designers and engineers making better decisions which lead to better products and eventually, better outcomes.
Our job as product leaders is to figure out what our customers want.
One of the main rules of product research is to prepare to be wrong and go into [product research] with the intent to prove yourself wrong. “If you can't prove yourself wrong you might be onto something right.”
Always take into account the audience you’re presenting to. Remember that you’re talking to busy executives, so share your conclusions up front so that they don’t miss the main points.
Resources C Todd Lombardo | LinkedIn | Twitter Openly
Episode Transcript
Melissa Perri: Creating great products isn't just about product managers and their day-to-day interactions with developers. It's about how an organization supports products as a whole. The systems, the processes, and cultures in place that help companies deliver value to their customers. With the help of some boundary-pushing guests and inspiration from your most pressing product questions, we'll dive into this system from every angle and help you think like a great product leader. This is the Product Thinking Podcast. Here's your host, Melissa Perry.
Melissa Perri: Hey, C. Todd. Thanks so much for being on the podcast with us today.
C.
Todd Lombardo: Hey, thank you so much for having me, Melissa. It's great to be here.
Melissa Perri: So I've known you for quite a few years. I think we met at Mind the Product for the first time. And then as I came up to Boston a lot, you know, working at Harvard and with Athena Health, we ran into each other so much. So I'm excited to be moving up your way later this year.
C.
Todd Lombardo: Yeah, I look forward to having you back up in Massachusetts and hopefully this pandemic will be over and we'll be able to enjoy a beer or wine outside somewhere in Cambridge like we did last time.
Melissa Perri: That would be so nice. I can't wait. I'm just like, I'm ready to get out of this office one day. So I'm so excited for everybody to learn more about you, more about your books. Can you tell us a little bit about who you are, what kind of product leadership you do, and we'll get into talking about your new book, too.
C.
Todd Lombardo: Yeah, great. Thank you. So I recently started a new job as VP of Product and Experience at Openly, which is an insurance tech company, initially based in the Boston area, but now we're like most companies have gone remote. So we have employees all across the United States. And I've been in product for a while.
I'm no spring chicken in this regard. In one of my first PM jobs, I had this crazy platform that was like this half a million dollar machine with some hardware and software and chemistry and biology. And so the levels of complexity was crazy, but I actually really liked it, enjoyed that sort of trying to bring those things together.
And so a variety of different product jobs and even design jobs along the way have sort of led me to here, where I'm at now, leading a team of product managers and designers here at this place called Openly.
Melissa Perri: That's great. And then how'd you get into product?
C.
Todd Lombardo: Yeah, so that reference I mentioned before, the product manager, basically leading that product, she quit and went to become a whitewater rafting instructor in Maine. There was a gap and there was nobody leading that product at all for a while. And I guess I was one of the more knowledgeable engineers.
And the sales team kept coming to me. Maybe I was friendly enough to them. I was able to explain things to them to the point where one of the sales reps said, hey, you really should take Robin's old job. You're kind of doing half of it. So I went to the, at the time, all the product managers reported to the marketing manager.
I went to Rich, the marketing manager, and I said, so can I have Robin's job? I've been told I'm doing half of it. And he kind of just chuckled and said, yeah, I think you are. I think you are. And then he said, well, we'll have you interview with everything. But he's like, I think you're a shoo-in, but at the same rate, I can't just immediately hire you.
I'm going to have to put you through the interview process that we're putting everyone else through. So yeah, I did that. And eventually I got hired as a formally a product manager. So it was certainly quite crazy because not only was I managing or inheriting this really complex product to manage, I actually said to Rich, I said, this is my first product manager job.
I'm sure I've been like kind of doing it quasi on the side, helping the sales team, but that's only one small part of what product management is. I've never taken a marketing class. I've never taken a business class. Can you help me with this? And he's like, don't worry. I'll teach you everything you need to know.
I've been doing product management for the last 15 years in the BowTech space, blah, blah, blah, blah, blah. I'm like, awesome. Two weeks later, Rich gets asked to leave the company.
Melissa Perri: Oh my God.
C.
Todd Lombardo: So I basically had four weeks. I had two weeks left and basically had four weeks with this guy to teach me everything about product manager that I need to know. I'm kind of nuts because I spent the next probably six months to maybe a year without a direct boss. And I was at a company that was pretty large.
And the person above me was like senior vice president of sales and marketing. So I just barely ever had any time from him or guidance or anything. I think my product overall was like maybe a five or $6 million product, annual revenue. And then he was dealing with like a $350 million portfolio he had to manage.
So I was such a small part. He had very little time for me, but it was interesting to try just having to learn how do you communicate across all sorts of different departments and stakeholders and personalities and really try to be that glue, which a lot of what product managers are. So it was a very formative experience that a little bit of masochism, but I kind of enjoyed it.
When I look back, I learned a lot in those two years because it was really just such a fire pit of learning. I treaded water for a long, long time for a good year before finally actually starting to maybe do a couple of strokes and actually make headway. A lot of fun. And doing only quote unquote software does seem like it's easier just because the bar was so much higher.
The levels of complexity were so much higher in my past. Not to say that software is not complex, but it just feels a lot less complex because software is a man-made thing. When you add in like chemistry and biology into the mix, it's like, whoa, I got this stuff that's really even more unpredictable.
Holy crap. So it's been a lot of fun. I've really enjoyed product and learning how to do it. And also the evolution over the last 15 or so years I've been doing this, it's really seen a drastic change, especially thanks to people like you, right? Teaching it, having podcasts with it, writing books, all that stuff has been super cool to see over the last decade or so.
Melissa Perri: Yeah. What do you think is the biggest change you've seen in product management over the last decade, like you said?
C.
Todd Lombardo: One of the biggest things is the legitimacy of the position. Previously, I think a lot of people looked at product management as a glorified project manager, as just somebody who was supposed to just get stuff done for this product. And yes, we're all effectively shepherds of our product, but I think it didn't have maybe like UX and design finally has a seat at the table.
I think products in the last decade or so, it really has a bigger seat at the table. You're seeing more chief product officer roles. You're seeing more VPs of product reporting straight to the CEO or straight to the C-suite. There's much more of that direct line of products being a discipline that people need to hire and grow in their organizations.
I say that's the macro trend I've seen over the last 15 years.
Melissa Perri: Yeah, I've definitely seen that too. And I think a lot of the students who come through Harvard to the MBAs, I think it's like 30% of them want to be product managers eventually, which is great, which is huge. But nobody used to go get their MBA for that before. So I love that we're finally having a role for us and we don't have to justify our existence over and over again.
C.
Todd Lombardo: Yeah, because even some of my students, I teach at a different MBA program and similar things. Some of them are like when they take my class, I call it data-driven innovation, but it's basically a variation of product management without getting into the... We have a couple of modules around product management specifically, but it really is how do I train them to show what is it like to be a product manager? And they kind of get a taste of it without framing it as product management. And some of them are like, I want to do this. This is so cool.
Melissa Perri: Do you also get people who go, I don't want to do this because I've been finding that a lot lately?
C.
Todd Lombardo: Yeah, I've got a few students who are like, okay, this is definitely not for me. And it's good, right? You know that if it's not for you, that's great. And that's a good... One of the great things about any education is that you can learn, oh, I can test the waters to see if this is something I want to do or not. I've definitely had some students like, you know what? No, I'm going into sales or no, I'm going into marketing. I'm glad I did this because I don't want that position.
Melissa Perri: That's great. That's what I get too, which I love because, you know, nobody wants to go to a job and then spend a year hating it or things like that. Plus you're probably in there going, is it me? Is it the job? Am I not doing it right? You know, what's going on? So I love that, you know, we both get to teach students about product management and what they can do going forward. It's so nice that they're actually putting these into schools now.
C.
Todd Lombardo: Yeah, and there's another degree program at the school I teach at is IE University. I think that's called now. They sort of officially rebranded in Madrid, and they have this other program called the I think MCXI for short, but it's Masters of Customer Experience and Innovation. And a lot of it is a mix of UX and product management. If I had to oversimplify their entire program, it really is like a mix of those two disciplines.
Melissa Perri: Wow, that's really cool. That's a neat one, too. Awesome. So in addition to teaching the MBA program, you've also helped people by writing a ton of product management books.
C.
Todd Lombardo: I wouldn't call it a ton.
Melissa Perri: I call it a ton because after writing one, I was like, I never want to do this again. But you have kept at it. I think your first book was on design sprints. Your second book was all about product road mapping, which I recommend to everybody. I love it. And then the latest book is on product research rules.
C.
Todd Lombardo: Yes.
Melissa Perri: Can you tell us a little bit about product research? I know everybody has heard of user research, customer research, but what is product research? How is it different?
C.
Todd Lombardo: Yeah, it's sort of like a, it's not a different thing, right? It's this whole separate category. It's actually like a mix of everything, right? As a product manager, product leader, you've got to be understanding what your users are doing. So you have to understand some basic user research, but you also have to frame that in the context of a market.
And so you have to understand what market research is. How do I frame this? Is this a growing market, a whole new market? What is it? What is it about the characteristics of that market we need to take into consideration? And then combine that with some level of quantitation around, if you have a product right now, most product managers are actively working on a product that's in market that has users that you can measure something, some KPIs around them.
So understanding all those analytics and then how to put those three things together, right? It's really the mix of those three things that any product manager or product leader needs to understand to be able to do their job. I think you have some people who are UX researchers and ethnographers, and they're so deep in qualitative analysis, but then you put a spreadsheet in front of them or a database in front of them, they're like, I don't know what to do with anything.
And then vice the flip side, right? You've got somebody who is just an analytics whiz. They can write SQL till the cows come home. They can do all sorts of great things with Python notebooks you never thought were possible, but then you actually ask them to interview a customer. They're like, hi, do you like this?
And so it's like, I'm painting extremes here, but the product manager needs to be competent in being able to flip between those two things and be able to understand, okay, I've seen the analysis. I've seen what analysis you've done, what the quantitative data shows me. Now how can I combine that with the qualitative insights I have and make the right decision for both my user and my business at the same time?
Product research is the gathering of information and trying to synthesize that in a meaningful way.
Melissa Perri: Yeah, I love that framing for it too, because when I explain what product managers do to a lot of companies who don't get it, I say like your job is to basically synthesize all this data and then figure out what path we should take, right? You're looking at that and it sounds like that. That's what you're getting into with the product research. So you've got your market research, you've got your customer research, you've got your analytics, like you said, but that seems like a lot of work for the product manager. So is it the product manager doing all this by themselves? Are they working with a team? How do you get all that done?
C.
Todd Lombardo: Yeah, it's rarely ever the product manager doing it all by themselves. I think you and I both know product is a team sport, right? And a lot of it is you're doing it in collaboration with them. So whether you're a product leader, you might be coaching your team on how to do this or coaching your product managers, how to think about this, how to frame the problem, how to work with the UX team to go out and conduct the research, but then flip side, making sure you're going to the internal data science and analytics teams to say, hey, this is another question.
I've framed it one way for the UX researchers. I've got to frame it a different way for the data science or data analyst teams so that they can do their analysis. And maybe it's bringing those together and saying, okay, I see these things happening. I see a trend here or some insights the UX research got, but then does that match up with what we have from the analysts?
And are they seeing the same thing? And it might be vice versa. It depends on where, you know, start with the data you have. So it might be actually starting with your analysts first to see anything interesting come up, like what patterns have you seen, or are we seeing drop-offs in a particular funnel or a flow and then sort of matching those two things together.
And then maybe walking across and saying, hey, marketing team, you know, tell me what you've got, what are you seeing for trends in your market? So it's really that team sport of working with all three. So it's rarely ever the product manager is doing it all by themselves unless they're at a very, very small startup.
Melissa Perri: Okay, for our product leaders out there, I was just talking about this with our CPO accelerator class yesterday about org design, right? Like what are the different things that you need besides just product managers? So with this, you know, you mentioned the marketing team, which is standard over there, but you've got the product managers as well. What other roles do you need to be able to do this effectively, or what would you recommend for other product leaders who want to start to implement this stuff?
C.
Todd Lombardo: Yeah, I wouldn't necessarily think it's about roles, it's really about skills, right? Because you could have certain roles do a variety of different skills, depending on the size of your organization. So if you're dealing with like, you know, a thousand-person company, that's going to look very, very different than, say, an 80 or 100-person company.
But it's really you need skill sets of the ability to go out and speak with customers, do that level of qualitative research, right? So it's that slight ethnography, understanding what's the difference between, let's say, generative research, descriptive research, and evaluative research, right? Do you understand those three things, and what are the differences, and when would you want to employ one versus the other, depending on which question you have, right?
So that would be one thing from the UX research side, from the sort of analyst side, depending on the type of product you have, like are you a very data-driven product, like a, let's say like a Zillow, right? Like they've got so much data around homes, and that is not just in all of the different numbers around that might tell how many square foot is a home, what was it last sold for, what the tax is, you know, all this stuff, all this stuff around the neighborhoods. And then you've got this, like, probably this whole wealth of images about each home, right?
So some insights you could probably derive there, so there's a whole bunch of different data there, right? And that might be one type, do you need a data, what type of data analysis do you need, or data analyst skills do you need to mine that data and surface some insights? So that could be, so again, so UX research, some kind of quantitative data analysis, and then where's your marketing team?
Like how do they know what trends are happening in the market? How do they help you understand what is the top of the funnel look like for getting your biz dev team or your sales team leads if you are sales led or marketing led or product led? Like what kind of organization are you? I'm sure you've probably talked to, I think you did talk to Ben about, Ben Foster about product led growth, right?
Yeah, so are you more of a product led growth company or do you actually need a biz dev team in sales? So there's a lot of, you know, if thens, and it's always hard to give you a straight answer about that, but it's really about the different skillsets you need to be able to do these. Now you can have one person who can kind of do all of it, right?
And as a product person, you should be able to roll up your sleeves and do some of this, not necessarily be completely an expert in any of them, but you have to be competent enough to like, all right, I can do this, kind of like hack my way through it and get an answer with some relative level of, high level of confidence. Sure, there's probably somebody with a specialist who could do a much better job than you. In larger companies, you probably have that luxury and in smaller companies, you probably don't.
Melissa Perri: Yeah, yeah, I think that's true. I think like product managers just need to be able to do those things competently, but as you scale, it's maybe like, you shouldn't be doing all those things, right? If you have the ability to have other people start to get that data. When I was working with a lot of the scale-up companies, we started to add one or two like product ops analysts into the teams.
Once we hit like 10 to 15 product managers, and that helps a lot from the analytics perspective, not so much a user research perspective, but these people were really good at like pulling the financials together and getting the data out of amplitude to start to synthesize it. And start surfacing up insights that way. Have you ever worked with any of those types of roles?
C.
Todd Lombardo: The product ops role? No, I don't think I have, at least not in the last five or six years, mostly because I'm going to guess the teams that I work with have been a bit smaller. When I was at Fresh Told Soil, we were working with some pretty big companies, but after that, I've been mostly in smaller startups that have been anywhere from five to, I think, 100 people or more, like in terms of total employees. So my experience as of late has been much more in the smaller roles.
Melissa Perri: So getting the product operations, that's a luxury.
C.
Todd Lombardo: Yeah, definitely at that point. I would love to have and aspire to someday. And actually, my hope is that someday at Openly, we will have to pull that role in. But right now, I think we're 80, 90 people. So we're just not there yet.
Melissa Perri: Yeah, you're still working that way. And you were just telling me before, too, that Openly, you have to hire now maybe four product managers.
C.
Todd Lombardo: That's correct.
Melissa Perri: So if anybody's listening, they want a job in Boston. See, Todd is hiring.
C.
Todd Lombardo: Actually, it could be anywhere. Actually, we've gone remote. The focus is, obviously, we have a, I shouldn't say focus, the headquarters is in Boston. And there are a handful of people in the Boston area, partly because I think it's spun out of Liberty Mutual, or at least one of the co-founders was from Liberty Mutual.
And they went through the Boston Techstars a number of years ago. But we have people all across the United States building out the product practice here. They've done a really good job in scaling in such a short period of time. But they've been very delivery-focused, right? They've been very much focused on, like, we just got to do this stuff.
And so it's been very project-focused. And they've done a good job at sort of shipping, you know, doing output. But instead of now they're out of the reflection part, we're like, OK, we can't, it can't just be shipping more stuff. It's actually, we need to be a little bit smarter about shipping the right stuff.
And so then adding more layers of discovery, adding that product practice, I think, will be really beneficial for the organization as we scale.
Melissa Perri: Yeah, for sure. So when you're working with an organization who hasn't done this before, like you said, very delivery-oriented, where do you start when you came in as the VP of product? Like, how are you shifting them into this mindset of discovery?
C.
Todd Lombardo: Part of it is just trying to, like, identify where is discovery needed? And of course, you know, the first question is, like, everywhere. But can you identify a place where, like, oh, I can get a quick win around this if I can show them, here's where we can do a little bit of discovery and actually get a better outcome, right?
I can help the designers and engineers maybe make some better choices or a better product that ultimately leads to a better outcome and also framing it away from just make this thing to, well, if we make this thing, the outcome we should see is this, right? And starting to make sure that you tie those and connect those dots. Because I think sometimes, especially in startups, when you're really just trying to get stuff out the door, sometimes the forest to the trees ends up getting missed.
Like, people just aren't paying attention to, like, wait, why are we doing this? Oh, yeah, we're doing this because, like, oh, you know, my boss said so, or we just got to get this thing out. And nobody's necessarily questioning, well, why do we have to do that right now? Or how does that connect to a KPI or a particular goal we have?
So finding that one small place to show them how discovery can really help them get a better outcome. I did that at my company I was just at at Machine Metrics. Similar thing. I was their first product hire VPR product coming in to build a product practice and organization. Similar thing. It kind of had to come in and say, all right, well, where are we struggling?
How can I help figure out where to start? I think it took a little bit of time to figure out exactly where to help them out. And they had a designer there kind of, like, cranking out all sorts of designs, but they weren't getting built. They didn't know what to build. And I was like, okay. I said to the designer, I said, Jared, have you been out and talked to customers yet?
And he's like, no. And he had been there for like two months. I was like, you haven't talked to any customers yet and you've been there for two months? I was like, pack your stuff up tomorrow. We're going to go to factory. The machine metrics makes software for factories. I said, we're going to go to factory and show them what you got and just see some feedback.
And that did two things to me. One, it helped gauge how good was the designer at putting together a prototype for a customer to see how he would interact with the customer, but also how the whole organization reacts to seeing like, wait, here's something that's not even built yet or in the product. We're actually going to go show it to a customer.
So I just seemed like their level of like, but it was one of those things like, okay, yeah. They kind of trusted the process and said, yeah. And they bringing that process of like, oh, getting customer feedback is really valuable because we can make sure we build the right things for them. And I think they had suffered from listening too much, too closely to their customers prior.
Like they were, like customers said, build me this feature. They would go off and build that feature and they wouldn't necessarily question exactly why or think it through, do that discovery process. And that led them to building a bunch of things that actually the one story there is they went off and spent two or three months building this feature or feature set, shipped it.
And then the customer didn't like it because it really wasn't what they needed. It was an interpretation of what they asked for and the customer churned.
Melissa Perri: Wow.
C.
Todd Lombardo: So it was like showing them like, hey, there's another way to do this. And it's not like that. And yes, we listen to customers, but we listen to that with a filter and we don't take what they say verbatim. We have to peel back those layers of an onion to what a customer tells you. And let's make sure we peel that back to figure out what the right layer is. And then we can think about how to build something that will solve it.
Melissa Perri: Yeah, that makes so much sense. And the thing that I like about this is, in your story, you describe taking Jared out there and starting to talk to these guys. I have met so many new product leaders, especially new ones, right? Who come into these types of organizations and they go, well, I can't do customer research, right?
Like I can't do this because our organization doesn't do it. I think the message here that I'm hearing from you and what I try to tell people too is like, you just got to go do it, right? Like that's your job as a product leader is to go figure out what the customer wants. So how do you like navigate that with an organization?
Did you actually ask for permission or did you just go out there and do it?
C.
Todd Lombardo: I kind of just went out there and did it. I just, I got connected to a handful of customers and said, yeah, I'm going to go. But the good thing is the CEO is pretty, you know, he wasn't against it by any means. And he was talking to customers himself to his credit. He actually, the first day, this was a few years ago now, but he said, oh, you know what?
We're going to go visit a factory. So we actually went and visited. So Machine Match was headquartered in Northampton, Massachusetts. And one of their early customers was in Gardner Mass, just maybe about a 20 minute drive up the road. So he was like, yeah, we're going to go visit them. And so it was great.
On my first day, he had me out in a factory talking to customers, right? And it's been hard the last probably year for most product people because you're kind of stuck in the Zoom land, right? That you have to interview your customers over Zoom. You can't actually go to their spaces because to me, for me as a product person, the ability to like see what are the walls, what does it smell like in their organization?
What color are the walls painted? You know, are there chips coming down from the ceiling? Who interrupts somebody? Like all these things you can't see on a Zoom screen when you actually go to where the customers are, meet them where they're at, and just, you get more than just the conversation. You really can get just, it's such a, from my perspective, it's like a sponge.
I'm like, I've taken so much. And then I can say, all right, now I have a really good sense of what this customer is up against. I'll just tell you another story. Like at Constant Contact, we went out to this, she had this, she basically ran this little boutique shop just outside of Boston in Newton, Massachusetts.
And we were interviewing her for like, basically loyalty programs. And when Constant Contact was looking at how they could expand their email marketing and add like more of a loyalty program offering, because a lot of the people who are on email lists end up becoming members of a loyalty program of sorts, especially for retailers. So we were doing some interviewing with her and we just said, oh, can you show me like where your office is and how you do things?
And she did like her whole demeanor just dropped. And she was like, well, she got really embarrassed and she didn't want to take any of us there. And then finally, like after enough coaxing, she just said, okay, fine. So she brings us down to this tiny little corner of this damp basement. There's papers everywhere and there's this laptop that I swear had Windows 98 on it.
And that's where she wrote her marketing copy and her marketing emails. And like, it was, I was like, oh my God. Like there was puddles on the floor. Like it was clearly like this really dungy little place. I could see why she was embarrassed, but that gave us is like, okay, wait a minute. You know, she's on a really old laptop with a small screen.
Like we can't be putting all these super whiz bang features with like expecting that our customers are going to have these, you know, 27 inch monitors or double monitors when they have this tiny little old laptop that they're just interfacing with the web to like write some marketing copy, write an email. So that was a really interesting insight that for us, it was one of those like, yeah, the environment matters, right? And that's where it goes beyond just the, it would be really hard to tell that from just over a Zoom call.
So sometimes getting out there is really important.
Melissa Perri: Yeah. So what would you suggest though? You know, a lot of places are still remote. I don't know what it's going to look like going forward. If you can't go out and visit people, what can you do to get some of that off of Zoom? Yeah. And what we actually, we got away from Zoom.
C.
Todd Lombardo: Although you still probably can use Zoom in conjunction. What we started doing at Machine Metrics and the reason why I'll refer to them because I've only got about a month at openly so far, but we actually started to use a lot of FaceTime and other ways. So we actually fire up Zoom for one thing, but we actually started, we asked the customers because it's factory floor, having them take out their phones, fire up FaceTime, and then show us where they were actually like looking at either inside the machines or around the factory floor.
So asking them to use their mobile phones more or their iPads or tablets to start to give us better sense of just the environment, right? And then that sort of, that helped a lot. And using like, all right, we have a Zoom call that maybe helps set up and record, but then we have them maybe either set up a FaceTime or set up something on a Zoom on their phone.
So like you'd have that one person might be calling in with like three other phones on the line, and they're all like maybe having a camera from their phone pointed at something else on the factory floor. So that was sort of how we started to get over the Zoom only thing. It wasn't ideal, but it certainly helped us try to bring the factory back to us.
Melissa Perri: Love that. I've never heard of anybody doing that. And I like the fact that your mobile phone is mobile so you can walk around with it. You can see what's going on there. You can see where everybody's standing and sitting. That's really cool.
C.
Todd Lombardo: Yeah, it definitely helped a lot. I think it helped a lot for a couple of specific projects that we actually needed to see, like, what was inside the machine? How is the machine operating? Would we actually be able to do something to the machine if we did something with our software? Like, and seeing all those different things happen in sort of a quasi prototype test was really helpful.
Melissa Perri: Oh, that's fantastic. Really cool. All right, I love that tidbit. So in the book, too, you write about a couple rules for product research. So what are the rules we should be following for product research?
C.
Todd Lombardo: All of them.
Melissa Perri: Which ones can we lose? Seriously, there's too many now.
C.
Todd Lombardo: No, I think they're like children. You love them all. But I think the first one is probably one of the most important ones for all of us. And I remind myself every day, right? It's prepare to be wrong because you are likely to be wrong. And that's probably one of the most important things. Come into it with that beginner's mindset, right?
I could be wrong about something. In fact, maybe go into it with an intent to prove yourself wrong. And if you can't prove yourself wrong, then you might be onto something right. That's probably one of the big things about why I became Shepard One, I guess. Because when you have that mindset, a lot of things can really flow from that.
I'd probably say the other big one is making sure you have a clear research question. This is a lot about product vision. I think that you talked a little bit about that with Ben, too. A product vision. If you have that clear product vision, you have that clear research question, right? It's like a product vision.
It gives you a direction. It gives you a key element that you need to answer that question somehow. You may have a million other questions to answer in service of that large question. But if you have that research question, it is like a guiding light for you to understand. So yeah, we need to understand how customers think about X or how customers react to Y.
That kind of research question will help guide you as to where you go next, the activities you do, and et cetera. Then I think we have another one around the team that analyzes together. It's don't go off and do research all by yourself. Product is a team sport, and so should research be. Bring people along, even if you can't bring them for every single moment of every single time, bring them along in the steps of the way.
One example that I would use with DesignSprint would be invite the executives to the first part on Monday morning with the DesignSprint. Invite them to that key goal setting and assumption storm and get the assumptions out of the executive's heads. Then check in with them again on Monday afternoon once you've gotten some sketches together.
Then when you're synthesizing your research from the interviews at the end of the DesignSprint, bring some of those executives back in and actually show them how you actually do the final analysis with them to say, hey, these were our assumptions, this is what we did, and this is what we learned from these interviews. Maybe you can even play them some clips of some of those interviews. That brings them along, even though they have to spend all 40 hours of their week there, they spent maybe two hours in total and you've brought them along for the ride.
That's another thing to think about is you don't want a team to go off and come back and say, move the search bar to the lower left and you're good to go.
Melissa Perri: That makes a lot of sense. Your last rule I really liked as well, which was all about the communication. How do you communicate this back to executives, tell people what you're getting at? That's one of the biggest things I see product managers struggle with. That's what they ask too. They're always like, how do I convince my executives of this?
How do I make them move in this direction? I think that's the key to actually having your research be taken seriously is being able to communicate it. What have you seen worked well going back to explain it to executives, get buy-in for the direction? What are some practices people can take away to do that?
C.
Todd Lombardo: I think if you write a 250-page report, it'll totally work. No, do not do that, because I did something similar. In the book, I talk about a story that constant contact rewrote a 55-page report, and my colleague Laura and I worked our tails off on it and think maybe three people in the organization read it.
It was just like, why do we spend so much time doing this? We shouldn't have done it. Insights are shared. How do you share the insights? How do you frame the why around what insights you got from your research? I think that's probably a couple of key things. Another one is making sure that you're also understanding your audience who you're presenting to.
If they are busy executives, put the conclusions up front. Get to the chase. Don't lay out your evidence and then finally get to the conclusion at the end, because what if they came to a very different conclusion than you did? That could be a bit challenging. Uh-oh, I'm presenting this conclusion, but everyone else thinks it's a different conclusion.
Wait, am I wrong? That doesn't look good. Cut to the chase and say, we're going to do this. We are going to focus on trying to add new features around calendaring in our app for whatever reason, and here's why we came to that conclusion. A lot of it is framing why. Why do we get these insights? Why does it help?
Why should we do this? How does it help our business move forward? How does it help our product, help our customers? Really tying all those things together.
Melissa Perri: Yeah. And when you're going to present this to, I hear conflicting opinions on whether or not you should do this, but do you ever send pre-reads to the executives with the presentation so that if they tune out, they're not losing it all? Is it more like, no, just do the presentation in person?
C.
Todd Lombardo: Yeah, I've heard both sides of that as well. But I think if you're going to send them a readable document prior to that, then if that's the case, then don't worry about presenting, just show up and be like, all right, I'm going to assume you've read it, and what questions do you have for me, right? If that's the kind of thing.
If I'm going to go through the effort of putting it, because the thing about a presentation is you need to be there to explain it, to tell that story and help frame the why. With a pre-read, you've got to format it in a way that somebody can read it without you there, or you've got to find a way that's going to bridge the two. All right, I'm going to read this, and then I'm going to have to present something.
They're going to come in with a certain preconceived notion, or they're going to come in primed with something. I can see it working in some situations. I can also see it completely blowing up in your face, and people are like, well, I've already read this, I don't need to pay attention to you type of thing.
What I don't suggest doing is repeating yourself, right? How can you do it in a way that if you've got a format that's going to be a readable document ahead of time, make sure it is a readable document for ahead of time, not let's say just a copy of your PowerPoint presentation, and then format and structure and facilitate the meeting in a way that's going to ultimately get your outcomes, right? Are you trying to get buy-in?
Are you trying to get a product direction or feature direction to go, what is your objective beyond just presenting your research? What do you want to do with it? And I think having a strong perspective point of view coming into that meeting will be helpful because that will say, I want to get this out of this meeting, and here's how we're going to do it.
Yeah, being very intentional
Melissa Perri: With what you're trying to get after. I like that. So we also talked about, right at the beginning of this when we were diving into the rules, not being wrong, right? Like being able to be wrong. Do you have any stories or experience working with an executive, let's say, the CEO who does not want to be wrong? And how do you navigate trying to show them with the research that you should go the other way?
C.
Todd Lombardo: Yeah, I very much have a story about this. Machine Metrics, the CEO, a great guy. I love him. He's awesome, and he's going to do great things for the company. But he's a very opinionated guy, like anything. Strong opinions, strongly held sometimes. But there was this one aspect of Machine Metrics product that he was convinced that, from a reporting perspective, that customers just needed to hook into our APIs and they can go push the data they have in our product out to BI tools like Clipfolio or Microsoft BI, Tableau, you name it.
And he's like, no, we just need better APIs. We don't need to do anything in our product. We don't need to build any reporting stuff. Say, I want to build another Clipfolio. I'm like, okay, great. But one of the pieces of feedback we kept getting was, hey, I need a better export. I need a better export.
And so we thought, all right, why are they exporting this data into a CSV file? Well, it turns out, these customers didn't have the skill set or capability to, one, connect to the API. They didn't have that skill set in-house. A lot of mid-tier factories don't necessarily have a set of developers that can just do this for them.
So a lot of what they were doing was downloading into the Excel, CSV into Excel, and then doing some manipulation, creating a report, and then sharing that report with their colleagues. And they weren't creating anything fancy. These were simple line charts and bar charts. And so we said, okay, we're going to actually go off and build a small reporting function that allowed our customers to basically query the data in every factory needs, has some different KPI and different measurements.
So you couldn't create an out-of-the-box report that would work for every single factory. So it's like, let's not try to solve for every factory's case. Let's just create a system that allows the customer to tailor what they need. We'll give them a couple of examples, and they can have at it. That turned out to be really, really powerful.
And it went against, but the CEO did not want to do this at all. So I had to distill the information that we found from the research around the CSV. So I had to show them, look, this is what they're asking for. And we also did a product market fit survey. And one of the top things was the reporting wasn't good.
So that whole question of the people who say, well, I'd be kind of bummed out if your product went away. The number one thing they kept asking for was, yeah, we just better reporting. I don't get the data I need. So it was a combination of that survey, a bit of qualitative research, and even just going off and doing a little bit of building something so that we could show them like, hey, this is what happened.
And then we actually built it in very incremental phase. The first phase was just query it and get a table back. Second phase was like, oh, query it, and then be able to save it. Because one thing was they could query it, but they couldn't actually save the report and come back to it again later. I was like, oh, but it validated that was the right thing to do.
And then our chief customer officer at the time, he basically said, he's like, I can't tell you how much this has helped our onboarding because I just basically tell the customer show up with your Excel file that you typically report on, and then I'll build it in our product. And he's like, and it totally wows them. So it was a good example of how it really did go against what the CEO did think about, but we were able to kind of push it through because it really did solve a true customer need.
Melissa Perri: Yeah, and it sounds like a very classic example of the users trying to solve their own problems, but not knowing how to solve it. So they just ask for whatever solution makes sense to them, but not really. And that's not their job. Like we talk about in product, it's your job to figure out what it is. So yeah, that's a fantastic example of product research at work there.
C.
Todd Lombardo: Yes, it is. And a little bit of negotiation. I think actually that's one of the things maybe if you're someday in the future when I actually can write another, feel like running another book and you want to write about negotiation for product managers, I think that's like a, exactly, see, exactly. Every time I'd say that to some another product manager, they're like, oh my God, yes, we need that book. I'm like, all right, well, who's going to write it? You, me, somebody else?
Melissa Perri: Yeah. Like, you know, the influencing, getting into like what we were talking about with the presenting, understanding where the CEO is coming from, trying to show them with the different data. It's so important for product managers. And it's something that I feel like we overlook so much, but you could do like all the research you want, but if you can't present it in the right way, or you can't empathize with people and try to bring them along, it'll never get implemented. Yeah.
C.
Todd Lombardo: Yeah. I always advise any aspiring PM, they, you know, like ask me what books should I read? Aside from recommending yours, but said you recommend a lot, by the way.
Melissa Perri: I recommend yours too.
C.
Todd Lombardo: Well, thanks. Mutual relationship here. But I also recommend Never Split the Difference by Chris Voss.
Melissa Perri: Oh, I love that book.
C.
Todd Lombardo: Because it is a great, one is it's just a great read. I mean, it's just, the stories are awesome. You know, it really talks about the techniques and approaching to think about for negotiating a really hard situations. And oftentimes in product, like you've got to really negotiate hard and not necessarily like being forceful.
And when I say hard, I don't mean like being a jerk or being mean, but you have to like really, you have to win the argument when it seems like the odds are not at your cost in your favor, just like you do with a terrorist, like terrorist has 12 hostages and they're demanding 10 million bucks. You can't just say, well, I only got five million, give me six, we'll call it a day. But that doesn't work.
And sometimes that's the same thing in product. Like you need a winner take all kind of approach. Sometimes not all the time, but sometimes you need that. And Chris's techniques are really, are really useful.
Melissa Perri: Yeah. I think that book completely blew my mind when I was reading it. I was like, this is so powerful for people in our craft. So I love that you like it too. Yeah.
C.
Todd Lombardo: That's great.
Melissa Perri: All right. So your books, product research role is the newest book. Where can everybody find more about your books? Where can they go buy them? Amazon, just search for CTOD on Amazon.
C.
Todd Lombardo: You'll find them like most things, but also I think you can get, you probably find them in O'Reilly too, but I think they just link you over to Amazon. But if you want to know more about me, CTOD.com, C-T-O-D-D.com is pretty straightforward. I usually have jumping points to like either articles I've written or talks I've given, podcasts I've been on, et cetera. And you can always follow me on Twitter where I am CTOD is probably the place I'm the most active most frequently.
Melissa Perri: Fantastic. Well, thanks so much for being on the podcast today, CTOD.
C.
Todd Lombardo: Thank you so much for having me. It's been so much fun.
