Kathy Pham is a computer scientist, a product management executive, and co-leads the Ethics and Responsible Tech at Mozilla. She also co-founded the Mozilla Fix the Internet Incubator, as well as the Product and Society at Mozilla, which focuses on product management, ethics, and the public interest. Kathy joins Melissa to discuss product ethics.
Show Notes
Melissa Perri’s guest on this week’s Product Thinking Podcast is Kathy Pham. Kathy is a computer scientist, a product management executive, and co-leads the Ethics and Responsible Tech at Mozilla. She also co-founded the Mozilla Fix the Internet Incubator, as well as the Product and Society at Mozilla, which focuses on product management, ethics, and the public interest. Kathy joins Melissa to discuss product ethics.
Here are some key points you’ll hear Melissa and Kathy talk about in this episode:
Interesting details about the product management class Kathy teaches at Harvard.
Kathy’s experiences working at The White House, Harvard, and in product management.
Ethics and its implications for product management.
Accessibility and security are important and need to be integrated into a product from the very beginning.
When designing products for an inclusive audience, you have to take into consideration the team culture that is in place and build out those personas to be as inclusive as possible. “I think it's important to build into the culture knowing that the moment something is built it's really hard to reverse,” Kathy tells Melissa.
How to respond to changes or challenges with your product platform when it happens in another country where no member of your team is from.
As product managers, we need to be listening to our users, and that includes listening to those users who are telling us what we don’t want to hear. It means not condemning or shutting down their feedback by telling them they’re wrong.
Having diverse perspectives within our product teams is very important for the decision-making process. It ensures that the concerns of the target audience are heard and are taken into account.
When the people building our program algorithms make terrible assumptions or have blind spots, they bake into them issues that already exist in the world and just propagate them through code.
Resources Kathy Pham | LinkedIn | Twitter
Episode Transcript
Melissa Perri: Creating great products isn't just about product managers and their day-to-day interactions with developers. It's about how an organization supports products as a whole. The systems, the processes, and cultures in place that help companies deliver value to their customers. With the help of some boundary-pushing guests and inspiration from your most pressing product questions, we'll dive into this system from every angle and help you think like a great product leader. This is the Product Thinking Podcast. Here's your host, Melissa Perry.
Melissa Perri: Hello, everyone. Today we're here with Kathy Pham, and today we're going to be talking about product ethics. So welcome, Kathy.
Kathy Pham: Hello.
Melissa Perri: It's exciting to have you here. I came across Kathy on Twitter, the best of places, but I also found out she was teaching at Harvard just like I was. So what do you teach at Harvard?
Kathy Pham: I teach a class called Product Management and Society, and unlike your class, which is a much deeper dive and they build very tangible products, this class is meant to be more of an overview. So I'm actually super grateful that your class exists because I'm like, oh, we can go get into that class. That would be amazing.
You can go much deeper. And my class is an overview of product principles, everything from how do you discover your sprints to understanding user experience research to a small spigot of what it's like to work with engineers to how to think about metrics, the lens of society and responsibility and inclusive design and equity by design kind of weaved into every single one of those components. So if you're doing a discovery sprint, for example, what communities might you be leaving in or excluding?
If you're thinking about a content strategy, how do you enable the words you have on the page are thoughtful enough that they can reach many different kinds of audiences? And so it's part a skills class, but it's also part how do you now weave in all of these concepts of building a better society into these really core product skills.
Melissa Perri: I love that. I think it's so important as well. And when I found out you were teaching it, it's in the Kennedy School, right? And it's part of, okay, so you are basically educating our future political leaders and people who want to get into government.
Kathy Pham: Yeah, it's in the public policy school. So I get about half students that are the public policy students. And then there are folks from the business school and the engineering program and the education school and others, as you know, like they find their way over. And they actually provide such a rich environment, because I think especially in product, having so many different perspectives is always so critical.
So someone can share something in the class with their engineering background or their policy background. And someone else weighs in with their architecture and design background. And they can very quickly dispel either myths that different groups have about each person's discipline or something. But at the core, it's a lot of public policy students who end up going into the tech sector, doing something in the policy space, in the nonprofit, or back into government.
And another big part of that is getting product managers into government and getting chief product officers and product people all throughout governments everywhere. That's another dream and goal of mine.
Melissa Perri: I saw that. I saw California is hiring a chief product officer for the first time.
Kathy Pham: Yeah, it's amazing. Health and Human Services had a chief product officer. The fun fact though is that even when hiring PMs into government, because of the way, at least in the US at the federal level, the way job titles are, sometimes we have to call them something different, like a product specialist or something else.
And then as we send it out, we don't have a byline that is like, but FYI, for those of you in tech, this really means a product manager. So we're getting there. But imagine the world in which every single government service had product people who deeply focused on how people use those services. And you go to the DMV or you sign up for social security or you try to separate from the military to go to Veterans Affairs or you sign up for small business loans.
And there's someone there who goes out through the entire product and user journey and understands what your needs are on a tech level. So that's another lens of all of this.
Melissa Perri: That sounds like a dream. That sounds like it would cut down on all the paperwork and all the horrible experiences we have dealing with government services on a daily basis.
Kathy Pham: Yeah, and one more thing I'll add too, is sometimes we'll think about private sector and government as separate. And in some ways they are. Once you start kind of digging in, there's a lot of blending. Also in many ways, the private sector has to think about how to build responsible products. And we'll talk about that, how we build services that millions and billions of people use that are more responsible, more ethical, don't cause harm.
And with government, you have people who have to use your product. And when you don't build them well, you also have the same side effects where people can't sign up for services. People have to wait years, decades to be able to get certain kinds of services, to pass immigration checks, to get healthcare they're entitled to, to any number of things.
So they're different, but in many ways it comes back to the core. When you don't build something well, and well-meaning, we can talk about it a bit later, but what well-means is different in different contexts. When we don't build something that serves people, there are so many different ways in both government and private sector and everything in between where we can cause a lot of unintended harm.
Melissa Perri: Yeah, and it sounds like that is also something that we as product managers need to be aware of in our daily jobs, right? Not just something that we look at when we're working in government or anywhere else, as you said, in the private sector. It's something that we have to look at as well. So how did you get roped into this? So how did you end up on the path that you're in? You've worked with the White House. You are teaching at Harvard. How did you get into product management and how did you get talking about ethics?
Kathy Pham: Yeah, I think it was a long time incubated. So I had a pretty, I'll say, quote, traditional. There isn't really a traditional PM path, but there is one path that is, go study computer science, maybe do a little bit of software engineering, go find a place that will take you on as a PM, a product manager, either like an associate product manager or like a more experienced product manager, and then go build things.
And I had that, right? I started out studying CS. I was software engineer and built flight simulation software for a while that was super interested in healthcare and was one of the first people on the Google Health team, learned a ton about building healthcare technology, which would later come into play, and also learned a ton about what it's like when a tech company that perhaps doesn't have the deep expertise in a really complicated field like healthcare, despite having a ton of tech talent, it's really, really hard.
And that was kind of early on. So I did, I was like Kathy, PM world engineering product. And then went and did some consulting and did product and data science and engineering work across hospital systems and then came back to Google and worked on Google search as well as Google people operations. I highly recommend, if you ever get a chance, even if it's a short tour, to go work in the HR department of any tech company, because you learn so much about how people are hired, how people information is stored and how to do it responsibly or not responsibly.
And so I saved a long time coming because I feel like there were nuggets of caring about healthcare. There were nuggets of, even when I was at Google or IBM or these other places, I would take on these side projects to work with immigrant communities to help them sign up for social services. Or I took on a 20% time where I worked with K through 12 students and did these side projects.
And then one day, there were a lot of things that happened, but the healthcare.gov website failing was one thing that a lot of people at Google took notice. There were a lot of people who were thinking about bringing technology into government for a long time now, recognizing that when technology services fail in government, millions of people are affected in really sometimes really terrible ways. But this particular event caught the attention of tech people.
That's when I was called to come help build out the US Digital Service at the White House. And people are like, well, why'd you leave Google? I'm like, well, wouldn't you leave Google to go and build out a new tech organization inside government to completely transform how our government does all of our tech services at the federal level.
And I think that's where everything kind of came together. It's like, oh, I don't really need that side project on health care or that side project with social service. Here's a role where I can come in with all of the training I happen to have at that point on engineering and product and bring it into health care and the Veterans Affairs, bring it into helping figure out how to hire team members who can work on, how to help students understand colleges that are best suited for them and so many other things that our government works on.
And I think that also brought to surface something that I think not that many people talk about is how deeply the government cares about people. So how deeply career public servants care about people who are housing insecure, veterans who need access to health care, people who are trying to start small businesses in a way that blew away any user experience research team I've ever seen in the private sector. And that's where I think there's just this moment of we have to build better tech in government and we have to think about how to help the private sector understand all the groups that we're building for.
And that was also the same time. So a few years later, it was around the same time that Cambridge Analytica happened. For those of you who are unfamiliar, it's just one of many examples where a big tech company has opened an API which allows third party entities to access its information and code and data.
And if you don't look closely at what they do with it, they kind of have free range for what they do with it. In this case, they were irresponsible with user information. And it was just one example of many out there. So it was this moment for me of, I have to think about all of this, of any role that I've played in the tech sector to build, to be what it is.
I have to think about what it means to build more responsible tech in the public sector. So that's when I think the pivotal moment was when I had left the first, I had left the US Digital Service and came to the Berkman Klein Center at Harvard. And built out the ethical tech working group, which by far, is the group that is like the most dearest group to my heart to date, along with Mary Gray, who wrote the book Ghost Work.
I highly recommend reading that. It's about all the people that really make tech work. So not just the gig workers, but everyone behind the scenes and the content, moderators, the good workers, to everyone who are not the tech engineers that make tech work. But it's an interdisciplinary group of engineers, product people, historians, philosophers, race and gender scholars, who are just thinking about tech.
And so I think that's really the pivotal point of everything that was before in my career led to this moment of starting this group. And then this whole movement around responsible tech, ethical tech, around the Responsible Computer Science Challenge at Mozilla. And yeah, I think it's a long time in the making.
I just didn't know where it would take me. And some of this didn't exist years ago, so.
Melissa Perri: Yeah, it's a really specialized area, but I think it has so many wide reaching implications. So I'm really excited that you get a chance to go full throttle and think about these things all the time and then help others learn about it too. Because I think I got really passionate about ethics and tech.
I feel like I've always been a little bit passionate about it, but especially recently, like when I read the article about the kid committing suicide because of Robin Hood, I sat there and I was like, wow, a product manager made that decision or a UX designer made that decision about what to show on your balances, right? And that's like an extreme version of worst case scenario, right? Like extreme, extreme worst case scenario.
But it comes down to what we do as product managers. It comes down to what we do as user experience designers. And you said with the government, it's not, that's just private sector. With the government, it's a lot of things that are a little closer to home with people too. It's about like, can you get a driver's license?
Can you go do all of these things? But I think sometimes we don't think about that so much in product management. So when you're trying to teach people about product management and how to consider ethics in there and how to think about the implications of it, like what advice do you give them? How do you start by orienting them towards, this is what you should be considering, in addition to how to make a beautiful UI, right?
What else goes into that?
Kathy Pham: Yeah, so one, I hope it slowly becomes less of a specialized area. I hear that college essays now are so often centered around tech and societal impacts in some way, which is really interesting, right? And so I wonder what it would look like if common conversations with coworkers and peers are not just about how to get the next X users, but is also like weaved in a lot of these concepts.
And so to answer your question, and this also goes back to how we hire product managers, right? If it's a group of like a bunch of engineers that are hoping to train to be product managers, that's one kind of narrative. If it's people with certain like communities or history or different kinds of backgrounds who can also be great PMs, it's also like a different starting point.
So I think some ways to start is I'll talk about core concepts of product and what I mean by that is discovery and design and metrics and content and stakeholders and testing and these things. If you go to any product conference, product road maps, you'll inevitably hear someone talk about it and then start questioning, well, let's do like a stakeholder mapping, for example. Who are all the people that you would consider a stakeholder?
And usually folks will start out with, you know, anyone that's like some executive or your manager, people that may be interested in your product, but then you start digging deeper. So for example, let's say you're working for a company that's selling policing software to a government. So that's both a private sector and a public sector thing, right?
It's a private sector company building something for the public sector. So who are your stakeholders? And they'll start out with, you know, like different police officers in the Department of Justice and then you start digging. They're like, let's keep talking through that. You start digging more and you're like, well, what about the people impacted by the actual software, the people being policed?
And then not just the people being policed, but then all like the other people in that ecosystem that are around the people being policed. And then if a region gets overly policed, what does that do to the businesses in the area? And like you started digging much deeper to get folks to think through like a much deeper layer of that particular topic, for example, stakeholders.
I think that's one way to get started in these conversations because part of the ethical discussion is who are we leaving out? Who are we prioritizing when we're building? We talk all the time in product about who's our 10X user. Well, if we only design for our 10X user, there are a ton of people we're gonna leave out.
And even if you're purely solely self-interested, there are also a ton of people you're gonna miss. So that's like one example. Another one is to start out with something that might be familiar. So security and accessibility in some ways went through similar versions of this where security is clearly very important.
But when you go to an engineering school, sometimes security is seen as like a second part, right? And then that will happen, product startup teams, and then also in companies as well. You build the product first and then maybe think about security later. So how do you weave it into the culture that you can't just ignore this aspect that's really critical.
It's gonna bring down your whole product and just say, I'll worry about it later. Same with accessibility, right? There are folks that are like, well, we'll worry about all this later. But then because there's been quite a movement around this there are all these frameworks where even if you don't care about it, there's like all these frameworks that you can take and you're like, all right, I'm just gonna follow this guide and I can at least get partway there.
I can at least at the very basic have like certain colors that make sense or make sure I have like text-to-speech or make sure I have like the alt text on my page. I think that's where you get to like these very seemingly small design product engineering decisions, like the alt text of a page or what kind of variables you're gonna use ultimately can have these cascading effects. And so I'll bring it back to maybe concepts even if you don't know really well or familiar enough, like, oh, I totally know what accessibility is and I can see how it's been used in like different products.
And then one framework I love to use is this, again, it applies to accessibility, but I find it works quite well in anything that deals with thinking about like more inclusive design. I don't know if you've seen it, but Microsoft came up with this years ago where it's different kinds of abilities where you have permanent, temporary, and then like in the moment, I forget the exact word they use. So like permanent would be, let's say you are missing an arm and then temporary is let's say you are, you broke your arm, so now you just have to wear a cast.
And then in the moment, as though you're just carrying a baby and like you can't like open the door. Someone is going to be in one of those categories and when you design for the permanent, so many other people will benefit. And in that case, that's like a physical ability, but the same applies for like different kinds of access to the internet.
So think of when we all had to go on Zoom with the pandemic. Some people permanently don't have internet. Some people maybe they're doing construction on their street and you like temporarily don't have internet or maybe there's a power. Like how do you design in such a way that you take into account what we sometimes call these quote edge cases, but they're not edge cases.
They're just like a scenario that you might not think is common, but how do you take into account these scenarios that people you think might be on the edge are experiencing, but we have to really take into account?
Melissa Perri: Yeah, I think this is a good topic to dive into. I hear a lot of companies and I think this is some people's best case or best practice product management what they advise to companies or people when they're starting a product, right? We're always told to focus, like pick your target user, pick your target person, narrow in on that and focus.
How do you differentiate, right? Or what would be your advice so that people would focus on a target persona, right? But also not leave out too many people who'd also be that target persona, right? But might not just be who's in your head. Like how do you balance that? How do you balance the focus versus all the different scenarios that you're talking about too?
Kathy Pham: I love these questions. I feel like I'm sure you get asked all the time. So, like who are my initial users and what are my user stories and what are my personas? I can't design for everyone. The tension with building something, getting an MVP out there so you get some users, versus like the, how do I design for a more inclusive or broad audience?
So I think that comes down to a couple of things. I think one is the team culture you put in place and then also building out those personas to begin with that are as inclusive as possible. And what that looks like is so many times, so last year I actually started an incubator at Mozilla called the Fix the Internet Incubator.
And within eight months we funded 50 new startups to build something that, quote fixes the internet. And ideally that means that all of these people are pretty mission focused. They're looking at all the ways in which they believe the internet was broken, whether it's with news online or search, or disinformation, misinformation, decentralized web, et cetera.
And it was really hard to balance building something within like a few months and showing proof of building something to also figure out how to make sure as many populations as possible are included. So I raised that because I want to just say out loud that it's a really hard thing because you're like survival versus including people. But I think it's also important to build into the culture knowing that the moment something is built, it's really hard to reverse, both mentally, because your team's already stuck on it, but also sometimes, if you build code a certain way to backtrack, it can be a lot harder.
So to have a team culture where you're like, you are so, so deep on, like we're really, when we say reiterate, we really mean it, that like this might be the first version, but we might realize in the second version that we left out a ton of stuff and we're going to have to be okay with going back and changing how we do all of our variables, for example. So if we decided one day that gender was a Boolean, yes or no, that's a very, that's very much an engineering decision, seems to make sense to a ton of people. We like put everywhere that field gender, some team will probably give it not a second thought because it's just common practice.
And then you're like, oh, that's maybe not the path we want to take. And if you like, be prepared to go ahead and like refactor all your code. And that has to be part of your culture, that it's okay and it's not scary. So I think that's one part of it. And the other part is I'm really big on early discovery, sprints early on.
So not just like cursing, like do like a solid two weeks, like all hands on deck, spend the time to like do as many interviews as possible, draw out all the personas you can possibly think of and interview someone and have them tell you about someone else you have to interview. Just keep digging and digging and digging and digging. It doesn't mean you're going to have to build for everyone, but you want to spend that time to understand as deeply as possible the problem space you're in, even if you're not even going to expand to like that, maybe that area yet, so that you're just armed with that information is always in the back of your mind.
Because all of those like subconscious, nuanced things can really drive how we structure our code, the product decisions we make, the types of user interviews we do, which in like a physical world, which regions we go to for all those things, I think, early on building out as many personas as possible. And ultimately you're going to, yeah, you might have to pick a few personas that you're going to have to start with, but then bear in mind that like, but then at the very least know what you might have left out, versus just not know. And then you can kind of start from there.
Can you have your backlog of all the things that you maybe left out?
Melissa Perri: Yeah, I don't think there's really excuse for super mature companies who've had a product that's like stable for a long time. They have a mass amount of people in there to forget about, you know, some portions of the population. Like, you know, when you're starting out, yes, you're going to have to prioritize and you might not be able to sell for everybody, but if you're finding out that that's something that you want to widely offer, your advice is to go back and make sure you're accounting for everybody, right? Make sure that you're actually making it something that everybody can use. And is that what I'm getting from it?
Kathy Pham: Yeah. And while you make a good point about bringing up mature companies, right? It would seem that if you're a mature tech company with millions or billions of revenue, there really is no excuse. But I think that goes back to the foundation of how your product is built and then who you have around building the product.
And let's say the majority of your users are in the West and something crazy happens in your platform in a different country, in a part of the world where maybe no one on your team is from. It can be actually pretty easy for a company with, like, the best engineering talent in the world, the best product talent, the best design talent, to take a really long time to respond, to not staff up that team, to not understand, like, the political, economic climate of a region. Like, one example we had talked about, Melissa, that...
But again, this is, like, not unique. I can think of examples in other places, but this one just has been written about, so you can find it online, is the Facebook and Myanmar case, right? Where, with the lack of understanding of a combination of things, like the way news is consumed and the growth of electronic devices and the access to Internet all combined together, it made it so that perhaps news feed might be the only source of news.
And for someone in the US, you might be like, we have all these other ways to consume news, but there's this environment where actually the news feed on the phone, especially if you're only able to access that, might be the only source of news. And now, if you're the product team for that, you have to really think of, well, what happens when my assumption is that, like, people using my product as, like, their source of truth always, and the only source of truth. And so, like, even with mature companies, arguably sometimes with mature companies, they all have blind spots.
They're not the only case, right? There's so many cases of other mature companies where you have these blind spots and then you're slow to address it. You might not have the expertise. They ultimately, you know, started ramping up that team and that's documented as well. But sometimes a newer company, I mean, I think in some ways we have to rely on some of these startups to think of things differently and build a really different foundation, to build it in your culture that, like, the moment something goes wrong, you have your, like, SRE version of a team to address that, right?
Like, if you have a security incident or someone hacked your system, there's usually a tech team, a DevOps team, a set of other teams, something that addresses that. So, like, what's a version of that for when something goes wrong with your product but in, like, a more squishy society issue versus just, like, not our problem. We didn't really design it that way.
People are using it in ways we didn't really design it for, not our issue. That's a big cultural thing, right? I mean, we talked about Robinhood. When you buy a Robinhood, it's an example of, like, mission-driven to bring more access to people. But what does more access look like? What does easier trading look like when you also don't...
You know, we talk about end-to-end testing and end-to-end user experience, right? So the ease of trading, yes. And what happens when the data you give back, you know, the data you display on the page, the content, is accidentally incorrect? Well, the repercussions for that can be really, really, really deep.
And when you have an engineer that might not really understand, like, nuances of trading so that you show certain data and maybe not other just things in ways that might not make sense, it could be perceived in ways that can be fatal or in many ways. So, yeah.
Melissa Perri: I was sitting here nodding very profusely at what you're saying. Did you know I have a course for product managers that you could take? It's called Product Institute. Over the past seven years, I've been working with individuals, teams, and companies to upscale their product chops through my fully online school. We have an ever-growing list of courses to help you work through your current product dilemma. Visit productinstitute.com and learn to think like a great product manager. Use code thinking to save $200 at checkout on our premier course Product Management Foundations.
Melissa Perri: So there's two things there, right? Like, there's the my users are stupid and they just don't understand this, right? And I I see this attitude a lot with product managers or UX designers where they're like, oh, they couldn't possibly use that that way. They used it wrong. And you're like, what's wrong though?
Right? Like, who constitutes what's right and what's wrong in these products? And I think we get such an attitude sometimes in product that the way that we think is the right way. And I think that goes against all the empathy that everybody teaches you that you need in product. But it can also lead you to not understand the experiences of others and how other people use it, right?
It's such a dismissive attitude. I hate that. It's like, no, the user's not wrong. You just don't understand the user. Like, you did a bad job. You're not a great product manager or UX designer if people don't know how to use your product. So you definitely see that. And then, you know, with the Robinhood type things too, it's very much like designing from a perspective of, you know, what stockbrokers look at every day but then marketing it to users who've never done stock trading before because we want to make it accessible, right?
Never thinking outside the box from a perspective of how do I get people started or what might be scary for them or how do I help people actually understand this? So I think that whole thing that you were talking about with the user not understanding the user is so spot on. I see it all the time. But that also leads me into the second thing.
How do we prevent it from happening, right? Like, how do we build our teams so that it's not like, oh, there was a crisis in Myanmar. Like, now we have to build a team around that. But how do we build teams that are going to preempt that rather than us be reactive to these situations that happen?
Kathy Pham: Yeah. You actually made me think of something with users. It seems like such a contradiction, right? Where I feel like in product and design, we're like, listen to the users. We listen to the users. But then, do we really? And then which users are we listening to? And then sometimes when users are telling us things we don't want to hear, we're like, well, you're wrong.
No, that is not listening to the user. Yeah, exactly. And how do we know? I actually have a pin someone gave me that said, I am not the user. And how often do we experience those scenarios? And I think that gets to your question of how do we prevent some of this? I think going back to also thinking about who normally becomes a product manager in tech.
There really is no direct path. But some of the big tech companies do have a formula they like to follow, right? Thankfully, a lot of them have now deviated from that. But for a while, it was engineering background with some product experience that maybe you got somewhere and hire you as a PM or someone who maybe was a small startup founder and then was later hired on as a product manager.
Or you were an engineer for a while. You went and got your MBA, came back, became a product manager or was consulted. There's like a pretty former-like thing. And so now you're looking at who's now really in the room making those decisions around what product decisions are. Oftentimes a lot of people with engineering or tech backgrounds to some degree.
And then with an added consulting business training on top of that. And that creates a lot of blind spots because we've narrowed our paths to who these big decision makers are. PMs are so pivotal on every team and every company, right? We're the ones that hold so much of what gets shipped and what doesn't as the ultimate voice.
And we talk often about how it's this seemingly powerful role where you're the voice of the user and you're ultimately the decision maker for making all these product decisions. And when that group is homogenous in the sense that it's very similar backgrounds. It's also homogenous in demographics, right?
But also with just similar backgrounds, which is related to demographics. What does that look like? So what instead do our teams look like if you have product teams that honor expertise equally across, for example, tech and engineering or also history and political science and economics in ways that you honor a product team and you may be debating which metrics do you want to track or which personas you might want to take into account right now.
Or even just the details of which region of the country or of the world you might want to fly to to do some initial testing, for example. And initial testing ultimately translate to who your priority users are, right? You have all these decisions you make. But what if you have people in the room with really complex backgrounds amongst all of them so they can really quickly raise the concerns of, well, you might think this is the group of user that you want.
But if you really think about it, there's a food desert in this part of the country and you really should go interview the people in the food desert because they might really benefit from, let's say, same-day grocery delivery. Whereas before, you're like, no, we're just going to find all of our power grocery stores, like shoppers or something. And then someone else brings up these really different perspectives.
But it has to be able to happen in a pretty easy, not the right word, but in a way that's natural and doesn't mean bringing in a whole other consultant and reaching out to find experts. It's really helpful when those people are also around and the team itself can riff off of each other in addition to bringing in people with experts and lived experiences and interviewing the right people. And some examples of that, I think, are those who want to look it up.
There's a scenario with Airbnb a few years ago where, as some of you recall, Airbnb while black, Airbnb while Asian, et cetera, were trending on various platforms because hosts were discriminating against guests. This is by no means to say that Airbnb has solved its race problem. But what they did at the time was they hired Laura Murphy, who was at the ACLU, the American Civil Liberties Union, who really was an expert on the topic.
Whereas most teams would have perhaps put her in the policy, law, trust and safety side of the org. They embedded her right smack inside the product team, so the product engineering team. And this, I think, also goes back to which areas hold power in your organization, in this case the product and design team.
And they came up with a slew of things that they tried out for everything from community guides and values that hosts and guests had to look at to host training to product decisions that I think are really interesting because they're such like product decisions. They looked at changing instant booking to be much more common. So now someone can just instantly book before a host could go and look at a photo and judge based on a topic.
And I think it's product decisions like that that can happen when there's someone in the room to really point that out. And I think we just need to have that happen over and over again. And I see this constantly with startup teams where your early stage startup is like four or five of you. Sometimes it's the people you know.
The perspective is limited. And there are all these perspectives missing, right? And to think of ways where you leave room for these perspectives to be added to your product and think through how a product decision like how easy it is to do instant booking versus verify by host completely changes the dynamic of that.
Another example I share that also I think would be like a dream product decision. It would be like a dream product meeting for example. This is a real scenario. Let's say you're working on a team for a ride sharing app somewhere in the world. And your team is super excited about using Google Street View and doing machine learning on the Google Street View images to determine a safety score because passengers and drivers are like, we're just worried about the safety of the places we're going if we don't really know what the area is.
And the team is just so excited about this tech that they can use. And they can use machine learning to just give a score of what safety is. And then this real scenario, in that room that day, there was a lawyer, a historian, a race and gender scholar, a social scientist, computer scientist, engineer, a few others who just happened to be brainstorming this product idea.
And within like 10 minutes, someone was like, well, have you heard the broken windows theory? This person was like, no, what's that? And they're like, well, there's this broken windows theory that you can't judge the safety of a neighborhood by broken windows. And then other people brought up like various concerns and safety.
And then finally someone, Jasmine McNeely, an amazing legal and media scholar, was like, well, safety for a home. And they're like, what do you mean? They're like, well, safety just has a different definition for different people. Some people feel safe in one neighborhood and unsafe in another. How are you going to take into account when you give them a safety score?
And all that happened in like 10 minutes. It was just like people bantering just as they would with which feature we want to build next. But it completely changed how this team thought about using machine learning on Google Street View images to have this huge feature of safety score in the product.
So what can that look like if our product teams were able to have those kinds of conversations to bring in the product feature, but also all these critical concepts that were rooted in history and inequality and all sorts of other research outside just like the build part
Melissa Perri: Of tech. Yeah, and I love those examples. I think they're so important. You hear more and more of things like that cropping up in the news and all around. One of the ones that was coming out was the algorithms with the facial identification that the police were using. And it was completely identifying the wrong people.
And at the end of the day a tech team built that and a tech team trained it. And I think there's also these debates I think we get into about, oh, you know, we just blame the algorithm, right? Like all the blaming of the algorithm stuff that has been floating around. I'm sure you have a lot of thoughts on this.
But at the end of the day, it's people who build those algorithms and train them, right? So having that diverse team that you're talking about, having that banter, having people go through it, it seems like that could help prevent some of that as well. Yeah. You know, one of the first things I ever built in like fresh out of college, Kathy, was the sentiment analysis engine.
You'll laugh because this was back in, I don't know, like mid-2000 or so. But it was, you use like all this open source machine learning sentiment analysis stuff to analyze Twitter, back whatever Twitter was back in like 2006 or so, to like give a score for how Twitter felt about a person. And I'll never forget because there was so much buzz and excitement about this.
We actually ended up like winning first place at this like international competition. But it was crap. Like it was just so like, what? We built this thing that like told the internet like how people felt about someone. Sentiment analysis is so hard, right? And we ship it and people believe that it's true and right and real, right?
People just adopt the technology like it's awesome. Like this is so amazing and like we're praising us. And I think that was my first look under the hood of, wow, you can build something. People who have never actually built it were like, this is awesome. But the code behind it actually is completely imperfect.
Kathy Pham: And then if it goes wrong or goes right, people are like, well, the algorithm is so great or the algorithm is so bad. And having built so many of these algorithms, it's all imperfect. The data we use is imperfect. The data models, a thing we talk less about is not just the data that goes into the code we built, but like the models that we have around like for the algorithm.
So for example, if one example I'll share is those of you who haven't seen it yet, ProPublica did the piece about Compass, which is the recidivism technology to ensure it. It's this piece of technology that two data scientists built that was sold to government that is now reported on that was supposed to be less biased than humans. And they built this whole thing to determine if someone should be released to go home or stay in prison.
To simplify, it's not that simple. But ultimately, they use things like a series of questions that they just ported over from paper stuff. So things like has anyone in your family ever been convicted? Have you ever been convicted? All these questions that are incredibly loaded. And so now you can blame the algorithms.
But ultimately, we as developers and builders put that in there. And it's not just the data, it's like our actual code and model that we are now asking these kinds of questions in our models. And yeah, we can at the end like blame the algorithm, but it's the people building the algorithm. And when we make terrible assumptions or we have total blind spots, we like bake into them issues that already have existed in the world and we just propagate them
Melissa Perri: Through code. Yeah, it's really interesting how people just assume the tech is infallible. It's true. Like you make something and people are like, oh, it works. Like this is an algorithm. This is something I can use. But why do you think that is?
Kathy Pham: I have to find, somebody wrote a piece on this, on the moment you build something, you get it out there and you give it to people. There's just this sense that it works. If somebody gives you something they buy or if somebody, you just assume that that thing works. So how many times, well actually I do, but I'd have you use something like maps to get you somewhere.
You just assume that the map is going to get you somewhere. I actually never trust any maps. What assumptions did they make behind the algorithm they did for me to get from home to school or something, right? But there is some really, I have to look at it, there's some really interesting studies around the moment you build and get it out the door and you sell it and someone buys it, there's an inherent, even if there is some distrust of this inherent feeling of well this is going to do what it's supposed to do.
Melissa Perri: It could be electronic health records in a hospital system. That's a whole other conversation around ethics and tech of how failed technology, healthcare services lead to deaths basically and have a lack of understanding. In that area for a while it is wild.
Kathy Pham: Oh Melissa, we can have a whole other conversations about some of these electronic health record companies did not have design teams for the longest time, which is wild. But you have all these scenarios of technology being built without fully understanding people. What was the original? Why do people trust the algorithm?
Yeah, and I think sometimes in part because an algorithm is like amorphous so you ship it and people are like, well you gave me this thing to use so I'm just going to assume that it does what it says it's going to, it's a map or it's a phone or it's a laptop or it isn't going to inherit trust that it's going to do what it's going to do until it doesn't. And so I think that's where there's a deep responsibility for us as builders to, if we say something is going to do what it's going to do, whether it's, you know, we talked about Robinhood to help you generate wealth, we talked about Airbnb to help you book a place to stay. You just, you're like well that's what they say they're going to do and then you start digging and you're like oh, it might not actually help me find a place to stay if I look a certain way.
But like, we just there are just reasons around like in this ecology of how we think of things and when someone gives us something that says it's going to do a thing, we just believe it's going to do a thing until we're proven otherwise.
Melissa Perri: Yeah, and that's a lot of pressure for product managers, right, to make sure we get this right and to think about those things too I think before doing it. So if you had to give some advice to a product manager who wants to learn more about tech ethics and how to think through these implications, where would you say to start? Yeah, I think
Kathy Pham: A place to start is to broadly look at tech ethics in generally the field of products, ethics and inclusive products is we're building it right now. I'm building out product and society at Harvard and there are but I think some of the resources I love looking at are is Kat has something called Design Ethically and she goes through like amazing frameworks around design thinking and ethical design. Stephanie Nguyen also did some really great work around inclusive design and privacy by design as well.
I send this to all my students. There's an equity by design framework that exists out there and what it does is it takes like a human centered, and I think when you do this with product, it takes a human centered design framework and you probably have seen it. It came out of the D school at Stanford years ago.
It's the one with a bunch of like either octagons or pentagons all connected together. Most people have seen it but it takes that and then it takes like each step of that like empathize and define and it brings in, well here's how you can bring an equity lens on that. Here's how you can bring in historical context.
Here's how you can push your team to check themselves and their assumptions. And it's a really great, I think, framework for anyone new to thinking about this topic to take something familiar and like, oh, I can see I can fit some of these ideas into this framework that I'm already thinking about as well. And there's a ton of, for those who are more into teaching, there's a ton of stuff that's coming out now at ResponsibleCS.org where computing faculty are looking at ways to integrate ethics into core computing concepts like algorithms, data, design, etc.
And I think a lot of that content is incredibly useful to like non-computing fields as well. And I can share more with you, maybe somewhere else, but those are a few things that pop. And a promising thing is I also am now seeing tech and product conferences start adding like an ethics track or a responsible track somewhere in their conferences.
So that's also fun to see as well.
Melissa Perri: Yeah, I'm really excited to see that too. And I'm starting to put product ethics into PM101 at HBS too. So I'm sure there's lots more for us to collaborate on in the future. But thank you so much for being with us on the podcast today. If everybody wants to follow you, where can they find you?
Kathy Pham: Read more about your work. Yeah, so I think two good places are just productandsociety.com or just on Twitter at Kathy T.
Melissa Perri: Great. Thanks so much, Kathy.
