It’s interesting that you’re talking about how, in these kinds of scenarios, you have to actively design distrust into the system to make it more safe.
Yes, that’s what you have to do. We’re actually trying an experiment right now around the idea of denial of service. We don’t have results yet, and we’re wrestling with some ethical concerns. Because once we talk about it and publish the results, we’ll have to explain why sometimes you may not want to give AI the ability to deny a service either. How do you remove service if someone really needs it?
But here’s an example with the Tesla distrust thing. Denial of service would be: I create a profile of your trust, which I can do based on how many times you deactivated or disengaged from holding the wheel. Given those profiles of disengagement, I can then model at what point you are fully in this trust state. We have done this, not with Tesla data, but our own data. And at a certain point, the next time you come into the car, you’d get a denial of service. You do not have access to the system for X time period.
It’s almost like when you punish a teenager by taking away their phone. You know that teenagers will not do whatever it is that you didn’t want them to do if you link it to their communication modality.
What are some other mechanisms that you’ve explored to enhance distrust in systems?
The other methodology we’ve explored is roughly called explainable AI, where the system provides an explanation with respect to some of its risks or uncertainties. Because all of these systems have uncertainty—none of them are 100%. And a system knows when it’s uncertain. So it could provide that as information in a way a human can understand, so people will change their behavior.
As an example, say I’m a self-driving car, and I have all my map information, and I know certain intersections are more accident prone than others. As we get close to one of them, I would say, “We’re approaching an intersection where 10 people died last year.” You explain it in a way where it makes someone go, “Oh, wait, maybe I should be more aware.”
We’ve already talked about some of your concerns around our tendency to overtrust these systems. What are others? On the flip side, are there also benefits?
The negatives are really linked to bias. That’s why I always talk about bias and trust interchangeably. Because if I’m overtrusting these systems and these systems are making decisions that have different outcomes for different groups of individuals—say, a medical diagnosis system has differences between women versus men—we’re now creating systems that augment the inequities we currently have. That’s a problem. And when you link it to things that are tied to health or transportation, both of which can lead to life-or-death situations, a bad decision can actually lead to something you can’t recover from. So we really have to fix it.
The positives are that automated systems are better than people in general. I think they can be even better, but I personally would rather interact with an AI system in some situations than certain humans in other situations. Like, I know it has some issues, but give me the AI. Give me the robot. They have more data; they are more accurate. Especially if you have a novice person. It’s a better outcome. It just might be that the outcome isn’t equal.
In addition to your robotics and AI research, you’ve been a huge proponent of increasing diversity in the field throughout your career. You started a program to mentor at-risk junior high girls 20 years ago, which is well before many people were thinking about this issue. Why is that important to you, and why is it also important for the field?
It’s important to me because I can identify times in my life where someone basically provided me access to engineering and computer science. I didn’t even know it was a thing. And that’s really why later on, I never had a problem with knowing that I could do it. And so I always felt that it was just my responsibility to do the same thing for those who have done it for me. As I got older as well, I noticed that there were a lot of people that didn’t look like me in the room. So I realized: Wait, there’s definitely a problem here, because people just don’t have the role models, they don’t have access, they don’t even know this is a thing.
And why it’s important to the field is because everyone has a difference of experience. Just like I’d been thinking about human-robot interaction before it was even a thing. It wasn’t because I was brilliant. It was because I looked at the problem in a different way. And when I’m talking to someone who has a different viewpoint, it’s like, “Oh, let’s try to combine and figure out the best of both worlds.”
Airbags kill more women and kids. Why is that? Well, I’m going to say that it’s because someone wasn’t in the room to say, “Hey, why don’t we test this on women in the front seat?” There’s a bunch of problems that have killed or been hazardous to certain groups of people. And I would claim that if you go back, it’s because you didn’t have enough people who could say “Hey, have you thought about this?” because they’re talking from their own experience and from their environment and their community.
How do you hope AI and robotics research will evolve over time? What is your vision for the field?
If you think about coding and programming, pretty much everyone can do it. There are so many organizations now like Code.org. The resources and tools are there. I would love to have a conversation with a student one day where I ask, “Do you know about AI and machine learning?” and they say, “Dr. H, I’ve been doing that since the third grade!” I want to be shocked like that, because that would be wonderful. Of course, then I’d have to think about what is my next job, but that’s a whole other story.
But I think when you have the tools with coding and AI and machine learning, you can create your own jobs, you can create your own future, you can create your own solution. That would be my dream.
Recent Comments