The Marie Kondo Approach to Security

Posted on in Presentations

Can security decisions really “spark joy”? Well-traveled CISO Bob Lord would argue yes, and that basic tidying decisions can help achieve significant, immediate, measurable improvements. Join this fast-paced fireside chat to hear wide-ranging discussion from memory safety issues and vulnerabilities to MFA and security controls to decision making and leadership. This unique keynote offers something for everyone.

Video Transcript

>> SPEAKER: Please welcome Hugh Thompson and Bob Lord.

>> BOB LORD: Hello. Hello. Hello.

>> HUGH THOMPSON: Hi, Bob. How's everything?

>> BOB LORD: It's well. It's good. And, you know, by the way, dude, you do not have to arrange this. You can just call me to get caught up.

>> HUGH THOMPSON: Yeah, that's true.

>> BOB LORD: It's been three years.

>> HUGH THOMPSON: Hey, Bob. Let's get together.

>> BOB LORD: You could actually do that.

>> HUGH THOMPSON: While we're at RSA conference.

>> BOB LORD: You could actually do that. I give you permission.

>> HUGH THOMPSON: Okay. All right. Thank you. I'll remember that for next time. Well, first, thank you for being here. Thank you to the audience for being here, too. It's the last day of RSA. How was RSA so far? Good? Okay. That's good.

>> BOB LORD: I agree. I agree.

>> HUGH THOMPSON: And so, Bob, we had a very interesting conversation a few years ago when you were the CISO, of the DNC, and you were talking about the challenge of managing this, or, I guess, advising, I think, is the way you put it, this very distributed, you know, group of folks, and you are ultimately responsible for security. But, you know, it's like, gee, some of these races are very minimally funded. And -- and it was absolutely fascinating. And so, when we talked, it's like, we've got to come back and see what the stories were, what the experiences were. But I've got to tell you, when we talked about this, and then I saw the program, and the title of this session, the Marie Kondo Approach to Security, which you wrote.

>> BOB LORD: Yes, I did.

>> HUGH THOMPSON: As the title.

>> BOB LORD: Guilty.

>> HUGH THOMPSON: I was slightly concerned. I've got to --I'll just say it. I'm going to just go -- say it -- say it -- say it just like that.

>> BOB LORD: Concerned about me?

>> HUGH THOMPSON: Yeah. Just generally.

>> BOB LORD: Like my mental health?

>> HUGH THOMPSON: Yeah, like that kind of a thing.

>> BOB LORD: Oh, I didn't know you cared.

>> HUGH THOMPSON: So, can you tell me about this Marie Kondo, like sparking joy.

>> BOB LORD: So, I'm not an expert, but I had been giving an interview to -- a series of interviews to Nicole Perlroth from the New York Times, and so she wanted to write an article about the transformations that we were making at the DNC. So, all that is fine. And in the course of describing what I was doing, you know, I think a lot of people, myself included, thought that when I would show up to the DNC, I'd be pulling in all of my start-up friends, do deep machine learning and AI to try to find –


>> BOB LORD: Seemed a reasonable thing.


>> BOB LORD: That didn't turn out to be how it played out. And so, it played out by me going around and trying to kill things and trying to turn things off. You can't hack it if it's turned off. It just turns out that in the course of trying to –- same thing everybody would do, discover the assets and then try to prioritize the risk. Ended up turning a lot of things off. So, I'll give you an example.

When I got there, there were no fewer than four different ways that staff would trade documents back and forth and with outside people. And so, you know, that didn't make me happy. It didn't make me happy as somebody who is also, in charge of IT and had to deal with costs. It didn't make me happy from a management standpoint. It did not make me happy from the perspective of IT because you're never going to get enough IT.

>> HUGH THOMPSON: Did you standardize on Pastebin?

>> BOB LORD: No, no, not quite. But in the course of talking to her, she said, yeah, you're just throwing out the stuff that doesn't spark joy. You're kind of the Marie Kondo of cybersecurity. And I thought, okay, if the shoe fits.

>> HUGH THOMPSON: I'd take that.

>> BOB LORD: So, do you know Marie Kondo?

>> HUGH THOMPSON: Well, okay. So, all right. So, I'll -- I will share this. Many years ago, I was just browsing the New York Times bestseller list and I saw this book that was number one, and the title of it was something like The Art of Tidying Up. I think that was her first book. I'm like, this is crazy. Like, why is this a number one bestselling book? And so, I actually downloaded it, and I'm thinking, okay, it was out of curiosity more than anything, and I was totally hooked. Like I was all in. And I don't know if you're familiar with this method.

>> BOB LORD: A little bit, yeah.

>> HUGH THOMPSON: Well, you are the Marie Kondo of cybersecurity.

>> BOB LORD: Inadvertently. I sort of backed into the whole practice.

>> HUGH THOMPSON: You would be familiar with it. But the way that it works, and I actually did this, is you start with your clothes. And the process is you take all the clothes that you have in the entire house, like even stuff that you have in a box, in storage, everywhere, and you just dump it on the floor, right, in one central location, and then you go through each piece of clothes, and you have like an emotional reunion with the clothes. That's literally how it works. And you touch it, you know, you feel it. You're like, geez, you know, when did I wear this? And then you take a picture of it. And if you can emotionally part with it, you get rid of it.

>> BOB LORD: Okay.

>> HUGH THOMPSON: It was -- I don't know. It was an amazing experience. It was very cathartic. My wife thought I was completely insane. But I probably got rid of like 60% of the stuff.

>> BOB LORD: Yeah, it really works.

     >> HUGH THOMPSON: It sparked –- it did spark joy, even though this was pre the sparking joy.

     >> BOB LORD: So, you would look at something and say does this spark joy? And if not, off you go.

     >> HUGH THOMPSON: Yeah. Yeah. Yeah.

     >> BOB LORD: Did you take pictures of them to –

     >> HUGH THOMPSON: I did. Of course. Yeah.

     >> BOB LORD: So that you can – if you need to go back, you can take a photo?

     >> HUGH THOMPSON: Yes. Yes. Yeah. I have a whole library.

     >> BOB LORD: I love that. I didn't quite do that with those servers that we were turning off. I didn't quite –- I mean, I didn't -– so, maybe I still have a physical token. Here's, you know, the service that we're going to decommission. I didn't quite take -- well, you know, next time. So, I would say that I'm developing into the Marie Kondo.

     >> HUGH THOMPSON: Okay. All right.

     >> BOB LORD: Maybe I have a little bit.

>> HUGH THOMPSON: It's a title you have to grow into.

>> BOB LORD: It is.

>> HUGH THOMPSON: And so, what have you been up to since the DNC? You were the CISO of Yahoo. You've been in massive organizations. You were then in the DNC, which was just fascinating because of how you described it, just very distributed, you have influence, very little budget, regionally. What have you been up to?

>> BOB LORD: So, like you said, you know, Yahoo is a very big company, and the DNC was just a few hundred people, and then, to your point, the state parties are separate legal entities, and so I didn't get to tell them what to do, so I had to try to explain things and be charming and provide them [indiscernible 00:07:04].

>> HUGH THOMPSON: You are charming.

>> BOB LORD: Well, you know, it's not – it's not so charming when I'm telling you I'm taking away your toys, right? I mean, there's only so much, you know, when I say I need to turn this thing off. But then, of course, the campaigns, they're also, separate legal entities, and they all have their own way of doing things, so influencing all of them was very challenging. And then I was -- I got a call from Jen Easterly, and she wanted to talk about some of the things that I had done, and I think maybe she had seen some of these videos. I don't know if she saw the one from three years ago.

>> HUGH THOMPSON: Probably not. Probably not.

>> BOB LORD: On Twitter. I don't know if you saw. But I made some faces because she was telling some alarming stories, and so I tweeted that out this morning. But, yeah, we talked a little bit about what I was looking to do, and I think I described, you know, full disclosure a little bit, of dissatisfaction with the maturity, the maturing of our industry. It doesn't take too much reading to determine that we still have a long way to go before we have things that are truly safe.


>> BOB LORD: And I talked to her a little bit about some of the reasons why I thought that. And so, she invited me to come join CISA to try to be part of that solution. And, obviously, if you have been doing anything at RSA, you already know her background and her reputation and her ability to think about -- think deeply about culture and -- as well as all of the elements of people, process, and technology. And so, I just couldn't say no.

>> HUGH THOMPSON: Yeah. It's fantastic. And she's building out such a great organization. And, you know, the feedback that I'm getting is that the public-private partnerships that we've been talking about for 20 years at that RSA conference, they're actually manifesting into something very material now, and that's -- that's huge.

>> BOB LORD: I'm only six weeks in.

>> HUGH THOMPSON: So, you can't take credit.

>> BOB LORD: So, I can't take credit for any of that, but I'm absolutely --

>> HUGH THOMPSON: I've been hearing more in the last six weeks, but yeah.

>> BOB LORD: Yes, we've been very vocal for a while. No. I think -- I think that's right. I think we're starting to see real not just activity but progress.

>> HUGH THOMPSON: So, talk to me about -- about sparking joy, which, again, you had in the title of this abstract, which I should have read before we printed it. But what is it -- that's not typically the description that I hear from a company when they say we're embarking on a new security program, and, actually, our intent is to spark joy. Like I've not heard that as a speech, but I'd like to, and I'd like -- I'd like to -- I'd like to understand.

>> BOB LORD: I think there's something there.

>> HUGH THOMPSON: I have joy in it personally.

>> BOB LORD: Yeah. I don't have a complete answer. I think -- I think so often we're associated with doom and gloom and saying no and I think a lot of what I try to do is to not just say does this spark joy with me because, as a cybersecurity professional, a lot of things will not spark joy with me because I've seen a lot of bad things. But one of the really great things was people at the DNC and in the community were really sensitized to security issues and wanted to do the right thing, and they were very -- they were very hungry for the leadership that was going to get them where they needed to be. And so, you know, again, I can't take credit for the title, and this wasn't a conscious thought when I was doing it. I think it's more of a conscious thought now after Nicole mentioned that name, but I really try to bring people along for the journey. One of the things I do constantly is talk about how the attacks work. And so, people will always ask me questions about, hey, you know, so if I'm not supposed to use wi-fi in the airports, what should I be doing? And, you know, I spend time teaching people how the hacks happen, and that's not how they're going to get you. They're going to get you, but they're not going to get you in the ways that you think. So, let's talk about how the attacks actually work, let's talk about these probabilities, and let's figure out what's most important. Spoil alert: MFA.


>> BOB LORD: So, we can talk about that in a minute. But a lot of what I try to do is bring them along as much as I could for the journey so that they would understand that this didn't spark joy for them either. And a lot of times, people kind of already know. Like they're -- they're paying for a thing they only use occasionally, or it's not hooked up to our single sign-on provider and I explain what the benefits of, you know, the single sign-on system are. You can kind of see that they're like, yeah, like I don't want to be the next patient zero. Right. Right. And so, and I admit that this may have been a different environment because people were so highly sensitized. When I got there in 2018, they were sensitized to the 2016 events. But -- but people were interested in coming along for the journey. And a lot of times, the biggest bit of friction was just -- was just the work to get whatever we knew that we needed to get done to get done. Right? It just -- it takes time. You have to renegotiate contracts. You have to deal with the billing issues. It just takes time. That, for us, was, in many cases, not all, but in many cases, the biggest impediment.

>> HUGH THOMPSON: I'm going to ask you, just to contrast that with my own Marie Kondo experience. When I got rid of the tie-dye shirt that I had that was a size medium, which, you know, I got rid of it for --

>> BOB LORD: We do want to see those pictures.

>> HUGH THOMPSON: -- multiple reasons. I do have the picture, but still.

>> BOB LORD: Twitter.

>> HUGH THOMPSON: But I really -- it did not have utility for me anymore, too. It wasn't just a joy thing. It was a utility thing. And we live in such a fascinating industry in the sense that if you take away a control, it's not like the problem that that control was trying to solve originally went away. Right? Like, let's say we stopped doing spam filters, for example, which we've had for a very long time. You're going to get a lot of spam again. Right? Even though we've made progress with DMARC and all of these, how do you -- how do you go about the process of actually thinking about trimming controls without the fear that there is something that this control was here for, and maybe we're going to leave some massive --

>> BOB LORD: Yeah.

>> HUGH THOMPSON: Ghost of Christmas Past that comes and visits us?

>> BOB LORD: Well, I think what we were doing was not really removing the security controls, but really removing the thing that required security controls. And so -- so, for example, we had at least three ways that I can recall to send bulk email to constituents and for the communications team to email reporters, and having three different ways of doing the same thing means if you're doing it right, you have to have three different sets of controls, you have to have three different bills associated with that, and you have to support three different systems within IT. So, consolidation was what we wanted to do, and, of course, we had to bring people along for the ride, and they had to change their workflow. And that's -- that's the hardest thing. And I don't know that we do enough of that in the industry where we go to people and say I'm not taking things away from you, but we do need to consolidate, and we do need to help you change your workflow. We have a tendency as technologists to try to add more technology to what is a people and process problem, and that just -- it just rarely works. And so, again, the philosophy wasn't to try to trim the controls, but to try to trim the things that needed control so that we could actually watch those controls. If you have five different ways of doing things, you probably don't have five comparable ways of dealing with the controls. You're just -- you're just not. You have to -- you have to be honest with yourself and say like, I know that somewhere there's going to be a process that is -- that's going to lapse and we're going to have a problem. So, cut, cut, cut.

>> HUGH THOMPSON: How do you -- how do you convince people to actually do that? Like, you know, I'm thinking about now let's move back from the DNC into a sophisticated and large organization like t the time? When you go, and I -- you know, I've had this experience -- you go to a development organization, for example, software development organization, inside of the company, and you say, look, you know, we've been doing some utilization metrics and there's some systems here that just aren't being used, we literally are running an ancient operating system, it cannot be patched anymore, it's no longer supported by the vendor, and we -- we have to get rid of it, and you don't even use it, right, like according to RRA. My experience is that you get massive pushback because it's almost like a knee-jerk reaction. You don't even know why you still need it in some cases, but you're just afraid of actually decommissioning it.

>> BOB LORD: Yeah. I had exactly those kinds of issues, well, I mean, at every company, and I'd talk to other CISOs and they have exactly the same thing. There was -- I think we got good at that at Yahoo by coming up with a real risk register, so we had like our top ten list of the biggest areas of concern, the biggest risks. And exactly what you said. We went to the teams who were responsible for these systems that were underutilized or needed patching or whatever. And, you know, what happens in a lot of organizations is whenever there is a reorg, that is the opportunity for technical debt to creep in. A system that's working well, it self-heals, it reboots itself when it needs to, it gets left alone, and as teams move around, you start to develop these systems for which there isn't an easily identifiable owner. And so, we ran into a few of those cases. And exactly what you said. We talked to the development team, and we got the antibodies forming. We got the pushback, right, the pushback. And, you know, people, without hearing what it was that we were concerned about, knew that we were going to take away their toys and they didn't have time for that.

And partly, that's, I think, a function of the way some security teams work, and so we just had to like unlearn them, those -- those attitudes. And we sat with them and we'd ask them questions, like what would it take if you were to solve this problem and make this a fully supported system that you would be proud of? And they'd say, we don't have time, Bob, you know, we just -- we don't have the money. We don't have the manpower. The people who had originally designed this have left the organization. We'd have to do some archaeological. It's too much. I'd say, but -- but humor me. Like what would it take if the CEO said, I will make it happen. What would that look like?

>> HUGH THOMPSON: And is the answer like a personal Ferrari?

>> BOB LORD: Yeah. The first answer, yes. And then the second answer is a bunch of people with these kinds of skills. Like how many people? What kind of skills? How long would it take? And, you know, pad it out a little bit. But what would it take? And it took us multiple meetings to get to the point where they actually thought about what it would take and then gave us a roadmap and said, okay, so if the CEO says this, we will fund this, this is what would take this thing from a high-risk system to one that everyone would be proud of. I said, yes. Okay. And so, I would take the top ten list to the CEO every month and say, here's the top ten list. And -- and one of those systems popped up, and we had the -- the -- the -- the, you know, the bill for the estimate for what it would take to fix -- fix this car, and, on one occasion, the CEO said, just turn it off, I don't care. And I said, I'm sorry, ma'am. If you look over here in this column, this is the number of millions of dollars per year that this system is supporting. She said, this is a multi-billion dollar a year company. I don't have time for a couple million dollars, which was exactly the right answer. I delivered the message back to the team, and, sure enough, they were able to, you know, over the course of a few weeks, they were able to come up with a truly brilliant way to do what they needed to do. And so, sometimes constraints can create tremendous -- tremendous creativity. It's -- it can be inspiring, in a way. And so, they didn't lose the money, but we got a system that was on par with the rest of the standards in the organization.

>> HUGH THOMPSON: Let me shift gears for a second because there's been a lot of discussion this week about a term that Wendy Nather, who is just amazing, coined many, many years ago of the security poverty line.

>> BOB LORD: Yeah. Right? So, the -- the place under which you don't have the skills, you don't have the capabilities, you -- you can't actually apply security controls. And I can imagine you witnessed this at the DNC, especially in state and local races. You want them to do MFA, for example, you want them -- do they have the resources? In many cases, I'm sure the answer is no.

>> BOB LORD: Right.

>> HUGH THOMPSON: How do you think we deal with this problem systemically?

>> BOB LORD: Have you seen the movie Moneyball?

>> HUGH THOMPSON: Of course. Yeah.

>> BOB LORD: It's great.

>> HUGH THOMPSON: Have you all seen this movie?

>> BOB LORD: Yeah. Yeah. Did you like it?

>> HUGH THOMPSON: That's Billy Beane.

>> BOB LORD: Yeah.

>> HUGH THOMPSON: Yeah. Oakland A's.

>> BOB LORD: Yeah. Oakland A's, Billy Beane.

>> HUGH THOMPSON: Yeah. Yeah. Yeah.

>> BOB LORD: So, every now and then, I have to take a break from security. I don't know if you all do. You just have to kind of do something different. I usually learn my lesson and I go back and watch black and white films because there's no mention of Twitter, there's no mention of anything, you know, rotary phones, like, okay, I can actually decompress. I had the good fortune of picking this, which I'd never seen before, and good things about it. So, I watched this. And there's a scene. It's so -- to set up for if you haven't seen it. Billy Beane is the general manager of the Oakland A's, losing team, and they just lost their best player, and they're trying to figure out how do we attract a player of that caliber to fill this spot, and how are we going to pay for it, knowing that they have a budget which was, I forget, like a quarter or a fifth of some of the bigger teams?


>> BOB LORD: And so --

>> HUGH THOMPSON: We actually had him here in 2013, actually.

>> BOB LORD: I need to watch that video.

>> HUGH THOMPSON: Please Please.

>> BOB LORD: Yeah. So, in the movie, what happens is Billy Beane is talking to all the -- the managers and the scouts and he's asking them, what do you think the problem is? What do you think the problem is? And he keeps saying, that's not the problem. It's like, fine, Billy, what is the problem? And he says, the problem is that there are the rich teams, then there are the poor teams, then there's 25 feet of crap, and then there's us.


>> BOB LORD: That's aspirational speech.

>> BOB LORD: That's the problem. And he says -- and he says, this is an unfair game. This game is rigged against us. We will not be able to find a player to fit that role. And if we did, he would not join because we do not have the money. It is an unfair game. And if we play this game the way that everyone else plays the game, the one thing I know is we will lose. And what we need to do is change the way we think about the rules and play differently. And this was a -- for me, this was -- this was amazing. He said, we need to find those undervalued players that are flying under the radar of all the other teams who we can put in unusual configurations at a -- at a price we can afford, then we will win. And I thought -- I stopped the video, and my wife looks at me, and she said, you know this is a movie about cybersecurity.

>> HUGH THOMPSON: That's every movie.

>> BOB LORD: This is about the cybersecurity poverty line. And she's very nice, and so she let me just talk about it for a little bit. And then she goes, okay, you can hit play again. And so, I did. And I stopped it three or four other times because the movie is about the cybersecurity poverty line. And so, what do you do? Well, you have to look for these undervalued tools. You can't play the game the way that other people are playing it. And I wish I had seen this movie before I joined the DNC because then I could have given this talk to people and, you know, short-circuited a whole bunch of longer conversations. But -- but part of what we had to do is to really focus the spotlight of people's attentions on the -- on the undervalued tools that were going to dramatically change their ability to have a good security posture. And one of those was MFA. And we were able to -- you know, we have the numbers on this -- we were able to move the needle in ways that I don't think we would have been able to if we hadn't, like I said, explained about the dedicated human adversaries, and if we hadn't explained how the hacks actually happened, and then explained why things like MFA are going to disproportionately raise the cost for the attack. And so, that's part of -- of what we did.

We -- and I have to be careful when I talk about this because it's -- this is a -- you can help me wordsmith this, but the -- the challenge is that we also have to worry about the organizations that are valued enough that hackers will go the extra mile. And so, when we think about MFA, there's different kinds of MFA. And so, it's important to note that all forms of MFA are better than no MFA. That we can say quite clearly. That's -- that is true. Microsoft says that of their -- the account compromises that they look at, 99% -- 99 point whatever 1% did not have MFA on it. So, we know that it helps. Having said that, the more people who move onto MFA, the more that the attackers will engage in MFA bypasses where they will take into account the fact that you're going to type not just your name and password, but they're going to ask you for the six-digit authenticator code as well. And so, what we moved the DNC to and other organizations to was Fido security keys. And I assume most people are familiar with -- with Fido security keys. But those are phishing-resistant MFA options. And that is -- for us, that was critical.

But I also want to make sure that people are thinking about the way that the attacks work, so they understand that they need to move towards phishing-resistant MFA from whatever they're doing today, and that's something that can be challenging for a variety of reasons, or at least historically was challenging because of cost and other issues. Now, you can use your mobile phone as your security key. So, now, if you have a smartphone, now you have a whole range of new options. But -- but those were some of the things that -- that were -- that were really concerning about living below the security poverty line. The other that you've probably seen me tweet about is this thing called SSO, tax. Have we talked about this?


>> BOB LORD: Yeah, single sign-on tax.


>> BOB LORD: There's actually a website called

>> HUGH THOMPSON: Oh, okay.

>> BOB LORD: Somebody set up a website. It's very good. And the issue is, so I talked about some of the boring things that we do as security practitioners to reduce the risk, deal with identity and access management. That's one of those things. And when you spend a lot of time looking at things right of boom, you realize identity and access management is kind of key. You have to know who the people are who are logging into the systems and when they logged in. And if you -- if you stand up a different service for -- for, you know, mail, or calendar, or file sharing, or signing documents. And if each one of these has a different name and password for your staff, and if you have no ability to monitor what the controls are, you have no ability to enforce MFA, this -- this is a real problem. Luckily, there's a thing called single sign-on where you can go and purchase a service and all of your employees log into that, and then that bounces back to all of the other services. So, this sounds great, but the problem is that many vendors of a variety of services will put single sign-on support in the -- in the gold tier in the enterprise tier, and the groups that I was working with wanted the basic tier for a variety of reasons, including cost. And so, what happens is, you know, you'd go to this vendor and say, like, hey, I want the basic tier, but I want the single sign-on, they say, yeah, you should go to the enterprise tier. And, in many cases, the -- the enterprise tier is much more expensive than the basic tier. Sometimes it's 2x-3x. But it gets worse because sometimes there's a minimum number of seats that you have to buy. I want to buy five seats, but the minimum for the -- for the enterprise tier is 25. So, you know, do the math and you realize we can't afford that. And a lot of organizations just literally cannot afford that. There were several instances at the DNC where I had to go to bat for a team that was using a service, I wanted to migrate it to the single sign-on provider. And it was going to cost 2x 3x. Now, luckily, because I was doing all this other cost-cutting, we were saving, you know, that was about $1.2 million of annual recurring savings. So, when we started cutting everything, we were actually saving over a million dollars a year in continuing costs. So, that Marie Kondo program really helped us. And so, I was able to go take money that we had been saving and then apply it to some of these other services. Not every other organization can do that. And so, this is one of those problems that causes individuals and organizations that are below the security poverty line to stay below the security poverty line, they can't get up because they can never get control of their identity and access management. This is a real structural problem with our industry, and it's one of several things that we just don't talk enough about. And, you know, I understand the arguments from the vendors, you know, cost more to support or whatever, but they don't charge more for TLS, and that costs more to support, too. So, you know, I think my -- my hope is that -- that people who hear this, the product managers who are building these great products that we -- that we really love, that they'll start to think about decoupling because security shouldn't be a luxury good, it should not be priced like a luxury good. It should be more like a, I don't know, customer right sort of thing. You know, these security basics should be closer to -- to customer rights than -- than luxury goods.

>> HUGH THOMPSON: I agree with you. But it's, you know, how do you get the right incentives in place, even with the service provider vendors, to actually do it? Right? It's -- it's so baffling that if you look at our industry and how it's evolved, it's matured in so many ways, but one of the ways that I think we haven't matured very much is in metrics. It's very, very difficult, even for a security vendor that has a solution that they know will actually help you, to tell you how much better off you are because you have this security solution.

>> BOB LORD: Yeah.

>> HUGH THOMPSON: And I would imagine it's the same challenge when you go to somebody and you say, hey, look, if you implement this control, it's going to be good. Now, can I tell you how good? No, I can't really tell you that, but it's going to cost you some money. Right? So, the -- you can -- you can quantify the pain in terms of cost.

>> BOB LORD: Yeah.

>> HUGH THOMPSON: Maybe a usability cost in it, too. But we've just had a very, very difficult time articulating the value of the security center. How did you? How did you get around that? How did you approach that?

>> BOB LORD: So, when I figure out security metrics and have an answer for you, I'll write a book and then you can come.

>> HUGH THOMPSON: Okay. All right. Co-authored by Marie Kondo.

>> BOB LORD: Quite possibly could be. I mean, metrics are hard for a number of reasons. I mean, a lot of metrics that people talk about are like weather reports. They're -- they're not actionable. It's like, you know, this many pings, this many hacking attempts. Like, okay, like, what am I supposed to do in reaction to that?

>> HUGH THOMPSON: Well, let me -- and I'll just give you an example of this from this morning. So, I've got -- I've got the Ring doorbell thing at home. How many people here have Ring or have used Ring? Okay. Wow. I didn't expect applause, but, yeah, no, it is quite -- yeah. There is -- there is something when you open up the Ring app that's -- I forgot what it's called, it's like My Neighborhood, and it's basically threat intel for your neighborhood. That's what it is. Right? And it's fascinating the variety of things that you get. Like today, I got one, and it, you know, just popped up on the phone, and it said possible mountain lion, question mark. Right? That was the title of it. And then because it's Ring, it's got the video of this like just shadowy object at a distance, and I'm just panicked, right, because I've got five little kids. I'm like, oh, my gosh, is there a mountain lion loose in the neighborhood?

And so, then I have to go into it. And I'm reading through and there's this huge debate from, you know, even one of our actual direct neighbors, like, look, you know, it actually looks like a cat. There was somebody else that was a veterinarian, actually, that weighed in, or claimed to be a veterinarian. I haven't validated.

>> BOB LORD: You've got wild neighbors.

>> HUGH THOMPSON: Yeah. And, you know, they were saying that, you know, look, actually, the shape and the gait --

>> BOB LORD: Gait.

>> HUGH THOMPSON: The gait.

>> BOB LORD: It's all about the gait.

>> HUGH THOMPSON: Yeah, the gait of the animal indicated that it might be some kind of like raccoon. Right? And, you know, this debate goes on. And that's probably the most -- one of the most useful alerts that I've gotten from Ring.

>> BOB LORD: Now you get to know who's the vet in your neighborhood.

>> HUGH THOMPSON: Yeah, right. You get to know that.

>> BOB LORD: This is good intel.

>> HUGH THOMPSON: But you just, but you don't know how to act. Right? Okay. So, this is great that there is a possible mountain lion. There's some, you know, debate about whether, you know, this is the case. What do I do? Do I, you know, do I bring all the kids inside? Do I call my wife? It's like, look, mountain lion alert, you know, let's go to mountain lion DEFCON five, you know.

>> BOB LORD: That's serious.

>> HUGH THOMPSON: Action the plan we've been practicing and all that.

>> BOB LORD: Because you have your -- your -- your mountain lion incident response plan.

>> HUGH THOMPSON: Of coursed. Yeah, of course. You've got to have that.

>> BOB LORD: Yes.

>> HUGH THOMPSON: But what -- we are inundated with so much information, and you can call it threat intelligence. There's lots of different names for it. But it's hard to know what you actually react to because a reaction has a cost.

>> BOB LORD: It does.

>> HUGH THOMPSON: And I can imagine for folks that are below the security poverty line, there's actually not very -- even if you wanted to react to it, there's not very many ways that you can.

>> BOB LORD: Man.

>> HUGH THOMPSON: Geez. I've got to wait for the book.

>> BOB LORD: Man. So, a couple things. One is that is absolutely true. But I also, again, having worked at Yahoo where we had to deal with all sorts of -- all sorts of metrics and, you know, we saw lots of virtual mountain lions and -- and so we had to do that stuff.

>> HUGH THOMPSON: Oh, wow. All right. I didn't realize it's such a systemic problem.

>> BOB LORD: It is. And so -- and so, what's interesting is when I went to a smaller organization, it turned out that, yes, there are all sorts of things that could be going on. The -- the good news is that the basics are still the basics. The basics are hard. They're hard to get done. But we just -- we're -- we're just completely obsessed with focusing the spotlight of people's attention on the basics, and we wouldn't let them deviate. So, when they call up and ask questions about how to configure things, we'd say have you done our checklist, which was three things you need to do to dramatically reduce the chances of getting hacked. And you all can probably tell what those are? Like, what's the first one? That's MFA. Pad your stuff.

>> HUGH THOMPSON: Patrick, that was a good one.

>> BOB LORD: Pad your stuff. Use a password manager. And so, if you do those things, you've dramatically increased the cost of -- of attack. And -- and what's -- what's challenging for us today is, in 2022, we still don't have broad acceptance for MFA. I don't know if you know this, but did you see last year, last summer, Microsoft did one of their annual threat briefing reports?


>> BOB LORD: It's great. It's great. And in it, in one of the later pages, they talk about the Azure AD users that they have. And so, these are enterprise users. And one of the things that they say is of their Azure AD enterprise users, only 20% of them use MFA.

>> HUGH THOMPSON: Yeah. That's an amazing stat.

>> BOB LORD: Well, hang on a minute because I've got another one for you.

>> HUGH THOMPSON: Okay, great. I thought this was supposed to be the fun, relaxing, like end of pick-me-up.

>> BOB LORD: Yeah. All right. Well, it will be. We'll finish on a high note. But the scary stat is that the global administrators who are the ones who, again, you know, the global administrators, they can do all the things that -- they can do all the things -- they could delete the entire incidence. Whatever. Only 30% of the global admins - three zero percent -- not even one-third were using MFA.

>> HUGH THOMPSON: That's incredible. So, you know, we talk about MFA and -- and people say, I'm going to do a little Seinfeld callback here, but like people said, Bob, we understand the value proposition of MFA. And I say, I don't think you do. I don't think you do.

>> HUGH THOMPSON: You're not acting like you actually --

>> BOB LORD: If you understood the value of MFA -- anybody can talk about MFA. You deploy the MFA. You have to enable the MFA. That's really the most important part of the MFA is the enabling of the MFA.

>> HUGH THOMPSON: And joy of MFA, communicating the joy.

>> BOB LORD: Yes.

>> HUGH THOMPSON: You're sparking joy kind of comment.

>> BOB LORD: It is joyful to not get hacked.

>> HUGH THOMPSON: I have not heard many people describe it as joyful, but I could see -- I could see how you could position that.

>> BOB LORD: This is a very difficult thing. And so, while we should also think about some of the more esoteric things that we can and should do, if we're failing at the basics, all that other stuff may not matter, or it may not matter as much as you've invested time and energy and money into it. So.

>> HUGH THOMPSON: Well, I've got to bring up something that we actually talked about in the last discussion that we had on this topic, which is an example that I ran into. It was here at RSA Conference. And I want to say -- it was a long time ago. It was at least seven or eight years ago. I was given a talk, somebody came up afterwards, and it was somebody that had, you know, told me that they had -- I'm trying to get the exact words, but they had innovated on MFA. And I'm like, okay, great. You know, this is a startup, you know, maybe innovation sandbox would be a good candidate. But this was just, you know, a person who worked for a company that had implemented MFA. They were using physical tokens in this case. And -- and he said, I figured out how to solve the usability problem of MFA. I'm like, wow, this is -- this is amazing, you know. What did, you know, what did you do? Did you patent it? What is it? And so he goes -- he goes to his phone and he pulls up a website that is not password protected, by the way, and this may be relevant in a minute. And so, it's just -- it's just a website. I don't even think it was HTTPS. And in it is just a streaming video -- okay, streaming video -- and it's a video of his desk at home that has the physical token sitting on the desk, so that at any time, so he doesn't have the inconvenience of carrying the physical token with him.

>> BOB LORD: That's solid.

>> HUGH THOMPSON: He can go and look at the number that was generated by the token.

>> BOB LORD: The good news is if you deploy Fido security keys, that doesn't happen.

>> HUGH THOMPSON: Oh, yes, that's very true. That's very true. But I guess my meta point is there's -- there is convincing the organization that there are some basic hygienic things that you can do to improve, but there really is some level of awareness that has to happen at the individual.

>> BOB LORD: So -- so, we think about shared responsibility between individuals and the organizations, and what we're starting to see now is more examples of -- of vendors taking ownership of the security outcomes for their customers. And one example of that is Microsoft that released those numbers, thank you, Microsoft, for doing the research to get the numbers because it sounds easy when you put it in a PDF, like here are the numbers, but actually, at scale, doing the analysis to get those numbers is actually non-trivial. So, thank you to Microsoft for doing that. Thank you for publishing numbers that weren't so great. And then a couple weeks ago, they announced that they're going to start changing the defaults for some of these services to be more secure by design. And so, I think that's an example of a company that is taking ownership of security outcomes for their customers because what you're kind of implying is there's a limit to how much we can convince the individual. There's a limit to how much we can convince the organizations to do these things, for whatever reasons, and I have a bunch of theories. But at some point, it doesn't matter. If the organizations that are building and maintaining the systems, if they can take ownership of those -- those controls, everything can change. And we see Salesforce did that. So, they mandated that their customers use MFA. GitHub has also announced that they're on that journey. And they've -- I've been told that they have already secured the first -- or the top 500 packages by downloads. So, the most popular packages are already protected by MFA. So, they're not -- they're not likely to get hacked in the ways that -- that they could have been before. And we all -- you probably sat in multiple sessions about supply chain compromise, like, we want to -- you want to know how that happens? It happens because somebody didn't have MFA turned on. And so, hopefully they all go to Fido MFA because those top 500 are so critical. But no --

>> HUGH THOMPSON: No webcam.

>> BOB LORD: And no webcam, please.

>> HUGH THOMPSON: Webcam proof. Well, no, it is great to see systemically boats rising. Right? So -- so the actual technology vendors are building this in. And I've got to ask you, you know, one of the things that -- that you'd mentioned to me is memory security.

>> BOB LORD: Yes.

>> HUGH THOMPSON: And how important you think that is. Can you expand on that?

>> BOB LORD: I think it's -- so, I'm starting -- so, here's one of my theories. One of my theories is the reason that we still have so much work to do is because we don't talk about the big problems, that we literally just ignore the big problems. And so, I learned last year about a problem that I think is one of the larger problems. And so, you should actually do this little thought experiment right now in the audience. If you think about CVs that come out that describe vulnerabilities that a particular product has.

>> HUGH THOMPSON: There were three new ones this morning, actually.

>> BOB LORD: I don't doubt that.

>> HUGH THOMPSON: No, they're actually -- Dave, Danding, and Wise, by your organization actively exploited.

>> BOB LORD: Actively exploited. So, which is another example of -- of data being mined to be more useful.


>> BOB LORD: So, it's -- it's -- that's -- we need more of that. So -- but if you think about these CVs, some percentage of those are tied to memory safety issues, so buffer overruns, use after free, those kinds of things. In your mind, just think about what percentage you think that is. Obviously, it's not like 5 or 10%. I wouldn't be talking about it. It's also not 90-95% because we'd all be dead. But what -- what number would you pick in your -- in your head to say this is the percentage of CVs that are related to memory safety? And I'll tell you the answer. The answer is about two-thirds. Two-thirds. And this all goes back to the programming languages that -- that people select, and C and C++ are kind of the standard memory unsafe languages. And there are memory safe languages where you don't have these kinds of problems. Rust is one emerging language that is -- that is memory safe. It forces you to engage in hygiene during the programming process, then the compiler won't let you do things that are unsafe. It won't let you do things now that are going to show up as problems later. There's also Go and Python and others. And so, this just struck me as one of those big problems that we just don't talk enough about. And I'm on a little crusade to start talking about some of these things to give people a little bit of perspective so that we can -- we can start to solve these problems. Now, you know, I don't know how much C code is out there. Is it a billion lines of C code? Is it a trillion? It's a lot. Whatever it is, it's a lot. And nobody's going to go through and rewrite all that stuff. But we've seen people from some of the major companies engage in very thoughtful work to find the pieces of code that are likely to get targeted by adversaries, that if they actually got a foothold in here, that would be bad. And they're starting to rewrite pieces of those in languages. The one -- ones that I've heard most recently about is Rust. And so, those are the kinds of things where if you can do 20% of the work and get 80% of the value, or, in some cases, you can do 2% of the work and still get 80% of the value, those are the kinds of things that we need to talk more about. And we should be talking to vendors, like what's your memory safety plan for the next generation of products? We just need to have the conversation. Things will get better if we're just open and honest about what some of these problems are.

>> HUGH THOMPSON: I agree with you. There's so many of those I'd say systemic ones of an infrastructure. In this case, just the way that C allows you to manipulate memory directly, which is a wonderful thing on the one hand.

>> BOB LORD: Yeah.

>> HUGH THOMPSON: It's really easy to make a mistake. A stir copy that, you know, like buffers a little too small, data is a little too big.

>> BOB LORD: Need a little bit of elbow room there.

>> HUGH THOMPSON: A little elbow room.

>> BOB LORD: Yeah.

>> HUGH THOMPSON: But, you know, we have such more even profound problems like BGP and how it was designed and just that you could broadcast a route and have the whole internet thrown through. There is a set of these problems that are so entrenched that, I agree with you, we need to talk about it more.

>> BOB LORD: We need to have a list, like somebody -- somebody needs to come up with a list. I was talking to somebody the other day and I told them I only learned about the memory safety stat a year ago, and he couldn't believe that because -- I said why do you look like that? He said everybody knows this. And I said, I don't think they do. I really don't think that they know this. I think you know this. I think there's a small group of people who are in certain communities and they know this. But if you want to get this -- if you want to solve these problems, you've got to -- you've got to find ways to -- to broadcast the problem. And also, by the way, you're solving it, so it's not like you're just complaining. It's like you have existence proof that if you do these things, you can literally eliminate these risks. And we should be talking more about that. So, your BGP is another one, but we should -- we should have a top ten list, and we should talk more about those.

>> HUGH THOMPSON: So, Bob, we are almost out of time. You promised you would leave us on a high note. I don't know if you remember that promise.

>> BOB LORD: Yeah.

>> HUGH THOMPSON: Maybe you have to just make something up right now.

>> BOB LORD: No. I think --


>> BOB LORD: So, I think there's --

>> HUGH THOMPSON: That's what everyone is expecting.

>> BOB LORD: So, I don't have one specific thing. I do think that, like I said, we've seen vendors start to take ownership of the security outcomes for their customers. We started to see that. I want to see more of them. We've seen -- and I'm a little bit biased here, but I think we've seen tremendous work coming out of CISA. We've seen companies starting to step up in ways that I deeply appreciate. And so, I think -- I think this is kind of the year, I'm hoping that this is the year when we start to see the beginning of that elbow and in the graph, and we're starting to -- to move up and -- and really start to -- to solve some of these systemic problems. I'm pretty optimistic. I wouldn't be continuing to be in this business if I wasn't, you know, ultimately, an optimist.

>> HUGH THOMPSON: I can tell by the title --

>> BOB LORD: Yeah.

>> HUGH THOMPSON: -- that you picked.

>> BOB LORD: Yes, we can do this. We can do big things. We can do hard things.

>> HUGH THOMPSON: Bob, thanks so much for being here. Thanks so much for sharing those experiences.

>> BOB LORD: And, you know, you don't have to do it. You just call me and we can talk about these things.

>> HUGH THOMPSON: No, it's better. It's better as a keynote event.

>> BOB LORD: Okay.

>> HUGH THOMPSON: Thank you, Bob. Thank you all.

>> BOB LORD: Really appreciate it.

Bob Lord


Senior Technical Advisor, Cybersecurity and Infrastructure Security Agency (CISA)

Hugh Thompson


Executive Chairman, RSAC, Program Committee Chair, RSA Conference

Share With Your Community