Pioneers of a More Data Driven Union

Alongside distinguished speakers Nick Sinai, Senior Advisor at Insight Partners, and Dr. John Elder, Founder of Elder Research, host Christina Ho explores questions about how Artificial Intelligence (AI) is used to help businesses and government agencies solve complex and challenging problems.

Listen to the Podcast

In this episode you will learn:

• What is AI? How has it evolved? [2:19]
• Does AI really live up to its hype? How is it different in industry than in the government? [3:38]
• How significant is the opportunity for AI to solve complex problems for public good? What are the key areas it could benefit? [5:58]
• What are the government’s challenges in leveraging AI to solve problems and produce outcomes? [8:16]
• What are examples of complex government/public good problems that were previously impossible to tackle, but with AI became possible? [11:08]
• What can the federal government do to facilitate progress in AI? [15:37]

Learn more about why we created Pioneers of a More Data Driven Union.

Christina Ho (00:01):

Welcome to Pioneers a More Data Driven Union. The series created by Elder Research to spotlight government leaders leveraging analytics and technology to solve complex problems for public good. Our first series is titled AI for public good and today is our first episode. I’m Christina Ho, your host, and our topic today is what is AI and how is it used in industry and government? For our inaugural episode, we’ve have two distinguished speakers with us, John Elder and Nick Sinai. John is the founder of Elder Research, a data science consulting firm with 25 years of history as a recognized leader in delivering advanced analytics, machine learning, and AI solutions. John was a data scientist before data science was a known discipline. He has authored books that received a book of the year award. John also served for five years on a panel appointed by president Bush to guide national security technology. Thank you for joining me, John.

John Elder (01:26):

Thanks, Christina. It’s an honor.

Christina Ho (01:28):

Nick is a senior advisor at Insight Partners, a leading global venture capital and private equity firm investing in high growth software solution and companies. Nick joined Insight in 2014 from the White House where he was the US Deputy Chief Technology Officer. Among many accomplishments, he led president Obama’s open data initiatives, helped start and grow the presidential innovation fellows program which brings entrepreneurs, innovators, and technologists to government. Nick is also a senior fellow and former adjunct faculty at Harvard Kennedy School. Thank you Nick for joining me.

Nick Sinai (02:16):

Hey Christina, great to see you again.

Christina Ho (02:19):

So AI is a buzz word these days that may mean different things to different people. So I think it’s important for us to first define what it is. So John, what is AI? Can you speak to its evolution briefly?

John Elder (02:37):

So the history is actually kind of a story of a war between two worlds, the sort of digital versus analog, the expert systems and logic programming versus the equations and neural net side of the world; perceptrons versus lisp and the digital side is kind of logic based and deductive reasoning, which is sort of using rules and how systems work versus the equations which are using data and using inductive reasoning and connecting the dots between data points and figuring out how things work with equations. So the successes in recent years is really resulting from a combination, a marriage, of those two warring families. And that’s where you’re getting some of these recent breakthroughs. And that’s what’s really exciting.

Christina Ho (03:38):

Well, speaking like a true data scientist, John, thank you, Nick. So does AI really live up to its hype? And what are you seeing in the industry and how is that different from the government?

Nick Sinai (03:53):

There is a tremendous amount of hype about AI. And I think that has actually been true for a while. We just call it big data a decade ago. So there has been, and probably something before that. So there has been a tremendous amount of hype, but I think it’s warranted because there is tremendous promise in a wide variety of applications. So, if you think about predictive maintenance in the armed services, if you think about fraud and do not pay, and helping make a more efficient government. If you think about better predicting what citizens and residents and veterans need. There are so many opportunities here in government and in the private sector, where I am now working with software companies, machine learning, and artificial intelligence are increasingly part of the product suite. That is, it is not this extra thing that companies or governments are buying necessarily. It is part of software. So just to give you a quick example in cybersecurity, you know, if you’re buying cybersecurity software these days here may be machine learning algorithms that help predict whether a file is malware. So it’s not so much that you need to have the signature of every known piece of bad malware, but there’s actually algorithms that help predict whether a file is a good file or a bad file. And it’s that kind of prediction that you can then say, okay, well, we’re going to block this or quarantine, this file. And that’s just built into a cybersecurity product that is trying to protect the endpoint or protect your network, right? And so that kind of machine learning and artificial intelligence that is part of commercial software products is what I’m seeing, and what I spend a lot of my time with. But there’s also an important piece of where John spends a lot of time, the data scientists of the world, right? Who of course are going to be looking at a lot of data and doing data science and data engineering as well.

Christina Ho (05:58):

How significant would you say is the opportunity for AI to solve some real complex problem? What do you think are some of the areas that could benefit?

John Elder (06:09):

The question always comes up, well, what can you do with artificial intelligence and data science? And the question is similar to saying, what can you do with electricity? You know, it’s good for everything that ails you. And that’s why I made the joke about, you know, this ointment is good for everything. This thing can do anything, you know but let’s start at the top. What is some of the most important problems that you face? And I would say that one of the best matches for the data science or artificial intelligence type of problem is your needle in a haystack problem. You’ve got a trained investigator looking into say fraud, waste, and abuse. And they’re worried about boiling the ocean. They worried about the vast set of things out there that they have to look at. And the beauty is that the machine, once you’ve trained it to look for anomalies can look at literally everything and they can score it for you, and they can rate every single strand of hay in terms of its needliness, and show you the most needle-like strands for you to spend your precious time on, as investigators. So, fraud discovery is one of the most important and powerful applications. Similarly for cleared agencies, insider threat, looking for anomalies, looking for folks that might be betraying the country, betraying secrets, is a very, very powerful application, similar to that, a needle in a haystack type problem. So prioritization of workloads for your precious resource of that trained investigator is a major area. But it’s been used for, you know, discovering new drugs from promising compounds out of the vast infinite set of possible ways of putting compounds together. It’s been used to predict oil and gas wells that are about to freeze up months in advance. It’s been used to identify people who might be in danger of committing harm to themselves through stress in the battlefield, all sorts of mazing applications are out there. They’re basically infinite.

Christina Ho (08:16):

I am seeing quite a lot of applications also in the operational efficiency area in the government. So Nick what do you see are some of the challenges for government to leverage AI to solve that kind of problems and increase its own outcomes?

Nick Sinai (08:36):

So we’re going through a digital transformation in government, and this is not unique, large organizations of any kind, I mean, it’s cliche or pretty standard to talk about digital transformations, but it’s particularly true in large government agencies that are making this transition from paper to digital. And one of the things that I worry about is we don’t want to take paper-based processes and just make them electronic, right? We want to actually think about how these processes can be re-imagined in a digital sphere where we can think about how services might be more personalized. We might have information about you. So take the VA for example, the VA already knows perhaps a lot about a veteran, so they might be able to better predict what services they are eligible for, what they might be interested in all of those kinds of things. And so the ability to rethink how we’re going to deliver services to our veterans, or students, or small businesses is something that I’m really excited about and in the context of this big digital transformation, I think we have this opportunity to rethink service delivery and to use machine learning and AI to do the hard work and make it easier. I mean, we’re used to Google suggesting the end of our searches, or Facebook suggesting who we may know as friends. And there are of course, challenges to that, right? Sometimes Google leads us down a wrong way in a predicted search or something like that, but there’s a lot of value that consumers get every day from AI and machine learning. And so we want to bring that to government service delivery. And the challenge is we just don’t want to be doing these big digital transformations and not be thinking about the opportunities for machine learning and AI in them.

Christina Ho (10:34):

You remind me of a big mistake that I made with Google predictive. One time I took my daughter to go to an amusement park and then put in the address, but didn’t pay attention to the town. And it took us four hours instead of two hours to get to our final destination. And my daughter, at the time who was four, says mom, this is unbelievable. So that’s one story about not paying attention to the predictive technology. Well, now I have some questions for both of you. If you can come up with one complex problem that, you know, before was impossible to tackle, but now with AI in these technologies it became possible. What is that problem?

John Elder (11:29):

You know, humans are super clever and they can do things still that computers can’t. But what the computer advantage has is scale. And so we have some amazing things that we can do, but we can’t do it at the same scale. So we humans can solve some problems and leave them as examples for the computer. Then the computer can study those examples and then scale it out by a thousand fold and look at every case and learn from those examples and then say, okay, I’m going to follow your lead Mr. Expert and I’m going to try to mimic that as best I can, a thousand fold over this scale and then show you some things that take your lessons to heart. And what that enables then, is you take a group that’s using its expertise on the cases it has time for, and then it’s productivity is scaled enormously. So for instance, at the postal service, we did some anti-fraud work and they immediately tripled the amount of gains they were able to get with the same amount of investigative effort. At the IRS we did some anti-fraud work and they factored by a factor of 25 times the amount of fraud, they were able to recover. A factor of 25 times the amount of fraud they were able to recover with the same amount of investigative effort! And in every single case when someone does something like this, what has happened is they’ve been able to expand their mandate. Their productivity has increased so much that the group has been able to attack new problems that they weren’t able to get before. And not only that, the group has been rewarded with a larger mandate, a larger workforce. They’re now a profit center basically. And so they’re rewarded with more budget, more people, more of a mandate. AI hasn’t led to a reduction in people, it’s led to an expansion in the group and an expansion of their task. In every group that we’ve worked with in government and in industry, it’s led to a larger footprint of people and mandate. Just the opposite of what people fear with A taking over our jobs. They’ve created new jobs.

Nick Sinai (13:54):

There’s so many important problems. So maybe I’ll just touch on a few quickly. So in the military the big focus is readiness, right? How do you have the people and how do you have the machines, you know, the ships and planes, and so forth ready for conflict. We don’t want conflict, but we have to have everyone trained for it, right? And so that means that we really have to understand our supply chain when parts are going to fail on a plane or on a ship. And the more that we can anticipate that the more that we can get people ready for whatever they’re trained to do. So machine learning and AI can bring a lot to that side of the house. I spoke a little bit on the benefits side of the house and when you think about it, a lot of government is a case management, right, where there’s a particular or adjudication that government is trying to do. And, you know, we traditionally have had very rules based approaches where human beings have looked at case files, say a veteran, or an immigrant, or someone and tried to apply a set of rules. Well, to John’s point machines can help us with this and we can also predict if there’s a challenge in the application, right, if there’s a problem in the application, let’s go earlier. Why have a veteran wait several years to tell them, Hey, there’s a problem in this appeal. Let’s predict that this is going to be a problem and get it fixed earlier. That can help people get the benefits that they’ve earned and deserve.

Christina Ho (15:37):

Yeah, that’s great. So here’s the last question we have. What can the federal government do to facilitate progress in AI? Why don’t you go first this time Nick?

Nick Sinai (15:54):

Yeah, I have a few things. I think we tend to have this vision of AI and machine learning as this special thing that only a few special people can do. Right. And so we celebrate the data scientists and with full love and respect, John they’re important, great people, but what I want to do is make sure that we can democratize analytics across government. And so that really means how do we broaden access to data inside of government and outside of government? You know, I was very focused on president Obama’s open data initiatives. So how can we make it easier for government officials? And that could be an information analyst. It could be a data engineer, and yes, it could be a data scientist. It could be someone who’s not trained in the data field, right?

Nick Sinai (16:46):

How can they have access to the relevant government data, and then how can they have access to the tools? And then rapidly up-skill where they, if they want to learn a little Python or R or something over the weekend and kind of work on a little problem that they have, you know, they don’t have to go to central IT, they don’t have to go to their agency, chief data officers, right? I’m passionate about how do we democratize access to data inside of agencies while of course putting the proper safeguards around privacy and national security. And then how do we democratize access to tools? I think those two things will get us away from the kind of centralized: we’re going to create one data lake, we’re going to do some very fancy data science on that data, etc.

Nick Sinai (17:39):

And those, those approaches probably haven’t shown as much ROI and mission impact with what we care about in the public sector. So to the extent that we can democratize because every function in government is going to be its own special snowflake, right? So the regulatory state, for example John was talking about the ability to kind of predict things, right? So as we’re doing enforcement in the regulatory state, being able to better predict where the inspectors should go so that we can keep our air clean and our waters clean and those kinds of things. But that’s the kind of thing that the regulators are going to have to do and they don’t have large data science teams necessarily. And so the more that we can empower them and upskill them, and yes, there’ll be buying a great commercial software as part of that, but they’ll also be applying data engineering and data science to those problems basis and being able to have more effective inspectors in our regulatory regime as just one small example.

John Elder (18:51):

That’s a great thing. You mentioned opening data up and Christina, you and Nick did great work to that Open Data Initiative. And maybe you can say another word about that before we close, but taking good care of data is the best thing that the government could do and opening it up to folks and allowing a thousand analysts to look at things and use their creativity to see things and make suggestions. That’s awesome because data has value. And then acting on the analytics insights that come from that, you know, is still today, when people find wonderful things in data and have great ideas even when companies are or agencies pay for modeling still only about a third of those actually get acted on. So it’s very, very hard to change even in industry.

John Elder (19:45):

Sometimes the industry does a little better than governments. Sometimes government does better, but acting on and making changes and making decisions based on data is still a change management problem, which is very hard to do. And so that will and decision to do the rational thing and to make decisions that are based on learning from data, even when the models and the analysts show that it’s the right thing to do it often it’s hard to make a bureaucracy or an organization change the way it does things. So we’re studying how to do that. We’ve, we’ve gotten much better at it. We have more of a 90% success at getting our clients to actually make decisions based on our models and it’s taken us a long time, we’ve been at 25 years, so it’s taken us a long time to get to that level. So we’ve learned some secrets about how to, how to get people involved at every stage, so that they’re comfortable with the decision making, but it’s more than just solving the problem and handing the solution over. You have to work with people all along the way to get them comfortable with making decisions in a new way, but that’s when you do that it works.

Nick Sinai (21:10):

I’m sorry to jump in here but one thing that I’m super passionate about that you just prompted in this is this idea of a briefing on live data. And Christina, maybe you might appreciate this from your tenure in government too, is there’s too much briefing on static data and static PowerPoints, right? And to the extent we can overplay dashboards. I think there’s a danger of being so dashboard focused that we’re not actually talking about how the data is being used because often oftentimes how it’s used beyond dashboards is equally or more important.

Nick Sinai (21:50):

But I do think that one way to start changing the culture is briefing on live dashboards rather than static power PowerPoints, because that’s kind of, that was the currency when I served and I’m hopeful that government is increasingly moving to live data briefings.

John Elder (22:10):

And I’ll just add one thing, a story. We are story people. We love to hear people’s stories. And, you know, a person’s story is one data point. That’s one data point. There’s a cloud of stories and a model is like a super story, and so the model should be even more powerful than a story, but it’s really not to us. People were used to stories. So we need to find the super story and then tell stories that exemplify the super story, if you will. So there’s an art to conveying the message that is the truth in the data and it’s really wonderful when you get that live data and you get people learning and deciding, and the light bulbs are going off. And it’s a very exciting thing when you’ve seen that happen. And a huge progress can be made in a very short time when you have a room where that’s happening.

Christina Ho (23:13):

When I was leading the implementation of the Data Act I spoke about a simple formula data plus use equals the value, and as an open data advocate myself the Data Act was about unlocking the data, setting the data free so that more people can use it. Using it requires people, like data engineers and data scientists, techniques, and software solutions, and together, then we can deliver the value to solve these problems because we are living in a time that we have very limited resources and the more we could apply technology to help us be more efficient, the better. So thank you so much for joining me today. And I really appreciate all of the contributions you guys do in your role, and we need more of that. I would thank you. Thank you for joining me for pioneers of a more data-driven union.