Pentagon Problems: What Happens When Military Software Becomes Too Advanced?

James Hasik

Security,

A problem we did not see coming? 

Psibernetix, it seems, has built an artificial intelligence smarter than a fighter pilot. As I mentioned here at the beginning of the month, the company hatched at the University of Cincinnati has developed software for a Raspberry Pi machine that has defeated at least one retired USAF fighter pilot in simulated combat. Yes, there’s something inherently scary about that. But before the RAAF and every other air force get too excited about drones that could turn on us, we might consider what “inelegant messes” dwell at the heart of the software suites that enable their capabilities. Samuel Arbesman, scientist-in-residence at Lux Capital and a senior fellow at the University of Colorado, argues in his new book Overcomplicated: Technology at the Limits of Comprehension (Current, 2016) that engineers’ endlessly adding to software without understanding the underlying code is “making the world indecipherable.” The question is what can be done about it.