The quote is long to provide context, but the bold part is what worries me. Ruh-roh.
A recent study prepared for the Office of Naval Research by a team from the California Polytechnic State University said that robot ethics had not received the attention it deserved because of a "rush to market" mentality and the "common misconception" that robots will do only what they have been programmed to do.
"Unfortunately, such a belief is sorely outdated, harking back to the time when computers were simpler and their programs could be written and understood by a single person," the study says. "Now programs with millions of lines of code are written by teams of programmers, none of whom knows the entire program; hence, no individual can predict the effect of a given command with absolute certainty since portions of programs may interact in unexpected, untested ways."
It’s not bad enough that we’re giving the robots guns, we’re not entirely sure what they’ll do when we tell them to shoot.