“But again, I don’t care. Why? Because the failures are so obvious compared to those of most algorithms. Dead people by the side of the road constitute public tragedies. They make headlines. They damage companies’ reputations and market valuations. This creates inherent and continuous pressure on the data scientists who build the algorithms to get it right. It’s self-regulating nirvana, to be honest. I don’t care because the self-driving car companies have to care for me.

By contrast, companies that own and deploy other algorithms – algorithms that decide who gets a job, who gets fired, who gets a credit card and with what interest rate, who pays what for car or life insurance – have shockingly little incentive to care.

The problem stems from the subtlety of most algorithmic failures. Nobody, especially not the people being assessed, will ever know exactly why they didn’t get that job or that credit card. The code is proprietary. It’s typically not well understood, even by the people who build it. There’s no system of appeal and often no feedback to improve decision-making over time. The failures could be getting worse and we wouldn’t know it.

Source: Don’t Worry About the Ethics of Self-Driving Cars – Bloomberg