Ethics of Big Data: How Developers Play a Role in Data-Handling Practices

When I first started writing code, software application developers were called programmers. In the boardroom, the term programmer wasn't a term of endearment, and the term geek was a derogatory insult. Today, developers typically play a role in business operations and geek T-shirts are worn as a matter of pride. Times have changed.

Many big data experts will tell you that 90 percent of data has been captured in the past five years. Andrew Brust, founder of Blue Badge Insights, is one of those experts. He told me, "The data has always been there, but we used to just monitor it and then throw it away. Now we save and analyze it because the processing power and the storage are effective and affordable."

Ethical Considerations Surrounding Big Data

Obviously, tools, plumbing, and platforms have changed, too. They have become so powerful that software, once solely capable of being benign CRUD applications, now frequently crosses into ethical territory. This forces developers to make new decisions and develop new discussions. If a company has the capability to determine when one of its customers is pregnant, does this mean that it should act on this knowledge for the company's interests? If my company, InterKnowlogy, can use the Microsoft Kinect for Windows to watch and uniquely identify and track a consumer's shopping habits in several stores to create unique advertising and couponing strategies for that specific consumer, does that mean we should? Is that type of development creepy, or is it powerful?

It's the pure power of today's software, which is why I enjoyed a new book written by Kord Davis called Ethics of Big Data (O'Reilly, 2012). I read a lot of technical publications, but I've never read something so thorough and provocative on technology problems that aren't purely technical. This is what motivated me to reach out to Kord, and I was privileged to have a long chat on the phone with him.

A key message from the book is that that a balance needs to be maintained between the benefits of the innovations that big data provides and the risks of unintended consequences. Developers are some of the most knowledgeable people about how to maintain that balance because they are on the front lines of understanding the technical details of both the value of the innovations and the potential risks.

I quickly asked Kord, "This book has to be a labor of love. I mean, you're not going to get rich on this book, right?" Kord laughed and agreed. He stated that a key motivation for writing the book was to apply the tools that philosophy and ethical inquiry provide us to evolve and shape technology into tools that can help us live better and easier lives.

In his book, Kord Davis is points out that the word ethics is a loaded word that can sometimes imply judgment. "It's a highly abstract topic with very real-world implications," he told me. For instance, Kord mentions in his book that two security researchers announced that iPhones were regularly recording the position of each device to a hidden file. Although Apple readily acknowledged that the claim was true, the resulting uproar made it clear that it was the method by which the file was generated and stored that caused security concerns. The decision to use that technological method had clear and direct ethical consequences in the real world.

And it's not only big corporations that have the capability and resources to take big data to the next level of ethical structure. Kord noted that anyone can host a Hadoop cluster on commodity hardware in his or her own living rooms.

With this book, Kord sets the stage for ongoing discussion about ethics and big data. Kord points out that there are several grass roots movements and well-organized equivalents from the White House that are attempting to build a digital bill of rights to protect the consumer.

One key concept from Kord's book is that an organization's data-handling practices can actually be thought of as a physical manifestation of its values. Kord drives home the message that developers are a critical part of explicit ethical inquiry into big data handling practices. As the people who actually build the systems to instantiate an organization's processes, another key message from the book is that systems, processes, and actions must align with commonly held values if we're to meet the ethical challenges that big data presents.

Resources for Tackling Big Data & Ethical Challenges

In addition to getting your hands on companion website, which will be a dynamic work in progress that highlights articles that cover this ongoing problem.

Kord also runs a workshop that helps organizations develop their commonly held values and identify practical data-handling practices. Finally, the workshop helps organizations understand if those values and practices are in alignment. If they aren't, then the workshop provides organizations with an action plan to get them going in the right direction. You can keep up with Kord through his Twitter handle, @KDavis, and his blog.