100 Best Practices in Big Data Security and Privacy

By Ryan Bergsma, Research Intern, CSA

‘Big data’ refers to the massive amounts of digital information companies and governments collect about human beings and our environment. Experts anticipate that the amount of data generated will double every two years, from 2500 exabytes in 2012 to 40,000 exabytes in 2020. Security and privacy issues are magnified by the volume, variety, and velocity of big data. As big data expands through streaming cloud technology, traditional security mechanisms tailored to secure small-scale, static data on firewalled and semi-isolated networks offer inadequate protection.

Recently our Big Data Working Group led by Sreeranga Rajan and Daisuke Mashim released the “Big Data Security and Privacy Handbook: 100 Best Practices in Big Data Security and Privacy,” outlining the 100 best practices that should be followed by any big data service provider to fortify their infrastructure. The handbook presents 10 compelling solutions for each of the top 10 challenges in big data security and privacy, which the working group previously identified in the 2012 CSA document titled “Top Ten Big Data Security and Privacy Challenges.”

New Security Challenges

It is not merely the existence of large amounts of data that creates new security challenges. In reality, big data has been collected and utilized for several decades. The current uses of big data are novel because organizations of all sizes now have access to the information and the means to collect it. In the past, big data was limited to very large users such as governments and big enterprises that could afford to create and own the infrastructure necessary for hosting and mining large amounts of data. These infrastructures were typically proprietary and isolated from general networks. Today, big data is cheaply and easily accessible to organizations of all sizes through public cloud infrastructure.

Software infrastructure developers can easily leverage thousands of computing nodes to perform data-parallel computing. Combined with the ability to buy computing power on-demand from public cloud providers, the adoption of big data mining methodologies is greatly accelerated. Large-scale cloud infrastructures, diversity of data sources and formats, the streaming nature of data acquisition and high-volume, inter-cloud migration all play a role in the creation of unique security vulnerabilities.

Big Data Best Practices

Now that we have enormous amounts of data and know the security and privacy risks it presents, what can enterprises do to secure their information? This CSA handbook provides a roster of 100 best practices, ranging from typical cybersecurity measures, such as authentication and access control, to state-of-the-art cryptographic technologies. In each section, CSA presents 10 solutions for each of the top 10 major challenges in big data security and privacy. Each section addresses what is the best practice, why these security measures are needed and should be followed and how they can be implemented.

Read the entire “Big Data Security and Privacy Handbook: 100 Best Practices in Big Data Security and Privacy” handbook. Learn more about CSA.