The Era of Big Data: It Was the Best of Times, It Was the Worst of Times


In the context of the rapid development of the Internet, the era of big data affects each of us. Big data is closely related to our life. Because of the powerful capabilities of data, it has gradually become one of the valuable resources in the 21st century. While we are enjoying the convenience by big data, the potential dangers brought by those are always there (Flew, 2021).

(Image Source:

What is Big Data?

The formation of big data is due to the continuous development of science and technology, and a large amount of data with various types is becoming a new issue. Big data not only includes data information but also has the technical ability to process this complex information (Niebel, Rasel & Viete, 2019). The display of these technical capabilities usually comes from the challenges of three dimensions:(1)volume, (2)variety, and (3)velocity (Laney, 2001). These datasets show the enormous amount of data, a wide variety of data straight from diverse sources and data processing speed. It is because of the continuous advancement of computing power, storage capacity and software that big data technology has also kept improving.

Even in just one minute, the vast amount of data presented by the internet is beyond what the human brain can process. For instance, in 2022, Domo shared the date showing each minute of Internet, Google recorded 5.9 million searches, Twitter had 347.2K tweets shared, and there were 66K photos uploaded to Instagram. Due to the complex variety of data types, traditional data management tools are not capable of processing such data.

(Image Source: Data Never Sleeps 10.0, Domo)

However, technically, big data cannot be simply defined by the 3V since it’s not a stable platform but a dynamic technology that changes with the defining technologies. Big data is the result of the rapid development of technology and databases, which continuously update and merge with each other (Strydom & Buckley, 2020).

The Best of Times: Prediction and Innovation

The development of big data has brought us the “best of times”, and its most dazzling aspect is the predictive function and innovation ability of big data. According to Alazab & Gupta (2022), big data plays a crucial role in the healthcare field. By studying past datasets, the predictive function of big data can prevent similar situations or curb the situation getting worse. At the same time, big data can also develop new standards and regulations or analyze correct solutions based on past data and experience to deal with similar situations. For example, the continuous evolution of the COVID-19 virus poses a great threat to human health, but healthcare big data can better understand patients’ needs through comprehensive analysis of patients. At the same time, based on the settings for different age groups, it can also analyze the population groups most susceptible to coronavirus infection in different age groups. Therefore, when a pandemic like the coronavirus occurs again, people can use the predictive function of big data to take measures to deal with the widespread spread of the virus in advance or provide more reasonable methods to help people affected by the virus. This will help healthcare providers and even governments greatly improve their emergency response efficiency, benefiting more people from the predictive function of big data.

(Image Source:

Big data has made our lives better by providing predictive capabilities and innovative solutions. In the healthcare industry, big data is crucial. It can predict and prevent similar situations from happening by analyzing past data sets. Additionally, big data can create new standards, regulations, and solutions based on previous experiences. For instance, the continuous evolution of the corona virus is a significant threat to human health. However, healthcare big data can analyze patients’ needs and the population groups most susceptible to coronavirus infection, based on age groups. If a similar pandemic occurs, big data’s predictive function can help people take necessary measures to prevent the virus’s spread, benefitting healthcare providers and governments, and improving emergency response efficiency.

The predictive function of big data allows us to better guide the future based on experience. However, big data not only teaches us to do “right” things based on experience. It also brings us many “new” solutions. Of course, this cannot be simply understood as big data will innovate. We need to analyze and apply big data to achieve innovation. As Niebel, Rasel, and Viete (2019) mentioned, big data analysis is not only an important determinant for companies to become product innovators, but also an important factor for successful market innovation. Especially in manufacturing and service industries, the constantly updating experience of big data can help companies correct errors and make wiser choices. For example, by monitoring different information based on past consumption patterns and social network data, companies can improve new products or address existing product defects. This is beneficial for personalized services or innovative development of companies. For example, Tesla has achieved a series of enterprise goals such as cost reduction, discovering market opportunities, meeting customer needs, developing new products, and improving cars through big data. Connected vehicles can collect more data and based on the near-real-time information collected, Tesla can often predict and handle issues before they occur (Abdoullaev, 2021).

(Image Source:

The Worst of Times: Privacy Regulation, Data Security, and Data Inequality

Big data can be seen as a shining light that illuminates the path to the future, but there is also hidden dark side that we cannot see. Issues such as privacy regulation, data security, and data inequality constantly raise people’s concerns about the development of big data.

Privacy regulation is one of the most concerning issues for people. According to Saetra (2021), the invasion of privacy by utilizing big data is a significant reason why big data poses a threat to a free society. On the one hand, the information collected through data collection may disturb to personal freedom, and some private companies or enterprises may potentially influence people’s choices using data and achieve their own interests. According to Jiang (2022), big data includes data records such as atmospheric data, call detail records, internet search indexes, medical records, military surveillance, sensor network data, and social network data. Based on the sensitivity of these data, they often lead to many privacy issues. For example, social network data usually records people’s names, ages, interests, email addresses, etc. When some people are not willing to disclose information excessively, some social platforms may restrict their use of the platform’s functions. In China, a video website called Bilibili requires users to log in to the website to select high-definition videos. Logging in usually requires registering an account, which generally requires collecting a phone number or email address.

(Image Source: Screenshot from website

Another example of concern is in the development of smart cities, where the privacy notice and consent model of “notice and consent” cannot be realized because there are many sensors in the city and the sources and future processing of the data are unclear. Perhaps notification to the public can be made for some data sources, but it is almost impossible to ask people to confirm their consent to collect their large amount of data in front of all sensors, let alone the reuse and future processing of datasets (Löfgren & Webster, 2020).

However, if a large amount of data is collected without informed consent, or if data is stolen or lost, it will be another blow to our lives, as data security is threatened, and these records may be misused and violate personal intentions. Do you often wonder why you receive spam emails or scam phone calls? How do these people get your information? Data can be transferred in countless ways that individuals cannot fully understand, and because data is persistent, it can have a negative impact on individuals’ future (Saetra, 2021). In 2022, there was a large-scale customer data leak at Medibank, which involved millions of current and former Medibank, ahm, and international student account holders. Medibank first confirmed the data leak in a statement on October 13, 2022, and released information that may have been leaked, including customer names, dates of birth, phone numbers and email addresses, some Medicare card numbers, some passport numbers and health claim data (service provider names and locations, where customers received certain medical services, and codes related to diagnosis and management procedures) and some close relative contact details and healthcare provider details for My Home Hospital patients, including names, provider numbers and addresses. Of course, Medibank will also pay a corresponding price. It may be ordered to pay compensation to affected customers to compensate for their losses and damages caused by data leaks, including pain and shame caused by data leaks. However, for customers who have suffered harm, the mental damage is continuous and difficult to reverse.

(Image Source:

Similarly, data inequality is also a bad aspect. Löfgren and Webster(2020) believe that some ways of collecting data may cause many problems, such as the motivation for data collection on social platforms under the business model, or the quality problems caused by bias in data collection. For example, companies and businesses are only interested in customer data that benefits themselves. Or the data collection quality for affluent and non-affluent areas on map platforms is vastly different. Information in affluent areas is often completer and more detailed, while information in non-affluent areas is not. The regulatory issues caused by these problems are also very complex, and it is difficult to solve them perfectly under the current legal and regulatory framework. For individuals, big data may provide different results and recommendations based on their personal characteristics such as socioeconomic status, race, gender, and sexual orientation. For example, in the recruitment process, when certain unfair factors such as a person’s race or family background are considered in credit scoring, this may result in a lower score for that individual. Such inequality may affect people’s lives and career development and may also lead to more injustice and social instability.

How to find a good way to accept it?

Is big data an angel or a demon? Perhaps it is neither. Big data itself is neither good nor bad, it is simply a tool and a resource, much like fire, which can cause destruction and danger but can also be used for cooking food and providing warmth and light. By utilizing the advantages of big data, we can bring convenience and quality to our lives, such as improving productivity in business, enhancing scientific research, and improving healthcare. In addressing the drawbacks of big data, such as privacy infringement, data insecurity, and unfairness, we can continually improve the regulation of data usage through the law and strengthen our awareness of data protection to further reduce the bias caused by data inequality, ensuring that the use of big data has more positive effects on our society. In the long history of humanity, fire has often been symbolic of the light of hope. And how will we describe big data in the future?


Alazab, M., & Gupta, M. (2022). Trust, Security and Privacy for Big Data. In CRC Press eBooks. Informa.

Data Never Sleeps 10.0 | Domo. (n.d.-b).

Flew, T. (2021). Regulating Platforms. John Wiley & Sons.

Jiang, R., Bouridane, A., Li, C., Crookes, D., Boussakta, S., Hao, F., & Edirisinghe, E. A. (2022). Big Data Privacy and Security in Smart Cities. Springer Nature.

Laney, D. (2001). “3D Data Management: Controlling Data Volume, Velocity and Variety.” Application Delivery Strategies, 6 February 2001. META Group. (n.d.). Medibank Data Breach 2022 Investigation. Maurice Blackburn.

Niebel, T., Rasel, F., & Viete, S. (2019). BIG data – BIG gains? Understanding the link between big data analytics and innovation. Economics of Innovation and New Technology, 28(3), 296–316.

Saetra, H. S. (2021). Big Data’s Threat to Liberty: Surveillance, Nudging, and the Curation of Information. Academic Press.

Strydom, M., & Buckley, S. (2020). AI and big data’s potential for disruptive innovation (M. Strydom & S. Buckley, Eds.). IGI Global.

Be the first to comment

Leave a Reply