Back بيانات ضخمة Arabic বিগ ডাটা Assamese Böyük verilənlər Azerbaijani Вялікія даныя Byelorussian Big data Bulgarian বড় উপাত্ত Bengali/Bangla Velika količina podataka BS Dades massives Catalan زلدراوە CKB Velká data Czech

Big data

This article is about large collections of data. For the band, see Big Data (band). For the practice of buying and selling of personal and consumer data, see Surveillance capitalism.

Non-linear growth of digital global information-storage capacity and the waning of analog storage[1][needs update]

Big data primarily refers to data sets that are too large or complex to be dealt with by traditional data-processing software. Data with many entries (rows) offer greater statistical power, while data with higher complexity (more attributes or columns) may lead to a higher false discovery rate.[2] Though used sometimes loosely partly due to a lack of formal definition, the best interpretation is that it is a large body of information that cannot be comprehended when used in small amounts only.[3]

Big data analysis challenges include capturing data, data storage, data analysis, search, sharing, transfer, visualization, querying, updating, information privacy, and data source. Big data was originally associated with three key concepts: volume, variety, and velocity.[4] The analysis of big data presents challenges in sampling, and thus previously allowing for only observations and sampling. Thus a fourth concept, veracity, refers to the quality or insightfulness of the data.[5] Without sufficient investment in expertise for big data veracity, the volume and variety of data can produce costs and risks that exceed an organization's capacity to create and capture value from big data.[6]

Current usage of the term big data tends to refer to the use of predictive analytics, user behavior analytics, or certain other advanced data analytics methods that extract value from big data, and seldom to a particular size of data set. "There is little doubt that the quantities of data now available are indeed large, but that's not the most relevant characteristic of this new data ecosystem."[7] Analysis of data sets can find new correlations to "spot business trends, prevent diseases, combat crime and so on".[8] Scientists, business executives, medical practitioners, advertising and governments alike regularly meet difficulties with large data-sets in areas including Internet searches, fintech, healthcare analytics, geographic information systems, urban informatics, and business informatics. Scientists encounter limitations in e-Science work, including meteorology, genomics,[9] connectomics, complex physics simulations, biology, and environmental research.[10]

The size and number of available data sets have grown rapidly as data is collected by devices such as mobile devices, cheap and numerous information-sensing Internet of things devices, aerial (remote sensing) equipment, software logs, cameras, microphones, radio-frequency identification (RFID) readers and wireless sensor networks.[11][12] The world's technological per-capita capacity to store information has roughly doubled every 40 months since the 1980s;[13] as of 2012, every day 2.5 exabytes (2.17×260 bytes) of data are generated.[14] Based on an IDC report prediction, the global data volume was predicted to grow exponentially from 4.4 zettabytes to 44 zettabytes between 2013 and 2020. By 2025, IDC predicts there will be 163 zettabytes of data.[15] According to IDC, global spending on big data and business analytics (BDA) solutions is estimated to reach $215.7 billion in 2021.[16][17] While Statista report, the global big data market is forecasted to grow to $103 billion by 2027.[18] In 2011 McKinsey & Company reported, if US healthcare were to use big data creatively and effectively to drive efficiency and quality, the sector could create more than $300 billion in value every year.[19] In the developed economies of Europe, government administrators could save more than €100 billion ($149 billion) in operational efficiency improvements alone by using big data.[19] And users of services enabled by personal-location data could capture $600 billion in consumer surplus.[19] One question for large enterprises is determining who should own big-data initiatives that affect the entire organization.[20]

Relational database management systems and desktop statistical software packages used to visualize data often have difficulty processing and analyzing big data. The processing and analysis of big data may require "massively parallel software running on tens, hundreds, or even thousands of servers".[21] What qualifies as "big data" varies depending on the capabilities of those analyzing it and their tools. Furthermore, expanding capabilities make big data a moving target. "For some organizations, facing hundreds of gigabytes of data for the first time may trigger a need to reconsider data management options. For others, it may take tens or hundreds of terabytes before data size becomes a significant consideration."[22]

  1. ^ Hilbert, Martin; López, Priscila (2011). "The World's Technological Capacity to Store, Communicate, and Compute Information". Science. 332 (6025): 60–65. Bibcode:2011Sci...332...60H. doi:10.1126/science.1200970. PMID 21310967. S2CID 206531385. Archived from the original on 14 April 2016. Retrieved 13 April 2016.
  2. ^ Breur, Tom (July 2016). "Statistical Power Analysis and the contemporary "crisis" in social sciences". Journal of Marketing Analytics. 4 (2–3). London, England: Palgrave Macmillan: 61–65. doi:10.1057/s41270-016-0001-3. ISSN 2050-3318.
  3. ^ Mahdavi-Damghani, Babak (2019). Data-Driven Models & Mathematical Finance: Apposition or Opposition? (DPhil thesis). Oxford, England: University of Oxford. p. 21. SSRN 3521933.
  4. ^ McAfee, Andrew; Brynjolfsson, Erik (1 October 2012). "Big data: the management revolution". Harvard Business Review. 90 (10): 60–66, 68, 128. ISSN 0017-8012. PMID 23074865.
  5. ^ Cite error: The named reference :0 was invoked but never defined (see the help page).
  6. ^ Cappa, Francesco; Oriani, Raffaele; Peruffo, Enzo; McCarthy, Ian (2021). "Big Data for Creating and Capturing Value in the Digitalized Environment: Unpacking the Effects of Volume, Variety, and Veracity on Firm Performance". Journal of Product Innovation Management. 38 (1): 49–67. doi:10.1111/jpim.12545. ISSN 0737-6782. S2CID 225209179.
  7. ^ boyd, dana; Crawford, Kate (21 September 2011). "Six Provocations for Big Data". Social Science Research Network: A Decade in Internet Time: Symposium on the Dynamics of the Internet and Society. doi:10.2139/ssrn.1926431. S2CID 148610111. Archived from the original on 28 February 2020. Retrieved 12 July 2019.
  8. ^ Cite error: The named reference Economist was invoked but never defined (see the help page).
  9. ^ "Community cleverness required". Nature. 455 (7209): 1. September 2008. Bibcode:2008Natur.455....1.. doi:10.1038/455001a. PMID 18769385.
  10. ^ Reichman OJ, Jones MB, Schildhauer MP (February 2011). "Challenges and opportunities of open data in ecology". Science. 331 (6018): 703–5. Bibcode:2011Sci...331..703R. doi:10.1126/science.1197962. PMID 21311007. S2CID 22686503. Archived from the original on 19 October 2020. Retrieved 12 July 2019.
  11. ^ Hellerstein, Joe (9 November 2008). "Parallel Programming in the Age of Big Data". Gigaom Blog. Archived from the original on 7 October 2012. Retrieved 21 April 2010.
  12. ^ Segaran, Toby; Hammerbacher, Jeff (2009). Beautiful Data: The Stories Behind Elegant Data Solutions. O'Reilly Media. p. 257. ISBN 978-0-596-15711-1. Archived from the original on 12 May 2016. Retrieved 31 December 2015.
  13. ^ Hilbert M, López P (April 2011). "The world's technological capacity to store, communicate, and compute information" (PDF). Science. 332 (6025): 60–5. Bibcode:2011Sci...332...60H. doi:10.1126/science.1200970. PMID 21310967. S2CID 206531385. Archived (PDF) from the original on 19 August 2019. Retrieved 11 May 2019.
  14. ^ "IBM What is big data? – Bringing big data to the enterprise". ibm.com. Archived from the original on 24 August 2013. Retrieved 26 August 2013.
  15. ^ Reinsel, David; Gantz, John; Rydning, John (13 April 2017). "Data Age 2025: The Evolution of Data to Life-Critical" (PDF). seagate.com. Framingham, MA, US: International Data Corporation. Archived (PDF) from the original on 8 December 2017. Retrieved 2 November 2017.
  16. ^ "Global Spending on Big Data and Analytics Solutions Will Reach $215.7 Billion in 2021, According to a New IDC Spending Guide". Archived from the original on 23 July 2022. Retrieved 31 July 2022.
  17. ^ "Big data and business analytics revenue 2022".
  18. ^ "Global big data industry market size 2011-2027".
  19. ^ a b c Big data: The next frontier for innovation, competition, and productivity McKinsey Global Institute May 2011
  20. ^ Oracle and FSN, "Mastering Big Data: CFO Strategies to Transform Insight into Opportunity" Archived 4 August 2013 at the Wayback Machine, December 2012
  21. ^ Jacobs, A. (6 July 2009). "The Pathologies of Big Data". ACMQueue. Archived from the original on 8 December 2015. Retrieved 21 April 2010.
  22. ^ Magoulas, Roger; Lorica, Ben (February 2009). "Introduction to Big Data". Release 2.0 (11). Sebastopol, CA: O'Reilly Media. Archived from the original on 2 November 2021. Retrieved 26 February 2021.

Developed by StudentB