大数据系统综述
A survey on big data systems
查看参考文献240篇
|
文摘
|
随着科学、技术和工程的迅猛发展,近20年来,许多领域(如光学观测、光学监控、健康医护、传感器、用户数据、互联网和金融公司以及供应链系统)都产生了海量的数据(更恰当的描述或许是"无限"的数据,例如,在光学观测和监控等应用中,数据都是源源不断而来的,形成了"数据灾难"),大数据的概念也随之再次引起重视.与传统的数据相比,除了大容量等表象特点,大数据还具有其他独特的特点,例如大数据通常是无结构的,并且需要得到实时分析,因此大数据的发展需要全新的体系架构,用于处理大规模数据的获取、传输、存储和分析.本文对大数据分析平台进行了尽可能详尽的文献调研,首先介绍了大数据的基本定义和大数据面临的一些挑战;然后提出了大数据系统框架,将大数据系统分解为数据生成、数据获取、数据存储和数据分析等4个模块,这4个模块也构成了大数据价值链;随后讨论了学术界和工业界中和大数据相关的方法和机制;最后介绍了典型的大数据系统基准和大数据的一些科学问题.本文意图为非专业读者提供大数据的全景知识,也为高级读者定制自己的大数据解决方案提供辅助思想,希望能够对大数据相关的科技和工程人员起到一些参考作用. |
|
其他语种文摘
|
With the development of the latest technologies, a large amount of data is generated from various domains (such as optical observation and control, healthcare, sensors, user-generated data, Internet and financial companies, supply chain systems, etc.) during the last two decades. (A more appropriate description could be"infinite" data, e.g., in the application of optical observation and control, data are continuously generated, creating a data disaster.) The term of big data is coined to capture the profound meaning of this emerging trend. Compared with traditional data, big data exhibits some unique characteristics besides the sheer volume, such as commonly un-structured data and more real-time analysis requirements. The development of big data calls for new system architectures for data storage and large-scale data processing mechanisms. In this paper, we present a literature survey of big data analytics. Firstly, the definition of big data and big data challenges are presented. Secondly, a systematic framework to decompose big data system into four sequential modules, namely data generation, data acquisition, data storage, and data analytics, which form the value chain for big data, is proposed. A detailed survey of numerous approaches and mechanisms related to each module, from research and industry communities is discussed. Finally, some evaluation benchmarks and potential scientific problems in big data systems are outlined. |
|
来源
|
中国科学. 信息科学
,2015,45(1):1-44 【核心库】
|
|
关键词
|
大数据
;
数据获取
;
数据存储
;
数据处理
;
数据分析
|
|
地址
|
1.
中国科学院西安光学精密机械研究所光学影像分析与学习中心(OPTIMAL), 西安, 710119
2.
电子科技大学计算机科学与工程学院, 成都, 611731
|
|
语种
|
中文 |
|
文献类型
|
综述型 |
|
ISSN
|
1674-7267 |
|
学科
|
自动化技术、计算机技术 |
|
基金
|
国家自然科学基金
|
|
文献收藏号
|
CSCD:5374891
|
参考文献 共
240
共12页
|
|
1.
Gantz J. Extracting value from chaos.
IDC iView,2011:1-12
|
被引
16
次
|
|
|
|
|
2.
Manyika J.
Big data: the next frontier for innovation, competition, and productivity,2011
|
被引
40
次
|
|
|
|
|
3.
Cukier K. Data, data everywhere.
Economist,2010,394:3-16
|
被引
1
次
|
|
|
|
|
4.
Lohr S. The age of big data.
New York Times,2012:11
|
被引
4
次
|
|
|
|
|
5.
Noguchi Y. Following digital breadcrumbs to big data gold.
National Public Radio,2011
|
被引
2
次
|
|
|
|
|
6.
Noguchi Y. The search for analysts to make sense of big data.
National Public Radio,2011
|
被引
2
次
|
|
|
|
|
7.
White House.
Fact Sheet: Big Data Across the Federal Government,2012
|
被引
1
次
|
|
|
|
|
8.
Howard J H. Scale and performance in a distributed file system.
ACM Trans Comput Syst,1988,6:51-81
|
被引
30
次
|
|
|
|
|
9.
Cattell R. Scalable SQL and NoSQL data stores.
SIGMOD Rec,2011,39:12-27
|
被引
34
次
|
|
|
|
|
10.
Dean J. MapReduce: simplified data processing on large clusters.
Commun ACM,2008,51:107-113
|
被引
742
次
|
|
|
|
|
11.
White T.
Hadoop: the definitive guide,2012
|
被引
23
次
|
|
|
|
|
12.
Zikopoulos P.
Understanding big data: analytics for enterprise class hadoop and streaming data,2011
|
被引
10
次
|
|
|
|
|
13.
Meijer E. The world according to LINQ.
Commun ACM,2011,54:45-51
|
被引
3
次
|
|
|
|
|
14.
Laney D.
3D Data Management: Controlling Data Volume, Velocity and Variety.Gartner,2001
|
被引
1
次
|
|
|
|
|
15.
Cooper M.
Tackling Big Data.NIST,2012
|
被引
1
次
|
|
|
|
|
16.
Team O R. Big Data Now: Current Perspectives from O'Reilly Radar.
O'Reilly Media,2011
|
被引
1
次
|
|
|
|
|
17.
Marche S. Is Facebook making us lonely.
Atlantic,2012,309:60-69
|
被引
1
次
|
|
|
|
|
18.
Borkar V R. Big data platforms: what's next?.
XRDS: Crossroads, The ACM Magazine for Students,2012,19:44-49
|
被引
1
次
|
|
|
|
|
19.
Borkar V. Inside Big Data management: ogres, onions, or parfaits?.
Proceedings of the 15th International Conference on Extending Database Technology,2012:3-14
|
被引
2
次
|
|
|
|
|
20.
Dewitt D J. Parallel database systems: the future of high performance database systems.
Commun ACM,1992,35:85-98
|
被引
22
次
|
|
|
|
|
了解气象卫星更多信息，请访问