基于NoSQL的FITS文件头元数据存储和查询研究
Research on metadata storage and querying of FITS file based on NoSQL
查看参考文献28篇
|
文摘
|
随着大型天文望远镜的投入使用,观测台站正面临PB量级的海量数据存储、快速检索难题;同时由于在数据检索中起着关键作用的FITS文件头的可变性,导致难以使用传统的关系型数据库来建立可适应这种变化需求的非结构化数据模型。针对这个难题,提出了使用NoSQL对天文上广泛使用的FITS文件头中所包含的可变元数据信息进行存储和查询;讨论了关系型数据模型存储可变FITS文件头的不足;分析了NoSQL存储可变FITS头元数据信息的可行性;使用形式化的关系型代数对这种存储查询方式进行了一般化的讨论。通过具体查询实例验证了该方案在存储天文可变FITS文件头的有效性和可行性。 |
|
其他语种文摘
|
With a large telescope in use,observing stations are facing problems of PB-level data storage and data retrieval.Because of the variability of the FITS file header,it is difficult to utilize the traditional relational database to store and retrieve the unstructured data efficiently.To address these problems,this paper proposed using NoSQL to store and query the metadata of the FITS file header.It discussed the shortcomings of using relational data model to store the variable header of a FITS file.It described the process of data storage and query,analyzed the feasibility of using NoSQL to store the variable FITS header,and further employed the method of relational algebra to express the process formally.A specific query instance proves the effectiveness and feasibility of using NoSQL to store the variable astronomical FITS header. |
|
来源
|
计算机应用研究
,2015,32(2):461-465 【扩展库】
|
|
关键词
|
天文数据存储
;
元数据
;
海量数据查询
;
非关系型数据库
|
|
地址
|
1.
中国科学院云南天文台, 云南省计算机技术应用重点实验室, 昆明, 650011
2.
昆明理工大学, 云南省计算机技术应用重点实验室, 昆明, 650500
|
|
语种
|
中文 |
|
文献类型
|
研究性论文 |
|
ISSN
|
1001-3695 |
|
学科
|
自动化技术、计算机技术 |
|
基金
|
国家自然科学基金-联合基金资助项目
;
国家自然科学基金资助项目
;
云南省应用基础研究计划重点项目
|
|
文献收藏号
|
CSCD:5349740
|
参考文献 共
28
共2页
|
|
1.
Szalay A S. Designing and mining multi-terabyte astronomy archives: the sloan digital sky survey.
ACM SIGMOD Record,2000,29(2):451-462
|
CSCD被引
1
次
|
|
|
|
|
2.
Hey T.
The data deluge: an e-science perspective,2003:809-824
|
CSCD被引
1
次
|
|
|
|
|
3.
Kaiser N. Moore's law takes on the universe; new astronomy with giga-pixel imagers and peta-byte data archives.
Proc of Aerospace Conference,2009:1-2
|
CSCD被引
1
次
|
|
|
|
|
4.
Beasley A J. Astronomy: the United States must rejoin the SKA.
Nature,2012,489(7416):363
|
CSCD被引
1
次
|
|
|
|
|
5.
Gray J. Scientific data management in the coming decade.
ACM SIGMOD Record,2005,34(4):34-41
|
CSCD被引
7
次
|
|
|
|
|
6.
Berriman G B. How will astronomy archives survive the data tsunami?.
Communications of the ACM,2011,54(12):52-56
|
CSCD被引
2
次
|
|
|
|
|
7.
Stonebraker M. SQL databases v. NoSQL databases.
Communications of the ACM,2010,53(4):10-11
|
CSCD被引
26
次
|
|
|
|
|
8.
Goodrich B D. Gathering headers in a distributed environment.
Astronomical Telescopes + Instrumentation,2008:70192N-70192N-10
|
CSCD被引
1
次
|
|
|
|
|
9.
崔辰州. FITS数据文件的检索和访问.
天文研究与技术,2008,5(2):116-123
|
CSCD被引
12
次
|
|
|
|
|
10.
Kitaeff V V. SkuareView: clientserver framework for accessing extremely large radio astronomy image data.
Proc of Workshop on High-Performance Computing for Astronomy Date,2012:25-32
|
CSCD被引
1
次
|
|
|
|
|
11.
Becla J. Designing a multi-petabyte database for LSST.
Astronomical Telescopes and Instrumentation,2006:62700R-62700R-8
|
CSCD被引
1
次
|
|
|
|
|
12.
Murphy T. Data storage,processing, and visualization for the Australia telescope compact array.
Publications of the Astronomical Society of Australia,2006,23(1):25-32
|
CSCD被引
3
次
|
|
|
|
|
13.
Gosink L. HDF5-FastQuery: accelerating complex queries on HDF datasets using fast bitmap indices.
Proc of the 18th International Conference on Scientific and Statistical Database Management,2006:149-158
|
CSCD被引
1
次
|
|
|
|
|
14.
Chou J. Parallel index and query for large scale data analysis.
Proc of International Conference for High Performance Computing,Networking,Storage and Analysis,2011:1-11
|
CSCD被引
1
次
|
|
|
|
|
15.
Fu B. Indexing and fast near-matching of billions of astronomical objects.
Proc of the 4th Workshop on Interfaces and Architecture for Scientific Data Storage,2012
|
CSCD被引
1
次
|
|
|
|
|
16.
Baruffolo A. R-trees for astronomical data indexing.
Astronomical Data Analysis Software and Systems VIII,1999:375
|
CSCD被引
1
次
|
|
|
|
|
17.
Chilingarian I. PostgreSQL: the suitable DBMS solution for astronomy and astrophysics.
Astronomical Data Analysis Software and Systems ,2004:225
|
CSCD被引
1
次
|
|
|
|
|
18.
Pence W D. Definition of the flexible image transport system (FITS),version 3. 0.
Astronomy and Astrophysics,2010,524:Article No. A42
|
CSCD被引
7
次
|
|
|
|
|
19.
Strauch C.
NoSQL databases,2012
|
CSCD被引
4
次
|
|
|
|
|
20.
Leavitt N. Will NoSQL databases live up to their promise?.
Computer,2010,43(2):12-14
|
CSCD被引
12
次
|
|
|
|
|
了解气象卫星更多信息，请访问