基于MPI的大规模遥感影像金字塔并行构建方法
An MPI-based Parallel Pyramid Building Algorithm for Large-scale RS Image
查看参考文献22篇
|
文摘
|
影像金字塔是实现影像数据多分辨率组织的重要方式,是提高影像可视化性能的有效手段。传统串行金字塔构建算法,对大规模影像数据的构建性能已无法满足遥感影像快速浏览的预处理需求。故此,其成为一个亟待解决的问题,而利用多核、多节点的高性能集群计算环境和并行机制是一个重要的技术途径。本文在共享外存的高性能集群环境下,提出使用消息传递接口(MPI)的金字塔并行构建算法,对构建遥感影像金字塔过程中的重采样与I/O过程进行并行处理,大大缩短了遥感影像金字塔构建时间。实验结果表明:(1)该算法比传统串行构建方法的加速效果明显,对于单波段遥感影像,其加速效果可达到GDAL的5倍以上,而对于多波段遥感影像,加速效果可达到GDAL的2倍以上;(2)遥感影像数据量越大,并行构建算法加速效果越显著,对于大规模的遥感影像,本文提出的金字塔并行构建算法的速度可达到GDAL的10倍左右。 |
|
其他语种文摘
|
With the rapid development of remote sensing (RS) image acquisition and processing technology, the spatial resolution and temporal resolution of RS image have been greatly improved. For large-scale RS image, traditional sequential pyramid building algorithms have been found difficult to meet the quick browsing requirements. Technologies and facilities of high performance computing have become more and more feasible to researchers. Taking the advantages of multi-core, multi-node cluster computing environments and parallel processing mechanism is turning to be an inevitable trend. Some of recent works explore the efficiency and flexibility of parallel pyramid building methods. However, these methods all have deficiencies. For example, GPU-based parallel method is hardware-aware, the improvement of its performance is limited on a single node, and the system architecture will be too complicated when applied in a cluster environment. Whereas the distributed cluster-based method requires the data to be distributed to be stored in different nodes, and the complete pyramid file needs to be merged, which is excessive time consuming. Therefore, using the high-performance disk-shared cluster is an alternative mechanism for achieving the parallel building pyramid of large-scale RS images. In this paper, we proposed a parallel algorithm based on Message Passing Interface (MPI). Based on this, the whole pyramid building task is decomposed into several subtasks. The result of each subtask can be written to the same pyramid file simultaneously by incorporating MPI/IO. The algorithm can greatly improve the performance of pyramid building through parallel resampling and parallel I/O. Specially, with regard to the multi-band pyramid file stored in BIP format, a parallel I/O strategy using file view was proposed to improve the performance of parallel writing. Experimental results show that our algorithm has better acceleration effect compared to the sequential method, and there is a positive correlation between the acceleration effect and the image size. For large remote sensing images (in our case it is 46G), the performance of our parallel algorithm can be approximately 10 times faster than GDAL. |
|
来源
|
地球信息科学学报
,2015,17(5):515-522 【核心库】
|
|
DOI
|
10.3724/SP.J.1047.2015.00515
|
|
关键词
|
遥感影像金字塔
;
消息传递接口
;
并行
;
集群
|
|
地址
|
国防科学技术大学电子科学与工程学院, 长沙, 410073
|
|
语种
|
中文 |
|
文献类型
|
研究性论文 |
|
ISSN
|
1560-8999 |
|
学科
|
测绘学;自动化技术、计算机技术 |
|
基金
|
国家自然科学基金
;
国家863计划
|
|
文献收藏号
|
CSCD:5431083
|
参考文献 共
22
共2页
|
|
1.
苏光日. GDAL在地理国情监测项目中的应用.
测绘与空间地理信息,2014,37(1):86-88
|
被引
2
次
|
|
|
|
|
2.
Qin C Z. How to apply the Geospatial Data Abstraction Library (GDAL) properly to parallel geospatial raster I/O?.
Transactions in GIS,2014,18(6):950-957
|
被引
4
次
|
|
|
|
|
3.
Zhang S Q. Research on performances of parallel programming models based on chip multi-processor.
2012 IEEE International Conference on Oxide Materials for Electronic Engineering (OMEE),2012:632-635
|
被引
2
次
|
|
|
|
|
4.
Wang S. CyberGIS:Blueprint for integrated and scalable geospatial software ecosystems.
International Journal of Geographical Information Science,2013,27(11):2119-2121
|
被引
5
次
|
|
|
|
|
5.
Yang C. Spatial cloud computing:how can the geospatial sciences use and help shape cloud computing?.
International Journal of Digital Earth,2011,4(4):305-329
|
被引
16
次
|
|
|
|
|
6.
康俊锋. 基于GPU加速的遥感影像金字塔创建算法及其在土地遥感影像管理中的应用.
浙江大学学报:理学版,2011,38(6):695-700
|
被引
5
次
|
|
|
|
|
7.
Chen G D. Research on orthorectification of remote sensing images using GPU-CPU cooperative processing.
2011 International Symposium on Image and Data Fusion (ISIDF),2011:1-4
|
被引
1
次
|
|
|
|
|
8.
刘小利. 结合MapReduce和HBase的遥感图像并行分布式查询.
地理与地理信息科学,2014,30(5):26-28
|
被引
4
次
|
|
|
|
|
9.
刘义. 利用MapReduce进行批量遥感影像瓦片金字塔构建.
武汉大学学报:信息科学版,2013(3):278-282
|
被引
16
次
|
|
|
|
|
10.
Ma Y. A new framework of cluster-based parallel processing system for high-performance geo-computing.
IEEE International Geoscience and Remote Sensing Symposium, 4,2009:IV49-IV52
|
被引
1
次
|
|
|
|
|
11.
Guo W. A Multi-granularity parallel model for unified remote sensing image processing WebServices.
Transactions in GIS,2012,16(6):845-866
|
被引
4
次
|
|
|
|
|
12.
Guan X. A parallel framework for processing massive spatial data with a split-and-merge paradigm.
Transactions in GIS,2012,16(6):829-843
|
被引
1
次
|
|
|
|
|
13.
Thakur R. Optimizing noncontiguous accesses in MPI-IO.
Parallel Computing,2002,28(1):83-105
|
被引
4
次
|
|
|
|
|
14.
Didelot S. Improving MPI communication overlap with collaborative polling.
Computing,2014,96(4):263-278
|
被引
2
次
|
|
|
|
|
15.
申焕. 基于MPI的海量遥感影像并行处理技术探析.
全球定位系统,2013,37(6):73-76
|
被引
1
次
|
|
|
|
|
16.
欧阳柳. 地理栅格数据的并行访问方法研究.
计算机科学,2012,39(11):116-121
|
被引
3
次
|
|
|
|
|
17.
Porwal S. Performance evaluation of various resampling techniques on IRS imagery.
2014 Seventh International Conference on Contemporary Computing (IC3),2014:489-494
|
被引
4
次
|
|
|
|
|
18.
Parker J A. Comparison of interpolating methods for image resampling.
IEEE Transactions on Medical Imaging,1983,2(1):31-39
|
被引
38
次
|
|
|
|
|
19.
Chen Y. Improving parallel I/O performance with data layout awareness.
2010 IEEE International Conference on Cluster Computing (CLUSTER),2010:302-311
|
被引
1
次
|
|
|
|
|
20.
Ferhatosmanoglu H. Optimal data-space partitioning of spatial data for parallel I/O.
Distributed and Parallel Databases,2005,17(1):75-101
|
被引
2
次
|
|
|
|
|
了解气象卫星更多信息，请访问