|
|
基于MapReduce的JP算法设计与实现
Design and Implementation of JP Algorithm Based on MapReduce
查看参考文献10篇
|
文摘
|
针对大规模文本聚类分析所面临的海量、高维、稀疏等难题,提出一种基于云计算的海量文本聚类解决方案。选择经典聚类算法Jarvis-Patrick(JP)作为案例,采用云计算平台的MapReduce编程模型对JP聚类算法进行并行化改造,利用搜狗实验室提供的语料库在Hadoop平台上进行实验验证。实验结果表明,JP算法并行化改造可行,且相对于单节点环境,该算法在处理大规模文本数据时具有更好的时间性能。 |
|
其他语种文摘
|
This paper analyzes the prevalent problems such as massiveness, high-dimension and sparse of feature vector of the ordinary algorithms in clustering textual data, then proposes a massive text clustering based on cloud computing technology as a feasible solution. The classical Jarvis-Patrick(JP) algorithm is chosen as a case. It is implemented using MapReduce programming mode and is testified on the cloud computing platform-Hadoop with Sogou corpus provided by Sogou laboratory. Experimental results indicate that the JP algorithm can be paralleled in MapReduce framework and paralled algorithm can handle massive textual data and get a better time performance than single-node environment. |
|
来源
|
计算机工程
,2012,38(24):14-16 【核心库】
|
|
关键词
|
文本挖掘
;
聚类分析
;
文本聚类
;
海量数据
;
云计算
;
并行数据挖掘
|
|
地址
|
国防科学技术大学信息系统与管理学院, 长沙, 410073
|
|
语种
|
中文 |
|
ISSN
|
1000-3428 |
|
学科
|
自动化技术、计算机技术 |
|
文献收藏号
|
CSCD:4726337
|
参考文献 共
10
共1页
|
|
1.
Dean J. MapReduce: Simplified Data Processing on Large Clusters.
Communications of the ACM,2008,51(1):107-113
|
被引
742
次
|
|
|
|
|
2.
江小平. 云计算环境下朴素贝叶斯文本分类算法的实现.
计算机应用,2011,31(9):2551-2554
|
被引
4
次
|
|
|
|
|
3.
邓倩妮. 云计算及其关键技术.
计算机应用,2009,29(9):2562-2567
|
被引
2
次
|
|
|
|
|
4.
陈康. 云计算:系统实例与研究现状.
软件学报,2009,20(5):1337-1348
|
被引
277
次
|
|
|
|
|
5.
Liu Yang. A MapReduce-based Distributed LSI.
Proc. of the 7th International Conference on Fuzzy Systems and Knowledge Discovery,2010
|
被引
1
次
|
|
|
|
|
6.
Jarvis R A. Clustering Using a Similarity Measure Based on Shared Nearest Neighbors.
IEEE Transactions on Computer,1973,22(11):1025-1034
|
被引
29
次
|
|
|
|
|
7.
Venner J.
Pro Hadoop,2009
|
被引
9
次
|
|
|
|
|
8.
Ertoz L. A New Shared Nearest Neighbor Clustering Algorithm and Its Application in Workshop on Clustering High Dimensional Data and Its Applications.
Proc. of the 1st SIAM International Conference on Data Mining,2001
|
被引
1
次
|
|
|
|
|
9.
搜狗实验室.
互联网语料库,2012
|
被引
1
次
|
|
|
|
|
10.
高小平.
Imdict-chinese-analyzer,2012
|
被引
1
次
|
|
|
|
|
|
了解气象卫星更多信息，请访问