基于网络爬虫的地名数据库维护方法
Method of Toponym Database Updating Based on Web Crawler
查看参考文献21篇
|
文摘
|
目前,我国地名数据库建设存在大、中颗粒度地名集中,小颗粒度地名较为缺乏,地名资料陈旧、时效性较低,简称、别名等非标准地名信息和地名的相对位置信息缺失等问题。而地名数据库的更新维护工作主要通过人工测绘手段完成,存在周期长、成本高、效率低等缺点。针对这一问题,本文以现有地名数据库和空间关系词汇为基础,基于Google搜索引擎服务,提出一种以网页资源为数据源,利用网络爬虫技术和地名识别技术,进行地名数据库更新维护的方法。首先,设计以地名为主题的网络爬虫,实现非结构化的网页数据中海量空间敏感网页文本的主动获取;然后,采用HTML DOM技术解析空间敏感网页并应用CRF地名识别模型自动识别网页文本中地名;最后,设计相关算法进行网页文本中地名信息的自动解析,实现新地名和地名空间位置信息的获取,进行地名数据库的更新维护。以"南京师范大学仙林宾馆+西北"为空间检索实例,验证了此方法的可行性。 |
|
其他语种文摘
|
Generally,toponym database provides description information on place names and its spatial location and feature type.It provides basic information for national administration,economic development,domestic and foreign exchanges,etc.It is a basis for public place name services,particularly for Location-Based-Service(LBS)with a growing demand.Therefore,a toponym database with complete and timely place name information is a premise and guarantee for efficient LBS services.However,currently,there are some problems about place names in our national toponym database.Most of the place names are with a big particle size,and small particle sized and non-standard place names are in shortage,and there are no relative position descriptions of place names in toponym database.Moreover,toponym database updating is based on manual surveying with disadvantages of long cycle,high cost,low efficiency and time consuming.In this paper,a new method for toponym database updating is explored on the technology combination of search engine,web crawler and place name recognition.Firstly,a mass of space-sensitive web pages are obtained by a web crawler which is based on Google search engine and a spatial search subject of "place name" or "place name + spatial relation terms".Secondly,after analysis of web pages with a DOM tree method,place name recognition is completed based on Conditional Random Fields(CRF)recognition model.Finally,automatic spatial location interpretation of place names is completed from candidate web texts which include new place names and spatial location information of place names.This paper also presents a case study with a spatial search subject of "Nanjing Normal University,Xianlin hotel + northwest".The experiment result shows that this method is feasible and effective.However,timely and accurately locating of place names in web pages are in challenge,because publishing time of web pages and change time of place names driven by events in web pages are not considered in this paper.This may result in potential lag of place name information and can’t ensure the completeness and consistency of toponym database.In recent years,public participation internet maps can provide accurate and real-time place name source,especially coordinate information,such as GoogleMap,GoogleEarth,OpenStreetMap,etc.Our future work will focus on time attribute interpretation of place names from web pages and obtaining of place names as well as their coordinates from internet maps.Moreover,an integration of place names from different data sources will provide a more effective toponym database updating. |
|
来源
|
地球信息科学学报
,2011,13(4):492-499 【核心库】
|
|
关键词
|
地名数据库
;
网络爬虫
;
地名识别
;
主题相关性
|
|
地址
|
南京师范大学, 虚拟地理环境教育部重点实验室, 南京, 210046
|
|
语种
|
中文 |
|
文献类型
|
研究性论文 |
|
ISSN
|
1560-8999 |
|
学科
|
自动化技术、计算机技术 |
|
基金
|
国家自然科学基金项目
|
|
文献收藏号
|
CSCD:4300637
|
参考文献 共
21
共2页
|
|
1.
Goodchild M F. Introduction to Digital Gazetteer Research.
Geographical Information Science,2008,22(10):1039-1044
|
被引
14
次
|
|
|
|
|
2.
张雪英. 地理命名实体分类体系的设计与应用分析.
地球信息科学学报,2010,12(2):220-227
|
被引
7
次
|
|
|
|
|
3.
陈钻. 基于XML的无线位置服务地理信息服务器的实现.
地球信息科学,2004,6(4):100-104
|
被引
1
次
|
|
|
|
|
4.
.
http:∥www.alexandria.ucsb.edu/adl/
|
被引
1
次
|
|
|
|
|
5.
.
http:∥nhd.usgs.gov/gnis.html
|
被引
1
次
|
|
|
|
|
6.
.
http:∥www.ga.gov.au/place-name/
|
被引
1
次
|
|
|
|
|
7.
狄琳. 全国1:25万地名数据库的设计与建立.
测绘通报,2010,10:32-33
|
被引
1
次
|
|
|
|
|
8.
陈春华. 1∶5万地名数据库到1:1万地名数据库转换的研究与开发.
测绘通报,2006,5:71-72
|
被引
1
次
|
|
|
|
|
9.
张保钢. 北京市地名数据库的维护更新.
北京测绘,2010,3:28-30
|
被引
1
次
|
|
|
|
|
10.
Palkowsky B. A New Approach to Information Discovery-Geography Really Does Matter.
Proceedings of the SPE Annual Technical Conference and Exhibition,2005
|
被引
4
次
|
|
|
|
|
11.
Hill L L. Core Elements of Digital Gazetteers:Place Names,Categories,and Footprints.
Research and Advanced Technology for Digital Libraries,2000:280-290
|
被引
2
次
|
|
|
|
|
12.
李金良. 汉语地名时空信息的一体化表达.
地理与地理信息科学,2010,26(6):6-10
|
被引
5
次
|
|
|
|
|
13.
陈丛丛.
主题爬虫搜索策略研究,2009
|
被引
1
次
|
|
|
|
|
14.
李勇. 主题搜索引擎中网络爬虫的搜索策略研究.
计算机工程与科学,2008,30(3):4-6
|
被引
8
次
|
|
|
|
|
15.
陈财森. 基于搜索引擎调用的主题搜索设计与实现.
计算机工程与设计,2008,29(21):5627-5629
|
被引
1
次
|
|
|
|
|
16.
Diligenti M. Focused Crawling Using Context Graphs.
Proceedings of the 26th International Conference on Very Large Data-Bases,2000:527-534
|
被引
1
次
|
|
|
|
|
17.
刘秉权. 基于结构树解析的网页正文抽取方法.
黑龙江省计算机学会2007年学术交流年会,2007:14-17
|
被引
1
次
|
|
|
|
|
18.
周俊生. 自然语言信息抽取中的机器学习方法研究.
计算机科学,2005,32(3):186-199
|
被引
3
次
|
|
|
|
|
19.
张小衡. 中文机构名称的识别与分析.
中文信息学报,1997,11(4):21-32
|
被引
27
次
|
|
|
|
|
20.
王志强.
基于条件随机域的中文命名实体识别研究,2006
|
被引
4
次
|
|
|
|
|
了解气象卫星更多信息，请访问