为OpenStreetMap瓦片服务器添加3弧秒全球DEM图层
2017-03-16 12:44
761 查看
在上一篇文章里,我们试着测试了局部3弧秒(即1/1200度)分辨率的 DEM等高线嵌入OSM图层的效果。下面,来分享一下批量导入全球的DEM数据。
* 经笔者测试,如果一次性导入全球OpenStreetMap PBF(32GB,2017-01)、全球3弧秒 DEM(hgt压缩包转为PBF 约28GB),需要一块 2TB 的 SSD。尽管最终的数据库规模可通过缩减中间表,压缩到300GB,但导入过程中会吃掉约1TB多的空间。若在导入OSM后进行一次收缩,而后导入DEM,则1TB SSD 勉强刚好。*
导入全球数据的原始DEM、转换后PBF、导入后数据库镜像(pg_dump)以及完整虚拟机镜像可从ArchOSM共享地址下载。为了多线程压缩,其使用了7z16.0版本LZMA2算法。解压缩需要512MB空闲内存。我的缓存服务器位置在这里RaspberryPI OSM 缓存服务器,如果渲染服务器没有开机,仅能看到缓存的数据。
viewfinderHgtIndex_3.txt
这个文件里记录了所有的原始数据地址,类似:
中括号里的都是地址。二话不说,复制所有内容到 Wps 电子表格,排序,便收集到了所有的网址。查找替换清除[]后,在下载工具里批量下载即可。
修改好后的viewfinderHgtIndex_3.txt类似下面的样子。
使用命令生成全球PBF
此时,该程序会自动读取已经存在的viewfinderHgtIndex_3.txt,从本地服务器下载、执行。
转换过程需要约1天,最终生成一个 PBF文件,28GB
连南极都有了
* 经笔者测试,如果一次性导入全球OpenStreetMap PBF(32GB,2017-01)、全球3弧秒 DEM(hgt压缩包转为PBF 约28GB),需要一块 2TB 的 SSD。尽管最终的数据库规模可通过缩减中间表,压缩到300GB,但导入过程中会吃掉约1TB多的空间。若在导入OSM后进行一次收缩,而后导入DEM,则1TB SSD 勉强刚好。*
导入全球数据的原始DEM、转换后PBF、导入后数据库镜像(pg_dump)以及完整虚拟机镜像可从ArchOSM共享地址下载。为了多线程压缩,其使用了7z16.0版本LZMA2算法。解压缩需要512MB空闲内存。我的缓存服务器位置在这里RaspberryPI OSM 缓存服务器,如果渲染服务器没有开机,仅能看到缓存的数据。
1.使用下载工具批量下载DEM压缩包
有心人会注意到,在使用phyghtmap下载局部数据时,会得到一个任务列表,位于hgt文件夹下。viewfinderHgtIndex_3.txt
这个文件里记录了所有的原始数据地址,类似:
# VIEW3 index file, VERSION=4 ... [http://viewfinderpanoramas.org/ANTDEM3/01-15.zip] ... S61W091 ...
中括号里的都是地址。二话不说,复制所有内容到 Wps 电子表格,排序,便收集到了所有的网址。查找替换清除[]后,在下载工具里批量下载即可。
2.从本地FTP服务器创建全球高程PBF
下载完毕后,把所有ZIP文件拷贝到一个独立的文件夹,设置为FTP地址。而后,把上述文件viewfinderHgtIndex_3.txt的所有地址替换为本地FTP。这样处理后,下载全球数据就变得非常迅速啦。修改好后的viewfinderHgtIndex_3.txt类似下面的样子。
# VIEW3 index file, VERSION=4 [ftp://192.168.1.10/zip/01-15.zip] ... [ftp://192.168.1.10/zip/16-30.zip] ...
使用命令生成全球PBF
phyghtmap -a -180:-85:180:85 --source=view3 --pbf --max-nodes-per-tile 0
此时,该程序会自动读取已经存在的viewfinderHgtIndex_3.txt,从本地服务器下载、执行。
转换过程需要约1天,最终生成一个 PBF文件,28GB
3.导入全球PBF
留好磁盘空间,而后,进行导入:[archosm@archosm ~]$ osm2pgsql -c -s -S"./contour/contour.style" -C24000 -dcontours --drop --flat-nodes "./cache/nodes/flat_node" ./contour/lon-180.00_180.00lat-85.00_85.00_view3.osm.pbf osm2pgsql version 0.93.0-dev (64 bit id space) Using built-in tag processing pipeline Using projection SRS 3857 (Spherical Mercator) Setting up table: planet_osm_point Setting up table: planet_osm_line Setting up table: planet_osm_polygon Setting up table: planet_osm_roads Allocating memory for dense node cache Allocating dense node cache in one big chunk Allocating memory for sparse node cache Sharing dense sparse Node-cache: cache=24000MB, maxblocks=384000*65536, allocation method=11 Mid: loading persistent node cache from ./cache/nodes/flat_node Allocated space for persistent node cache file Maximum node in persistent node cache: 0 Mid: pgsql, scale=100 cache=24000 Setting up table: planet_osm_nodes Setting up table: planet_osm_ways Setting up table: planet_osm_rels Reading in file: ./contour/lon-180.00_180.00lat-85.00_85.00_view3.osm.pbf Using PBF parser. Processing: Node(10198222k 6803.4k/s) Way(141003k 11.63k/s) Relation(0 0.00/s) parse time: 13621s Node stats: total(10198222219), max(10208222218) in 1499s Way stats: total(141003653), max(151003652) in 12122s Relation stats: total(0), max(0) in 0s Maximum node in persistent node cache: 10208935935 Committing transaction for planet_osm_point Committing transaction for planet_osm_line Committing transaction for planet_osm_polygon Committing transaction for planet_osm_roads Mid: loading persistent node cache from ./cache/nodes/flat_node Maximum node in persistent node cache: 10208935935 Setting up table: planet_osm_nodes Setting up table: planet_osm_ways Setting up table: planet_osm_rels Using built-in tag processing pipeline Mid: loading persistent node cache from ./cache/nodes/flat_node Maximum node in persistent node cache: 10208935935 Setting up table: planet_osm_nodes Setting up table: planet_osm_ways Setting up table: planet_osm_rels Using built-in tag processing pipeline Mid: loading persistent node cache from ./cache/nodes/flat_node Maximum node in persistent node cache: 10208935935 Setting up table: planet_osm_nodes Setting up table: planet_osm_ways Setting up table: planet_osm_rels Using built-in tag processing pipeline Mid: loading persistent node cache from ./cache/nodes/flat_node Maximum node in persistent node cache: 10208935935 Setting up table: planet_osm_nodes Setting up table: planet_osm_ways Setting up table: planet_osm_rels Using built-in tag processing pipeline Going over pending ways... 0 ways are pending Using 4 helper-processes Finished processing 0 ways in 0 s Committing transaction for planet_osm_point Committing transaction for planet_osm_line Committing transaction for planet_osm_polygon Committing transaction for planet_osm_roads Committing transaction for planet_osm_point Committing transaction for planet_osm_line Committing transaction for planet_osm_polygon Committing transaction for planet_osm_roads Committing transaction for planet_osm_point Committing transaction for planet_osm_line Committing transaction for planet_osm_polygon Committing transaction for planet_osm_roads Committing transaction for planet_osm_point Committing transaction for planet_osm_line Committing transaction for planet_osm_polygon Committing transaction for planet_osm_roads Going over pending relations... 0 relations are pending Using 4 helper-processes Finished processing 0 relations in 0 s Committing transaction for planet_osm_point Committing transaction for planet_osm_line Committing transaction for planet_osm_polygon Committing transaction for planet_osm_roads Committing transaction for planet_osm_point Committing transaction for planet_osm_line Committing transaction for planet_osm_polygon Sorting data and creating indexes for planet_osm_polygon Stopping table: planet_osm_nodes Sorting data and creating indexes for planet_osm_roads Sorting data and creating indexes for planet_osm_point Stopping table: planet_osm_ways Stopping table: planet_osm_rels Sorting data and creating indexes for planet_osm_line Stopped table: planet_osm_rels in 0s Stopped table: planet_osm_nodes in 0s Copying planet_osm_polygon to cluster by geometry finished Creating geometry index on planet_osm_polygon Copying planet_osm_roads to cluster by geometry finished Creating geometry index on planet_osm_roads Creating indexes on planet_osm_polygon finished Creating indexes on planet_osm_roads finished Copying planet_osm_point to cluster by geometry finished Creating geometry index on planet_osm_point Creating indexes on planet_osm_point finished All indexes on planet_osm_point created in 0s Completed planet_osm_point All indexes on planet_osm_roads created in 0s Completed planet_osm_roads All indexes on planet_osm_polygon created in 0s Completed planet_osm_polygon Stopped table: planet_osm_ways in 1s Copying planet_osm_line to cluster by geometry finished Creating geometry index on planet_osm_line Creating indexes on planet_osm_line finished All indexes on planet_osm_line created in 6804s Completed planet_osm_line Maximum node in persistent node cache: 10208935935 Maximum node in persistent node cache: 10208935935 Maximum node in persistent node cache: 10208935935 Maximum node in persistent node cache: 10208935935 node cache: stored: 3145729419(30.85%), storage efficiency: 100.00% (dense blocks: 384000, sparse nodes: 2432), hit rate: 30.78% Osm2pgsql took 20429s overall
4 显示效果
显示效果还是很棒的,我们看看南美洲的山脉连南极都有了
相关文章推荐
- 使用Phyghtmap为OpenStreetMap添加 DEM 高程图层
- 使用1角分高程数据为OpenStreetMap服务器添加海洋等深线
- ArchLinux-- OpenStreetMap瓦片服务器的新宿主
- ArchLinux Openstreetmap 瓦片数字高程服务器演示网址变更解释
- OpenLayers 添加OpenStreetMap(OSM)瓦片层示例
- OpenStreetMap/Google/百度/Bing瓦片地图服务(TMS)
- OpenStreetMap/Google/百度/Bing瓦片地图服务(TMS)
- OpenStreetMap/Google/百度/Bing瓦片地图服务(TMS)
- arcgis api for silverlight自己写个图层,实现对OpenStreetMap的访问
- OpenStreetMap(osm) Google 瓦片计算规则探究
- Qt 插件综合编程-基于插件的OpenStreetMap瓦片查看器客户端(1)-墨卡托投影与坐标控制
- OpenStreetMap/Google/百度/Bing瓦片地图服务(TMS)
- OpenStreetMap初探(八)——制作地图瓦片Kosmos及Maperitive使用
- 搭建Openstreetmap地图服务器(Ubuntu 16.04LTS)
- Qt 插件综合编程-基于插件的OpenStreetMap瓦片查看器客户端(5) 小结
- OpenStreetMap/Google/百度/Bing瓦片地图服务(TMS)
- 转 OpenStreetMap初探(八)——制作地图瓦片Kosmos及Maperitive使用
- OpenStreetMap/Google/百度/Bing瓦片地图服务(TMS)
- OpenStreetMap/Google/百度/Bing瓦片地图服务(TMS)