您的位置:首页 > 数据库

sh调取sqoop脚本每天增量跑数据 从hadoop到关系型数据库

2015-11-27 09:44 330 查看
sh中调取sqoop每天增量懂啊关系型数据库中,看看吧

#!/bin/bash

source ~/.bashrc
set -x

DAY=`date  -d "$1 days ago" +%y-%m-%d`
DAYI=`date -d "$1 days ago" +%Y%m%d`
APPHOME="/opt/pig_home/bshare_etl/hdetl/app/pv_placement/"
PROG=`basename $0`
MAIL="dba@i-click.com"
EXPORT="/shortdata/pv_placement/15-10-01"
USER="xmo_summarizer"
PWD="YCt452uz"
URL="jdbc:postgresql://10.1.1.230:5432/xmo_dw"

sqoop eval --connect ${URL} --username ${USER} --password ${PWD} -e "delete from bshare_placement where date_i=${DAY}"

sqoop export --connect ${URL} --username ${USER} --password ${PWD} --export-dir ${EXPORT} --table bshare_placement --fields-terminated-by '\001' --input-null-non-string '\\N' --input-null-string '\\N' --columns date_i,adx,city,province,placement_id,pv -- --schema xmo_dw

#sqoop export --connect 'jdbc:postgresql://10.1.1.230:5432/xmo_dw' --username 'mo_summarizer' --password 'YCt452uz' --export-dir '/shortdata/pv_placement/15-10-01' --table bshare_placement --fields-terminated-by '\001' --input-null-non-string '\\N' --input-null-string '\\N' --columns date_i,adx,city,province,placement_id,pv -- --schema xmo_dw
sqoop list-databases --connect "jdbc:postgresql://10.1.1.230:5432/xmo_dw" --username 'xmo_summarizer' --password 'YCt452uz'
内容来自用户分享和网络整理,不保证内容的准确性,如有侵权内容,可联系管理员处理 点击这里给我发消息
标签: