外部表（二）相关实验

实验一，使用sql*loader生成创建外部表的命令

sqlldr有一个参数：external_table
这个参数有3个属性：

not_used：不适用外部表，通过常规路径或直接加载数据，默认值

generate_only：sqlldr并不执行加载，而是生成创建外部表的sql和处理数据的sql，并保存在log文件中。dba修改后可以拿到sql*plus中执行

execute：首先创建外部表，然后通过外部表加载数据

1.创建数据文件

[oracle@secdb1 scripts]$ vi /home/oracle/t.dat

P,James,31,
P,Thomas,22,
E,Pat,38,93645,1122,Engineering,
P,Bill,19,
P,Scott,55,
S,Judy,45,27316,English,
S,Karen,34,80356,History,
E,Karen,61,90056,1323,"Manufa,cturing",
S,Pat,29,98625,Spanish,
S,Cody,22,99743,Math,
P,Ted,43,
E,Judy,44,87616,1544,Accounting,
E,Bob,50,63421,1314,Shipping,
S,Bob,32,67420,Psychology,
E,Cody,33,25143,1002,"Hum,an Resources",

~

2，创建控制文件

[oracle@secdb1 scripts]$ vi /home/oracle/t.ctl

load data
infile '/home/oracle/t.dat'
badfile '/home/oracle/t.bad'
discardfile '/home/oracle/t.dec'
append
into table t
fields terminated by ',' optionally enclosed by '"'
trailing nullcols
(
x1,
x2,
x3,
x4,
x5,
x6
)

3，创建表T

SYS@PROD> conn scott/tiger
Connected.
SCOTT@PROD> create table t(
2 x1 varchar2(20),
3 x2 varchar2(20),
4 x3 varchar2(20),
5 x4 varchar2(20),
6 x5 varchar2(20),
7 x6 varchar2(20)
8 );

Table created.

4，使用sql*loader生成创建外部表的命令

[oracle@secdb1 ~]$ sqlldr scott/tiger control=/home/oracle/t.ctl external_table=GENERATE_ONLY

SQL*Loader: Release 10.2.0.1.0 - Production on Mon Apr 1 18:33:51 2013

Copyright (c) 1982, 2005, Oracle. All rights reserved.

5，在/home/oracle目录下面找同名的日志文件t.log

[oracle@secdb1 ~]$ ll | grep t.log
-rw-r--r-- 1 oracle oinstall 3243 Apr 1 18:34 t.log

查看：（节选创建外部表的一部分）（红色为需要修改的地方）

CREATE TABLE statement for external table:
------------------------------------------------------------------------
CREATE TABLE "SYS_SQLLDR_X_EXT_T"
(
"X1" VARCHAR2(20),
"X2" VARCHAR2(20),
"X3" VARCHAR2(20),
"X4" VARCHAR2(20),
"X5" VARCHAR2(20),
"X6" VARCHAR2(20)
)
ORGANIZATION external
(
TYPE oracle_loader
DEFAULT DIRECTORY SYS_SQLLDR_XT_TMPDIR_00000
ACCESS PARAMETERS
(
RECORDS DELIMITED BY NEWLINE CHARACTERSET US7ASCII
BADFILE 'SYS_SQLLDR_XT_TMPDIR_00000':'t.bad'
DISCARDFILE 'SYS_SQLLDR_XT_TMPDIR_00000':'t.dec'
LOGFILE 't.log_xt'
READSIZE 1048576
FIELDS TERMINATED BY "," OPTIONALLY ENCLOSED BY '"' LDRTRIM
MISSING FIELD VALUES ARE NULL
REJECT ROWS WITH ALL NULL FIELDS
(
"X1" CHAR(255)
TERMINATED BY "," OPTIONALLY ENCLOSED BY '"',
"X2" CHAR(255)
TERMINATED BY "," OPTIONALLY ENCLOSED BY '"',
"X3" CHAR(255)
TERMINATED BY "," OPTIONALLY ENCLOSED BY '"',
"X4" CHAR(255)
TERMINATED BY "," OPTIONALLY ENCLOSED BY '"',
"X5" CHAR(255)
TERMINATED BY "," OPTIONALLY ENCLOSED BY '"',
"X6" CHAR(255)
TERMINATED BY "," OPTIONALLY ENCLOSED BY '"'
)
)
location
(
't.dat'
)
)REJECT LIMIT UNLIMITED；

稍加修改后便能使用创建外部表
创建SYS_SQLLDR_XT_TMPDIR_00000并授权

SYS@PROD> create directory SYS_SQLLDR_XT_TMPDIR_00000 as '/home/oracle';

Directory created.

SYS@PROD> grant read,write on directory SYS_SQLLDR_XT_TMPDIR_00000 to scott;

Grant succeeded.

稍微修改上面的名字就行了

[oracle@secdb1 ~]$ vi ext.sql
#指定表结构
CREATE TABLE ext_t
(
"X1" VARCHAR2(20),
"X2" VARCHAR2(20),
"X3" VARCHAR2(20),
"X4" VARCHAR2(20), //创建表，表的结构。指定表结构
"X5" VARCHAR2(20),
"X6" VARCHAR2(20)
)
ORGANIZATION external
#指定访问的驱动和路径
(
TYPE oracle_loader //指定驱动类型，两种：还有一种oracle_datapump
DEFAULT DIRECTORY SYS_SQLLDR_XT_TMPDIR_00000 //指定数据文件所在路径对应的Driectroy对象
#指定加载参数
ACCESS PARAMETERS //设置数据源文件与表中行之间的映射关系
(
RECORDS DELIMITED BY NEWLINE CHARACTERSET US7ASCII //设置文件中的记录分隔符和字符接
BADFILE 'SYS_SQLLDR_XT_TMPDIR_00000':'t.bad' //设置坏文件的存放路径和文件名
DISCARDFILE 'SYS_SQLLDR_XT_TMPDIR_00000':'t.dec' //设置废弃文件的存放目录和文件名
LOGFILE 't.log_xt' //日志文件名
READSIZE 1048576 //读取日志文件的缓存区大小
FIELDS TERMINATED BY "," OPTIONALLY ENCLOSED BY '"' LDRTRIM //设置文件中的字段分割符
MISSING FIELD VALUES ARE NULL //设置文件中无值字段的处理
REJECT ROWS WITH ALL NULL FIELDS //如果整行都为空时，不进行加载
(
"X1" CHAR(255)
TERMINATED BY "," OPTIONALLY ENCLOSED BY '"',
"X2" CHAR(255)
TERMINATED BY "," OPTIONALLY ENCLOSED BY '"',
"X3" CHAR(255)
TERMINATED BY "," OPTIONALLY ENCLOSED BY '"',
"X4" CHAR(255)
TERMINATED BY "," OPTIONALLY ENCLOSED BY '"',
"X5" CHAR(255)
TERMINATED BY "," OPTIONALLY ENCLOSED BY '"',
"X6" CHAR(255)
TERMINATED BY "," OPTIONALLY ENCLOSED BY '"'
)
)
#指定数据来源
location
(
't.dat'
)
)REJECT LIMIT UNLIMITED；

@/home/oracle/ext.sql

SCOTT@PROD> set lines 200
SCOTT@PROD> select * from ext_t;

X1 X2 X3 X4 X5 X6
-------------------- -------------------- -------------------- -------------------- -------------------- --------------------
P James 31
P Thomas 22
E Pat 38 93645 1122 Engineering
P Bill 19
P Scott 55
S Judy 45 27316 English
S Karen 34 80356 History
E Karen 61 90056 1323 Manufacturing
S Pat 29 98625 Spanish
S Cody 22 99743 Math
P Ted 43

X1 X2 X3 X4 X5 X6
-------------------- -------------------- -------------------- -------------------- -------------------- --------------------
E Judy 44 87616 1544 Accounting
E Bob 50 63421 1314 Shipping
S Bob 32 67420 Psychology
E Cody 33 25143 1002 Human Resources

15 rows selected.

实验二，创建一个外部表，加载外部表日志内容

1，修改生成指定的日志文件

SCOTT@PROD> alter table ext_t access parameters
2 (
3 records delimited by newline
4 skip 6
5 logfile 't.log_xt'
6 fields terminated by ","
7 );

Table altered.
说明：如果要修改access parameter中的参数，该子句中参数必须重新指定

2，创建一个访问日志文件的外部表

create table ext_t_log(lname varchar2(4000))
organization external(
type oracle_loader
default directory SYS_SQLLDR_XT_TMPDIR_00000
access parameters
(
records delimited by newline
nologfile
nobadfile
nodiscardfile
fields terminated by ","
(lname position(1:4000))
)
location('t.log_xt')
);

3.查询

SCOTT@PROD> select * from ext_t_log where rownum<=10;

LNAME
--------------------------------------------------------------------------------

LOG file opened at 04/01/13 18:58:00

KUP-04080: directory object def_dir1 not found


LOG file opened at 04/01/13 19:04:58

KUP-04080: directory object def_dir1 not found

10 rows selected.

对外部表进行修改

第一部分：表结构的修改和普通表的修改方式相同，但是修改后一定要对access parameters子句的相关参数可能也要适当修改

第二部分：访问驱动一般不需要修改，默认目录直接 修改

SCOTT@PROD> alter table ext_t default directory def_dir1;

Table altered.

第三部分：对访问参数的修改，修改必须指定全部参数子句

第四部分：加载路径location，直接修改

SCOTT@PROD> alter table ext_t_log location ('t.ctl');

Table altered.

查询一下

SCOTT@PROD> set pages 999
SCOTT@PROD> select * from ext_t_log;

LNAME
--------------------------------------------------------------------------------
load data
infile '/home/oracle/t.dat'
badfile '/home/oracle/t.bad'
discardfile '/home/oracle/t.dec'
append
into table t
fields terminated by ',' optionally enclosed by '"'
trailing nullcols
(
x1,
x2,
x3,
x4,
x5,
x6
)

16 rows selected.

修改参数reject limit

SCOTT@PROD> alter table ext_t reject limit 10;

Table altered.

查询外部表的信息

包含数据库中外部表参数设置信息

相关视图：dba_external_tables,user_external_tables,all_external_tables

包含外部表所对应的外部数据文件源信息

相关的视图有dba_external_locations，all_external_locations，user_external_locations

SCOTT@PROD> set lines 160
SCOTT@PROD> col table_name for a30
SCOTT@PROD> col location for a20
SCOTT@PROD> col DIRECTORY_OWNER for a30
SCOTT@PROD> col directory_name for a50
SCOTT@PROD> select * from user_external_locations;

TABLE_NAME LOCATION DIRECTORY_OWNER DIRECTORY_NAME
------------------------------ -------------------- ------------------------------ --------------------------------------------------
EMP_LOAD info.dat SYS DEF_DIR1
EXT_T t.dat SYS DEF_DIR1
EXT_T_LOG t.ctl SYS SYS_SQLLDR_XT_TMPDIR_00000

查询表对象的定义，使用函数DBMS_METADATA.GET_DDL实现

SCOTT@PROD> set long 20000
SCOTT@PROD> set pagesize 0
SCOTT@PROD> select DBMS_METADATA.GET_DDL('TABLE','EXT_T') from dual;

CREATE TABLE "SCOTT"."EXT_T"
( "X1" VARCHAR2(20),
"X2" VARCHAR2(20),
"X3" VARCHAR2(20),
"X4" VARCHAR2(20),
"X5" VARCHAR2(20),
"X6" VARCHAR2(20)
)
ORGANIZATION EXTERNAL
( TYPE ORACLE_LOADER
DEFAULT DIRECTORY "DEF_DIR1"
ACCESS PARAMETERS
( records delimited by newline
skip 6
logfile 'ext_t.log'
fields terminated by ","
)
LOCATION
( 't.dat'
)
)
REJECT LIMIT 10

外部表加载多个文件

加载格式相同的多个文件，只要修改location参数即可

如，我们的ext_t_log同时把告警日志文件内容也加载进去

1.拷贝告警日志文件到Directory对应的目录下
2.修改参数location

SCOTT@PROD> alter table ext_t_log location('t.ctl','alert_PROD.log');

Table altered.

查看：

SCOTT@PROD> select * from user_external_locations where table_name='EXT_T_LOG';

TABLE_NAME LOCATION DIRECTORY_OWNER DIRECTORY_NAME
-------------------- -------------------- ------------------------------ --------------------------------------------------
EXT_T_LOG alert_PROD.log SYS SYS_SQLLDR_XT_TMPDIR_00000
EXT_T_LOG t.ctl SYS SYS_SQLLDR_XT_TMPDIR_00000

小知识：

如下插入方式，不写入日志

使用外部表实现数据的加载/卸载

1.卸载
SYS@PROD> create table ext_bigtbl_dp
2 organization external
3 (
4 type oracle_datapump
5 default directory def_dir1
6 location('ext_bigtbl_dp.dmp')
7 )
8 as
9 select * from dba_objects;

Table created.

2，使用DBMS_METADATA.GET_DDL函数生成创建外部表的语句

SYS@PROD> select dbms_metadata.get_ddl('TABLE','EXT_BIGTBL_DP') from dual;

DBMS_METADATA.GET_DDL('TABLE','EXT_BIGTBL_DP')
--------------------------------------------------------------------------------

CREATE TABLE "SYS"."EXT_BIGTBL_DP"
( "OWNER" VARCHAR2(30),
"OBJECT_NAME" VARCHAR2(128),
"SUBOBJECT_NAME" VARCHAR2(30),
"OBJECT_ID" NUMBER,
"DATA_OBJECT_ID" NUMBER,
"OBJECT_TYPE" VARCHAR2(19),
"CREATED" DATE,
"LAST_DDL_TIME" DATE,
"TIMESTAMP" VARCHAR2(19),
"STATUS" VARCHAR2(7),
"TEMPORARY" VARCHAR2(1),
"GENERATED" VARCHAR2(1),
"SECONDARY" VARCHAR2(1)
)
ORGANIZATION EXTERNAL
( TYPE ORACLE_DATAPUMP
DEFAULT DIRECTORY "DEF_DIR1"
LOCATION
( 'ext_bigtbl_dp.dmp'
)
)

执行生成的语句

CREATE TABLE "SCOTT"."EXT_BIGTBL_DP"
( "OWNER" VARCHAR2(30),
"OBJECT_NAME" VARCHAR2(128),
"SUBOBJECT_NAME" VARCHAR2(30),
"OBJECT_ID" NUMBER,
"DATA_OBJECT_ID" NUMBER,
"OBJECT_TYPE" VARCHAR2(19),
"CREATED" DATE,
"LAST_DDL_TIME" DATE,
"TIMESTAMP" VARCHAR2(19),
"STATUS" VARCHAR2(7),
"TEMPORARY" VARCHAR2(1),
"GENERATED" VARCHAR2(1),
"SECONDARY" VARCHAR2(1)
)
ORGANIZATION EXTERNAL
( TYPE ORACLE_DATAPUMP
DEFAULT DIRECTORY "DEF_DIR1"
LOCATION
( 'ext_bigtbl_dp.dmp'
)
);

测试插入速度
开启时间：
SYS@PROD> set timing on

SCOTT@PROD> conn / as sysdba
Connected.
SYS@PROD> create table scott.objects as select * from dba_objects;

Table created.

SYS@PROD> conn scott/tiger
Connected.
SCOTT@PROD> select * from tab;

TNAME TABTYPE CLUSTERID
------------------------------ ------- ----------
DEPT TABLE
EMP TABLE
BONUS TABLE
SALGRADE TABLE
EMP1 TABLE
OBJECTS TABLE
EMP_LOAD TABLE
EXT_T TABLE
T TABLE
EXT_T_LOG TABLE
EXT_BIGTBL_DP TABLE

11 rows selected.

SCOTT@PROD> insert /*+ APPEND*/ into objects select * from ext_bigtbl_dp;

9458 rows created.