Joiner transformation(2) 优化
2016-01-05 14:48
169 查看
Joiner transformation 使用注意点:
1)使用场景: 不同数据源( 异构源)
2)Join type: Normal Join, Master Outer Join, Detail Outer Join or Full Outer Join.
3) 用行数少的重复记录少的表为 master, 反之为detail
4) 对输入项做排序
5) 理解 blocking transformation,cache
minimizing the number of disk IOs. you can see good performance when worked with large data sets.
Steps to follow for configuring the sorted input option
Sort the master and detail source either by using the source qualifier transformation or sorter transformation.
Sort both the source on the ports to be used in join condition either in ascending or descending order.
Specify the Sorted Input option in the joiner transformation properties tab.
Unsorted Joiner Transformation
In case of unsorted joiner transformation, the integration service first reads all the master rows before it reads the detail rows. The integration service blocks the detail source while it caches the all the master rows. Once it reads all the master
rows, then it unblocks the detail source and reads the details rows.
Sorted Joiner Transformation
Blocking logic may or may not possible in case of sorted joiner transformation. The integration service uses blocking logic if it can do so without blocking all sources in the target load order group. Otherwise, it does not use blocking logic.
If possible, perform joins in a database. Performing joins in a database is faster than performing joins in a session.
You can improve the session performance by configuring the Sorted Input option in the joiner transformation properties tab.
Specify the source with fewer rows and with fewer duplicate keys as the master and the other source as detail.
1)使用场景: 不同数据源( 异构源)
2)Join type: Normal Join, Master Outer Join, Detail Outer Join or Full Outer Join.
3) 用行数少的重复记录少的表为 master, 反之为detail
4) 对输入项做排序
5) 理解 blocking transformation,cache
Sorted Input
Use the sorted input option in the joiner properties tab when both the master and detail are sorted on the ports specified in the join condition. You can improve the performance by using the sorted input option as the integration service performs the join byminimizing the number of disk IOs. you can see good performance when worked with large data sets.
Steps to follow for configuring the sorted input option
Sort the master and detail source either by using the source qualifier transformation or sorter transformation.
Sort both the source on the ports to be used in join condition either in ascending or descending order.
Specify the Sorted Input option in the joiner transformation properties tab.
Why joiner transformation is called as blocking transformation
The integration service blocks and unblocks the source data depending on whether the joiner transformation is configured for sorted input or not.Unsorted Joiner Transformation
In case of unsorted joiner transformation, the integration service first reads all the master rows before it reads the detail rows. The integration service blocks the detail source while it caches the all the master rows. Once it reads all the master
rows, then it unblocks the detail source and reads the details rows.
Sorted Joiner Transformation
Blocking logic may or may not possible in case of sorted joiner transformation. The integration service uses blocking logic if it can do so without blocking all sources in the target load order group. Otherwise, it does not use blocking logic.
Joiner Transformation Performance Improve Tips
To improve the performance of a joiner transformation follow the below tipsIf possible, perform joins in a database. Performing joins in a database is faster than performing joins in a session.
You can improve the session performance by configuring the Sorted Input option in the joiner transformation properties tab.
Specify the source with fewer rows and with fewer duplicate keys as the master and the other source as detail.
相关文章推荐
- Swoole:重新定义PHP
- 越长大越孤单
- hibernate 在tomcat7.X 下配置mysql数据源
- VS2012工程移值到VS2010
- Java [Leetcode 231]Power of Two
- cocos2d-x里的关于C++的笔记
- 使用shareperference实现缓存cache功能
- java回调
- tomcat 启动脚本
- oracle修改序列
- 框架笔记----数据字典Dictionary
- 黑马程序员————java基础之网络编程
- Unity3D教程:编辑器编程
- 数据结构03--线性表
- XStream 用法详解 java 类与 XML 互换
- 使用websocket做视频直播
- CSS基础—样式
- 100%会用到的angularjs的知识点【新手可mark】
- java多文件上传plupload控件实现多图片上传(一)
- 养成良好的编程习惯