optimizer hints
2016-06-23 11:47
288 查看
In version MySQL 5.7.7 Oracle presented a new promising feature: optimizer hints. However it did not publish any documentation about the hints. The only note which I found in the user manual about the hints is:
It is now possible to provide hints to the optimizer by including
There are also three worklogs: WL #3996, WL #8016 and WL #8017. But they describe the general concept and do not have much information about which optimizations can be used and how. More light on this provided by slide 59 from Øystein Grøvlen’s session at Percona Live. But that’s all: no “official” full list of possible optimizations, no use cases… nothing.
I tried to sort it out myself.
My first finding is the fact that slide #59 really lists six of seven possible index hints. Confirmation for this exists in one of two new files under sql directory of MySQL source tree, created for this new feature.
Looking into file sql/opt_hints.cc we can find out what these optimizations give not much choice: either enable or disable.
A choice for the way to include hints into SQL statements: inside comments with sign “+”
, – is compatible with style of optimizer hints which Oracle uses.
We actually had access to these hints before: they were accessible via variable optimizer_switch. At least such optimizations like BKA, BNL, ICP, MRR. But with new syntax we cannot only modify this access globally or per session, but can turn on or off particular optimization for a single table and column in the query. I can demonstrate it on this quite artificial but always accessible example:
I used one more hint, which we could not turn on or off directly earlier: range optimization.
One more “intuitively” documented feature is the ability to turn on or off a particular optimization. This works only for BKA, BNL, ICP and MRR: you can specify NO_BKA(table[[, table]…]), NO_BNL(table[[, table]…]), NO_ICP(table indexes[[, table indexes]…]) and NO_MRR(table indexes[[, table indexes]…]) to avoid using these algorithms for particular table or index in the JOIN.
MAX_EXECUTION_TIME does not require any table or key name inside. Instead you need to specify maximum time in milliseconds which query should run:
QB_NAME is more complicated. WL #8017 tells us this is custom context. But what is this? The answer is in the MySQL test suite! Tests for optimizer hints exist in file t/opt_hints.test For QB_NAME very first entry is query:
So we can specify custom QB_NAME for any subquery and specify optimizer hint only for this context.
To conclude this quick overview I want to show a practical example of when query hints are really needed. Last week I worked on an issue where a customer upgraded from MySQL version 5.5 to 5.6 and found some of their queries started to work slower than before. I wrote an answer which could sound funny, but still remains correct: “One of the reasons for such behavior is optimizer improvements. While they all are made for better performance, some queries – optimized for older versions – can start working slower than before.”
To demonstrate a public example of such a query I will use my favorite source of information: MySQL Community Bugs Database. In a search for Optimizer regression bugs that are still not fixed we can find bug #68919 demonstrating regression in case the MRR algorithm is used for queries with LIMIT. In run queries, shown in the bug report, we will see a huge difference:
With MRR query execution takes 6.88 seconds and 0 if MRR is not used! But the bug report itself suggests using
as a workaround. And this will work perfectly well if you are OK to run
every time you are running a query which will take advantage of having it OFF. With optimizer hints you can have one or another algorithm to be ON for particular table in the query and OFF for another one. I, again, took quite an artificial example, but it demonstrates the method:
The post Optimizer hints in MySQL 5.7.7 – The missed manual appeared first on MySQL Performance Blog.
It is now possible to provide hints to the optimizer by including
/*+ ... */comments following the
SELECT,
INSERT,
REPLACE,
UPDATE, or
DELETEkeyword of SQL statements. Such statements can also be used with
EXPLAIN. Examples:
SELECT /*+ NO_RANGE_OPTIMIZATION(t3 PRIMARY, f2_idx) */ f1 FROM t3 WHERE f1 > 30 AND f1 < 33; SELECT /*+ BKA(t1, t2) */ * FROM t1 INNER JOIN t2 WHERE ...; SELECT /*+ NO_ICP(t1) */ * FROM t1 WHERE ...;
There are also three worklogs: WL #3996, WL #8016 and WL #8017. But they describe the general concept and do not have much information about which optimizations can be used and how. More light on this provided by slide 59 from Øystein Grøvlen’s session at Percona Live. But that’s all: no “official” full list of possible optimizations, no use cases… nothing.
I tried to sort it out myself.
My first finding is the fact that slide #59 really lists six of seven possible index hints. Confirmation for this exists in one of two new files under sql directory of MySQL source tree, created for this new feature.
$cat sql/opt_hints.h ... /** Hint types, MAX_HINT_ENUM should be always last. This enum should be synchronized with opt_hint_info array(see opt_hints.cc). */ enum opt_hints_enum { BKA_HINT_ENUM= 0, BNL_HINT_ENUM, ICP_HINT_ENUM, MRR_HINT_ENUM, NO_RANGE_HINT_ENUM, MAX_EXEC_TIME_HINT_ENUM, QB_NAME_HINT_ENUM, MAX_HINT_ENUM };
Looking into file sql/opt_hints.cc we can find out what these optimizations give not much choice: either enable or disable.
$cat sql/opt_hints.cc ... struct st_opt_hint_info opt_hint_info[]= { {"BKA", true, true}, {"BNL", true, true}, {"ICP", true, true}, {"MRR", true, true}, {"NO_RANGE_OPTIMIZATION", true, true}, {"MAX_EXECUTION_TIME", false, false}, {"QB_NAME", false, false}, {0, 0, 0} };
A choice for the way to include hints into SQL statements: inside comments with sign “+”
/*+ NO_RANGE_OPTIMIZATION(t3 PRIMARY, f2_idx) */
, – is compatible with style of optimizer hints which Oracle uses.
We actually had access to these hints before: they were accessible via variable optimizer_switch. At least such optimizations like BKA, BNL, ICP, MRR. But with new syntax we cannot only modify this access globally or per session, but can turn on or off particular optimization for a single table and column in the query. I can demonstrate it on this quite artificial but always accessible example:
mysql> use mysql Database changed mysql> explain select * from user where host in ('%', '127.0.0.1'); +----+-------------+-------+------------+-------+---------------+---------+---------+------+------+----------+-----------------------+ | id | select_type | table | partitions | type | possible_keys | key | key_len | ref | rows | filtered | Extra | +----+-------------+-------+------------+-------+---------------+---------+---------+------+------+----------+-----------------------+ | 1 | SIMPLE | user | NULL | range | PRIMARY | PRIMARY | 180 | NULL | 2 | 100.00 | Using index condition | +----+-------------+-------+------------+-------+---------------+---------+---------+------+------+----------+-----------------------+ 1 row in set, 1 warning (0.01 sec) mysql> explain select /*+ NO_RANGE_OPTIMIZATION(user PRIMARY) */ * from user where host in ('%', '127.0.0.1'); +----+-------------+-------+------------+------+---------------+------+---------+------+------+----------+-------------+ | id | select_type | table | partitions | type | possible_keys | key | key_len | ref | rows | filtered | Extra | +----+-------------+-------+------------+------+---------------+------+---------+------+------+----------+-------------+ | 1 | SIMPLE | user | NULL | ALL | PRIMARY | NULL | NULL | NULL | 5 | 40.00 | Using where | +----+-------------+-------+------------+------+---------------+------+---------+------+------+----------+-------------+ 1 row in set, 1 warning (0.00 sec)
I used one more hint, which we could not turn on or off directly earlier: range optimization.
One more “intuitively” documented feature is the ability to turn on or off a particular optimization. This works only for BKA, BNL, ICP and MRR: you can specify NO_BKA(table[[, table]…]), NO_BNL(table[[, table]…]), NO_ICP(table indexes[[, table indexes]…]) and NO_MRR(table indexes[[, table indexes]…]) to avoid using these algorithms for particular table or index in the JOIN.
MAX_EXECUTION_TIME does not require any table or key name inside. Instead you need to specify maximum time in milliseconds which query should run:
mysql> select /*+ MAX_EXECUTION_TIME(1000) */ sleep(1) from user; ERROR 3024 (HY000): Query execution was interrupted, max_statement_time exceeded mysql> select /*+ MAX_EXECUTION_TIME(10000) */ sleep(1) from user; +----------+ | sleep(1) | +----------+ | 0 | | 0 | | 0 | | 0 | | 0 | +----------+ 5 rows in set (5.00 sec)
QB_NAME is more complicated. WL #8017 tells us this is custom context. But what is this? The answer is in the MySQL test suite! Tests for optimizer hints exist in file t/opt_hints.test For QB_NAME very first entry is query:
EXPLAIN SELECT /*+ NO_ICP(t3@qb1 f3_idx) */ f2 FROM (SELECT /*+ QB_NAME(QB1) */ f2, f3, f1 FROM t3 WHERE f1 > 2 AND f3 = 'poiu') AS TD WHERE TD.f1 > 2 AND TD.f3 = 'poiu';
So we can specify custom QB_NAME for any subquery and specify optimizer hint only for this context.
To conclude this quick overview I want to show a practical example of when query hints are really needed. Last week I worked on an issue where a customer upgraded from MySQL version 5.5 to 5.6 and found some of their queries started to work slower than before. I wrote an answer which could sound funny, but still remains correct: “One of the reasons for such behavior is optimizer improvements. While they all are made for better performance, some queries – optimized for older versions – can start working slower than before.”
To demonstrate a public example of such a query I will use my favorite source of information: MySQL Community Bugs Database. In a search for Optimizer regression bugs that are still not fixed we can find bug #68919 demonstrating regression in case the MRR algorithm is used for queries with LIMIT. In run queries, shown in the bug report, we will see a huge difference:
mysql> SELECT * FROM t1 WHERE i1>=42 AND i2<=42 LIMIT 1; +----+----+----+----+ | pk | i1 | i2 | i3 | +----+----+----+----+ | 42 | 42 | 42 | 42 | +----+----+----+----+ 1 row in set (6.88 sec) mysql> explain SELECT * FROM t1 WHERE i1>=42 AND i2<=42 LIMIT 1; +----+-------------+-------+------------+-------+---------------+------+---------+------+---------+----------+----------------------------------+ | id | select_type | table | partitions | type | possible_keys | key | key_len | ref | rows | filtered | Extra | +----+-------------+-------+------------+-------+---------------+------+---------+------+---------+----------+----------------------------------+ | 1 | SIMPLE | t1 | NULL | range | idx | idx | 4 | NULL | 9999958 | 33.33 | Using index condition; Using MRR | +----+-------------+-------+------------+-------+---------------+------+---------+------+---------+----------+----------------------------------+ 1 row in set, 1 warning (0.00 sec) mysql> SELECT /*+ NO_MRR(t1) */ * FROM t1 WHERE i1>=42 AND i2<=42 LIMIT 1; +----+----+----+----+ | pk | i1 | i2 | i3 | +----+----+----+----+ | 42 | 42 | 42 | 42 | +----+----+----+----+ 1 row in set (0.00 sec)
With MRR query execution takes 6.88 seconds and 0 if MRR is not used! But the bug report itself suggests using
optimizer_switch="mrr=off";
as a workaround. And this will work perfectly well if you are OK to run
SET optimizer_switch="mrr=off";
every time you are running a query which will take advantage of having it OFF. With optimizer hints you can have one or another algorithm to be ON for particular table in the query and OFF for another one. I, again, took quite an artificial example, but it demonstrates the method:
mysql> explain select /*+ MRR(dept_emp) */ * from dept_emp where to_date in (select /*+ NO_MRR(salaries)*/ to_date from salaries where salary >40000 and salary <45000) and emp_no >10100 and emp_no < 30200 and dept_no in ('d005', 'd006','d007'); +----+--------------+-------------+------------+--------+------------------------+------------+---------+----------------------------+---------+----------+-----------------------------------------------+ | id | select_type | table | partitions | type | possible_keys | key | key_len | ref | rows | filtered | Extra | +----+--------------+-------------+------------+--------+------------------------+------------+---------+----------------------------+---------+----------+-----------------------------------------------+ | 1 | SIMPLE | dept_emp | NULL | range | PRIMARY,emp_no,dept_no | dept_no | 8 | NULL | 10578 | 100.00 | Using index condition; Using where; Using MRR | | 1 | SIMPLE | <subquery2> | NULL | eq_ref | <auto_key> | <auto_key> | 3 | employees.dept_emp.to_date | 1 | 100.00 | NULL | | 2 | MATERIALIZED | salaries | NULL | ALL | salary | NULL | NULL | NULL | 2838533 | 17.88 | Using where | +----+--------------+-------------+------------+--------+------------------------+------------+---------+----------------------------+---------+----------+-----------------------------------------------+ 3 rows in set, 1 warning (0.00 sec)
The post Optimizer hints in MySQL 5.7.7 – The missed manual appeared first on MySQL Performance Blog.
相关文章推荐
- 【转载】我所理解的BeanUtils.copyProperties() 用法
- 车机系统架构图
- docker深入2-使用自定义的网络来配置zookeeper集群
- nginx源码分析——事件模块
- Shell的RANDOM变量
- linux ssh key登陆
- 用nginx搭建流媒体服务器
- 爱牛网站
- Linux系统访问控制流程架构图
- 使用Nginx代理restful实现SSL链路加密
- aix hpux linux 命令收集
- Anaconda配置opencv
- openwrt中ubus的具体使用
- Hadoop中map数的计算
- Linux下打包压缩war和解压war包 (mac)
- Idea 使用maven+tomcat的时候,编译指定的Profile
- Jenkins+SVN+Maven+Shell 实现项目一键发布
- OpenResty学习笔记(四) string库
- 修改(或伪装)你的Linux发行版
- thinkphp nginx 配置