When does MySQL data get loaded in and out of cache?
2013-12-13 10:28
573 查看
When does MySQL data get loaded in and out of cache?
A cold cache, or a poorly tuned cache can be responsible for a number of performance problems. If we look at the data and indexes of InnoDB, the cache responsible is called the InnoDB buffer pool.In earlier versions of MySQL the default value for the setting
innodb_buffer_pool_sizewas
8M - which is a little out of date when you consider the recommended value to be 50-80% of system memory. In MySQL 5.5 the default value was increased to 128M, which is a comprimise made for users that may install MySQL on a development machine and not have
it running as a dedicated server. In production, it is unlikely that you will buy a new server with less than 64GB of RAM, so it is quite typical that this setting is 50GB+
So lets talk about the behavior of the InnoDB buffer pool -
Up until and including MySQL 5.5
When MySQL starts up, the buffer pool is completely empty. As queries are executed, MySQL will determine which pages are required - and if they are not in memory, they will be loaded from disk.Initially, performance will be very poor - since there will be 100% cache misses, but over time as the buffer pool fills this should reduce.
Provided there is enough memory - MySQL will just keep loading pages into cache, and warm to a state where it will not have to perform any reads to the disk at all. Only writes will go to disk.
From MySQL 5.6 onwards
Because server warm time is becoming a more serious problem (it always existed, but it is exacerbated by us now having an abundance of RAM, but if we are using hard drives, they are not really any faster), Oracle introduced a feature in MySQL 5.6 - buffer poolwarming.
What it does is on demand or on
shutdown saves the addresses (
space_id+
page_id)
of InnoDB pages to permanent storage.
Upon startup, these pages can then automatically be loaded back into memory
straight away so that we can have cache misses for less time.
This feature is not turned on by default, but I suspect in a future version it may be. There are a number of other advantages to pre-warming such as being able to sort and merge read requests and load the data in much faster. Also, since it is only the addresses
of pages being saved it only takes 8 bytes to point to each 16KB page, and there are no risks if there have been modifications since the last address saving operation ran.
Cache evictions
So we’ve described the “best case” so far, which is that data only gets loaded into cache and performance keeps getting better and better. However in practice, our cache is never unlimited. When our buffer pool gets to the point of being completely full itwill need to free space in order to load new pages into memory. The behavior of this ‘eviction’ is version specific.
Up until MySQL 5.5
InnoDB previously used a classic Least Recently Used (LRU) algorithm. What this means is that each page hasan order in a list. I like to think of it as a “Hot or not” score. Each time a page is accessed it gets promoted up the list to be considered “more hot”. When it comes to freeing space, InnoDB just picks the least hot page and frees it from memory to make
space for the next page.
In typical operation MySQL will not free pages from memory unless it needs to make space for another page. I say ‘typical’, because pages will be freed if a table is dropped for example.
MySQL 5.5 onwards
There are a number of operational limitations with using a classic LRU implementation on a database which has many users, and typically more than one workload. For example lets say that everything is working fine and we’ve figured out what is “hot or not”,noting that we do not need as much memory as we do data because in most cases there will be pages that are not accessed frequently. Now imagine that a series of queries come in from mysqldump that want to run tablescans to export the data. What can happen,
is the pages loaded in from the tablescans push out some of the good pages in our buffer pool, and even worse, as soon as the mysqldump operation is complete, they will no longer be required. So now post-mysqldump we have random IO as we try and re-settle
cache-contents again.
So in MySQL 5.5 the LRU was split into two sublists:
A ‘young’ sublist for our frequently accessed pages (default: 63%)
An ‘old’ sublist for our table scan page accesses (default: 37%)
The configuration variable
innodb_old_blocks_pctwas introduced to configure
the old/new split, and a variable
innodb_old_blocks_time(default in 5.5:
0, in 5.6: 1000) was introduced to specify a minimum amount of milliseconds that a freshly loaded page must stay in the old sublist before it is ellible to be promoted to the young sublist.
As mentioned in the manual, and in blog
posts it frequently makes sense to configure the
innodb_old_blocks_timeto
a larger value, for example 1000 (ms).
A note on page churn
If you do not have enough memory to hold all of your working set in memory, what will happen is that the buffer pool will start juggling as it churns pages out of memory only to load them back in soon after. A small amount of this is probably okay, but it canstart to degrade performance. The traditional rule of thumb is called ”the 5 minute rule”. This means that if you load a page into
memory and are going to need it again in the next five minutes - it should not need to be churned out.
转自:http://www.tocker.ca/2013/05/03/when-does-MySQL-data-get-loaded-in-and-out-of-cache.html
作者:
Morgan Tocker
MySQL Community Manager
相关文章推荐
- Get data out of excel and into R with readxl
- InnoDB: MySQL and InnoDB data dictionaries are out of sync + InnoDB: AUTOINC next value generation
- Moving data in and out of hadoop
- 海洋工作室——网站建设专家:The version of SQL Server in use does not support datatype datetime2 and the Entity Framework.
- Structure of a C-Program in Memory | How Heap,Stack,Data and Code segments are stored in memory?
- Get the content of an Iframe in Javascript – crossbrowser solution for both IE and Firefox
- 100.In which situations does the Oracle Data Pump use external tables and not the direct path load w
- Getting System.OutOfMemoryException when using ReportViewer contol in Local mode
- 转载:Character data is represented incorrectly when the code page of the client computer differs from the code page of the database in SQL Server 2005
- 论文阅读:Automatic Detection and Classication of Teeth in CT Data
- EF 6.0 The conversion of a datetime2 data type to a datetime data type resulted in an out-of-range value. c#中的时间
- 解决1418 - This function has none of DETERMINISTIC, NO SQL, or READS SQL DATA in its declaration and b
- in-band network management and out-of-band management
- [转]Comparing features of C#/Delphi and .NET/VCL Where does your Delphi knowledge fit in ?
- get you in, and out and back to life---手机的未来是社交的聚合
- Get and display the size of file and directory in Linux system using du command 获取和现实linux文件大小(三)
- Java in Science: Data Interpolation and Extrapolation Using Numerical Methods of Polynomial Fittings
- [Nuxt] Display Vuex Data Differently in Each Page of Nuxt and Vue.js
- conversion of a char data type to a datetime data type resulted in an out-of-range datetime value.
- How to get the data from a cell when I click on the GridButtonColumn of the same row