InnoDB On-Disk Structures(五)-- Redo Log & Undo Logs (转载)
1.Redo Log
The redo log is a disk-based data structure used during crash recovery to correct data written by incomplete transactions. During normal operations, the redo log encodes requests to change table data that result from SQL statements or low-level API calls. Modifications that did not finish updating the data files before an unexpected shutdown are replayed automatically during initialization, and before the connections are accepted.
By default, the redo log is physically represented on disk by two files named
ib_logfile0and
ib_logfile1. MySQL writes to the redo log files in a circular fashion. Data in the redo log is encoded in terms of records affected; this data is collectively referred to as redo. The passage of data through the redo log is represented by an ever-increasing LSN value.
1.1 Changing the Number or Size of Redo Log Files
To change the number or the size of redo log files, perform the following steps:
-
Stop the MySQL server and make sure that it shuts down without errors.
-
Edit
my.cnf
to change the log file configuration. To change the log file size, configureinnodb_log_file_size
. To increase the number of log files, configureinnodb_log_files_in_group
. -
Start the MySQL server again.
If
InnoDBdetects that the
innodb_log_file_sizediffers from the redo log file size, it writes a log checkpoint, closes and removes the old log files, creates new log files at the requested size, and opens the new log files.
1.2 Group Commit for Redo Log Flushing
InnoDB, like any other ACID-compliant database engine, flushes the redo log of a transaction before it is committed.
InnoDBuses group commit functionality to group multiple such flush requests together to avoid one flush for each commit. With group commit,
InnoDBissues a single write to the log file to perform the commit action for multiple user transactions that commit at about the same time, significantly improving throughput.
1.3 Redo Log Archiving
Backup utilities that copy redo log records may sometimes fail to keep pace with redo log generation while a backup operation is in progress, resulting in lost redo log records due to those records being overwritten. This issue most often occurs when there is significant MySQL server activity during the backup operation, and the redo log file storage media operates at a faster speed than the backup storage media. The redo log archiving feature, introduced in MySQL 8.0.17, addresses this issue by sequentially writing redo log records to an archive file in addition to the redo log files. Backup utilities can copy redo log records from the archive file as necessary, thereby avoiding the potential loss of data.
If redo log archiving is configured on the server, MySQL Enterprise Backup, available with the MySQL Enterprise Edition, uses the redo log archiving feature when backing up a MySQL server.
Enabling redo log archiving on the server requires setting a value for the
innodb_redo_log_archive_dirssystem variable. The value is specified as a semicolon-separated list of labeled redo log archive directories. The
label:directory
pair is separated by a colon (:). For example:
mysql> SET GLOBAL innodb_redo_log_archive_dirs='label1:directory_path1[;label2:directory_path2;…]';
The
labelis an arbitrary identifier for the archive directory. It can be any string of characters, with the exception of colons (:), which are not permitted. An empty label is also permitted, but the colon (:) is still required in this case. A
directory_pathmust be specified. The directory that is selected for the redo log archive file must exist when redo log archiving is activated, or an error is returned. The path can contain colons (':'), but semicolons (;) are not permitted.
The
innodb_redo_log_archive_dirsvariable must be configured before the redo log archiving can be activated. The default value is
NULL, which does not permit activating redo log archiving.
注意事项:
The archive directories that you specify must satisfy the following requirements. (The requirements are enforced when redo log archiving is activated.):
-
Directories must exist. Directories are not created by the redo log archive process. Otherwise, the following error is returned:
ERROR 3844 (HY000): Redo log archive directory '
directory_path1
' does not exist or is not a directory -
Directories must not be world-accessible. This is to prevent the redo log data from being exposed to unauthorized users on the system. Otherwise, the following error is returned:
ERROR 3846 (HY000): Redo log archive directory '
directory_path1
' is accessible to all OS users -
Directories cannot be those defined by
datadir
,innodb_data_home_dir
,innodb_directories
,innodb_log_group_home_dir
,innodb_temp_tablespaces_dir
,innodb_tmpdir
innodb_undo_directory
, orsecure_file_priv
, nor can they be parent directories or subdirectories of those directories. Otherwise, an error similar to the following is 670 returned:ERROR 3845 (HY000): Redo log archive directory '
directory_path1
' is in, under, or over server directory 'datadir' - '/path/to/data_directory
'
When a backup utility that supports redo log archiving initiates a backup, the backup utility activates redo log archiving by invoking the
innodb_redo_log_archive_start()user-defined function.
If you are not using a backup utility that supports redo log archiving, redo log archiving can also be activated manually, as shown:
mysql> SELECT innodb_redo_log_archive_start('label', 'subdir'); +------------------------------------------+ | innodb_redo_log_archive_start('label') | +------------------------------------------+ | 0 | +------------------------------------------+
Or:
mysql> DO innodb_redo_log_archive_start('label', 'subdir'); Query OK, 0 rows affected (0.09 sec)
注意事项:
The MySQL session that activates redo log archiving (using
innodb_redo_log_archive_start()) must remain open for the duration of the archiving. The same session must deactivate redo log archiving (using
innodb_redo_log_archive_stop()). If the session is terminated before the redo log archiving is explicitly deactivated, the server deactivates redo log archiving implicitly and removes the redo log archive file.
where
labelis a label defined by
innodb_redo_log_archive_dirs;
subdiris an optional argument for specifying a subdirectory of the directory identified by
labelfor savi ad8 ng the archive file; it must be a simple directory name (no slash (/), backslash (\), or colon (:) is permitted).
subdircan be empty, null, or it can be left out.
Only users with the
INNODB_REDO_LOG_ARCHIVEprivilege can activate redo log archiving by invoking
innodb_redo_log_archive_start(), or deactivate it using
innodb_redo_log_archive_stop(). The MySQL user running the backup utility or the MySQL user activating and deactivating redo log archiving manually must have this privilege.
The redo log archive file path is
directory_identified_by_label
/[subdir/]archive.
serverUUID.000001.log, where
directory_identified_by_label
is the archive directory identified by the label
argument for innodb_redo_log_archive_start().
subdir
is the optional argument used for innodb_redo_log_archive_start().
For example, the full path and name for a redo log archive file appears similar to the following:
/directory_path/subdirectory/archive.e71a47dc-61f8-11e9-a3cb-080027154b4d.000001.log
After the backup utility finishes copying
InnoDBdata files, it deactivates redo log archiving by calling the
innodb_redo_log_archive_stop()user-defined function.
If you are not using a backup utility that supports redo log archiving, redo log archiving can also be deactivated manually, as shown:
mysql> SELECT innodb_redo_log_archive_stop(); +--------------------------------+ | innodb_redo_log_archive_stop() | +--------------------------------+ | 0 | +--------------------------------+
Or:
mysql> DO innodb_redo_log_archive_stop(); Query OK, 0 rows affected (0.01 sec)
After the stop functi 3666 on completes successfully, the backup utility looks for the relevant section of redo log data from the archive file and copies it into the backup.
After the backup utility finishes copying the redo log data and no longer needs the redo log archive file, it deletes the archive file.
Removal of the archive file is the responsibility of the backup utility in normal situations. However, if the redo log archiving operation quits unexpectedly before
innodb_redo_log_archive_stop()is called, the MySQL server removes the file.
1.4 Performance Considerations
Activating redo log archiving typically has a minor performance cost due to the additional write activity.
On Unix and Unix-like operating systems, the performance impact is typically minor, assuming there is not a sustained high rate of updates. On Windows, the performance impact is typically a bit higher, assuming the same.
If there is a sustained high rate of updates and the redo log archive file is on the same storage media as the redo log files, the performance impact may be more significant due to compounded write activity.
If there is a sustained high rate of updates and the redo log archive file is on slower storage media than the redo log files, performance is impacted arbitrarily.
Writing to the redo log archive file does not impede normal transactional logging except in the case that the redo log archive file storage media operates at a much slower rate than the redo log file storage media, and there is a large backlog of persisted redo log blocks waiting to be written to the redo log archive file. In this case, the transactional logging rate is reduced to a level that can be managed by the slower storage media where the redo log archive file resides.
2. Undo Logs
An undo log is a collection of undo log records associated with a single read-write transaction. An undo log record contains information about how to undo the latest change by a transaction to a clustered index record. If another transaction needs to see the original data as part of a consistent read operation, the unmodified data is retrieved from undo log records. Undo logs exist within undo log segments, which are contained within rollback segments. Rollback segments reside in undo tablespacesand in the global temporary tablespace.
Undo logs that reside in the global temporary tablespace are used for transactions that modify data in user-defined temporary tables. These undo logs are not redo-logged, as they are not required for crash recovery. They are used only for rollback while the server is running. This type of undo log benefits performance by avoiding redo logging I/O.
Each undo tablespace and the global temporary tablespace individually support a maximum of 128 rollback segments. The
innodb_rollback_segmentsvariable defines the number of rollback segments.
The number of transactions that a rollback segment supports depends on the number of undo slots in the rollback segment and the number of undo logs required by each transaction.
The number of undo slots in a rollback segment differs according to
InnoDBpage size.
InnoDB Page Size | Number of Undo Slots in a Rollback Segment (InnoDB Page Size / 16) |
---|---|
4096 (4KB) |
256 |
8192 (8KB) |
512 |
16384 (16KB) |
1024 |
32768 (32KB) |
2048 |
65536 (64KB) |
4096 |
A transaction is assigned up to four undo logs, one for each of the following operation types:
-
INSERT
operations on user-defined tables -
UPDATE
andDELETE
operations on user-defined tables -
INSERT
operations on user-defined temporary tables -
UPDATE
andDELETE
operations on user-defined temporary tables
Undo logs are assigned as needed. For example, a transaction that performs
INSERT,
UPDATE, and
DELETEoperations on regular and temporary tables requires a full assignment of four undo logs. A transaction that performs only
INSERToperations on regular tables requires a single undo log.
A transaction that performs operations on regular tables is assigned undo logs from an assigned undo tablespace rollback segment. A transaction that performs operations on temporary tables is assigned undo logs from an assigned global temporary tablespace rollback segment.
An undo log assigned to a transaction remains tied to the transaction for its duration. For example, an undo log assigned to a transaction for an
INSERToperation on a regular table is used for all
INSERToperations on regular tables performed by that transaction.
Given the factors described above, the following formulas can be used to estimate the number of concurrent read-write transactions that
InnoDBis capable of supporting.
-
If each transaction performs either an
INSERT
or anUPDATE
orDELETE
operation, the number of concurrent read-write transactions thatInnoDB
is capable of supporting is:(innodb_page_size / 16) * innodb_rollback_segments * number of undo tablespaces
-
If each transaction performs an
INSERT
and anUPDATE
orDELETE
operation, the number of concurrent read-write transactions thatInnoDB
is capable of supporting is:(innodb_page_size / 16 / 2) * innodb_rollback_segments * number of undo tablespaces
-
If each transaction performs an
INSERT
operation on a temporary table, the number of concurrent read-write transactions thatInnoDB
is capable of supporting is:(innodb_page_size / 16) * innodb_rollback_segments
-
If each transaction performs an
INSERT
and anUPDATE
orDELETE
operation on a temporary table, the number of concurrent read-write transactions thatInnoDB
is capable of supporting is:
(innodb_page_size / 16 / 2) * innodb_rollback_segments
转载、节选于 https://dev.mysql.com/doc/refman/8.0/en/innodb-redo-log.html
https://dev.mysql.com/doc/refman/8.0/en/innodb-undo-logs.html
- innodb-undo-logs,innodb-redo-log,The Binary Log
- binlog,redo log,undo log区别
- InnoDB的Redo Undo Log
- mysql_Innodb的undo_log和redo_log
- Mysql Innodb中undo-log和MVCC多版…
- mysql_Innodb的undo_log和redo_log
- MySQL · 引擎特性 · InnoDB undo log 漫游
- InnoDB的Redo Undo Log
- How Logs Work On MySQL With InnoDB Tables
- InnoDB undo log原理之事务提交时undo page相关操作
- Mysql Innodb中undo-log和MVCC多版本一致性读的实现(源码分析)
- sap logon错误: "password logon no longer possible"
- mysql-innodb-undo和redo
- 【转载】MySQL 日志 undo | redo
- Mysql Innodb的undo redo操作过程
- innodb undo--update undo log
- undo log与redo log原理分析
- MySQL系列:innodb源码分析之redo log恢复
- MySQL · 引擎特性 · InnoDB undo log 漫游
- 14.5.7 Storing InnoDB Undo Logs in Separate Tablespaces 存储InnoDB Undo logs 到单独的表空间