您的位置:首页 > 数据库 > MySQL

postgresql9.6 mysql5.7基准性能测试对比记录一

2018-02-11 18:29 846 查看
测试了在50并发下的情况,结果日志中报错了。

ERROR: requested WAL segment 00000001000000010000002D has already been removed

ERROR: duplicate key value violates unique constraint “sbtest100_pkey”

DETAIL: Key (id)=(49782) already exists.

STATEMENT: INSERT INTO sbtest100 (id, k, c, pad) VALUES (1,1,2, 3,3,4)

ERROR: requested WAL segment 00000001000000010000002D has already been removed

ERROR: duplicate key value violates unique constraint “sbtest4_pkey”

DETAIL: Key (id)=(50144) already exists.

STATEMENT: INSERT INTO sbtest4 (id, k, c, pad) VALUES (1,1,2, 3,3,4)

ERROR: requested WAL segment 00000001000000010000002D has already been removed

LOG: checkpoints are occurring too frequently (23 seconds apart)

HINT: Consider increasing the configuration parameter “max_wal_size”.

ERROR: requested WAL segment 00000001000000010000002D has already been removed

ERROR: duplicate key value violates unique constraint “sbtest55_pkey”

DETAIL: Key (id)=(50151) already exists.

STATEMENT: INSERT INTO sbtest55 (id, k, c, pad) VALUES (1,1,2, 3,3,4)

ERROR: requested WAL segment 00000001000000010000002D has already been removed

ERROR: requested WAL segment 00000001000000010000002D has already been removed

Threads started!

SQL statistics:

queries performed:

read: 7601286

write: 2171542

other: 1086030

total: 10858858

transactions: 542888 (904.17 per sec.)

queries: 10858858 (18085.27 per sec.)

ignored errors: 61 (0.10 per sec.)

reconnects: 0 (0.00 per sec.)

General statistics:

total time: 600.4255s

total number of events: 542888

Latency (ms):

min: 5.34

avg: 55.27

max: 767.45

95th percentile: 95.81

sum: 30004086.20

Threads fairness:

events (avg/stddev): 10857.7600/52.20

execution time (avg/stddev): 600.0817/0.10

需要优化下检查点的配置,在将max_wal_size的大小调整到5G后的压测效果,没有报错,但是效果上看跟调整之前差不多

Threads started!

SQL statistics:

queries performed:

read: 7761726

write: 2217386

other: 1108934

total: 11088046

transactions: 554342 (923.31 per sec.)

queries: 11088046 (18468.12 per sec.)

ignored errors: 67 (0.11 per sec.)

reconnects: 0 (0.00 per sec.)

General statistics:

total time: 600.3883s

total number of events: 554342

Latency (ms):

min: 6.97

avg: 54.13

max: 2081.00

95th percentile: 84.47

sum: 30008119.53

Threads fairness:

events (avg/stddev): 11086.8400/41.71

execution time (avg/stddev): 600.1624/0.14

添加了opm的监控配置,再次压测了一下,配置的参数文件如下:

listen_addresses = ‘*’

max_connections = 1000 # (change requires restart)

unix_socket_directories = ‘/tmp’

unix_socket_group = ” # (change requires restart)

unix_socket_permissions = 0777

shared_buffers = 1024MB

temp_buffers = 8MB

max_prepared_transactions = 100

work_mem = 4MB # min 64kB

maintenance_work_mem = 64MB # min 1MB

replacement_sort_tuples = 150000 # limits use of replacement selection sort

autovacuum_work_mem = -1 # min 1MB, or -1 to use maintenance_work_mem

max_stack_depth = 2MB # min 100kB

dynamic_shared_memory_type = posix

temp_file_limit = -1

max_files_per_process = 1000

wal_level = hot_standby

fsync = on

synchronous_commit = on

wal_sync_method = fsync

full_page_writes = on # recover from partial page writes

wal_compression = off # enable compression of full-page writes

wal_log_hints = off

wal_buffers = -1

wal_writer_delay = 200ms # 1-10000 milliseconds

wal_writer_flush_after = 1MB # measured in pages, 0 disables

checkpoint_timeout = 5min # range 30s-1d

max_wal_size = 5GB

min_wal_size = 80MB

checkpoint_completion_target = 0.5 # checkpoint target duration, 0.0 - 1.0

checkpoint_flush_after = 256kB # measured in pages, 0 disables

checkpoint_warning = 30s # 0 disables

archive_mode = on

archive_command = ‘test ! -f /data/pg/archivedir/%f && cp %p /data/pg/archivedir/%f’

max_wal_senders = 5

effective_cache_size = 4GB

log_destination = ‘csvlog’

logging_collector = on

log_directory = ‘pg_log’

log_filename = ‘postgresql-%Y-%m-%d_%H%M%S.log’

log_rotation_age = 1d

log_rotation_size = 100MB

client_min_messages = notice

log_min_messages = warning

log_min_duration_statement = 60

log_checkpoints = on

log_connections = off

log_disconnections = off

log_duration = off

log_error_verbosity = default # terse, default, or verbose messages

log_line_prefix = ‘%m’

log_lock_waits = on # log lock waits >= deadlock_timeout

log_statement = ‘ddl’

log_timezone = ‘Asia/Shanghai’

datestyle = ‘iso, mdy’

timezone = ‘Asia/Shanghai’

lc_messages = ‘en_US.UTF-8’

lc_monetary = ‘en_US.UTF-8’ # locale for monetary formatting

lc_numeric = ‘en_US.UTF-8’ # locale for number formatting

lc_time = ‘en_US.UTF-8’ # locale for time formatting

default_text_search_config = ‘pg_catalog.english’

os配置:

4 Intel Xeon E312xx (Sandy Bridge)

8g内存

SQL statistics:

queries performed:

read: 6675900

write: 1907199

other: 953799

total: 9536898

transactions: 476799 (794.00 per sec.)

queries: 9536898 (15881.56 per sec.)

ignored errors: 51 (0.08 per sec.)

reconnects: 0 (0.00 per sec.)

General statistics:

total time: 600.5013s

total number of events: 476799

Latency (ms):

min: 5.38

avg: 62.94

max: 6793.34

95th percentile: 101.13

sum: 30008242.17

Threads fairness:

events (avg/stddev): 9535.9800/37.44

execution time (avg/stddev): 600.1648/0.15

看到添加opm的监控并记录日志后,还是对pg有很大的影响的,tps下降了130,下降了14%,毕竟是想opm库中写了了数据,还有一些指标的查询。下面是一些监控信息。



























我们能看到在测试期间,磁盘io的使用率在60%,70%左右已经很繁忙,出现了iowait,缓存池的命中率还是可以的在99.3%,但是期间执行的事务有很多的在等待wal写锁,有非常多的访问共享锁,高峰期有30左右的排他锁,wal的写每秒也不是很多高峰是2.8MB,在慢日志中我们看到单条语句的执行也不是很快,都是几百毫秒,这个对单条语句来说太长了,另外看到很多的commit的时间都是上了秒了,略显夸张。

找个具体的

duration: 180.945 ms bind sbstmt2102261739-333900483: SELECT DISTINCT c FROM sbtest12 WHERE id BETWEEN 1AND1AND2 ORDER BY c

这个sql花了180ms,看下表结构

myDB=# \d sbtest12
Table "public.sbtest12"
Column |      Type      |                       Modifiers
--------+----------------+-------------------------------------------------------
id     | integer        | not null default nextval('sbtest12_id_seq'::regclass)
k      | integer        | not null default 0
c      | character(120) | not null default ''::bpchar
pad    | character(60)  | not null default ''::bpchar
Indexes:
"sbtest12_pkey" PRIMARY KEY, btree (id)
"k_12" btree (k)


下面是mysql的压测情况,mysql主要配置:

sync_binlog | 1

innodb_flush_log_at_trx_commit | 1

innodb_buffer_pool_size | 1073741824

下面是第一次的压测记录,没有预热

Running the test with following options:

Number of threads: 50

Initializing random number generator from current time

Initializing worker threads…

Threads started!

SQL statistics:

queries performed:

read: 4552632

write: 1300752

other: 650376

total: 6503760

transactions: 325188 (541.84 per sec.)

queries: 6503760 (10836.76 per sec.)

ignored errors: 0 (0.00 per sec.)

reconnects: 0 (0.00 per sec.)

General statistics:

total time: 600.1573s

total number of events: 325188

Latency (ms):

min: 5.24

avg: 92.26

max: 462.44

95th percentile: 150.29

sum: 30003291.86

Threads fairness:

events (avg/stddev): 6503.7600/1010.96

execution time (avg/stddev): 600.0658/0.03

下面是第二次预热了的测试结果

Running the test with following options:

Number of threads: 50

Initializing random number generator from current time

Initializing worker threads…

Threads started!

SQL statistics:

queries performed:

read: 4606392

write: 1316112

other: 658056

total: 6580560

transactions: 329028 (548.24 per sec.)

queries: 6580560 (10964.90 per sec.)

ignored errors: 0 (0.00 per sec.)

reconnects: 0 (0.00 per sec.)

General statistics:

total time: 600.1479s

total number of events: 329028

Latency (ms):

min: 7.79

avg: 91.19

max: 333.12

95th percentile: 142.39

sum: 30003076.00

Threads fairness:

events (avg/stddev): 6580.5600/1117.61

execution time (avg/stddev): 600.0615/0.04

看到测试出来的结果比pg的要差很多。





慢查询的情况,看到commit的是占用的时间最多的。而且上面看到是在等待aio的完成
# 200ms user time, 10ms system time, 20.19M rss, 165.36M vsz
# Current date: Tue Feb 27 17:37:26 2018
# Hostname: perftest1
# Files: slow.log
# Overall: 442 total, 10 unique, 0 QPS, 0x concurrency ___________________
# Attribute          total     min     max     avg     95%  stddev  median
# ============     ======= ======= ======= ======= ======= ======= =======
# Exec time            52s   100ms   196ms   118ms   141ms    17ms   110ms
# Lock time           42ms       0    21ms    94us    49us     1ms       0
# Rows sent          4.72k       0     100   10.94   97.36   30.26       0
# Rows examine      11.04k       0     300   25.57  192.76   67.14       0
# Query size        17.31k       6     244   40.10  234.30   59.08    5.75

# Profile
# Rank Query ID           Response time Calls R/Call V/M   Item
# ==== ================== ============= ===== ====== ===== ==============
#    1 0x813031B8BBC3B329 30.5927 58.5%   257 0.1190  0.00 COMMIT
#    2 0xD30AD7E3079ABCE7  5.6074 10.7%    48 0.1168  0.00 UPDATE sbtest?
#    3 0x84D1DEE77FA8D4C3  4.0989  7.8%    36 0.1139  0.00 SELECT sbtest?
#    4 0xF1256A27240AEFC7  3.1427  6.0%    26 0.1209  0.00 INSERT sbtest?
#    5 0x558CAEF5F387E929  3.0988  5.9%    25 0.1240  0.01 SELECT sbtest?
#    6 0xE96B374065B13356  2.0156  3.9%    18 0.1120  0.00 UPDATE sbtest?
#    7 0x6EEB1BFDCCF4EBCD  1.2529  2.4%    10 0.1253  0.00 SELECT sbtest?
# MISC 0xMISC              2.4724  4.7%    22 0.1124   0.0 <3 ITEMS>

# Query 1: 0 QPS, 0x concurrency, ID 0x813031B8BBC3B329 at byte 11202 ____
# Scores: V/M = 0.00
# Attribute    pct   total     min     max     avg     95%  stddev  median
# ============ === ======= ======= ======= ======= ======= ======= =======
# Count         58     257
# Exec time     58     31s   100ms   179ms   119ms   141ms    16ms   116ms
# Lock time      0       0       0       0       0       0       0       0
# Rows sent      0       0       0       0       0       0       0       0
# Rows examine   0       0       0       0       0       0       0       0
# Query size     8   1.51k       6       6       6       6       0       6
# String:
# Hosts        10.200.142.231
# Time         2018-02-27... (2/0%), 2018-02-27... (1/0%)... 254 more
# Users        root
# Query_time distribution
#   1us
#  10us
# 100us
#   1ms
#  10ms
# 100ms  ################################################################
#    1s
#  10s+
COMMIT\G

# Query 2: 0 QPS, 0x concurrency, ID 0xD30AD7E3079ABCE7 at byte 17466 ____
# Scores: V/M = 0.00
# Attribute    pct   total     min     max     avg     95%  stddev  median
# ============ === ======= ======= ======= ======= ======= ======= =======
# Count         10      48
# Exec time     10      6s   101ms   196ms   117ms   128ms    18ms   105ms
# Lock time      4     2ms    26us    64us    39us    49us     8us    38us
# Rows sent      0       0       0       0       0       0       0       0
# Rows examine   0      48       1       1       1       1       0       1
# Query size    10   1.87k      39      40   39.94   38.53       0   38.53
# String:
# Hosts        10.200.142.231
# Time         2018-02-27... (1/2%), 2018-02-27... (1/2%)... 46 more
# Users        root
# Query_time distribution
#   1us
#  10us
# 100us
#   1ms
#  10ms
# 100ms  ################################################################
#    1s
#  10s+
# Tables
#    SHOW TABLE STATUS LIKE 'sbtest4'\G
#    SHOW CREATE TABLE `sbtest4`\G
UPDATE sbtest4 SET k=k+1 WHERE id=51873\G
# Converted for EXPLAIN
# EXPLAIN /*!50100 PARTITIONS*/
select  k=k+1 from sbtest4 where  id=51873\G

# Query 3: 0 QPS, 0x concurrency, ID 0x84D1DEE77FA8D4C3 at byte 43919 ____
# Scores: V/M = 0.00
# Attribute    pct   total     min     max     avg     95%  stddev  median
# ============ === ======= ======= ======= ======= ======= ======= =======
# Count          8      36
# Exec time      7      4s   100ms   139ms   114ms   128ms     9ms   110ms
# Lock time     86    36ms    23us    21ms     1ms     3ms     4ms    30us
# Rows sent     74   3.52k     100     100     100     100       0     100
# Rows examine  63   7.03k     200     200     200     200       0     200
# Query size    13   2.32k      65      66   65.89   65.89    0.50   65.89
# String:
# Hosts        10.200.142.231
# Time         2018-02-27... (2/5%), 2018-02-27... (1/2%)... 33 more
# Users        root
# Query_time distribution
#   1us
#  10us
# 100us
#   1ms
#  10ms
# 100ms  ################################################################
#    1s
#  10s+
# Tables
#    SHOW TABLE STATUS LIKE 'sbtest60'\G
#    SHOW CREATE TABLE `sbtest60`\G
# EXPLAIN /*!50100 PARTITIONS*/
SELECT c FROM sbtest60 WHERE id BETWEEN 49783 AND 49882 ORDER BY c\G

# Query 4: 0 QPS, 0x concurrency, ID 0xF1256A27240AEFC7 at byte 15816 ____
# Scores: V/M = 0.00
# Attribute    pct   total     min     max     avg     95%  stddev  median
# ============ === ======= ======= ======= ======= ======= ======= =======
# Count          5      26
# Exec time      6      3s   101ms   187ms   121ms   134ms    20ms   116ms
# Lock time      2     1ms    24us   120us    41us    57us    20us    36us
# Rows sent      0       0       0       0       0       0       0       0
# Rows examine   0       0       0       0       0       0       0       0
# Query size    35   6.19k     243     244  243.96  234.30       0  234.30
# String:
# Hosts        10.200.142.231
# Time         2018-02-27... (1/3%), 2018-02-27... (1/3%)... 24 more
# Users        root
# Query_time distribution
#   1us
#  10us
# 100us
#   1ms
#  10ms
# 100ms  ################################################################
#    1s
#  10s+
# Tables
#    SHOW TABLE STATUS LIKE 'sbtest81'\G
#    SHOW CREATE TABLE `sbtest81`\G
INSERT INTO sbtest81 (id, k, c, pad) VALUES (50127, 50435, '08676761217-51396177425-76348423335-71804524459-43800611424-22353735473-39492254361-97825670350-84120689198-79488444414', '45792476941-65368585208-89891477291-85749766483-87928795270')\G

# Query 5: 0 QPS, 0x concurrency, ID 0x558CAEF5F387E929 at byte 15585 ____
# Scores: V/M = 0.01
# Attribute    pct   total     min     max     avg     95%  stddev  median
# ============ === ======= ======= ======= ======= ======= ======= =======
# Count          5      25
# Exec time      5      3s   100ms   190ms   124ms   180ms    26ms   116ms
# Lock time      2   893us    12us   350us    35us    52us    65us    17us
# Rows sent      0      25       1       1       1       1       0       1
# Rows examine   0      25       1       1       1       1       0       1
# Query size     5     922      36      37   36.88   36.69    0.50   36.69
# String:
# Hosts        10.200.142.231
# Time         2018-02-27... (1/4%), 2018-02-27... (1/4%)... 23 more
# Users        root
# Query_time distribution
#   1us
#  10us
# 100us
#   1ms
#  10ms
# 100ms  ################################################################
#    1s
#  10s+
# Tables
#    SHOW TABLE STATUS LIKE 'sbtest99'\G
#    SHOW CREATE TABLE `sbtest99`\G
# EXPLAIN /*!50100 PARTITIONS*/
SELECT c FROM sbtest99 WHERE id=39015\G


统计了下pg的慢查询的平均执行时间约500ms,远比mysql的平均时间长。

因为mysql要写binlog和redolog,写日志的次数比pg的要多,所以在配置了数据不丢失的条件下,pg的性能要比mysql的性能高。
内容来自用户分享和网络整理,不保证内容的准确性,如有侵权内容,可联系管理员处理 点击这里给我发消息
标签: