MySQL分表優(yōu)化試驗(yàn)
2024-07-24 12:40:20
供稿：網(wǎng)友
我們的項(xiàng)目中有好多不等于的情況。今天寫這篇文章簡(jiǎn)單的分析一下怎么個(gè)優(yōu)化法。 這里的分表邏輯是根據(jù)t_group表的user_name組的個(gè)數(shù)來(lái)分的。 因?yàn)檫@種情況單獨(dú)user_name字段上的索引就屬于爛索引。起不了啥名明顯的效果。 1、試驗(yàn)PROCEDURE. DELIMITER $$ DROP PROCEDURE `t_girl`.`sp_split_table`$$ CREATE PROCEDURE `t_girl`.`sp_split_table`() BEGIN declare done int default 0; declare v_user_name varchar(20) default ''; declare v_table_name varchar(64) default ''; -- Get all users' name. declare cur1 cursor for select user_name from t_group group by user_name; -- Deal with error or warnings. declare continue handler for 1329 set done = 1; -- Open cursor. open cur1; while done <> 1 do fetch cur1 into v_user_name; if not done then -- Get table name. set v_table_name = concat('t_group_',v_user_name); -- Create new extra table. set @stmt = concat('create table ',v_table_name,' like t_group'); prepare s1 from @stmt; execute s1; drop prepare s1; -- Load data into it. set @stmt = concat('insert into ',v_table_name,' select * from t_group where user_name = ''',v_user_name,''''); prepare s1 from @stmt; execute s1; drop prepare s1; end if; end while; -- Close cursor. close cur1; -- Free variable from memory. set @stmt = NULL; END$$ DELIMITER ; 2、試驗(yàn)表。 我們用一個(gè)有一千萬(wàn)條記錄的表來(lái)做測(cè)試。 mysql> select count(*) from t_group; +----------+ | count(*) | +----------+ | 10388608 | +----------+ 1 row in set (0.00 sec) 表結(jié)構(gòu)。 mysql> desc t_group; +-------------+------------------+------+-----+-------------------+----------------+ | Field | Type | Null | Key | Default | Extra | +-------------+------------------+------+-----+-------------------+----------------+ | id | int(10) unsigned | NO | PRI | NULL | auto_increment | | money | decimal(10,2) | NO | | | | | user_name | varchar(20) | NO | MUL | | | | create_time | timestamp | NO | | CURRENT_TIMESTAMP | | +-------------+------------------+------+-----+-------------------+----------------+ 4 rows in set (0.00 sec) 索引情況。 mysql> show index from t_group; +---------+------------+------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+ |Table | Non_unique | Key_name | Seq_in_index | Column_name |Collation | Cardinality | Sub_part | Packed | Null | Index_type |Comment | +---------+------------+------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+ |t_group | 0 | PRIMARY | 1 | id |A | 10388608 | NULL | NULL | | BTREE | | | t_group | 1 | idx_user_name | 1 | user_name | A | 8 | NULL | NULL | |BTREE | | | t_group | 1 | idx_combination1| 1 | user_name | A | 8 | NULL |NULL | | BTREE | | | t_group | 1 |idx_combination1 | 2 | money | A | 3776| NULL | NULL | | BTREE | | +---------+------------+------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+ 4 rows in set (0.00 sec) PS: idx_combination1 這個(gè)索引是必須的，因?yàn)橐獙?duì)user_name來(lái)GROUP BY。此時(shí)屬于松散索引掃描！當(dāng)然完了后你可以干掉她。 idx_user_name 這個(gè)索引是為了加快單獨(dú)執(zhí)行constant這種類型的查詢。 我們要根據(jù)用戶名來(lái)分表。 mysql> select user_name from t_group where 1 group by user_name; +-----------+ | user_name | +-----------+ | david | | leo | | livia | | lucy | | sarah | | simon | | sony | | sunny | +-----------+ 8 rows in set (0.00 sec) 所以結(jié)果表應(yīng)該是這樣的。 mysql> show tables like 't_group_%'; +------------------------------+ | Tables_in_t_girl (t_group_%) | +------------------------------+ | t_group_david | | t_group_leo | | t_group_livia | | t_group_lucy | | t_group_sarah | | t_group_simon | | t_group_sony | | t_group_sunny | +------------------------------+ 8 rows in set (0.00 sec) 3、對(duì)比結(jié)果。 mysql> select count(*) from t_group where user_name = 'david'; +----------+ | count(*) | +----------+ | 1298576 | +----------+ 1 row in set (1.71 sec) 執(zhí)行了將近2秒。 mysql> select count(*) from t_group_david; +----------+ | count(*) | +----------+ | 1298576 | +----------+ 1 row in set (0.00 sec) 幾乎是瞬間的。 mysql> select count(*) from t_group where user_name <> 'david'; +----------+ | count(*) | +----------+ | 9090032 | +----------+ 1 row in set (9.26 sec) 執(zhí)行了將近10秒，可以想象，這個(gè)是實(shí)際的項(xiàng)目中是不能忍受的。 mysql> select (select count(*) from t_group) - (select count(*) from t_group_david) as total; +---------+ | total | +---------+ | 9090032 | +---------+ 1 row in set (0.00 sec) 幾乎是瞬間的。 我們來(lái)看看聚集函數(shù)。 對(duì)于原表的操作。 mysql> select min(money),max(money) from t_group where user_name = 'david'; +------------+------------+ | min(money) | max(money) | +------------+------------+ | -6.41 | 500.59 | +------------+------------+ 1 row in set (0.00 sec) 最小，最大值都是FULL INDEX SCAN。所以是瞬間的。 mysql> select sum(money),avg(money) from t_group where user_name = 'david'; +--------------+------------+ | sum(money) | avg(money) | +--------------+------------+ | 319992383.84 | 246.417910 | +--------------+------------+ 1 row in set (2.15 sec) 其他聚集函數(shù)的結(jié)果就不是FULL INDEX SCAN了。耗時(shí)2.15秒。 對(duì)于小表的操作。 mysql> select min(money),max(money) from t_group_david; +------------+------------+ | min(money) | max(money) | +------------+------------+ | -6.41 | 500.59 | +------------+------------+ 1 row in set (1.50 sec) 最大最小值完全是FULL TABLE SCAN,耗時(shí)1.50秒，不劃算。以此看來(lái)。 mysql> select sum(money),avg(money) from t_group_david; +--------------+------------+ | sum(money) | avg(money) | +--------------+------------+ | 319992383.84 | 246.417910 | +--------------+------------+ 1 row in set (1.68 sec) 取得這兩個(gè)結(jié)果也是花了快2秒，快了一點(diǎn)。 我們來(lái)看看這個(gè)小表的結(jié)構(gòu)。 mysql> desc t_group_david; +-------------+------------------+------+-----+-------------------+----------------+ | Field | Type | Null | Key | Default | Extra | +-------------+------------------+------+-----+-------------------+----------------+ | id | int(10) unsigned | NO | PRI | NULL | auto_increment | | money | decimal(10,2) | NO | | | | | user_name | varchar(20) | NO | MUL | | | | create_time | timestamp | NO | | CURRENT_TIMESTAMP | | +-------------+------------------+------+-----+-------------------+----------------+ 4 rows in set (0.00 sec) 明顯的user_name屬性是多余的。那么就干掉它。 mysql> alter table t_group_david drop user_name; Query OK, 1298576 rows affected (7.58 sec) Records: 1298576 Duplicates: 0 Warnings: 0 現(xiàn)在來(lái)重新對(duì)小表運(yùn)行查詢 mysql> select min(money),max(money) from t_group_david; +------------+------------+ | min(money) | max(money) | +------------+------------+ | -6.41 | 500.59 | +------------+------------+ 1 row in set (0.00 sec) 此時(shí)是瞬間的。 mysql> select sum(money),avg(money) from t_group_david; +--------------+------------+ | sum(money) | avg(money) | +--------------+------------+ | 319992383.84 | 246.417910 | +--------------+------------+ 1 row in set (0.94 sec) 這次算是控制在一秒以內(nèi)了。 mysql> Aborted 小總結(jié)一下：分出的小表的屬性盡量越少越好。大膽的去干吧。
