MySQL数据库性能调优、架构优化
我有一个 MySQL 数据库,其架构非常简单。有 parent
、child
和 access
表。
parent
存储 51 个字段,除 4 个 longtext
字段和主键 (一个bigint。除了主键之外,还有其他 3 个字段的索引。一个这样子表就可以将其作为外键引用。
child
存储 23 个字段,其中大部分是 varchar
以及一些 text
字段。 varchar(256)
字段用作外键以将其链接到父级。不过,外键字段的实际内容预计都少于 60 个字符。
accesss
有一个 bigint
字段和一个 varchar
字段,它们一起构成主键,bigint
字段是主键将其链接到parent
的外键。
access
表用于指定哪些用户有权访问父级
中的哪些记录。可能有多个用户应该有权访问任何记录。
parent
中有大约 2e6 行(因此 ,access
)child
中有大约 2e7 行。编辑:抱歉,access
有 5329840 行。即,parent
中的每一行在 access
中都有一行或多行。
上述架构基于旧的 FileMaker 数据库,我们希望将其迁移到 MySQL。我确信这并不理想,但我不知道具体原因。
查询的速度快慢取决于参数。因此,例如,如果 bob
可以访问多条记录,则以下查询将花费一两秒的时间。但是,如果用户 bob
没有访问权限的记录(例如,如果没有名为 bob
的用户),则查询将需要几分钟(例如 12 分钟):
SELECT
p."RecordID", p."SerialNumber", p."Folder", p."NoteType",
p."FirstName", p."LastName", p."DOB", p."Body", p."From",
p."DateTxt", a."UserName" AS Access
FROM parent AS p
INNER JOIN access AS a ON a."RecordID" = p."RecordID"
WHERE p."RecordID" > 123
AND a."UserName" = 'bob'
ORDER BY p."RecordID"
LIMIT 500;
以下是 EXPLAIN
关于查询的说法:
+----+-------------+-------+--------+---------------+---------+---------+---------------------+--------+--------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------+--------+---------------+---------+---------+---------------------+--------+--------------------------+
| 1 | SIMPLE | p | range | PRIMARY | PRIMARY | 8 | NULL | 731752 | Using where |
| 1 | SIMPLE | a | eq_ref | PRIMARY | PRIMARY | 74 | db.p.RecordID,const | 1 | Using where; Using index |
+----+-------------+-------+--------+---------------+---------+---------+---------------------+--------+--------------------------+
2 rows in set (0.01 sec)
那么,有什么方法可以加快不匹配查询的速度吗?这可能是由于对所有内容都使用 varchar/text 字段造成的吗?使用 varchar(256) 字段作为外键会导致问题吗(尽管上面的查询中没有使用它)?或者是查询惹的祸?
编辑:我刚刚意识到 access
表上的 PRIMARY KEY ("RecordID", "UserName")
未用于 SELECT ... FROM 访问 WHERE UserName = 'blah'
。我已经在 UserName 列上创建了一个索引,这似乎已经解决了问题。如果有人有建议,我仍然会很感激。
EXPLAIN
的当前输出:
+----+-------------+-------+--------+---------------+---------+---------+---------------------+--------+--------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------+--------+---------------+---------+---------+---------------------+--------+--------------------------+
| 1 | SIMPLE | p | range | PRIMARY | PRIMARY | 8 | NULL | 605020 | Using where |
| 1 | SIMPLE | a | eq_ref | PRIMARY,UNidx | PRIMARY | 74 | db.p.RecordID,const | 1 | Using where; Using index |
+----+-------------+-------+--------+---------------+---------+---------+---------------------+--------+--------------------------+
2 rows in set (0.00 sec)
编辑:@DRapp 的建议确实产生了巨大的差异。无论有或没有 access.UserName 上的索引,查询都很快。如果我删除索引并在没有 STRAIGHT_JOIN
的情况下尝试 DRapp 的查询,那么它会再次变慢。
DRapp 的查询在 access.UserName 上没有索引:
mysql> explain SELECT STRAIGHT_JOIN p."RecordID", p."SerialNumber", p."Folder", p."NoteType", p."FirstName", p."LastName", p."DOB", p."Body", p."From", p."DateTxt", a."UserName" AS Access FROM access as a, parent AS p where a."UserName" = 'bob' and a."RecordID" > 123 and a."RecordID" = p."RecordID" order by a."RecordID" limit 500;
+----+-------------+-------+--------+---------------+---------+---------+---------------+---------+--------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------+--------+---------------+---------+---------+---------------+---------+--------------------------+
| 1 | SIMPLE | a | range | PRIMARY | PRIMARY | 8 | NULL | 2382668 | Using where; Using index |
| 1 | SIMPLE | p | eq_ref | PRIMARY | PRIMARY | 8 | bb.a.RecordID | 1 | |
+----+-------------+-------+--------+---------------+---------+---------+---------------+---------+--------------------------+
2 rows in set (0.00 sec)
与 access.UserName
上的索引相同的查询:
mysql> explain SELECT STRAIGHT_JOIN ...;
+----+-------------+-------+--------+---------------+---------+---------+---------------+---------+--------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------+--------+---------------+---------+---------+---------------+---------+--------------------------+
| 1 | SIMPLE | a | ref | PRIMARY,UNidx | UNidx | 66 | const | 1209780 | Using where; Using index |
| 1 | SIMPLE | p | eq_ref | PRIMARY | PRIMARY | 8 | db.a.RecordID | 1 | |
+----+-------------+-------+--------+---------------+---------+---------+---------------+---------+--------------------------+
2 rows in set (0.00 sec)
在 access.UserName
上没有索引且没有 STRAIGHT_JOIN< 的相同查询/代码>:
mysql> explain SELECT p."RecordID", p."SerialNumber", p."Folder", p."NoteType", p."FirstName", p."LastName", p."DOB", p."Body", p."From", p."DateTxt", a."UserName" AS Access FROM access as a, parent AS p where a."UserName" = 'zzz' and a."RecordID" > 123 and a."RecordID" = p."RecordID" order by a."RecordID" limit 500;
+----+-------------+-------+--------+---------------+---------+---------+---------------------+--------+----------------------------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------+--------+---------------+---------+---------+---------------------+--------+----------------------------------------------+
| 1 | SIMPLE | p | range | PRIMARY | PRIMARY | 8 | NULL | 484016 | Using where; Using temporary; Using filesort |
| 1 | SIMPLE | a | eq_ref | PRIMARY | PRIMARY | 74 | db.p.RecordID,const | 1 | Using where; Using index |
+----+-------------+-------+--------+---------------+---------+---------+---------------------+--------+----------------------------------------------+
2 rows in set (0.00 sec)
I have a MySQL database with a pretty simple schema. There are parent
, child
and access
tables.
parent
stores 51 fields that are all varchar
(ranging in length from 16 to 512) except for for 4 longtext
fields and the primary key which is a bigint
. Besides the primary key, there are indexes on 3 other fields. One so that the child
table can refer to it as a foreign key.
child
stores 23 fields that are mostly varchar
with some text
fields. A varchar(256)
field is used as the foreign key to link it to parent. The actual contents of the foreign key field are all expected to be shorter than 60 characters, though.
accesss
has a bigint
field and a varchar
field that together make up the primary key and the bigint
field is the foreign key that links it to parent
.
The access
table is used to specify which users have access to which records from parent
. There may be multiple users who should have access to any record.
There are around 2e6 rows in parent
(and therefore and around 2e7 rows in access
)child
. EDIT: Sorry, access
has 5329840 rows. i.e. there are one or more rows in access
for every row in parent
.
The above schema is based on an old FileMaker database that we're looking to migrate to MySQL. I am sure it is not ideal, but I don't know exactly why.
Queries are fast or pretty slow depending on the parameters. So, e.g. the following query will take a second or two if there are several records that bob
has access to. The query will take several minutes (e.g. 12 minutes), though, if there are no records that user bob
has access to (e.g. if there is no user called bob
):
SELECT
p."RecordID", p."SerialNumber", p."Folder", p."NoteType",
p."FirstName", p."LastName", p."DOB", p."Body", p."From",
p."DateTxt", a."UserName" AS Access
FROM parent AS p
INNER JOIN access AS a ON a."RecordID" = p."RecordID"
WHERE p."RecordID" > 123
AND a."UserName" = 'bob'
ORDER BY p."RecordID"
LIMIT 500;
Here's what EXPLAIN
says about the query:
+----+-------------+-------+--------+---------------+---------+---------+---------------------+--------+--------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------+--------+---------------+---------+---------+---------------------+--------+--------------------------+
| 1 | SIMPLE | p | range | PRIMARY | PRIMARY | 8 | NULL | 731752 | Using where |
| 1 | SIMPLE | a | eq_ref | PRIMARY | PRIMARY | 74 | db.p.RecordID,const | 1 | Using where; Using index |
+----+-------------+-------+--------+---------------+---------+---------+---------------------+--------+--------------------------+
2 rows in set (0.01 sec)
So, is there some way to speed up the non-matching queries? Could this be caused by using varchar/text fields for everything? Would using a varchar(256) field as a foreign key cause problems (although it's not used in the above query)? Or is the query to blame?
EDIT: I've just realised that the PRIMARY KEY ("RecordID", "UserName")
on the access
table is not being used for SELECT ... FROM access WHERE UserName = 'blah'
. I've created an index on the UserName column and that appears to have fixed the problem. I'd still appreciate it if anyone has advice.
Current output of EXPLAIN
:
+----+-------------+-------+--------+---------------+---------+---------+---------------------+--------+--------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------+--------+---------------+---------+---------+---------------------+--------+--------------------------+
| 1 | SIMPLE | p | range | PRIMARY | PRIMARY | 8 | NULL | 605020 | Using where |
| 1 | SIMPLE | a | eq_ref | PRIMARY,UNidx | PRIMARY | 74 | db.p.RecordID,const | 1 | Using where; Using index |
+----+-------------+-------+--------+---------------+---------+---------+---------------------+--------+--------------------------+
2 rows in set (0.00 sec)
EDIT: @DRapp's suggestion does make a huge difference. The query is fast with or without the index on access.UserName. If I drop the index and try DRapp's query without the STRAIGHT_JOIN
, then it is again slow.
DRapp's query without index on access.UserName:
mysql> explain SELECT STRAIGHT_JOIN p."RecordID", p."SerialNumber", p."Folder", p."NoteType", p."FirstName", p."LastName", p."DOB", p."Body", p."From", p."DateTxt", a."UserName" AS Access FROM access as a, parent AS p where a."UserName" = 'bob' and a."RecordID" > 123 and a."RecordID" = p."RecordID" order by a."RecordID" limit 500;
+----+-------------+-------+--------+---------------+---------+---------+---------------+---------+--------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------+--------+---------------+---------+---------+---------------+---------+--------------------------+
| 1 | SIMPLE | a | range | PRIMARY | PRIMARY | 8 | NULL | 2382668 | Using where; Using index |
| 1 | SIMPLE | p | eq_ref | PRIMARY | PRIMARY | 8 | bb.a.RecordID | 1 | |
+----+-------------+-------+--------+---------------+---------+---------+---------------+---------+--------------------------+
2 rows in set (0.00 sec)
Same query with the index on access.UserName
:
mysql> explain SELECT STRAIGHT_JOIN ...;
+----+-------------+-------+--------+---------------+---------+---------+---------------+---------+--------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------+--------+---------------+---------+---------+---------------+---------+--------------------------+
| 1 | SIMPLE | a | ref | PRIMARY,UNidx | UNidx | 66 | const | 1209780 | Using where; Using index |
| 1 | SIMPLE | p | eq_ref | PRIMARY | PRIMARY | 8 | db.a.RecordID | 1 | |
+----+-------------+-------+--------+---------------+---------+---------+---------------+---------+--------------------------+
2 rows in set (0.00 sec)
Same query without the index on access.UserName
and without the STRAIGHT_JOIN
:
mysql> explain SELECT p."RecordID", p."SerialNumber", p."Folder", p."NoteType", p."FirstName", p."LastName", p."DOB", p."Body", p."From", p."DateTxt", a."UserName" AS Access FROM access as a, parent AS p where a."UserName" = 'zzz' and a."RecordID" > 123 and a."RecordID" = p."RecordID" order by a."RecordID" limit 500;
+----+-------------+-------+--------+---------------+---------+---------+---------------------+--------+----------------------------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------+--------+---------------+---------+---------+---------------------+--------+----------------------------------------------+
| 1 | SIMPLE | p | range | PRIMARY | PRIMARY | 8 | NULL | 484016 | Using where; Using temporary; Using filesort |
| 1 | SIMPLE | a | eq_ref | PRIMARY | PRIMARY | 74 | db.p.RecordID,const | 1 | Using where; Using index |
+----+-------------+-------+--------+---------------+---------+---------+---------------------+--------+----------------------------------------------+
2 rows in set (0.00 sec)
如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱,如果其他用户或者作者回复了您的评论,将不能在第一时间通知您!
发布评论
评论(1)
我总是很幸运地使用 MySQL 中的“STRAIGHT_JOIN”子句,并将主表基础放在限定符列表中的第一位。在这种情况下,您的“访问”表会查找 Bob,然后查看 Bob 有权查看的记录。如果 ACCESS 查询失败,则无需深入查看。此外,由于连接基于相同的“RecordID”,因此我更改了对“a”的引用。桌子。由于此查询首先基于用户名,因此我也有一个密钥。我对它的解释和性能也很感兴趣。
I've always had good luck using the "STRAIGHT_JOIN" clause in MySQL, and putting the primary table basis as first in the list of qualifiers. In this case, your "Access" table looking for Bob, THEN looking at the records Bob has access to see. If it fails at the ACCESS query, no need to look deeper. Additionally, since the join is based on same "RecordID", I've changed references to the "a." table. Since this query is based on username first, I would have a key on it too. I'd be interested in its Explain, and performance too.