MySQL数据库性能调优、架构优化

发布于 2024-10-14 09:36:28 字数 7106 浏览 2 评论 0原文

我有一个 MySQL 数据库,其架构非常简单。有 parentchildaccess 表。

parent 存储 51 个字段,除 4 个 longtext 字段和主键 (一个bigint。除了主键之外,还有其他 3 个字段的索引。一个这样子表就可以将其作为外键引用。

child 存储 23 个字段,其中大部分是 varchar 以及一些 text 字段。 varchar(256) 字段用作外键以将其链接到父级。不过,外键字段的实际内容预计都少于 60 个字符。

accesss 有一个 bigint 字段和一个 varchar 字段,它们一起构成主键,bigint 字段是主键将其链接到parent的外键。

access 表用于指定哪些用户有权访问父级 中的哪些记录。可能有多个用户应该有权访问任何记录。

parent 中有大约 2e6 行(因此 accesschild 中有大约 2e7 行。编辑:抱歉,access 有 5329840 行。即,parent 中的每一行在 access 中都有一行或多行。

上述架构基于旧的 FileMaker 数据库,我们希望将其迁移到 MySQL。我确信这并不理想,但我不知道具体原因。

查询的速度快慢取决于参数。因此,例如,如果 bob 可以访问多条记录,则以下查询将花费一两秒的时间。但是,如果用户 bob 没有访问权限的记录(例如,如果没有名为 bob 的用户),则查询将需要几分钟(例如 12 分钟):

SELECT
    p."RecordID", p."SerialNumber", p."Folder", p."NoteType",
    p."FirstName", p."LastName", p."DOB", p."Body", p."From",
    p."DateTxt", a."UserName" AS Access
FROM parent AS p
INNER JOIN access AS a ON a."RecordID" = p."RecordID"
WHERE p."RecordID" > 123
AND a."UserName" = 'bob'
ORDER BY p."RecordID"
LIMIT 500;

以下是 EXPLAIN 关于查询的说法:

+----+-------------+-------+--------+---------------+---------+---------+---------------------+--------+--------------------------+
| id | select_type | table | type   | possible_keys | key     | key_len | ref                 | rows   | Extra                    |
+----+-------------+-------+--------+---------------+---------+---------+---------------------+--------+--------------------------+
|  1 | SIMPLE      | p     | range  | PRIMARY       | PRIMARY | 8       | NULL                | 731752 | Using where              |
|  1 | SIMPLE      | a     | eq_ref | PRIMARY       | PRIMARY | 74      | db.p.RecordID,const |      1 | Using where; Using index |
+----+-------------+-------+--------+---------------+---------+---------+---------------------+--------+--------------------------+
2 rows in set (0.01 sec)

那么,有什么方法可以加快不匹配查询的速度吗?这可能是由于对所有内容都使用 varchar/text 字段造成的吗?使用 varchar(256) 字段作为外键会导致问题吗(尽管上面的查询中没有使用它)?或者是查询惹的祸?

编辑:我刚刚意识到 access 表上的 PRIMARY KEY ("RecordID", "UserName") 未用于 SELECT ... FROM 访问 WHERE UserName = 'blah'。我已经在 UserName 列上创建了一个索引,这似乎已经解决了问题。如果有人有建议,我仍然会很感激。

EXPLAIN 的当前输出:

+----+-------------+-------+--------+---------------+---------+---------+---------------------+--------+--------------------------+
| id | select_type | table | type   | possible_keys | key     | key_len | ref                 | rows   | Extra                    |
+----+-------------+-------+--------+---------------+---------+---------+---------------------+--------+--------------------------+
|  1 | SIMPLE      | p     | range  | PRIMARY       | PRIMARY | 8       | NULL                | 605020 | Using where              |
|  1 | SIMPLE      | a     | eq_ref | PRIMARY,UNidx | PRIMARY | 74      | db.p.RecordID,const |      1 | Using where; Using index |
+----+-------------+-------+--------+---------------+---------+---------+---------------------+--------+--------------------------+
2 rows in set (0.00 sec)

编辑:@DRapp 的建议确实产生了巨大的差异。无论有或没有 access.UserName 上的索引,查询都很快。如果我删除索引并在没有 STRAIGHT_JOIN 的情况下尝试 DRapp 的查询,那么它会再次变慢。

DRapp 的查询在 access.UserName 上没有索引:

mysql> explain SELECT STRAIGHT_JOIN p."RecordID", p."SerialNumber", p."Folder", p."NoteType", p."FirstName", p."LastName", p."DOB", p."Body", p."From", p."DateTxt", a."UserName" AS Access     FROM access as a, parent AS p where a."UserName" = 'bob' and a."RecordID" > 123 and a."RecordID" = p."RecordID" order by a."RecordID" limit 500;
+----+-------------+-------+--------+---------------+---------+---------+---------------+---------+--------------------------+
| id | select_type | table | type   | possible_keys | key     | key_len | ref           | rows    | Extra                    |
+----+-------------+-------+--------+---------------+---------+---------+---------------+---------+--------------------------+
|  1 | SIMPLE      | a     | range  | PRIMARY       | PRIMARY | 8       | NULL          | 2382668 | Using where; Using index |
|  1 | SIMPLE      | p     | eq_ref | PRIMARY       | PRIMARY | 8       | bb.a.RecordID |       1 |                          |
+----+-------------+-------+--------+---------------+---------+---------+---------------+---------+--------------------------+
2 rows in set (0.00 sec)

access.UserName 上的索引相同的查询:

mysql> explain SELECT STRAIGHT_JOIN ...;
+----+-------------+-------+--------+---------------+---------+---------+---------------+---------+--------------------------+
| id | select_type | table | type   | possible_keys | key     | key_len | ref           | rows    | Extra                    |
+----+-------------+-------+--------+---------------+---------+---------+---------------+---------+--------------------------+
|  1 | SIMPLE      | a     | ref    | PRIMARY,UNidx | UNidx   | 66      | const         | 1209780 | Using where; Using index |
|  1 | SIMPLE      | p     | eq_ref | PRIMARY       | PRIMARY | 8       | db.a.RecordID |       1 |                          |
+----+-------------+-------+--------+---------------+---------+---------+---------------+---------+--------------------------+
2 rows in set (0.00 sec)

access.UserName 上没有索引且没有 STRAIGHT_JOIN< 的相同查询/代码>:

mysql> explain SELECT p."RecordID", p."SerialNumber", p."Folder", p."NoteType", p."FirstName", p."LastName", p."DOB", p."Body", p."From", p."DateTxt", a."UserName" AS Access FROM access as a, parent AS p where a."UserName" = 'zzz' and a."RecordID" > 123 and a."RecordID" = p."RecordID" order by a."RecordID" limit 500;
+----+-------------+-------+--------+---------------+---------+---------+---------------------+--------+----------------------------------------------+
| id | select_type | table | type   | possible_keys | key     | key_len | ref                 | rows   | Extra                                        |
+----+-------------+-------+--------+---------------+---------+---------+---------------------+--------+----------------------------------------------+
|  1 | SIMPLE      | p     | range  | PRIMARY       | PRIMARY | 8       | NULL                | 484016 | Using where; Using temporary; Using filesort |
|  1 | SIMPLE      | a     | eq_ref | PRIMARY       | PRIMARY | 74      | db.p.RecordID,const |      1 | Using where; Using index                     |
+----+-------------+-------+--------+---------------+---------+---------+---------------------+--------+----------------------------------------------+
2 rows in set (0.00 sec)

I have a MySQL database with a pretty simple schema. There are parent, child and access tables.

parent stores 51 fields that are all varchar (ranging in length from 16 to 512) except for for 4 longtext fields and the primary key which is a bigint. Besides the primary key, there are indexes on 3 other fields. One so that the child table can refer to it as a foreign key.

child stores 23 fields that are mostly varchar with some text fields. A varchar(256) field is used as the foreign key to link it to parent. The actual contents of the foreign key field are all expected to be shorter than 60 characters, though.

accesss has a bigint field and a varchar field that together make up the primary key and the bigint field is the foreign key that links it to parent.

The access table is used to specify which users have access to which records from parent. There may be multiple users who should have access to any record.

There are around 2e6 rows in parent (and therefore access) and around 2e7 rows in child. EDIT: Sorry, access has 5329840 rows. i.e. there are one or more rows in access for every row in parent.

The above schema is based on an old FileMaker database that we're looking to migrate to MySQL. I am sure it is not ideal, but I don't know exactly why.

Queries are fast or pretty slow depending on the parameters. So, e.g. the following query will take a second or two if there are several records that bob has access to. The query will take several minutes (e.g. 12 minutes), though, if there are no records that user bob has access to (e.g. if there is no user called bob):

SELECT
    p."RecordID", p."SerialNumber", p."Folder", p."NoteType",
    p."FirstName", p."LastName", p."DOB", p."Body", p."From",
    p."DateTxt", a."UserName" AS Access
FROM parent AS p
INNER JOIN access AS a ON a."RecordID" = p."RecordID"
WHERE p."RecordID" > 123
AND a."UserName" = 'bob'
ORDER BY p."RecordID"
LIMIT 500;

Here's what EXPLAIN says about the query:

+----+-------------+-------+--------+---------------+---------+---------+---------------------+--------+--------------------------+
| id | select_type | table | type   | possible_keys | key     | key_len | ref                 | rows   | Extra                    |
+----+-------------+-------+--------+---------------+---------+---------+---------------------+--------+--------------------------+
|  1 | SIMPLE      | p     | range  | PRIMARY       | PRIMARY | 8       | NULL                | 731752 | Using where              |
|  1 | SIMPLE      | a     | eq_ref | PRIMARY       | PRIMARY | 74      | db.p.RecordID,const |      1 | Using where; Using index |
+----+-------------+-------+--------+---------------+---------+---------+---------------------+--------+--------------------------+
2 rows in set (0.01 sec)

So, is there some way to speed up the non-matching queries? Could this be caused by using varchar/text fields for everything? Would using a varchar(256) field as a foreign key cause problems (although it's not used in the above query)? Or is the query to blame?

EDIT: I've just realised that the PRIMARY KEY ("RecordID", "UserName") on the access table is not being used for SELECT ... FROM access WHERE UserName = 'blah'. I've created an index on the UserName column and that appears to have fixed the problem. I'd still appreciate it if anyone has advice.

Current output of EXPLAIN:

+----+-------------+-------+--------+---------------+---------+---------+---------------------+--------+--------------------------+
| id | select_type | table | type   | possible_keys | key     | key_len | ref                 | rows   | Extra                    |
+----+-------------+-------+--------+---------------+---------+---------+---------------------+--------+--------------------------+
|  1 | SIMPLE      | p     | range  | PRIMARY       | PRIMARY | 8       | NULL                | 605020 | Using where              |
|  1 | SIMPLE      | a     | eq_ref | PRIMARY,UNidx | PRIMARY | 74      | db.p.RecordID,const |      1 | Using where; Using index |
+----+-------------+-------+--------+---------------+---------+---------+---------------------+--------+--------------------------+
2 rows in set (0.00 sec)

EDIT: @DRapp's suggestion does make a huge difference. The query is fast with or without the index on access.UserName. If I drop the index and try DRapp's query without the STRAIGHT_JOIN, then it is again slow.

DRapp's query without index on access.UserName:

mysql> explain SELECT STRAIGHT_JOIN p."RecordID", p."SerialNumber", p."Folder", p."NoteType", p."FirstName", p."LastName", p."DOB", p."Body", p."From", p."DateTxt", a."UserName" AS Access     FROM access as a, parent AS p where a."UserName" = 'bob' and a."RecordID" > 123 and a."RecordID" = p."RecordID" order by a."RecordID" limit 500;
+----+-------------+-------+--------+---------------+---------+---------+---------------+---------+--------------------------+
| id | select_type | table | type   | possible_keys | key     | key_len | ref           | rows    | Extra                    |
+----+-------------+-------+--------+---------------+---------+---------+---------------+---------+--------------------------+
|  1 | SIMPLE      | a     | range  | PRIMARY       | PRIMARY | 8       | NULL          | 2382668 | Using where; Using index |
|  1 | SIMPLE      | p     | eq_ref | PRIMARY       | PRIMARY | 8       | bb.a.RecordID |       1 |                          |
+----+-------------+-------+--------+---------------+---------+---------+---------------+---------+--------------------------+
2 rows in set (0.00 sec)

Same query with the index on access.UserName:

mysql> explain SELECT STRAIGHT_JOIN ...;
+----+-------------+-------+--------+---------------+---------+---------+---------------+---------+--------------------------+
| id | select_type | table | type   | possible_keys | key     | key_len | ref           | rows    | Extra                    |
+----+-------------+-------+--------+---------------+---------+---------+---------------+---------+--------------------------+
|  1 | SIMPLE      | a     | ref    | PRIMARY,UNidx | UNidx   | 66      | const         | 1209780 | Using where; Using index |
|  1 | SIMPLE      | p     | eq_ref | PRIMARY       | PRIMARY | 8       | db.a.RecordID |       1 |                          |
+----+-------------+-------+--------+---------------+---------+---------+---------------+---------+--------------------------+
2 rows in set (0.00 sec)

Same query without the index on access.UserName and without the STRAIGHT_JOIN:

mysql> explain SELECT p."RecordID", p."SerialNumber", p."Folder", p."NoteType", p."FirstName", p."LastName", p."DOB", p."Body", p."From", p."DateTxt", a."UserName" AS Access FROM access as a, parent AS p where a."UserName" = 'zzz' and a."RecordID" > 123 and a."RecordID" = p."RecordID" order by a."RecordID" limit 500;
+----+-------------+-------+--------+---------------+---------+---------+---------------------+--------+----------------------------------------------+
| id | select_type | table | type   | possible_keys | key     | key_len | ref                 | rows   | Extra                                        |
+----+-------------+-------+--------+---------------+---------+---------+---------------------+--------+----------------------------------------------+
|  1 | SIMPLE      | p     | range  | PRIMARY       | PRIMARY | 8       | NULL                | 484016 | Using where; Using temporary; Using filesort |
|  1 | SIMPLE      | a     | eq_ref | PRIMARY       | PRIMARY | 74      | db.p.RecordID,const |      1 | Using where; Using index                     |
+----+-------------+-------+--------+---------------+---------+---------+---------------------+--------+----------------------------------------------+
2 rows in set (0.00 sec)

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(1

疏忽 2024-10-21 09:36:28

我总是很幸运地使用 MySQL 中的“STRAIGHT_JOIN”子句,并将主表基础放在限定符列表中的第一位。在这种情况下,您的“访问”表会查找 Bob,然后查看 Bob 有权查看的记录。如果 ACCESS 查询失败,则无需深入查看。此外,由于连接基于相同的“RecordID”,因此我更改了对“a”的引用。桌子。由于此查询首先基于用户名,因此我也有一个密钥。我对它的解释和性能也很感兴趣。

SELECT STRAIGHT_JOIN
      p."RecordID",
      p."SerialNumber",
      p."Folder",
      p."NoteType",
      p."FirstName", 
      p."LastName", 
      p."DOB",
      p."Body",
      p."From",
      p."DateTxt", 
      a."UserName" AS Access 
   FROM 
      access as a,
      parent AS p 
   where 
          a."UserName" = 'bob'
      and a."RecordID" > 123
      and a."RecordID" = p."RecordID"
   order by
      a."RecordID"
   limit 
      500

I've always had good luck using the "STRAIGHT_JOIN" clause in MySQL, and putting the primary table basis as first in the list of qualifiers. In this case, your "Access" table looking for Bob, THEN looking at the records Bob has access to see. If it fails at the ACCESS query, no need to look deeper. Additionally, since the join is based on same "RecordID", I've changed references to the "a." table. Since this query is based on username first, I would have a key on it too. I'd be interested in its Explain, and performance too.

SELECT STRAIGHT_JOIN
      p."RecordID",
      p."SerialNumber",
      p."Folder",
      p."NoteType",
      p."FirstName", 
      p."LastName", 
      p."DOB",
      p."Body",
      p."From",
      p."DateTxt", 
      a."UserName" AS Access 
   FROM 
      access as a,
      parent AS p 
   where 
          a."UserName" = 'bob'
      and a."RecordID" > 123
      and a."RecordID" = p."RecordID"
   order by
      a."RecordID"
   limit 
      500
~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文