当前位置： 文江博客话题详情

MySQL查询性能——时间上的巨大差异

发布于 2024-08-27 03:58:35 字数 28039 浏览 15 评论 0原文

我有一个查询在两个数据集之间返回的时间差异很大。对于一组（数据库 A），它会在几秒钟内返回，对于另一组（数据库 B）......好吧，我还没有等待足够长的时间，但已经超过 10 分钟了。我已将这两个数据库转储到我的本地计算机，在那里我可以重现运行 MySQL 5.1.37 的问题。

奇怪的是，数据库 B 比数据库 A 小。

重现该问题的查询的精简版本是：

SELECT * FROM po_shipment ps 
JOIN po_shipment_item psi USING (ship_id) 
JOIN po_alloc pa ON ps.ship_id = pa.ship_id AND pa.UID_items = psi.UID_items 
JOIN po_header ph ON pa.hdr_id = ph.hdr_id 
LEFT JOIN EVENT_TABLE ev0 ON ev0.TABLE_ID1 = ps.ship_id AND ev0.EVENT_TYPE = 'MAS0' 
LEFT JOIN EVENT_TABLE ev1 ON ev1.TABLE_ID1 = ps.ship_id AND ev1.EVENT_TYPE = 'MAS1' 
LEFT JOIN EVENT_TABLE ev2 ON ev2.TABLE_ID1 = ps.ship_id AND ev2.EVENT_TYPE = 'MAS2' 
LEFT JOIN EVENT_TABLE ev3 ON ev3.TABLE_ID1 = ps.ship_id AND ev3.EVENT_TYPE = 'MAS3' 
LEFT JOIN EVENT_TABLE ev4 ON ev4.TABLE_ID1 = ps.ship_id AND ev4.EVENT_TYPE = 'MAS4' 
LEFT JOIN EVENT_TABLE ev5 ON ev5.TABLE_ID1 = ps.ship_id AND ev5.EVENT_TYPE = 'MAS5' 
WHERE ps.eta >= '2010-03-22'
GROUP BY ps.ship_id
LIMIT 100;

在大约 2 秒内返回的第一个数据库 (A) 的 EXPLAIN 查询计划是：

+----+-------------+-------+--------+----------------------------------------------------------------------------------------------------------------------------------------+----------------------------------+---------+------------------------------+------+----------------------------------------------+
| id | select_type | table | type   | possible_keys                                                                                                                          | key                              | key_len | ref                          | rows | Extra                                        |
+----+-------------+-------+--------+----------------------------------------------------------------------------------------------------------------------------------------+----------------------------------+---------+------------------------------+------+----------------------------------------------+
|  1 | SIMPLE      | ps    | range  | PRIMARY,IX_ETA_DATE                                                                                                                    | IX_ETA_DATE                      | 4       | NULL                         |  174 | Using where; Using temporary; Using filesort | 
|  1 | SIMPLE      | ev0   | ref    | IX_EVENT_ID_EVENT_TYPE                                                                                                                 | IX_EVENT_ID_EVENT_TYPE           | 36      | UNIVIS_PROD.ps.ship_id,const |    1 |                                              | 
|  1 | SIMPLE      | ev1   | ref    | IX_EVENT_ID_EVENT_TYPE                                                                                                                 | IX_EVENT_ID_EVENT_TYPE           | 36      | UNIVIS_PROD.ps.ship_id,const |    1 |                                              | 
|  1 | SIMPLE      | ev2   | ref    | IX_EVENT_ID_EVENT_TYPE                                                                                                                 | IX_EVENT_ID_EVENT_TYPE           | 36      | UNIVIS_PROD.ps.ship_id,const |    1 |                                              | 
|  1 | SIMPLE      | ev3   | ref    | IX_EVENT_ID_EVENT_TYPE                                                                                                                 | IX_EVENT_ID_EVENT_TYPE           | 36      | UNIVIS_PROD.ps.ship_id,const |    1 |                                              | 
|  1 | SIMPLE      | ev4   | ref    | IX_EVENT_ID_EVENT_TYPE                                                                                                                 | IX_EVENT_ID_EVENT_TYPE           | 36      | UNIVIS_PROD.ps.ship_id,const |    1 |                                              | 
|  1 | SIMPLE      | ev5   | ref    | IX_EVENT_ID_EVENT_TYPE                                                                                                                 | IX_EVENT_ID_EVENT_TYPE           | 36      | UNIVIS_PROD.ps.ship_id,const |    1 |                                              | 
|  1 | SIMPLE      | psi   | ref    | PRIMARY,IX_po_shipment_item_po_shipment1,FK_po_shipment_item_po_shipment1                                                              | IX_po_shipment_item_po_shipment1 | 4       | UNIVIS_PROD.ps.ship_id       |    1 |                                              | 
|  1 | SIMPLE      | pa    | ref    | IX_po_alloc_po_shipment_item2,IX_po_alloc_po_details_old,FK_po_alloc_po_shipment1,FK_po_alloc_po_shipment_item1,FK_po_alloc_po_header1 | FK_po_alloc_po_shipment1         | 4       | UNIVIS_PROD.psi.ship_id      |    5 | Using where                                  | 
|  1 | SIMPLE      | ph    | eq_ref | PRIMARY,IX_HDR_ID                                                                                                                      | PRIMARY                          | 4       | UNIVIS_PROD.pa.hdr_id        |    1 |                                              | 
+----+-------------+-------+--------+----------------------------------------------------------------------------------------------------------------------------------------+----------------------------------+---------+------------------------------+------+----------------------------------------------+

第二个数据库的 EXPLAIN 查询计划(B) 在 >600 秒内返回的是：

+----+-------------+-------+--------+----------------------------------------------------------------------------------------------------------------------------------------+----------------------------------+---------+--------------------------------+------+----------------------------------------------+
| id | select_type | table | type   | possible_keys                                                                                                                          | key                              | key_len | ref                            | rows | Extra                                        |
+----+-------------+-------+--------+----------------------------------------------------------------------------------------------------------------------------------------+----------------------------------+---------+--------------------------------+------+----------------------------------------------+
|  1 | SIMPLE      | ps    | range  | PRIMARY,IX_ETA_DATE                                                                                                                    | IX_ETA_DATE                      | 4       | NULL                           |   38 | Using where; Using temporary; Using filesort | 
|  1 | SIMPLE      | psi   | ref    | PRIMARY,IX_po_shipment_item_po_shipment1,FK_po_shipment_item_po_shipment1                                                              | IX_po_shipment_item_po_shipment1 | 4       | UNIVIS_DEV01.ps.ship_id        |    1 |                                              | 
|  1 | SIMPLE      | ev0   | ref    | IX_EVENT_ID_EVENT_TYPE                                                                                                                 | IX_EVENT_ID_EVENT_TYPE           | 36      | UNIVIS_DEV01.psi.ship_id,const |    1 |                                              | 
|  1 | SIMPLE      | ev1   | ref    | IX_EVENT_ID_EVENT_TYPE                                                                                                                 | IX_EVENT_ID_EVENT_TYPE           | 36      | UNIVIS_DEV01.psi.ship_id,const |    1 |                                              | 
|  1 | SIMPLE      | ev2   | ref    | IX_EVENT_ID_EVENT_TYPE                                                                                                                 | IX_EVENT_ID_EVENT_TYPE           | 36      | UNIVIS_DEV01.ps.ship_id,const  |    1 |                                              | 
|  1 | SIMPLE      | ev3   | ref    | IX_EVENT_ID_EVENT_TYPE                                                                                                                 | IX_EVENT_ID_EVENT_TYPE           | 36      | UNIVIS_DEV01.psi.ship_id,const |    1 |                                              | 
|  1 | SIMPLE      | ev4   | ref    | IX_EVENT_ID_EVENT_TYPE                                                                                                                 | IX_EVENT_ID_EVENT_TYPE           | 36      | UNIVIS_DEV01.psi.ship_id,const |    1 |                                              | 
|  1 | SIMPLE      | ev5   | ref    | IX_EVENT_ID_EVENT_TYPE                                                                                                                 | IX_EVENT_ID_EVENT_TYPE           | 36      | UNIVIS_DEV01.ps.ship_id,const  |    1 |                                              | 
|  1 | SIMPLE      | pa    | ref    | IX_po_alloc_po_shipment_item2,IX_po_alloc_po_details_old,FK_po_alloc_po_shipment1,FK_po_alloc_po_shipment_item1,FK_po_alloc_po_header1 | IX_po_alloc_po_shipment_item2    | 4       | UNIVIS_DEV01.ps.ship_id        |    4 | Using where                                  | 
|  1 | SIMPLE      | ph    | eq_ref | PRIMARY,IX_HDR_ID                                                                                                                      | PRIMARY                          | 4       | UNIVIS_DEV01.pa.hdr_id         |    1 |                                              | 
+----+-------------+-------+--------+----------------------------------------------------------------------------------------------------------------------------------------+----------------------------------+---------+--------------------------------+------+----------------------------------------------+

当数据库 B 正在运行时，我可以查看 MySQL 管理员，并且状态无限期地保持在“正在复制到 tmp 表”。数据库 A 也有这种状态，但只持续一秒钟左右。

这些数据库之间的表结构、索引、键等没有差异（我已经完成了显示创建表并对它们进行了比较）。

表的大小为：

database A:
po_shipment 1776
po_shipment_item 1945
po_alloc 36298
po_header 71642
EVENT_TABLE 1608

database B:
po_shipment 463
po_shipment_item 470
po_alloc 3291
po_header 56149
EVENT_TABLE 1089

需要注意的几点：

  • 删除 WHERE 子句会使 查询返回< 1秒。
  • 删除 GROUP BY 会使查询 返回< 1秒。
  • 删除 ev5、ev4、ev3 等使得 每一项查询都变得更快 已删除。

AJ 回答后更新： - 数据库 B 上的 Ship_id 大小（最大值 = 800002752）明显大于数据库 A（最大值 = 3489）。鉴于这些是 InnoDB 表，更改任何缓冲区是否有助于处理这种大小的键？ 进一步更新：我减小了按键的大小并重新分析，但性能仍然没有变化。

EVENT_TABLE 的 UPDATE desc：

请注意，这两个数据库中的 相同

+--------------------+--------------+------+-----+---------+----------------+
| Field              | Type         | Null | Key | Default | Extra          |
+--------------------+--------------+------+-----+---------+----------------+
| EVENT_TABLE_ID     | bigint(20)   | NO   | PRI | NULL    | auto_increment | 
| EVENT_TYPE         | varchar(10)  | NO   |     | NULL    |                | 
| TABLE_ID1          | int(11)      | NO   | MUL | NULL    |                | 
| TABLE_ID2          | int(11)      | YES  |     | NULL    |                | 
| TABLE_ID3          | int(11)      | YES  |     | NULL    |                | 
| TABLE_ID4          | int(11)      | YES  |     | NULL    |                | 
| EVENT_CREATED_DATE | datetime     | NO   |     | NULL    |                | 
| MESSAGE_REF        | varchar(100) | YES  |     | NULL    |                | 
+--------------------+--------------+------+-----+---------+----------------+

并且为了更好地衡量 SHOW CREATE TABLE EVENT_TABLE：

数据库之间唯一的区别是自动增量value

| EVENT_TABLE | CREATE TABLE `EVENT_TABLE` (
  `EVENT_TABLE_ID` bigint(20) NOT NULL AUTO_INCREMENT,
  `EVENT_TYPE` varchar(10) NOT NULL,
  `TABLE_ID1` int(11) NOT NULL,
  `TABLE_ID2` int(11) DEFAULT NULL,
  `TABLE_ID3` int(11) DEFAULT NULL,
  `TABLE_ID4` int(11) DEFAULT NULL,
  `EVENT_CREATED_DATE` datetime NOT NULL,
  `MESSAGE_REF` varchar(100) DEFAULT NULL,
  PRIMARY KEY (`EVENT_TABLE_ID`),
  KEY `IX_EVENT_ID_EVENT_TYPE` (`TABLE_ID1`,`EVENT_TYPE`)
) ENGINE=InnoDB AUTO_INCREMENT=1925 DEFAULT CHARSET=utf8 |

谁能建议如何解决这个问题？我错过了什么？

Michael Holzmann 提问后更新 以下是基于更新的 STRAIGHT_JOIN 查询的新查询计划。请注意，数据库 B 有“使用临时；使用文件排序”，而现在数据库 A 没有。这可能是由于按键太长或类似的原因造成的吗？

数据库A

+----+-------------+-------+--------+----------------------------------------------------------------------------------------------------------------------------------------+----------------------------------+---------+------------------------------+------+-------------+
| id | select_type | table | type   | possible_keys                                                                                                                          | key                              | key_len | ref                          | rows | Extra       |
+----+-------------+-------+--------+----------------------------------------------------------------------------------------------------------------------------------------+----------------------------------+---------+------------------------------+------+-------------+
|  1 | SIMPLE      | ps    | index  | PRIMARY,IX_ETA_DATE                                                                                                                    | PRIMARY                          | 4       | NULL                         |  168 | Using where | 
|  1 | SIMPLE      | ev0   | ref    | IX_EVENT_ID_EVENT_TYPE                                                                                                                 | IX_EVENT_ID_EVENT_TYPE           | 36      | UNIVIS_PROD.ps.ship_id,const |    1 |             | 
|  1 | SIMPLE      | ev1   | ref    | IX_EVENT_ID_EVENT_TYPE                                                                                                                 | IX_EVENT_ID_EVENT_TYPE           | 36      | UNIVIS_PROD.ps.ship_id,const |    1 |             | 
|  1 | SIMPLE      | ev2   | ref    | IX_EVENT_ID_EVENT_TYPE                                                                                                                 | IX_EVENT_ID_EVENT_TYPE           | 36      | UNIVIS_PROD.ps.ship_id,const |    1 |             | 
|  1 | SIMPLE      | ev3   | ref    | IX_EVENT_ID_EVENT_TYPE                                                                                                                 | IX_EVENT_ID_EVENT_TYPE           | 36      | UNIVIS_PROD.ps.ship_id,const |    1 |             | 
|  1 | SIMPLE      | ev4   | ref    | IX_EVENT_ID_EVENT_TYPE                                                                                                                 | IX_EVENT_ID_EVENT_TYPE           | 36      | UNIVIS_PROD.ps.ship_id,const |    1 |             | 
|  1 | SIMPLE      | ev5   | ref    | IX_EVENT_ID_EVENT_TYPE                                                                                                                 | IX_EVENT_ID_EVENT_TYPE           | 36      | UNIVIS_PROD.ps.ship_id,const |    1 |             | 
|  1 | SIMPLE      | psi   | ref    | PRIMARY,IX_po_shipment_item_po_shipment1,FK_po_shipment_item_po_shipment1                                                              | IX_po_shipment_item_po_shipment1 | 4       | UNIVIS_PROD.ps.ship_id       |    1 |             | 
|  1 | SIMPLE      | pa    | ref    | IX_po_alloc_po_shipment_item2,IX_po_alloc_po_details_old,FK_po_alloc_po_shipment1,FK_po_alloc_po_shipment_item1,FK_po_alloc_po_header1 | FK_po_alloc_po_shipment_item1    | 8       | UNIVIS_PROD.psi.UID_items    |    6 | Using where | 
|  1 | SIMPLE      | ph    | eq_ref | PRIMARY,IX_HDR_ID                                                                                                                      | PRIMARY                          | 4       | UNIVIS_PROD.pa.hdr_id        |    1 |             | 
+----+-------------+-------+--------+----------------------------------------------------------------------------------------------------------------------------------------+----------------------------------+---------+------------------------------+------+-------------+

数据库B

+----+-------------+-------+--------+----------------------------------------------------------------------------------------------------------------------------------------+----------------------------------+---------+-------------------------------+------+----------------------------------------------+
| id | select_type | table | type   | possible_keys                                                                                                                          | key                              | key_len | ref                           | rows | Extra                                        |
+----+-------------+-------+--------+----------------------------------------------------------------------------------------------------------------------------------------+----------------------------------+---------+-------------------------------+------+----------------------------------------------+
|  1 | SIMPLE      | ps    | range  | PRIMARY,IX_ETA_DATE                                                                                                                    | IX_ETA_DATE                      | 4       | NULL                          |   38 | Using where; Using temporary; Using filesort | 
|  1 | SIMPLE      | ev0   | ref    | IX_EVENT_ID_EVENT_TYPE                                                                                                                 | IX_EVENT_ID_EVENT_TYPE           | 36      | UNIVIS_DEV01.ps.ship_id,const |    1 |                                              | 
|  1 | SIMPLE      | ev1   | ref    | IX_EVENT_ID_EVENT_TYPE                                                                                                                 | IX_EVENT_ID_EVENT_TYPE           | 36      | UNIVIS_DEV01.ps.ship_id,const |    1 |                                              | 
|  1 | SIMPLE      | ev2   | ref    | IX_EVENT_ID_EVENT_TYPE                                                                                                                 | IX_EVENT_ID_EVENT_TYPE           | 36      | UNIVIS_DEV01.ps.ship_id,const |    1 |                                              | 
|  1 | SIMPLE      | ev3   | ref    | IX_EVENT_ID_EVENT_TYPE                                                                                                                 | IX_EVENT_ID_EVENT_TYPE           | 36      | UNIVIS_DEV01.ps.ship_id,const |    1 |                                              | 
|  1 | SIMPLE      | ev4   | ref    | IX_EVENT_ID_EVENT_TYPE                                                                                                                 | IX_EVENT_ID_EVENT_TYPE           | 36      | UNIVIS_DEV01.ps.ship_id,const |    1 |                                              | 
|  1 | SIMPLE      | ev5   | ref    | IX_EVENT_ID_EVENT_TYPE                                                                                                                 | IX_EVENT_ID_EVENT_TYPE           | 36      | UNIVIS_DEV01.ps.ship_id,const |    1 |                                              | 
|  1 | SIMPLE      | psi   | ref    | PRIMARY,IX_po_shipment_item_po_shipment1,FK_po_shipment_item_po_shipment1                                                              | IX_po_shipment_item_po_shipment1 | 4       | UNIVIS_DEV01.ps.ship_id       |    1 |                                              | 
|  1 | SIMPLE      | pa    | ref    | IX_po_alloc_po_shipment_item2,IX_po_alloc_po_details_old,FK_po_alloc_po_shipment1,FK_po_alloc_po_shipment_item1,FK_po_alloc_po_header1 | IX_po_alloc_po_shipment_item2    | 4       | UNIVIS_DEV01.ps.ship_id       |    3 | Using where                                  | 
|  1 | SIMPLE      | ph    | eq_ref | PRIMARY,IX_HDR_ID                                                                                                                      | PRIMARY                          | 4       | UNIVIS_DEV01.pa.hdr_id        |    1 |                                              | 
+----+-------------+-------+--------+----------------------------------------------------------------------------------------------------------------------------------------+----------------------------------+---------+-------------------------------+------+----------------------------------------------+

更新 这肯定是与数据相关的。我从数据库 A 转储数据并将其加载到数据库 B 使用：

SELECT * from <table> into outfile <file>

然后

LOAD DATA INFILE <file> into table <table>

数据库 B 查询运行得很快 - 即。与数据库 A 一样快。关于如何诊断数据可能出现的问题有什么想法吗？

更新 @newtover： 来自数据库 A：

+-----------------+---------------------+
| eta_selectivity | ship_id_selectivity |
+-----------------+---------------------+
|          0.0693 |              1.0000 | 
+-----------------+---------------------+
1 row in set (0.02 sec)

来自数据库 B（坏数据库）

+-----------------+---------------------+
| eta_selectivity | ship_id_selectivity |
+-----------------+---------------------+
|          0.1814 |              1.0000 | 
+-----------------+---------------------+
1 row in set (0.02 sec)

并且显示为 po_shipment 创建：

| po_shipment | CREATE TABLE `po_shipment` (
  `ship_id` int(11) NOT NULL DEFAULT '0',
  `ship_type` varchar(16) DEFAULT NULL,
  `foreign_agent` varchar(16) DEFAULT NULL,
  `agent_ref` varchar(16) DEFAULT NULL,
  `exporter_code` varchar(30) DEFAULT NULL,
  `importer_code` varchar(30) DEFAULT NULL,
  `carrier_code` varchar(30) DEFAULT NULL,
  `exporter_name` varchar(50) DEFAULT NULL,
  `importer_name` varchar(50) DEFAULT NULL,
  `carrier_name` varchar(50) DEFAULT NULL,
  `receipt` varchar(30) DEFAULT NULL,
  `pol_aol` varchar(50) DEFAULT NULL,
  `pod_aod` varchar(30) DEFAULT NULL,
  `final_dest` varchar(50) DEFAULT NULL,
  `vessel_flno` varchar(30) DEFAULT NULL,
  `ets` date DEFAULT NULL,
  `eta` date DEFAULT NULL,
  `pieces` int(11) DEFAULT '0',
  `weight` decimal(17,2) DEFAULT '0.00',
  `volume` decimal(17,2) DEFAULT '0.00',
  `marks` varchar(500) DEFAULT NULL,
  `goods_desc` varchar(500) DEFAULT NULL,
  `ship_terms` varchar(16) DEFAULT NULL,
  `ship_terms_desc` varchar(50) DEFAULT NULL,
  `house_hawb` varchar(30) DEFAULT NULL,
  `ocean_mawb` varchar(30) DEFAULT NULL,
  `booking_date` date DEFAULT NULL,
  `expected_cargo` date DEFAULT NULL,
  `mfrt_jobdisp` varchar(30) DEFAULT NULL,
  `ship_complete` date DEFAULT NULL,
  `user_id` varchar(30) DEFAULT NULL,
  `receipt_desc` varchar(60) DEFAULT NULL,
  `fin_dest_desc` varchar(60) DEFAULT NULL,
  `pol_aol_desc` varchar(60) DEFAULT NULL,
  `pod_aod_desc` varchar(60) DEFAULT NULL,
  `exporter_ref` varchar(26) DEFAULT NULL,
  `carrier_ref` varchar(26) DEFAULT NULL,
  `terms_conds` date DEFAULT NULL,
  `last_amended` date DEFAULT NULL,
  `user_amended` varchar(30) DEFAULT NULL,
  `package_type` varchar(24) DEFAULT NULL,
  `ext_cancelled` tinyint(1) NOT NULL DEFAULT '0',
  `ext_goh` tinyint(1) NOT NULL DEFAULT '0',
  `ext_arrival_date` date DEFAULT NULL,
  `ext_booking_ref` varchar(255) DEFAULT NULL,
  `ext_dc_booked_delivery_date` date DEFAULT NULL,
  `ext_dc_booked_delivery_time` varchar(10) DEFAULT NULL,
  `ext_comments` text,
  `deleted` tinyint(1) NOT NULL DEFAULT '0',
  `last_amended_time` int(10) DEFAULT NULL,
  `last_amended_uni` varchar(30) DEFAULT NULL,
  PRIMARY KEY (`ship_id`),
  KEY `IX_ETA_DATE` (`eta`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 |

更新 @chris_I 如果我通过删除 EVENT_TABLE 之外的所有其他连接来精简查询，我会得到相同的性能（即蹩脚）

SELECT * FROM po_shipment ps 
LEFT JOIN EVENT_TABLE ev0 ON ev0.TABLE_ID1 = ps.ship_id AND ev0.EVENT_TYPE = 'MAS0' 
LEFT JOIN EVENT_TABLE ev1 ON ev1.TABLE_ID1 = ps.ship_id AND ev1.EVENT_TYPE = 'MAS1' 
LEFT JOIN EVENT_TABLE ev2 ON ev2.TABLE_ID1 = ps.ship_id AND ev2.EVENT_TYPE = 'MAS2' 
LEFT JOIN EVENT_TABLE ev3 ON ev3.TABLE_ID1 = ps.ship_id AND ev3.EVENT_TYPE = 'MAS3' 
LEFT JOIN EVENT_TABLE ev4 ON ev4.TABLE_ID1 = ps.ship_id AND ev4.EVENT_TYPE = 'MAS4' 
LEFT JOIN EVENT_TABLE ev5 ON ev5.TABLE_ID1 = ps.ship_id AND ev5.EVENT_TYPE = 'MAS5' 
WHERE ps.eta >= '2010-03-22'
GROUP BY ps.ship_id
LIMIT 100;

更新 @Marcus Adams： 查询您所要求的已删除内连接的计划：

SELECT * FROM po_shipment ps 
LEFT JOIN EVENT_TABLE ev0 ON ev0.TABLE_ID1 = ps.ship_id AND ev0.EVENT_TYPE = 'MAS0' 
LEFT JOIN EVENT_TABLE ev1 ON ev1.TABLE_ID1 = ps.ship_id AND ev1.EVENT_TYPE = 'MAS1' 
LEFT JOIN EVENT_TABLE ev2 ON ev2.TABLE_ID1 = ps.ship_id AND ev2.EVENT_TYPE = 'MAS2' 
LEFT JOIN EVENT_TABLE ev3 ON ev3.TABLE_ID1 = ps.ship_id AND ev3.EVENT_TYPE = 'MAS3' 
LEFT JOIN EVENT_TABLE ev4 ON ev4.TABLE_ID1 = ps.ship_id AND ev4.EVENT_TYPE = 'MAS4' 
LEFT JOIN EVENT_TABLE ev5 ON ev5.TABLE_ID1 = ps.ship_id AND ev5.EVENT_TYPE = 'MAS5' 
WHERE ps.eta >= '2010-03-22'
GROUP BY ps.ship_id
LIMIT 100;

来自数据库 A 的查询计划（0.35 秒内响应）

+----+-------------+-------+-------+------------------------+------------------------+---------+------------------------------+------+----------------------------------------------+
| id | select_type | table | type  | possible_keys          | key                    | key_len | ref                          | rows | Extra                                        |
+----+-------------+-------+-------+------------------------+------------------------+---------+------------------------------+------+----------------------------------------------+
|  1 | SIMPLE      | ps    | range | IX_ETA_DATE            | IX_ETA_DATE            | 4       | NULL                         |  174 | Using where; Using temporary; Using filesort | 
|  1 | SIMPLE      | ev0   | ref   | IX_EVENT_ID_EVENT_TYPE | IX_EVENT_ID_EVENT_TYPE | 36      | UNIVIS_PROD.ps.ship_id,const |    1 |                                              | 
|  1 | SIMPLE      | ev1   | ref   | IX_EVENT_ID_EVENT_TYPE | IX_EVENT_ID_EVENT_TYPE | 36      | UNIVIS_PROD.ps.ship_id,const |    1 |                                              | 
|  1 | SIMPLE      | ev2   | ref   | IX_EVENT_ID_EVENT_TYPE | IX_EVENT_ID_EVENT_TYPE | 36      | UNIVIS_PROD.ps.ship_id,const |    1 |                                              | 
|  1 | SIMPLE      | ev3   | ref   | IX_EVENT_ID_EVENT_TYPE | IX_EVENT_ID_EVENT_TYPE | 36      | UNIVIS_PROD.ps.ship_id,const |    1 |                                              | 
|  1 | SIMPLE      | ev4   | ref   | IX_EVENT_ID_EVENT_TYPE | IX_EVENT_ID_EVENT_TYPE | 36      | UNIVIS_PROD.ps.ship_id,const |    1 |                                              | 
|  1 | SIMPLE      | ev5   | ref   | IX_EVENT_ID_EVENT_TYPE | IX_EVENT_ID_EVENT_TYPE | 36      | UNIVIS_PROD.ps.ship_id,const |    1 |                                              | 
+----+-------------+-------+-------+------------------------+------------------------+---------+------------------------------+------+----------------------------------------------

来自数据库 B 的查询计划（响应时间赶不上泡一杯茶的时间）

    +----+-------------+-------+-------+------------------------+------------------------+---------+-------------------------------+------+----------------------------------------------+
| id | select_type | table | type  | possible_keys          | key                    | key_len | ref                           | rows | Extra                                        |
+----+-------------+-------+-------+------------------------+------------------------+---------+-------------------------------+------+----------------------------------------------+
|  1 | SIMPLE      | ps    | range | IX_ETA_DATE            | IX_ETA_DATE            | 4       | NULL                          |   38 | Using where; Using temporary; Using filesort | 
|  1 | SIMPLE      | ev0   | ref   | IX_EVENT_ID_EVENT_TYPE | IX_EVENT_ID_EVENT_TYPE | 36
原文

I have a query that is returning in vastly different amounts of time between 2 datasets. For one set (database A) it returns in a few seconds, for the other (database B)....well I haven't waited long enough yet, but over 10 minutes. I have dumped both of these databases to my local machine where I can reproduce the issue running MySQL 5.1.37.

Curiously, database B is smaller than database A.

A stripped down version of the query that reproduces the problem is:

SELECT * FROM po_shipment ps 
JOIN po_shipment_item psi USING (ship_id) 
JOIN po_alloc pa ON ps.ship_id = pa.ship_id AND pa.UID_items = psi.UID_items 
JOIN po_header ph ON pa.hdr_id = ph.hdr_id 
LEFT JOIN EVENT_TABLE ev0 ON ev0.TABLE_ID1 = ps.ship_id AND ev0.EVENT_TYPE = 'MAS0' 
LEFT JOIN EVENT_TABLE ev1 ON ev1.TABLE_ID1 = ps.ship_id AND ev1.EVENT_TYPE = 'MAS1' 
LEFT JOIN EVENT_TABLE ev2 ON ev2.TABLE_ID1 = ps.ship_id AND ev2.EVENT_TYPE = 'MAS2' 
LEFT JOIN EVENT_TABLE ev3 ON ev3.TABLE_ID1 = ps.ship_id AND ev3.EVENT_TYPE = 'MAS3' 
LEFT JOIN EVENT_TABLE ev4 ON ev4.TABLE_ID1 = ps.ship_id AND ev4.EVENT_TYPE = 'MAS4' 
LEFT JOIN EVENT_TABLE ev5 ON ev5.TABLE_ID1 = ps.ship_id AND ev5.EVENT_TYPE = 'MAS5' 
WHERE ps.eta >= '2010-03-22'
GROUP BY ps.ship_id
LIMIT 100;

The EXPLAIN query plan for the first database (A) that returns in ~2 seconds is:

+----+-------------+-------+--------+----------------------------------------------------------------------------------------------------------------------------------------+----------------------------------+---------+------------------------------+------+----------------------------------------------+
| id | select_type | table | type   | possible_keys                                                                                                                          | key                              | key_len | ref                          | rows | Extra                                        |
+----+-------------+-------+--------+----------------------------------------------------------------------------------------------------------------------------------------+----------------------------------+---------+------------------------------+------+----------------------------------------------+
|  1 | SIMPLE      | ps    | range  | PRIMARY,IX_ETA_DATE                                                                                                                    | IX_ETA_DATE                      | 4       | NULL                         |  174 | Using where; Using temporary; Using filesort | 
|  1 | SIMPLE      | ev0   | ref    | IX_EVENT_ID_EVENT_TYPE                                                                                                                 | IX_EVENT_ID_EVENT_TYPE           | 36      | UNIVIS_PROD.ps.ship_id,const |    1 |                                              | 
|  1 | SIMPLE      | ev1   | ref    | IX_EVENT_ID_EVENT_TYPE                                                                                                                 | IX_EVENT_ID_EVENT_TYPE           | 36      | UNIVIS_PROD.ps.ship_id,const |    1 |                                              | 
|  1 | SIMPLE      | ev2   | ref    | IX_EVENT_ID_EVENT_TYPE                                                                                                                 | IX_EVENT_ID_EVENT_TYPE           | 36      | UNIVIS_PROD.ps.ship_id,const |    1 |                                              | 
|  1 | SIMPLE      | ev3   | ref    | IX_EVENT_ID_EVENT_TYPE                                                                                                                 | IX_EVENT_ID_EVENT_TYPE           | 36      | UNIVIS_PROD.ps.ship_id,const |    1 |                                              | 
|  1 | SIMPLE      | ev4   | ref    | IX_EVENT_ID_EVENT_TYPE                                                                                                                 | IX_EVENT_ID_EVENT_TYPE           | 36      | UNIVIS_PROD.ps.ship_id,const |    1 |                                              | 
|  1 | SIMPLE      | ev5   | ref    | IX_EVENT_ID_EVENT_TYPE                                                                                                                 | IX_EVENT_ID_EVENT_TYPE           | 36      | UNIVIS_PROD.ps.ship_id,const |    1 |                                              | 
|  1 | SIMPLE      | psi   | ref    | PRIMARY,IX_po_shipment_item_po_shipment1,FK_po_shipment_item_po_shipment1                                                              | IX_po_shipment_item_po_shipment1 | 4       | UNIVIS_PROD.ps.ship_id       |    1 |                                              | 
|  1 | SIMPLE      | pa    | ref    | IX_po_alloc_po_shipment_item2,IX_po_alloc_po_details_old,FK_po_alloc_po_shipment1,FK_po_alloc_po_shipment_item1,FK_po_alloc_po_header1 | FK_po_alloc_po_shipment1         | 4       | UNIVIS_PROD.psi.ship_id      |    5 | Using where                                  | 
|  1 | SIMPLE      | ph    | eq_ref | PRIMARY,IX_HDR_ID                                                                                                                      | PRIMARY                          | 4       | UNIVIS_PROD.pa.hdr_id        |    1 |                                              | 
+----+-------------+-------+--------+----------------------------------------------------------------------------------------------------------------------------------------+----------------------------------+---------+------------------------------+------+----------------------------------------------+

The EXPLAIN query plan for the second database (B) that returns in >600 seconds is:

+----+-------------+-------+--------+----------------------------------------------------------------------------------------------------------------------------------------+----------------------------------+---------+--------------------------------+------+----------------------------------------------+
| id | select_type | table | type   | possible_keys                                                                                                                          | key                              | key_len | ref                            | rows | Extra                                        |
+----+-------------+-------+--------+----------------------------------------------------------------------------------------------------------------------------------------+----------------------------------+---------+--------------------------------+------+----------------------------------------------+
|  1 | SIMPLE      | ps    | range  | PRIMARY,IX_ETA_DATE                                                                                                                    | IX_ETA_DATE                      | 4       | NULL                           |   38 | Using where; Using temporary; Using filesort | 
|  1 | SIMPLE      | psi   | ref    | PRIMARY,IX_po_shipment_item_po_shipment1,FK_po_shipment_item_po_shipment1                                                              | IX_po_shipment_item_po_shipment1 | 4       | UNIVIS_DEV01.ps.ship_id        |    1 |                                              | 
|  1 | SIMPLE      | ev0   | ref    | IX_EVENT_ID_EVENT_TYPE                                                                                                                 | IX_EVENT_ID_EVENT_TYPE           | 36      | UNIVIS_DEV01.psi.ship_id,const |    1 |                                              | 
|  1 | SIMPLE      | ev1   | ref    | IX_EVENT_ID_EVENT_TYPE                                                                                                                 | IX_EVENT_ID_EVENT_TYPE           | 36      | UNIVIS_DEV01.psi.ship_id,const |    1 |                                              | 
|  1 | SIMPLE      | ev2   | ref    | IX_EVENT_ID_EVENT_TYPE                                                                                                                 | IX_EVENT_ID_EVENT_TYPE           | 36      | UNIVIS_DEV01.ps.ship_id,const  |    1 |                                              | 
|  1 | SIMPLE      | ev3   | ref    | IX_EVENT_ID_EVENT_TYPE                                                                                                                 | IX_EVENT_ID_EVENT_TYPE           | 36      | UNIVIS_DEV01.psi.ship_id,const |    1 |                                              | 
|  1 | SIMPLE      | ev4   | ref    | IX_EVENT_ID_EVENT_TYPE                                                                                                                 | IX_EVENT_ID_EVENT_TYPE           | 36      | UNIVIS_DEV01.psi.ship_id,const |    1 |                                              | 
|  1 | SIMPLE      | ev5   | ref    | IX_EVENT_ID_EVENT_TYPE                                                                                                                 | IX_EVENT_ID_EVENT_TYPE           | 36      | UNIVIS_DEV01.ps.ship_id,const  |    1 |                                              | 
|  1 | SIMPLE      | pa    | ref    | IX_po_alloc_po_shipment_item2,IX_po_alloc_po_details_old,FK_po_alloc_po_shipment1,FK_po_alloc_po_shipment_item1,FK_po_alloc_po_header1 | IX_po_alloc_po_shipment_item2    | 4       | UNIVIS_DEV01.ps.ship_id        |    4 | Using where                                  | 
|  1 | SIMPLE      | ph    | eq_ref | PRIMARY,IX_HDR_ID                                                                                                                      | PRIMARY                          | 4       | UNIVIS_DEV01.pa.hdr_id         |    1 |                                              | 
+----+-------------+-------+--------+----------------------------------------------------------------------------------------------------------------------------------------+----------------------------------+---------+--------------------------------+------+----------------------------------------------+

When database B is running I can look at the MySQL Administrator and the state remains at "Copying to tmp table" indefinitely. Database A also has this state but for only a second or so.

There are no differences in the table structure, indexes, keys etc between these databases (I have done show create tables and diff'd them).

The sizes of the tables are:

database A:
po_shipment 1776
po_shipment_item 1945
po_alloc 36298
po_header 71642
EVENT_TABLE 1608

database B:
po_shipment 463
po_shipment_item 470
po_alloc 3291
po_header 56149
EVENT_TABLE 1089

Some points to note:

  • Removing the WHERE clause makes the
    query return < 1 sec.
  • Removing the GROUP BY makes the query
    return < 1 sec.
  • Removing ev5, ev4, ev3 etc makes the
    query get faster for each one
    removed.

UPDATE after AJ's answer:
- The size of the ship_id is significantly larger on database B (max value = 800002752) than database A (max value = 3489). Given that these are InnoDB tables would changing any buffer help with handling keys of this size? Further Update to this: I reduced the size of the keys and re-ANALYZEd but still no change in performance.

UPDATE desc of EVENT_TABLE:

Note that it is identical in both databases

+--------------------+--------------+------+-----+---------+----------------+
| Field              | Type         | Null | Key | Default | Extra          |
+--------------------+--------------+------+-----+---------+----------------+
| EVENT_TABLE_ID     | bigint(20)   | NO   | PRI | NULL    | auto_increment | 
| EVENT_TYPE         | varchar(10)  | NO   |     | NULL    |                | 
| TABLE_ID1          | int(11)      | NO   | MUL | NULL    |                | 
| TABLE_ID2          | int(11)      | YES  |     | NULL    |                | 
| TABLE_ID3          | int(11)      | YES  |     | NULL    |                | 
| TABLE_ID4          | int(11)      | YES  |     | NULL    |                | 
| EVENT_CREATED_DATE | datetime     | NO   |     | NULL    |                | 
| MESSAGE_REF        | varchar(100) | YES  |     | NULL    |                | 
+--------------------+--------------+------+-----+---------+----------------+

And for good measure the SHOW CREATE TABLE EVENT_TABLE:

The only thing to differ in this between databases is the auto increment value

| EVENT_TABLE | CREATE TABLE `EVENT_TABLE` (
  `EVENT_TABLE_ID` bigint(20) NOT NULL AUTO_INCREMENT,
  `EVENT_TYPE` varchar(10) NOT NULL,
  `TABLE_ID1` int(11) NOT NULL,
  `TABLE_ID2` int(11) DEFAULT NULL,
  `TABLE_ID3` int(11) DEFAULT NULL,
  `TABLE_ID4` int(11) DEFAULT NULL,
  `EVENT_CREATED_DATE` datetime NOT NULL,
  `MESSAGE_REF` varchar(100) DEFAULT NULL,
  PRIMARY KEY (`EVENT_TABLE_ID`),
  KEY `IX_EVENT_ID_EVENT_TYPE` (`TABLE_ID1`,`EVENT_TYPE`)
) ENGINE=InnoDB AUTO_INCREMENT=1925 DEFAULT CHARSET=utf8 |

Can anyone suggest how to resolve this issue? What have I missed?

UPDATE after question from Michael Holzmann
Here is the new Query plans based on his updated STRAIGHT_JOIN query. Note that database B has "Using temporary; Using filesort" whereas now database A doesn't. Could this be due to the long keys or something similar?

database A

+----+-------------+-------+--------+----------------------------------------------------------------------------------------------------------------------------------------+----------------------------------+---------+------------------------------+------+-------------+
| id | select_type | table | type   | possible_keys                                                                                                                          | key                              | key_len | ref                          | rows | Extra       |
+----+-------------+-------+--------+----------------------------------------------------------------------------------------------------------------------------------------+----------------------------------+---------+------------------------------+------+-------------+
|  1 | SIMPLE      | ps    | index  | PRIMARY,IX_ETA_DATE                                                                                                                    | PRIMARY                          | 4       | NULL                         |  168 | Using where | 
|  1 | SIMPLE      | ev0   | ref    | IX_EVENT_ID_EVENT_TYPE                                                                                                                 | IX_EVENT_ID_EVENT_TYPE           | 36      | UNIVIS_PROD.ps.ship_id,const |    1 |             | 
|  1 | SIMPLE      | ev1   | ref    | IX_EVENT_ID_EVENT_TYPE                                                                                                                 | IX_EVENT_ID_EVENT_TYPE           | 36      | UNIVIS_PROD.ps.ship_id,const |    1 |             | 
|  1 | SIMPLE      | ev2   | ref    | IX_EVENT_ID_EVENT_TYPE                                                                                                                 | IX_EVENT_ID_EVENT_TYPE           | 36      | UNIVIS_PROD.ps.ship_id,const |    1 |             | 
|  1 | SIMPLE      | ev3   | ref    | IX_EVENT_ID_EVENT_TYPE                                                                                                                 | IX_EVENT_ID_EVENT_TYPE           | 36      | UNIVIS_PROD.ps.ship_id,const |    1 |             | 
|  1 | SIMPLE      | ev4   | ref    | IX_EVENT_ID_EVENT_TYPE                                                                                                                 | IX_EVENT_ID_EVENT_TYPE           | 36      | UNIVIS_PROD.ps.ship_id,const |    1 |             | 
|  1 | SIMPLE      | ev5   | ref    | IX_EVENT_ID_EVENT_TYPE                                                                                                                 | IX_EVENT_ID_EVENT_TYPE           | 36      | UNIVIS_PROD.ps.ship_id,const |    1 |             | 
|  1 | SIMPLE      | psi   | ref    | PRIMARY,IX_po_shipment_item_po_shipment1,FK_po_shipment_item_po_shipment1                                                              | IX_po_shipment_item_po_shipment1 | 4       | UNIVIS_PROD.ps.ship_id       |    1 |             | 
|  1 | SIMPLE      | pa    | ref    | IX_po_alloc_po_shipment_item2,IX_po_alloc_po_details_old,FK_po_alloc_po_shipment1,FK_po_alloc_po_shipment_item1,FK_po_alloc_po_header1 | FK_po_alloc_po_shipment_item1    | 8       | UNIVIS_PROD.psi.UID_items    |    6 | Using where | 
|  1 | SIMPLE      | ph    | eq_ref | PRIMARY,IX_HDR_ID                                                                                                                      | PRIMARY                          | 4       | UNIVIS_PROD.pa.hdr_id        |    1 |             | 
+----+-------------+-------+--------+----------------------------------------------------------------------------------------------------------------------------------------+----------------------------------+---------+------------------------------+------+-------------+

database B

+----+-------------+-------+--------+----------------------------------------------------------------------------------------------------------------------------------------+----------------------------------+---------+-------------------------------+------+----------------------------------------------+
| id | select_type | table | type   | possible_keys                                                                                                                          | key                              | key_len | ref                           | rows | Extra                                        |
+----+-------------+-------+--------+----------------------------------------------------------------------------------------------------------------------------------------+----------------------------------+---------+-------------------------------+------+----------------------------------------------+
|  1 | SIMPLE      | ps    | range  | PRIMARY,IX_ETA_DATE                                                                                                                    | IX_ETA_DATE                      | 4       | NULL                          |   38 | Using where; Using temporary; Using filesort | 
|  1 | SIMPLE      | ev0   | ref    | IX_EVENT_ID_EVENT_TYPE                                                                                                                 | IX_EVENT_ID_EVENT_TYPE           | 36      | UNIVIS_DEV01.ps.ship_id,const |    1 |                                              | 
|  1 | SIMPLE      | ev1   | ref    | IX_EVENT_ID_EVENT_TYPE                                                                                                                 | IX_EVENT_ID_EVENT_TYPE           | 36      | UNIVIS_DEV01.ps.ship_id,const |    1 |                                              | 
|  1 | SIMPLE      | ev2   | ref    | IX_EVENT_ID_EVENT_TYPE                                                                                                                 | IX_EVENT_ID_EVENT_TYPE           | 36      | UNIVIS_DEV01.ps.ship_id,const |    1 |                                              | 
|  1 | SIMPLE      | ev3   | ref    | IX_EVENT_ID_EVENT_TYPE                                                                                                                 | IX_EVENT_ID_EVENT_TYPE           | 36      | UNIVIS_DEV01.ps.ship_id,const |    1 |                                              | 
|  1 | SIMPLE      | ev4   | ref    | IX_EVENT_ID_EVENT_TYPE                                                                                                                 | IX_EVENT_ID_EVENT_TYPE           | 36      | UNIVIS_DEV01.ps.ship_id,const |    1 |                                              | 
|  1 | SIMPLE      | ev5   | ref    | IX_EVENT_ID_EVENT_TYPE                                                                                                                 | IX_EVENT_ID_EVENT_TYPE           | 36      | UNIVIS_DEV01.ps.ship_id,const |    1 |                                              | 
|  1 | SIMPLE      | psi   | ref    | PRIMARY,IX_po_shipment_item_po_shipment1,FK_po_shipment_item_po_shipment1                                                              | IX_po_shipment_item_po_shipment1 | 4       | UNIVIS_DEV01.ps.ship_id       |    1 |                                              | 
|  1 | SIMPLE      | pa    | ref    | IX_po_alloc_po_shipment_item2,IX_po_alloc_po_details_old,FK_po_alloc_po_shipment1,FK_po_alloc_po_shipment_item1,FK_po_alloc_po_header1 | IX_po_alloc_po_shipment_item2    | 4       | UNIVIS_DEV01.ps.ship_id       |    3 | Using where                                  | 
|  1 | SIMPLE      | ph    | eq_ref | PRIMARY,IX_HDR_ID                                                                                                                      | PRIMARY                          | 4       | UNIVIS_DEV01.pa.hdr_id        |    1 |                                              | 
+----+-------------+-------+--------+----------------------------------------------------------------------------------------------------------------------------------------+----------------------------------+---------+-------------------------------+------+----------------------------------------------+

UPDATE It's definitely data related. I dumped the data from database A and loaded it into database B using:

SELECT * from <table> into outfile <file>

and

LOAD DATA INFILE <file> into table <table>

Then the database B query runs quickly - ie. as fast as database A. Any ideas on how to diagnose what could be wrong with the data??

UPDATE @newtover:
From database A:

+-----------------+---------------------+
| eta_selectivity | ship_id_selectivity |
+-----------------+---------------------+
|          0.0693 |              1.0000 | 
+-----------------+---------------------+
1 row in set (0.02 sec)

From database B (the bad one)

+-----------------+---------------------+
| eta_selectivity | ship_id_selectivity |
+-----------------+---------------------+
|          0.1814 |              1.0000 | 
+-----------------+---------------------+
1 row in set (0.02 sec)

And the show create for po_shipment:

| po_shipment | CREATE TABLE `po_shipment` (
  `ship_id` int(11) NOT NULL DEFAULT '0',
  `ship_type` varchar(16) DEFAULT NULL,
  `foreign_agent` varchar(16) DEFAULT NULL,
  `agent_ref` varchar(16) DEFAULT NULL,
  `exporter_code` varchar(30) DEFAULT NULL,
  `importer_code` varchar(30) DEFAULT NULL,
  `carrier_code` varchar(30) DEFAULT NULL,
  `exporter_name` varchar(50) DEFAULT NULL,
  `importer_name` varchar(50) DEFAULT NULL,
  `carrier_name` varchar(50) DEFAULT NULL,
  `receipt` varchar(30) DEFAULT NULL,
  `pol_aol` varchar(50) DEFAULT NULL,
  `pod_aod` varchar(30) DEFAULT NULL,
  `final_dest` varchar(50) DEFAULT NULL,
  `vessel_flno` varchar(30) DEFAULT NULL,
  `ets` date DEFAULT NULL,
  `eta` date DEFAULT NULL,
  `pieces` int(11) DEFAULT '0',
  `weight` decimal(17,2) DEFAULT '0.00',
  `volume` decimal(17,2) DEFAULT '0.00',
  `marks` varchar(500) DEFAULT NULL,
  `goods_desc` varchar(500) DEFAULT NULL,
  `ship_terms` varchar(16) DEFAULT NULL,
  `ship_terms_desc` varchar(50) DEFAULT NULL,
  `house_hawb` varchar(30) DEFAULT NULL,
  `ocean_mawb` varchar(30) DEFAULT NULL,
  `booking_date` date DEFAULT NULL,
  `expected_cargo` date DEFAULT NULL,
  `mfrt_jobdisp` varchar(30) DEFAULT NULL,
  `ship_complete` date DEFAULT NULL,
  `user_id` varchar(30) DEFAULT NULL,
  `receipt_desc` varchar(60) DEFAULT NULL,
  `fin_dest_desc` varchar(60) DEFAULT NULL,
  `pol_aol_desc` varchar(60) DEFAULT NULL,
  `pod_aod_desc` varchar(60) DEFAULT NULL,
  `exporter_ref` varchar(26) DEFAULT NULL,
  `carrier_ref` varchar(26) DEFAULT NULL,
  `terms_conds` date DEFAULT NULL,
  `last_amended` date DEFAULT NULL,
  `user_amended` varchar(30) DEFAULT NULL,
  `package_type` varchar(24) DEFAULT NULL,
  `ext_cancelled` tinyint(1) NOT NULL DEFAULT '0',
  `ext_goh` tinyint(1) NOT NULL DEFAULT '0',
  `ext_arrival_date` date DEFAULT NULL,
  `ext_booking_ref` varchar(255) DEFAULT NULL,
  `ext_dc_booked_delivery_date` date DEFAULT NULL,
  `ext_dc_booked_delivery_time` varchar(10) DEFAULT NULL,
  `ext_comments` text,
  `deleted` tinyint(1) NOT NULL DEFAULT '0',
  `last_amended_time` int(10) DEFAULT NULL,
  `last_amended_uni` varchar(30) DEFAULT NULL,
  PRIMARY KEY (`ship_id`),
  KEY `IX_ETA_DATE` (`eta`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 |

UPDATE @chris_I
If I strip the query down by removing all other joins aside from EVENT_TABLE I get the same performance (ie. crappy)

SELECT * FROM po_shipment ps 
LEFT JOIN EVENT_TABLE ev0 ON ev0.TABLE_ID1 = ps.ship_id AND ev0.EVENT_TYPE = 'MAS0' 
LEFT JOIN EVENT_TABLE ev1 ON ev1.TABLE_ID1 = ps.ship_id AND ev1.EVENT_TYPE = 'MAS1' 
LEFT JOIN EVENT_TABLE ev2 ON ev2.TABLE_ID1 = ps.ship_id AND ev2.EVENT_TYPE = 'MAS2' 
LEFT JOIN EVENT_TABLE ev3 ON ev3.TABLE_ID1 = ps.ship_id AND ev3.EVENT_TYPE = 'MAS3' 
LEFT JOIN EVENT_TABLE ev4 ON ev4.TABLE_ID1 = ps.ship_id AND ev4.EVENT_TYPE = 'MAS4' 
LEFT JOIN EVENT_TABLE ev5 ON ev5.TABLE_ID1 = ps.ship_id AND ev5.EVENT_TYPE = 'MAS5' 
WHERE ps.eta >= '2010-03-22'
GROUP BY ps.ship_id
LIMIT 100;

UPDATE @Marcus Adams:
Query for plans you have asked for with inner joins removed:

SELECT * FROM po_shipment ps 
LEFT JOIN EVENT_TABLE ev0 ON ev0.TABLE_ID1 = ps.ship_id AND ev0.EVENT_TYPE = 'MAS0' 
LEFT JOIN EVENT_TABLE ev1 ON ev1.TABLE_ID1 = ps.ship_id AND ev1.EVENT_TYPE = 'MAS1' 
LEFT JOIN EVENT_TABLE ev2 ON ev2.TABLE_ID1 = ps.ship_id AND ev2.EVENT_TYPE = 'MAS2' 
LEFT JOIN EVENT_TABLE ev3 ON ev3.TABLE_ID1 = ps.ship_id AND ev3.EVENT_TYPE = 'MAS3' 
LEFT JOIN EVENT_TABLE ev4 ON ev4.TABLE_ID1 = ps.ship_id AND ev4.EVENT_TYPE = 'MAS4' 
LEFT JOIN EVENT_TABLE ev5 ON ev5.TABLE_ID1 = ps.ship_id AND ev5.EVENT_TYPE = 'MAS5' 
WHERE ps.eta >= '2010-03-22'
GROUP BY ps.ship_id
LIMIT 100;

Query Plan from database A (responds in 0.35s)

+----+-------------+-------+-------+------------------------+------------------------+---------+------------------------------+------+----------------------------------------------+
| id | select_type | table | type  | possible_keys          | key                    | key_len | ref                          | rows | Extra                                        |
+----+-------------+-------+-------+------------------------+------------------------+---------+------------------------------+------+----------------------------------------------+
|  1 | SIMPLE      | ps    | range | IX_ETA_DATE            | IX_ETA_DATE            | 4       | NULL                         |  174 | Using where; Using temporary; Using filesort | 
|  1 | SIMPLE      | ev0   | ref   | IX_EVENT_ID_EVENT_TYPE | IX_EVENT_ID_EVENT_TYPE | 36      | UNIVIS_PROD.ps.ship_id,const |    1 |                                              | 
|  1 | SIMPLE      | ev1   | ref   | IX_EVENT_ID_EVENT_TYPE | IX_EVENT_ID_EVENT_TYPE | 36      | UNIVIS_PROD.ps.ship_id,const |    1 |                                              | 
|  1 | SIMPLE      | ev2   | ref   | IX_EVENT_ID_EVENT_TYPE | IX_EVENT_ID_EVENT_TYPE | 36      | UNIVIS_PROD.ps.ship_id,const |    1 |                                              | 
|  1 | SIMPLE      | ev3   | ref   | IX_EVENT_ID_EVENT_TYPE | IX_EVENT_ID_EVENT_TYPE | 36      | UNIVIS_PROD.ps.ship_id,const |    1 |                                              | 
|  1 | SIMPLE      | ev4   | ref   | IX_EVENT_ID_EVENT_TYPE | IX_EVENT_ID_EVENT_TYPE | 36      | UNIVIS_PROD.ps.ship_id,const |    1 |                                              | 
|  1 | SIMPLE      | ev5   | ref   | IX_EVENT_ID_EVENT_TYPE | IX_EVENT_ID_EVENT_TYPE | 36      | UNIVIS_PROD.ps.ship_id,const |    1 |                                              | 
+----+-------------+-------+-------+------------------------+------------------------+---------+------------------------------+------+----------------------------------------------

Query Plan from database B (doesn't respond in time it takes to make a cup of tea)

    +----+-------------+-------+-------+------------------------+------------------------+---------+-------------------------------+------+----------------------------------------------+
| id | select_type | table | type  | possible_keys          | key                    | key_len | ref                           | rows | Extra                                        |
+----+-------------+-------+-------+------------------------+------------------------+---------+-------------------------------+------+----------------------------------------------+
|  1 | SIMPLE      | ps    | range | IX_ETA_DATE            | IX_ETA_DATE            | 4       | NULL                          |   38 | Using where; Using temporary; Using filesort | 
|  1 | SIMPLE      | ev0   | ref   | IX_EVENT_ID_EVENT_TYPE | IX_EVENT_ID_EVENT_TYPE | 36

如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论， 你可以免费 注册 一个本站的账号。

评论（14）

绅刃 2024-09-03 03:58:35

尝试将 STRAIGHT_JOIN 添加到查询中以查看执行计划是否是问题所在。优化器为每个数据库选择不同的执行计划，这可能会导致问题。

SELECT STRAIGHT_JOIN * FROM po_shipment ps 
LEFT JOIN EVENT_TABLE ev0 ON ev0.TABLE_ID1 = ps.ship_id AND ev0.EVENT_TYPE = 'MAS0' 
LEFT JOIN EVENT_TABLE ev1 ON ev1.TABLE_ID1 = ps.ship_id AND ev1.EVENT_TYPE = 'MAS1' 
LEFT JOIN EVENT_TABLE ev2 ON ev2.TABLE_ID1 = ps.ship_id AND ev2.EVENT_TYPE = 'MAS2' 
LEFT JOIN EVENT_TABLE ev3 ON ev3.TABLE_ID1 = ps.ship_id AND ev3.EVENT_TYPE = 'MAS3' 
LEFT JOIN EVENT_TABLE ev4 ON ev4.TABLE_ID1 = ps.ship_id AND ev4.EVENT_TYPE = 'MAS4' 
LEFT JOIN EVENT_TABLE ev5 ON ev5.TABLE_ID1 = ps.ship_id AND ev5.EVENT_TYPE = 'MAS5' 
JOIN po_shipment_item psi USING (ship_id) 
JOIN po_alloc pa ON ps.ship_id = pa.ship_id AND pa.UID_items = psi.UID_items 
JOIN po_header ph ON pa.hdr_id = ph.hdr_id
WHERE ps.eta >= '2010-03-22'
GROUP BY ps.ship_id
LIMIT 100;

更新
1. 数据库 A 的新执行计划不需要文件排序或临时表，因为它使用主键。我会开始在查询中添加USE INDEX，看看是否可以加快数据库B的查询速度。ship_id是po_shipment的主键吗？如果是这样，您需要弄清楚按日期分组或筛选日期的成本更高。

SELECT STRAIGHT_JOIN * FROM po_shipment ps USE INDEX( PRIMARY )
LEFT JOIN EVENT_TABLE ev0 ON ev0.TABLE_ID1 = ps.ship_id AND ev0.EVENT_TYPE = 'MAS0' 
LEFT JOIN EVENT_TABLE ev1 ON ev1.TABLE_ID1 = ps.ship_id AND ev1.EVENT_TYPE = 'MAS1' 
LEFT JOIN EVENT_TABLE ev2 ON ev2.TABLE_ID1 = ps.ship_id AND ev2.EVENT_TYPE = 'MAS2' 
LEFT JOIN EVENT_TABLE ev3 ON ev3.TABLE_ID1 = ps.ship_id AND ev3.EVENT_TYPE = 'MAS3' 
LEFT JOIN EVENT_TABLE ev4 ON ev4.TABLE_ID1 = ps.ship_id AND ev4.EVENT_TYPE = 'MAS4' 
LEFT JOIN EVENT_TABLE ev5 ON ev5.TABLE_ID1 = ps.ship_id AND ev5.EVENT_TYPE = 'MAS5' 
JOIN po_shipment_item psi USING (ship_id) 
JOIN po_alloc pa ON ps.ship_id = pa.ship_id AND pa.UID_items = psi.UID_items 
JOIN po_header ph ON pa.hdr_id = ph.hdr_id
WHERE ps.eta >= '2010-03-22'
GROUP BY ps.ship_id
LIMIT 100;

如果这没有帮助，请尝试建议数据库 A 的执行计划中为数据库 B 使用更多索引。

Try adding STRAIGHT_JOIN to the query to see if the execution plan is the issue. The optimizer is choosing a different execution plan for each database and this might be causing the issue.

SELECT STRAIGHT_JOIN * FROM po_shipment ps 
LEFT JOIN EVENT_TABLE ev0 ON ev0.TABLE_ID1 = ps.ship_id AND ev0.EVENT_TYPE = 'MAS0' 
LEFT JOIN EVENT_TABLE ev1 ON ev1.TABLE_ID1 = ps.ship_id AND ev1.EVENT_TYPE = 'MAS1' 
LEFT JOIN EVENT_TABLE ev2 ON ev2.TABLE_ID1 = ps.ship_id AND ev2.EVENT_TYPE = 'MAS2' 
LEFT JOIN EVENT_TABLE ev3 ON ev3.TABLE_ID1 = ps.ship_id AND ev3.EVENT_TYPE = 'MAS3' 
LEFT JOIN EVENT_TABLE ev4 ON ev4.TABLE_ID1 = ps.ship_id AND ev4.EVENT_TYPE = 'MAS4' 
LEFT JOIN EVENT_TABLE ev5 ON ev5.TABLE_ID1 = ps.ship_id AND ev5.EVENT_TYPE = 'MAS5' 
JOIN po_shipment_item psi USING (ship_id) 
JOIN po_alloc pa ON ps.ship_id = pa.ship_id AND pa.UID_items = psi.UID_items 
JOIN po_header ph ON pa.hdr_id = ph.hdr_id
WHERE ps.eta >= '2010-03-22'
GROUP BY ps.ship_id
LIMIT 100;

UPDATE
1. The new execution plan for database A doesn't need a filesort or temporary table because it's using the primary key. I would start adding USE INDEX into the query to see if you can speed up the query on database B. Is ship_id the primary key of po_shipment? if so, you need to figure out what costs more the group by or filtering of the dates.

SELECT STRAIGHT_JOIN * FROM po_shipment ps USE INDEX( PRIMARY )
LEFT JOIN EVENT_TABLE ev0 ON ev0.TABLE_ID1 = ps.ship_id AND ev0.EVENT_TYPE = 'MAS0' 
LEFT JOIN EVENT_TABLE ev1 ON ev1.TABLE_ID1 = ps.ship_id AND ev1.EVENT_TYPE = 'MAS1' 
LEFT JOIN EVENT_TABLE ev2 ON ev2.TABLE_ID1 = ps.ship_id AND ev2.EVENT_TYPE = 'MAS2' 
LEFT JOIN EVENT_TABLE ev3 ON ev3.TABLE_ID1 = ps.ship_id AND ev3.EVENT_TYPE = 'MAS3' 
LEFT JOIN EVENT_TABLE ev4 ON ev4.TABLE_ID1 = ps.ship_id AND ev4.EVENT_TYPE = 'MAS4' 
LEFT JOIN EVENT_TABLE ev5 ON ev5.TABLE_ID1 = ps.ship_id AND ev5.EVENT_TYPE = 'MAS5' 
JOIN po_shipment_item psi USING (ship_id) 
JOIN po_alloc pa ON ps.ship_id = pa.ship_id AND pa.UID_items = psi.UID_items 
JOIN po_header ph ON pa.hdr_id = ph.hdr_id
WHERE ps.eta >= '2010-03-22'
GROUP BY ps.ship_id
LIMIT 100;

If that doesn't help try suggesting more of the indexes used in the execution plan of database A for database B.

回复 原文
白云不回头 2024-09-03 03:58:35

如果是数据问题，我无法告诉您确切的问题是什么，但这是我最喜欢的解决此类问题的策略：

尝试删除一半的连接。递归地重复，直到查询运行得很快。然后添加您在上一步中删除的连接的一半...（此策略比通过连接删除和添加连接所需的步骤要少得多。）

一旦发现“坏”连接，您可以尝试限制其值使用附加的“where”子句，直到查询再次快速运行...在每一步中，始终尝试将问题减少一半。

注意：很可能的情况是，即使数据库 B 中的数据总量较小，您也会获得更多连接中间结果的记录。

If it's a data problem, I can't tell you what the exact problem is, but here's my favorite strategy to solve this kind of problem:

Try to remove half of your joins. Repeat recursively, until the query runs fast. Then add half of the joins you removed in the last step... (This strategy will require much fewer steps than removing and adding join by join.)

Once you've found out the "bad" join, you can try restricting its values with an additional "where" clause, until the query runs fast again... In each step, always try to reduce the problem in half.

Note: It can very well be the case, that you get a lot more records for the intermediate results of your joins, even if the total amount of data is smaller in database B.

回复 原文
浪荡不羁 2024-09-03 03:58:35

一段时间没有接触 MySQL，但我猜问题与以下任一方面有关：

  1. 您检查过键/连接字段长度（实际数据）是否可能导致 sort_buffer 在大（大小）键上分页到磁盘连接？ （这听起来像数据问题......）
  2. 服务器设置，基本上是写入内存临时表。几年前我也经历过类似的事情。您是否扩大了 key_buffer_size、table_cache、read_rnd_buffer_size、sort_buffer、read_buffer_size 看看是否有帮助？

Haven't touched MySQL for a while but I am guessing the problem has to do with either

  1. Have you checked you key/join field lengths (actual data) maybe causing the sort_buffer to page to disk joining on large (size) keys? (this smells like a data issue...)
  2. The server settings, basically it is writting to an in memory temp table. I had something similar a few years back. Have you enlarged the key_buffer_size, table_cache, read_rnd_buffer_size, sort_buffer, read_buffer_size to see if that helps?
回复 原文
唔猫 2024-09-03 03:58:35

我认为可能是 GROUP BY 子句的手动排序（文件排序）导致了明显的缺陷。

尝试使用 SQL_BIG_RESULT 提示来查看 MySQL 是否会改进其处理 GROUP BY 的方法。

SELECT SQL_BIG_RESULT * FROM ...

I think it may be the manual sort (filesort) for the GROUP BY clause that is causing the noticeable deficit.

Try using the SQL_BIG_RESULT hint to see if MySQL will improve it's method for handling the GROUP BY.

SELECT SQL_BIG_RESULT * FROM ...
回复 原文
酒与心事 2024-09-03 03:58:35

更新您的统计数据。有一次我遇到了类似的问题，这对我有用。

Update your statistics. one time I have a similar issue, and this works for me.

回复 原文
皇甫轩 2024-09-03 03:58:35

由于它是 InnoDB，看起来它很可能是一个锁定问题。同时还发生了什么？

Since it's InnoDB it looks like it's uite possibly a locking problem. What else is going on at the same time?

回复 原文
很快妥协 2024-09-03 03:58:35

由于数据似乎存在一些问题，因此找出导致问题的数据可能会有所帮助。创建第三个数据库 C 并从数据库 B 中插入一半数据两次（因此行数相同）。如果数据库C较慢，则坏数据就在那里，否则就在另一半。重复使用越来越小的块大小，以帮助找到问题数据。

即使数据库 B 比数据库 A 小，表“po_header”和“EVENT_TABLE”也不会按比例变小。这可能与速度差异有关。

Since it seems like some issue with the data, it would probably be helpful to figure out what data is causing the problem. Make a third database C and insert half of the data from database B twice(so you have the same number of rows). If database C is slower, than the bad data is there, otherwise it is in the other half. Repeat with smaller and smaller chunk sizes to help find the problem data.

Even though database B is smaller that database A, the tables 'po_header' and 'EVENT_TABLE' are not proportionally smaller. That may have something to do with the speed difference.

回复 原文
攒一口袋星星 2024-09-03 03:58:35

必须尝试执行“检查表”和/或“优化表”吗？

我遇到了 类似情况，问题在于关键索引未正确更新并执行 CHECK TABLE 修复它们。虽然机会渺茫，但值得一试。

Have to tried doing "CHECK TABLE" and/or "OPTIMIZE TABLE"?

I had a similar situation where the problem was that the key indexes where not updated properly and doing CHECK TABLE fixed them. A long shot, but worth the try.

回复 原文
愛放△進行李 2024-09-03 03:58:35

首先，请确保您已在适当的字段上创建了索引。我确信你已经做到了。

接下来，尝试使用索引提示（使用索引）强制数据库正确使用索引。

我有一个 类似问题，其中我假设索引已设置并使用正确地通过mysql，但他们没有。我可以使用索引提示来解决它。

To begin with make sure that you have created indexes on the appropriate fields. I am sure you've done that.

Next, try using index hints (USE INDEX) to force the database to use the indexes properly.

I had a similar problem in which I assumed that the indexes were setup and used by mysql properly, but they were not. I was able to work around it using index hints.

回复 原文
情话已封尘 2024-09-03 03:58:35

po_shipment.eta 和 po_shipment.ship_id 中数据的选择性是什么？您能否从两个数据库中发布以下查询的结果：

SELECT
    count(distinct eta)/count(*) as eta_selectivity,
    count(distinct ship_id)/count(*) as ship_id_selectivity
FROM po_shipment;

通常，字段数据的选择性越强（越接近 1），索引的效果就越好。如果优化器缺乏必要的统计信息，原因也可能是 po_shipment.eta 中的数据分布非常不均匀（当您使用“2099-01-01”等作为特殊值时）。

顺便问一下，您能提供 SHOW CREATE TABLE po_shipment 吗？桌面上的指数可以提供一些线索。

UPD：当字段的选择性低至eta字段时，索引基本上没有用处。更糟糕的是，它可能会使优化器在选择上感到困惑，并减慢数据的插入和更新速度。

第一个建议是尝试删除 eta 字段上的索引并测量结果。可能对于 A 数据库优化器甚至不会尝试使用索引，因为选择性非常低，而对于 B 数据库它会尝试使用索引。

第二件让我困扰的事情是为什么你首先要按 Ship_id 进行分组？当需要隐式临时表并且字段中有 TEXT （如您的情况）或 BLOB 字段时，MySQL 将始终使用磁盘临时表进行排序（这在 GROUP BY 中隐式需要）。在您的情况下，ship_id 是聚集主键，结果无论如何都会按ship_id 排序。您需要的是提取一个基本查询（如果可以有多个对应项，则已经按 Ship_id 分组）并 JOIN po_shipment 与应用您的范围条件的基本查询，而不是使用顶部的分组。

第三个。当您在顶部使用 * 时，您真的需要所有字段吗？加入 10 张桌子后，您会收到很多字段。我几乎不相信你需要所有这些。即使从结果中排除 TEXT 字段也可能会提高查询性能。

What is the selectivity of data in po_shipment.eta and po_shipment.ship_id. Could you post the results of the following query from both databases:

SELECT
    count(distinct eta)/count(*) as eta_selectivity,
    count(distinct ship_id)/count(*) as ship_id_selectivity
FROM po_shipment;

Usually the more selective are the field data (closer to 1), the better index is working. The cause might also be very uneven distribution of data in po_shipment.eta (when you use '2099-01-01' or the like for as a special value) if the optimizer lacks necessary statistics.

By the way, could you provide SHOW CREATE TABLE po_shipment? Indices on the table could shed some light.

UPD: When the selectivity on the field is so low as for eta field, the index is basically useless. Even worse, it might confuse the optimizer on selects and slow down inserts and updates of the data.

The first suggestion is to try removing the index on the eta field and measure the results. It might be that for the A database optimizer does not even try to use the index because of the the very low selectivity and for the B database it gives the index a try.

The second thing that bothers me is why do you group by ship_id in the first place? When there is a need in an implicit temporary table and among the fields there is a TEXT (as in your case) or BLOB one, MySQL will always use on disk temporary tables for sorting (which is implicitly required in GROUP BY). In your case, ship_id is the clustered primary key and the results will anyway be sorted by ship_id. What you need is to extract a base query (already grouped by ship_id if there can be several correscpondences) and JOIN po_shipment with the base query applying your range condition and not using the group by at the top.

And the third. Do you realy need all the fields when you use * at the top? Having joined 10 tables, you receive a lot of fieds. I hardly believe you need al of them. Even exclusion of the TEXT field from the results might boost the query performance.

回复 原文
北音执念 2024-09-03 03:58:35

当使用范围选择器 小于 时，我也经历过同样的情况。和大于 >也。

实验：如果范围不太大，您是否尝试过将范围扩展为 IN (...,...,...) 语句 instad ？

例如

SELECT * FROM po_shipment ps USE INDEX (IX_ETA_DATE)
JOIN po_shipment_item psi USING (ship_id) 
JOIN po_alloc pa ON ps.ship_id = pa.ship_id AND pa.UID_items = psi.UID_items 
JOIN po_header ph ON pa.hdr_id = ph.hdr_id 
LEFT JOIN EVENT_TABLE ev0 ON ev0.TABLE_ID1 = ps.ship_id AND ev0.EVENT_TYPE = 'MAS0' 
LEFT JOIN EVENT_TABLE ev1 ON ev1.TABLE_ID1 = ps.ship_id AND ev1.EVENT_TYPE = 'MAS1' 
LEFT JOIN EVENT_TABLE ev2 ON ev2.TABLE_ID1 = ps.ship_id AND ev2.EVENT_TYPE = 'MAS2' 
LEFT JOIN EVENT_TABLE ev3 ON ev3.TABLE_ID1 = ps.ship_id AND ev3.EVENT_TYPE = 'MAS3' 
LEFT JOIN EVENT_TABLE ev4 ON ev4.TABLE_ID1 = ps.ship_id AND ev4.EVENT_TYPE = 'MAS4' 
LEFT JOIN EVENT_TABLE ev5 ON ev5.TABLE_ID1 = ps.ship_id AND ev5.EVENT_TYPE = 'MAS5' 
WHERE ps.eta IN ('2010-03-22','2010-03-21','2010-03-20',...)
GROUP BY ps.ship_id
LIMIT 100;

编辑：按照 Salman A 的建议添加了 USE INDEX() 提示。似乎 mysql 看到了可能的索引，但选择不使用它......值得测试。

I've experienced the same when using range selectors less than < and greater than > too.

Experiment: If the range is not too big, have you tried tried to expand the range into and an IN (...,...,...) statement instad ?

E.g

SELECT * FROM po_shipment ps USE INDEX (IX_ETA_DATE)
JOIN po_shipment_item psi USING (ship_id) 
JOIN po_alloc pa ON ps.ship_id = pa.ship_id AND pa.UID_items = psi.UID_items 
JOIN po_header ph ON pa.hdr_id = ph.hdr_id 
LEFT JOIN EVENT_TABLE ev0 ON ev0.TABLE_ID1 = ps.ship_id AND ev0.EVENT_TYPE = 'MAS0' 
LEFT JOIN EVENT_TABLE ev1 ON ev1.TABLE_ID1 = ps.ship_id AND ev1.EVENT_TYPE = 'MAS1' 
LEFT JOIN EVENT_TABLE ev2 ON ev2.TABLE_ID1 = ps.ship_id AND ev2.EVENT_TYPE = 'MAS2' 
LEFT JOIN EVENT_TABLE ev3 ON ev3.TABLE_ID1 = ps.ship_id AND ev3.EVENT_TYPE = 'MAS3' 
LEFT JOIN EVENT_TABLE ev4 ON ev4.TABLE_ID1 = ps.ship_id AND ev4.EVENT_TYPE = 'MAS4' 
LEFT JOIN EVENT_TABLE ev5 ON ev5.TABLE_ID1 = ps.ship_id AND ev5.EVENT_TYPE = 'MAS5' 
WHERE ps.eta IN ('2010-03-22','2010-03-21','2010-03-20',...)
GROUP BY ps.ship_id
LIMIT 100;

Edit: Added USE INDEX() hint as Salman A suggested. Seems like mysql see's the possible index, but choses not to use it... worth testing.

回复 原文
樱桃奶球 2024-09-03 03:58:35

我认为你的做法是错误的。当您执行 LEFT JOIN 时，您将返回 EVENT_TABLE 中的所有记录，无论与 po_shipment 是否匹配。

您已经对 EVENT_TABLE 运行了 LEFT JOIN 六次。每个查询都会返回 (6 * (SELECT count(*) FROM EVENT_TABLE) ) 条记录。由于数据库 A 的记录比数据库 B 少，当然查询在 A 上运行得更快。

我认为这样的操作会执行得更好：

DECLARE @TEMP_EVENT_TYPES table 
(
    EVENT_TYPE varchar(10) PRIMARY KEY
)
;
/*
INSERT VALUES 'MAS0', 'MAS1', 'MAS2', 'MAS3', 'MAS4', 'MAS5'
*/

;
SELECT * FROM po_shipment ps 
JOIN po_shipment_item psi USING (ship_id) 
JOIN po_alloc pa ON ps.ship_id = pa.ship_id AND pa.UID_items = psi.UID_items 
JOIN po_header ph ON pa.hdr_id = ph.hdr_id 
INNER JOIN 
    EVENT_TABLE et ON et.TABLE_ID1 = ps.ship_id
INNER JOIN 
    @TEMP_EVENT_TYPES tet ON tet.event_type = et.EVENT_TYPE
WHERE ps.eta >= '2010-03-22'
GROUP BY ps.ship_id

I think you're going at this the wrong way. When you do a LEFT JOIN, you're getting back all of the records in EVENT_TABLE regardless of a match to po_shipment.

And you've run a LEFT JOIN to EVENT_TABLE six times. You're getting back (6 * (SELECT count(*) FROM EVENT_TABLE) ) records with every query. Since db A has less records than db B, of course the query runs faster on A.

I think something like this would perform worlds better:

DECLARE @TEMP_EVENT_TYPES table 
(
    EVENT_TYPE varchar(10) PRIMARY KEY
)
;
/*
INSERT VALUES 'MAS0', 'MAS1', 'MAS2', 'MAS3', 'MAS4', 'MAS5'
*/

;
SELECT * FROM po_shipment ps 
JOIN po_shipment_item psi USING (ship_id) 
JOIN po_alloc pa ON ps.ship_id = pa.ship_id AND pa.UID_items = psi.UID_items 
JOIN po_header ph ON pa.hdr_id = ph.hdr_id 
INNER JOIN 
    EVENT_TABLE et ON et.TABLE_ID1 = ps.ship_id
INNER JOIN 
    @TEMP_EVENT_TYPES tet ON tet.event_type = et.EVENT_TYPE
WHERE ps.eta >= '2010-03-22'
GROUP BY ps.ship_id
回复 原文
抱猫软卧 2024-09-03 03:58:35

看起来您确实正在为 po_shipment 建模一个状态机，并为每个状态转换在 event_table 中记录事件。

您试图用此查询表达的业务逻辑是什么？

您应该能够向 po_shipment 添加一个 state 字段来完成此操作，该字段对 event_table 中的一系列事件进行汇总和非规范化。

构建验证给定记录的所有 6 个事件是否已发生的逻辑不应该在数据库中，而应该在您的模型中并保留在数据库中（模型的经典状态机模式）。

然后只需从 po_shipment 中选择，其中 state = 'MAS5' 且 eta >日期;

It sure looks like you're modeling a state machine for po_shipment w/ event records in event_table for each state transition.

What is the business logic you're trying to express with this query?

You should be able to add a state field to your po_shipment to accomplish this, which summarizes and denormalizes the series of events in the event_table.

Building the logic to verify all 6 events have taken place for a given record shouldn't be in the database, it should be in your model and persisted in the database (classic state machine pattern for a model).

Then it's just select from po_shipment where state = 'MAS5' and eta > date;

回复 原文
绅士风度i 2024-09-03 03:58:35

出于好奇 - B 中的原始数据是否有很多 NULL？

Out of curiosity - does the original data in B have many NULLs?

回复 原文
~没有更多了~

关于作者

绮筵

暂无简介

文章
评论
26 人气
发私信

相关话题

更多

热门标签

操作系统 程序设计 IT运维 Linux系统管理 JavaScript 服务器应用 solaris C/C++ PHP Shell BSD Vue.js aix Oracle Python HTML 系统管理 HTML5 CSS 前端
更多

推荐作者

佚名

文章 0 评论 0

今天

文章 0 评论 0

゛时过境迁

文章 0 评论 0

达拉崩吧

文章 0 评论 0

呆萌少年

文章 0 评论 0

孤者何惧

文章 0 评论 0

更多

友情链接

文江博客
    我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站，即表示您同意使用 Cookies 和您的相关数据。
    原文