PDO 准备在单个查询中插入多行
我在 MySQL 上使用这种类型的 SQL 在一个查询中插入多行值:
INSERT INTO `tbl` (`key1`,`key2`) VALUES ('r1v1','r1v2'),('r2v1','r2v2'),...
在 PDO 上的阅读中,准备好的语句应该比静态查询提供更好的安全性。
因此,我想知道是否可以使用准备好的语句生成“通过使用一个查询插入多行值”。
如果是,我该如何实施?
I am using this type of SQL on MySQL to insert multiple rows of values in one single query:
INSERT INTO `tbl` (`key1`,`key2`) VALUES ('r1v1','r1v2'),('r2v1','r2v2'),...
On the readings on PDO, prepared statements should give me a better security than static queries.
I would therefore like to know whether it is possible to generate "inserting multiple rows of values by the use of one query" using prepared statements.
If yes, how can I implement it?
如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱,如果其他用户或者作者回复了您的评论,将不能在第一时间通知您!
发布评论
评论(25)
这是解决此问题的另一个(简单)解决方案:
首先,您需要使用 count() 计算源数组(此处:$aData)的数据。 然后,您使用 array_fill() 并生成一个新数组,其中的条目数量与源数组的条目数量相同,每个条目的值为“(?,?)”(占位符的数量取决于您使用的字段;此处为:2)。 然后生成的数组需要内爆,并使用逗号作为粘合。
在 foreach 循环中,您需要根据您使用的占位符数量生成另一个索引(占位符数量 * 当前数组索引 + 1)。 您需要在每个绑定值之后将生成的索引加 1。
Here is another (slim) solution for this issue:
At first you need to count the data of the source array (here: $aData) with count(). Then you use array_fill() and generate a new array wich as many entries as the source array has, each with the value "(?,?)" (the number of placeholders depends on the fields you use; here: 2). Then the generated array needs to be imploded and as glue a comma is used.
Within the foreach loop, you need to generate another index regarding on the number of placeholders you use (number of placeholders * current array index + 1). You need to add 1 to the generated index after each binded value.
我是这样做的:
首先定义您将使用的列名,或者将其留空,pdo 将假设您想要使用表中的所有列 - 在这种情况下,您需要告知它们在桌子上出现的确切顺序。
现在,假设您已经准备好了一个二维数组。 迭代它,并用您的行值构造一个字符串,如下所示:
现在,您刚才所做的是检查 $rows 是否已定义,如果没有,则创建它并存储行值和必要的 SQL 语法,因此它将是有效的陈述。 请注意,字符串应该放在双引号和单引号内,这样它们就会被立即识别。
剩下要做的就是准备语句并执行,如下所示:
到目前为止已测试最多 2000 行,执行时间很短。 将进行更多测试,如果我有进一步的贡献,我会回到这里。
问候。
This is how I did it:
First define the column names you'll use, or leave it blank and pdo will assume you want to use all the columns on the table - in which case you'll need to inform the row values in the exact order they appear on the table.
Now, suppose you have a two dimensional array already prepared. Iterate it, and construct a string with your row values, as such:
Now, what you just did was check if $rows was already defined, and if not, create it and store row values and the necessary SQL syntax so it will be a valid statement. Note that strings should go inside double quotes and single quotes, so they will be promptly recognized as such.
All it's left to do is prepare the statement and execute, as such:
Tested with up to 2000 rows so far, and the execution time is dismal. Will run some more tests and will get back here in case I have something further to contribute.
Regards.
这是我的解决方案: https://github.com/sasha-ch/Aura.Sql
基于 auraphp/Aura.Sql 库。
使用示例:
欢迎报告错误。
Here is my solution: https://github.com/sasha-ch/Aura.Sql
based on auraphp/Aura.Sql library.
Usage example:
Bugreports are welcome.
由于尚未建议,我非常确定 LOAD DATA INFILE 仍然是加载数据的最快方法,因为它会禁用索引,插入所有数据,然后重新启用索引 - 所有这些都在单个请求中完成。
记住 fputcsv,将数据保存为 csv 应该是相当简单的。 MyISAM 是最快的,但在 InnoDB 中您仍然可以获得出色的性能。 不过,还有其他缺点,所以如果您要插入大量数据,并且不关心 100 行以下的数据,我会采用这种方法。
Since it has not been suggested yet, I'm pretty sure LOAD DATA INFILE is still the fastest way to load data as it disables indexing, inserts all data, and then re-enables the indexes - all in a single request.
Saving the data as a csv should be fairly trivial keeping in mind fputcsv. MyISAM is fastest, but you still get big performance in InnoDB. There are other disadvantages, though so I would go this route if you are inserting a lot of data, and not bother with under 100 rows.
虽然是一个老问题,但所有贡献对我帮助很大,所以这是我的解决方案,它在我自己的 DbContext 类中工作。
$rows
参数只是一个表示行或模型的关联数组的数组:field name => 插入值
。如果您使用的模式使用模型,那么当将模型数据作为数组传递时(例如从模型类中的
ToRowArray
方法传递),这非常适合。Although an old question all the contributions helped me a lot so here's my solution, which works within my own
DbContext
class. The$rows
parameter is simply an array of associative arrays representing rows or models:field name => insert value
.If you use a pattern that uses models this fits in nicely when passed model data as an array, say from a
ToRowArray
method within the model class.正如我前面的答案所证明的那样,使用普通的
PDO
来做到这一点很麻烦。 诸如 Nette 之类的数据库抽象层使任务变得更加容易,而不会对速度产生重大影响:As evidenced by the answers preceding mine, it's cumbersome to do it with vanilla
PDO
. Database abstractions layers such as Nette make the task much easier without significant impact on speed:对于可能无法理解长行代码的人:
确保为要插入的每一行都有一个数组。 例如:
<前><代码> $数据 = [
['name'=>'Sam', 'job'=>'fullstack', 'device'=>'hp'],
['name'=>'Joey', 'job'=>'ui', 'device'=>'apple']
]
//$data中的子数组也可以是索引数组
这个想法是能够轻松地形成多行的 SQL 插入查询,它应该看起来像这样:
希望我仍然有意义。
如果您要使用动态列名进行批量插入,则必须始终拥有预期列名的数组并为其分配默认值。 例如:
然后;
但如果你不使用动态列名进行批量插入,请跳过 $validcols、$allvalidcols、$col_arr 和 $columns,它们不是很有必要,然后将 $cols 设置为列数,在本例中为 3 ,然后将 $sql 查询调整为:
最后;
For people who might have issues understanding long rows of codes:
Make sure to have one array for each row to be inserted. Eg:
The idea is to be able to easily form the SQL Insert query for multiple rows, which is supposed to look like this:
Hope I'm still making sense.
If you'll be doing this batch insert with dynamic column names, you must always have an array of the expected column names and assign default values to them. Eg:
Then;
But if you are not doing the batch insert with dynamic column names, skip $validcols, $allvalidcols, $col_arr and $columns, they're not very necessary, then set $cols to the number of columns, which is 3 in this example, then adjust the $sql query to:
And finally;
我的真实世界示例将所有德国邮政编码插入到空表中(稍后添加城镇名称):
如您所见,它完全灵活。 您不需要检查列的数量或检查列所在的位置。 您只需要设置插入数据:
我对某些查询字符串构造函数感到自豪,因为它们无需像 array_merge 这样的繁重数组函数即可工作。 尤其是 vsprintf() 是一个很好的发现。
最后我需要添加 2x while() 以避免超出内存限制。 这取决于您的内存限制,但它是避免问题的一个很好的通用解决方案(并且有 10 个查询仍然比 10.000 个查询要好得多)。
My real world example to insert all german postcodes into an empty table (to add town names later):
As you can see its fully flexible. You don't need to check the amount of columns or check on which position your column is. You only need to set the insert data:
I'm proud of some of the query string constructors as they work without heavy array-functions like array_merge. Especially vsprintf() was a good find.
Finally I needed to add 2x while() to avoid exceeding the memory limit. This depends on your memory limit but at all its a good general solution to avoid problems (and having 10 queries is still much better than 10.000).
这是使用 PDO 插入多行的干净解决方案。
由于数组解构,它应该适用于 php 7.1+,但我想这部分可以很容易地更改。
样本数据:
Here is a clean solution to insert multiple rows using PDO.
It should work with php 7.1+, because of array destructuring, but this part could be change easily I guess.
Sample data:
我遇到了同样的问题,这就是我为自己完成的方法,我为自己制作了一个函数(如果对您有帮助,您可以使用它)。
示例:
INSERT INTO 国家(国家、城市)VALUES(德国、柏林)、(法国、巴黎);
If insertMultipleData($table, $multi_params)< /strong> 返回 TRUE,您的数据已插入到数据库中。
I had the same problem and this is how i accomplish for myself, and i made a function for myself for it ( and you can use it if that helps you).
Example:
INSERT INTO countries (country, city) VALUES (Germany, Berlin), (France, Paris);
If insertMultipleData($table, $multi_params) returns TRUE, your data has been inserted to your database.
这里给出的大多数用于创建准备好的查询的解决方案都比它们需要的更复杂。 使用 PHP 的内置函数,您可以轻松创建 SQL 语句,而无需大量开销。
给定
$records
,一个记录数组,其中每个记录本身就是一个索引数组(采用field => value
的形式),以下函数将把记录插入到PDO 连接$connection
上的给定表$table
,仅使用单个准备好的语句。 请注意,这是一个 PHP 5.6+ 解决方案,因为在对array_push
的调用中使用了参数解包:Most of the solutions given here to create the prepared query are more complex that they need to be. Using PHP's built in functions you can easily creare the SQL statement without significant overhead.
Given
$records
, an array of records where each record is itself an indexed array (in the form offield => value
), the following function will insert the records into the given table$table
, on a PDO connection$connection
, using only a single prepared statement. Note that this is a PHP 5.6+ solution because of the use of argument unpacking in the call toarray_push
:这对我有用
This worked for me
像这样的事情怎么样:
这背后的想法是循环遍历数组值,为准备好的语句占位符的每个循环添加“id 号”,同时将值添加到绑定参数的数组中。 如果您不喜欢使用数组中的“key”索引,可以在循环内添加 $i=0 和 $i++ 。 在此示例中,即使您有带有命名键的关联数组,只要键是唯一的,它仍然可以工作。 只需做一点工作,嵌套数组也可以。
**注意,substr 会删除 $sql 变量的最后一个空格和逗号,如果没有空格,则需要将其更改为 -1 而不是 - 2.
what about something like this:
The idea behind this is to cycle through your array values, adding "id numbers" to each loop for your prepared statement placeholders while at the same time, you add the values to your array for the binding parameters. If you don't like using the "key" index from the array, you could add $i=0, and $i++ inside the loop. Either works in this example, even if you have associative arrays with named keys, it would still work providing the keys were unique. With a little work it would be fine for nested arrays too..
**Note that substr strips the $sql variables last space and comma, if you don't have a space you'd need to change this to -1 rather than -2.
您可以使用此函数在单个查询中插入多行:
$row 是值数组的数组。
在您的情况下,您可以使用以下命令调用该函数。
这样做的好处是您可以使用准备好的语句,同时使用单个查询插入多行。 安全!
You can insert multiple rows in a single query with this function:
$row is an array of arrays of values.
In your case you would call the function with
This has the benefit that you use prepared statements, while inserting multiple rows with a single query. Security!
可以对多行使用单个准备好的语句和单个绑定参数,仅更改提供给语句的参数。 我的项目使用此方法(0.398s)获得了比使用长绑定参数构建长语句(2.508s)更好的结果。
重要提示:
使用@jamesvi示例,我在我的项目中使用了这种方法。
对于程序风格感到抱歉,我为此责怪我的项目的前辈。
Using a single prepared statement and a single bind params for multiple rows is possible, changing only the arguments provided to the statement. My project had better results using this method (0.398s) than building a long statement wth a long bind param (2.508s)
Important Note:
Using @jamesvi example I used this method for my project.
Sorry for the procedural style, I blame my project's predecessors for this.
使用 PDO 准备语句插入多个值
在一个执行语句中插入多个值。 为什么因为根据 此页面 它比常规插入。
更多数据值,或者您可能有一个填充数据的循环。
这基本上就是我们希望插入语句的样子:
因此,对于准备好的插入,您需要知道创建单个 VALUES 部分的字段数以及行数,以便知道重复它的次数。
现在,代码:
请注意,这种方法是 100% 安全的,因为查询完全由代码中显式编写的常量部分构成,尤其是列名。
Multiple Values Insert with PDO Prepared Statements
Inserting multiple values in one execute statement. Why because according to this page it is faster than regular inserts.
more data values or you probably have a loop that populates data.
That is basically how we want the insert statement to look like:
So with prepared inserts you need to know the number of fields to create a single VALUES part and the number of rows in order to know how many times to repeat it.
Now, the code:
Note that this approach is 100% secure, as the query is constructed entirely of constant parts explicitly written in the code, especially the column names.
与 Balagtas 先生的答案相同,稍微清楚一些...
最近版本的 MySQL 和 PHP PDO do 支持多行
INSERT
语句。SQL 概述
SQL 看起来像这样,假设您想要
INSERT
到一个 3 列表。即使使用多行插入,
ON DUPLICATE KEY UPDATE
也能按预期工作; 附加:PHP 概述
您的 PHP 代码将遵循通常的
$pdo->prepare($qry)
和$stmt->execute($params)
PDO 调用。$params
将是要传递给INSERT
的所有值的一维数组。在上面的例子中,它应该包含 9 个元素; PDO 将使用每组 3 个作为单行值。 (插入 3 行,每行 3 列 = 9 个元素数组。)
实现
下面的代码是为了清晰起见而编写的,而不是为了提高效率。 如果您愿意,可以使用 PHP
array_*()
函数来更好地映射或遍历数据。 假设执行单个查询并且每个查询本身就是一个事务,则不需要显式事务。假设:
$dataVals
- 多维数组,其中每个元素都是要插入示例代码Same answer as Mr. Balagtas, slightly clearer...
Recent versions MySQL and PHP PDO do support multi-row
INSERT
statements.SQL Overview
The SQL will look something like this, assuming a 3-column table you'd like to
INSERT
to.ON DUPLICATE KEY UPDATE
works as expected even with a multi-row INSERT; append this:PHP Overview
Your PHP code will follow the usual
$pdo->prepare($qry)
and$stmt->execute($params)
PDO calls.$params
will be a 1-dimensional array of all the values to pass to theINSERT
.In the above example, it should contain 9 elements; PDO will use every set of 3 as a single row of values. (Inserting 3 rows of 3 columns each = 9 element array.)
Implementation
Below code is written for clarity, not efficiency. Work with the PHP
array_*()
functions for better ways to map or walk through your data if you'd like. Given a single query is executed and each query being a transaction on its own, no explicit transaction is required.Assuming:
$dataVals
- mutli-dimensional array, where each element is a 1-d array of a row of values to INSERTSample Code
无论如何,我看到很多用户建议迭代 INSERT 语句,而不是像所选答案那样构建单个字符串查询。 我决定运行一个仅包含两个字段和一个非常基本的插入语句的简单测试:
虽然整个查询本身花费了几毫秒或更短的时间,但后一个(单字符串)查询始终快 8 倍或更多。 如果这是为了反映更多列上数千行的导入,那么差异可能是巨大的。
For what it is worth, I have seen a lot of users recommend iterating through INSERT statements instead of building out as a single string query as the selected answer did. I decided to run a simple test with just two fields and a very basic insert statement:
While the overall query itself took milliseconds or less, the latter (single string) query was consistently 8 times faster or more. If this was built out to say reflect an import of thousands of rows on many more columns, the difference could be enormous.
当 $data 数组很小时,Herbert Balagtas 接受的答案效果很好。 对于较大的 $data 数组,array_merge 函数变得异常缓慢。 我创建 $data 数组的测试文件有 28 列,大约 80,000 行。 最终脚本花了41s完成。
使用array_push()创建$insert_values而不是array_merge()导致100倍加速,执行时间为0.41s。
有问题的 array_merge():
为了消除对 array_merge() 的需要,您可以构建以下两个数组:
然后可以按如下方式使用这些数组:
The Accepted Answer by Herbert Balagtas works well when the $data array is small. With larger $data arrays the array_merge function becomes prohibitively slow. My test file to create the $data array has 28 cols and is about 80,000 lines. The final script took 41s to complete.
Using array_push() to create $insert_values instead of array_merge() resulted in a 100X speed up with execution time of 0.41s.
The problematic array_merge():
To eliminate the need for array_merge(), you can build the following two arrays instead:
These arrays can then be used as follows:
两种可能的方法:
或者:
如果所有行的数据都在一个数组中,我将使用第二种解决方案。 由于使用了事务,它几乎与第一个解决方案一样快。
Two possible approaches:
Or:
If the data for all the rows are in a single array, I would use the second solution. Thanks to using transactions, it's almost as fast as the first solution.
这根本不是您使用准备好的语句的方式。
每个查询插入一行是完全可以的,因为您可以使用不同的参数多次执行一个准备好的语句。 事实上,这是最大的优势之一,因为它允许您以高效、安全和舒适的方式插入大量行。
因此,也许可以实现您提出的方案,至少对于固定数量的行,但几乎可以保证这不是您真正想要的。
That's simply not the way you use prepared statements.
It is perfectly okay to insert one row per query because you can execute one prepared statement multiple times with different parameters. In fact that is one of the greatest advantages as it allows you to insert you a great number of rows in an efficient, secure and comfortable manner.
So it maybe possible to implement the scheme you proposing, at least for a fixed number of rows, but it is almost guaranteed that this is not really what you want.
一个更简短的答案:展平按列排序的数据数组,然后
当插入 1,000 条左右的记录时,当您需要的只是值的计数时,您不希望必须循环遍历每个记录来插入它们。
A shorter answer: flatten the array of data ordered by columns then
When inserting a 1,000 or so records you don't want to have to loop through every record to insert them when all you need is a count of the values.
这是我的简单方法。
Here is my simple approach.
这是我编写的一个类,使用清除选项进行多次插入:
Here's a class I wrote do multiple inserts with purge option:
根据我的实验,我发现在单个事务中包含多个值行的 mysql insert 语句是最快的。
但是,如果数据太多,那么 mysql 的 max_allowed_packet 设置可能会限制单个事务插入多个值行。 因此,当数据大于mysql的
max_allowed_packet
大小时,以下函数将失败:singleTransactionInsertWithRollback
singleTransactionInsertWithPlaceholders
singleTransactionInsert
最成功插入大数据场景中的一种是transactionSpeed方法,但它比上述方法更耗时。 因此,要解决这个问题,您可以将数据分割成更小的块并多次调用单个事务插入,或者使用
transactionSpeed
方法放弃执行速度。这是我的研究
仅包含两列的表的 100,000 个条目的结果如下
Based on my experiments I found out that mysql insert statement with multiple value rows in single transaction is the fastest one.
However, if the data is too much then mysql's
max_allowed_packet
setting might restrict the single transaction insert with multiple value rows. Hence, following functions will fail when there is data greater than mysql'smax_allowed_packet
size:singleTransactionInsertWithRollback
singleTransactionInsertWithPlaceholders
singleTransactionInsert
The most successful one in insert huge data scenario is
transactionSpeed
method, but it consumes time more the above mentioned methods. So, to handle this problem you can either split your data into smaller chunks and call single transaction insert multiple times or give up speed of execution by usingtransactionSpeed
method.Here's my research
The results for 100,000 entries for a table containing only two columns is as below