使用 phpcassa 从文件批量加载到 Cassandra 时出现问题
我正在尝试使用 phpcassa 将文本文件中的 1000000 条记录加载到 Cassandra 中。但在加载过程中途我收到以下错误。
PHP 致命错误:/usr/share/php/phpcassa/columnfamily.php 第 759 行超出最大执行时间 30 秒**
如何增加执行时间?我是否必须更改 columnfamily.php
中的任何参数? 请在下面找到我的代码。
<?
require_once('phpcassa/connection.php');
require_once('phpcassa/columnfamily.php');
try {
$servers = array("127.0.0.1:9160");
$pool = new ConnectionPool("Keyspace1", $servers);
$column_family = new ColumnFamily($pool, 'Product');
$cnt=0;
$files = fopen("dump.txt", "r");
$mtime = microtime();
$mtime = explode(" ",$mtime);
$mtime = $mtime[1] + $mtime[0];
$starttime = $mtime;
while (!feof($files)) {
$line = fgets($files);
$keyname="P".$cnt;
$split_line = explode("," , $line);
for ($i=0;$i<count($split_line);$i++) {
//echo $split_line[$i];
$column_family->insert(
$keyname, array(
'code' => $split_line[0] ,
'pname' => $split_line[1] ,
'price' => $split_line[2]
)
);
}
$cnt += 1;
}
$mtime = microtime();
$mtime = explode(" ",$mtime);
$mtime = $mtime[1] + $mtime[0];
$endtime = $mtime;
fclose($files);
$totaltime = ($endtime - $starttime);
echo "$cnt keys loaded in ".$totaltime." seconds";
}
catch (Exception $e)
{
echo 'Exception: ' . $e->getMessage();
}
?>
I am trying to load 1000000 records from a text file into Cassandra using phpcassa. But halfway through the loading process I got the following error.
PHP Fatal error: Maximum execution time of 30 seconds exceeded in /usr/share/php/phpcassa/columnfamily.php on line 759**
How do I increase the execution time? Do I have to change any parameter in columnfamily.php
?
Please find my code below.
<?
require_once('phpcassa/connection.php');
require_once('phpcassa/columnfamily.php');
try {
$servers = array("127.0.0.1:9160");
$pool = new ConnectionPool("Keyspace1", $servers);
$column_family = new ColumnFamily($pool, 'Product');
$cnt=0;
$files = fopen("dump.txt", "r");
$mtime = microtime();
$mtime = explode(" ",$mtime);
$mtime = $mtime[1] + $mtime[0];
$starttime = $mtime;
while (!feof($files)) {
$line = fgets($files);
$keyname="P".$cnt;
$split_line = explode("," , $line);
for ($i=0;$i<count($split_line);$i++) {
//echo $split_line[$i];
$column_family->insert(
$keyname, array(
'code' => $split_line[0] ,
'pname' => $split_line[1] ,
'price' => $split_line[2]
)
);
}
$cnt += 1;
}
$mtime = microtime();
$mtime = explode(" ",$mtime);
$mtime = $mtime[1] + $mtime[0];
$endtime = $mtime;
fclose($files);
$totaltime = ($endtime - $starttime);
echo "$cnt keys loaded in ".$totaltime." seconds";
}
catch (Exception $e)
{
echo 'Exception: ' . $e->getMessage();
}
?>
如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱,如果其他用户或者作者回复了您的评论,将不能在第一时间通知您!
发布评论
评论(3)
尝试将其添加到脚本顶部:
<代码>
设置时间限制(0);
Try and add this to top of your script:
set_time_limit(0);
请参阅set_time_limit。
更一般地说,在 Web 服务器环境中进行批量加载(这就是事实,对吧?)并不是最好的主意。也不是用单线程来完成的。但我想对于一百万行来说这已经足够了。
See set_time_limit.
More generally, doing bulk load inside a web server environment (that's what this is, right?) isn't the best idea. Neither is doing it with a single thread. But for just a million rows this will work adequately, I guess.
请增加
php.ini
文件中的max_execution_time
限制。这将避免
PHP 致命错误:最长执行时间为 30 秒
。默认情况下,Web 服务器上的最大执行时间仍为 30 秒。
您可以更改该数量,以便为服务器上的某些执行提供更多时间。
Please increase the
max_execution_time
limit inphp.ini
file.This will avoid
PHP Fatal error: Maximum execution time of 30 seconds
.By default, the maximum execution time on web servers remains at 30 seconds.
You can change that amount to provide more time for certain executions on your server.