MySql:在表上运行大量搜索查询的最佳方法

发布于 2024-11-30 20:52:23 字数 3073 浏览 1 评论 0原文

我有两个表,一个是我需要搜索的静态数据库,另一个是我将用来搜索第一个数据库的动态数据库。现在我有两个单独的查询。首先在页面加载时,第二个表中的值作为搜索词传递到第一个表,并且我使用 cURL“捕获”搜索结果。这是非常低效的,而且可能是错误的方法,所以我需要帮助来解决这个问题。目前页面(html、前端)加载需要 40 秒。

可能的解决方案:将其转化为功能,但仍然发出如此多的调用。将表加载到内存中,然后运行查询并在完成后卸载缓存。使用正则表达式来帮助加快查询速度?可以加入吗?但我是菜鸟,所以我只能想象...

搜索脚本:

require 'mysqlconnect.php';

    $id = NULL;
    if(isset($_GET['n'])) {     $id = mysql_real_escape_string($_GET['n']);     }
    if(isset($_POST['n'])) {    $id = mysql_real_escape_string($_POST['n']);    }

    if(!empty($id)){
        $getdata = "SELECT id, first_name, last_name, published_name,
                    department, telephone FROM $table WHERE id = '$id' LIMIT 1"; 

        $result = mysql_query($getdata) or die(mysql_error());
        $num_rows = mysql_num_rows($result);

        while($row = mysql_fetch_array($result, MYSQL_ASSOC))
        {
            echo <<<PRINTALL
            {$row[id]}~~::~~{$row[first_name]}~~::~~{$row[last_name]}~~::~~{$row[p_name]}~~::~~{$row[dept]}~~::~~{$row[ph]} 
PRINTALL;
        } 
    }

HTML 页面脚本:

require 'mysqlconnect.php';
    function get_data($url)
    {
      $ch = curl_init();
      $timeout = 5;
      curl_setopt($ch,CURLOPT_URL,$url);
      curl_setopt($ch,CURLOPT_RETURNTRANSFER,1);
      curl_setopt($ch,CURLOPT_CONNECTTIMEOUT,$timeout);
      $data = curl_exec($ch);
      curl_close($ch);
      return $data;
    }

    $getdata = "SELECT * FROM $table WHERE $table.mid != '1'ORDER BY $table.$sortbyme $o LIMIT $offset, $rowsPerPage";
    $result = mysql_query($getdata) or die(mysql_error());

    while($row = mysql_fetch_array($result, MYSQL_ASSOC))
    {
            $idurl = 'http://mydomain.com/dir/file.php?n='.$row['id'].'';
            $p_arr = explode('~~::~~',get_data($idurl));
            $p_str = implode(' ',$p_arr);

           //Use p_srt and p_arr if exists, otherwise just output rest of the
           //html code with second table values


    } 

如您所见,第二个表可能有也可能没有有效的 id,因此没有结果,但第二个表非常大,总而言之,我正在读取并输出 15k+ 表格单元格。正如您可能从代码中看到的那样,我尝试过分页,但该解决方案不符合我的需求。我必须将客户端的所有数据放在单个 html 页面中。所以请指教。

谢谢!

编辑

第一个表:

id_row    id          first_name   last_name    dept    telephone
1         aaa12345    joe          smith        ANS     800 555 5555
2         bbb67890    sarah        brown        ITL     800 848 8848

Second_table:

id_row    type        model        har               status    id         date         
1         ATX         Hybrion      88-85-5d-id-ss    y         aaa12345   2011/08/12
2         BTX         Savin        none              n         aaa12345   2010/04/05
3         Full        Hp           44-55-sd-qw-54    y         ashley a   2011/07/25
4         ATX         Delin        none              _         smith bon  2011/04/05

因此第二个表是读取和显示的表,如果 ID 正匹配,则读取第一个表并显示信息。 ID 仅在第一个中是唯一的,第二个具有多种格式输入,因此它可能是或不可能是 ID,也可能是重复的 ID。希望这能让我更好地理解我的需要。再次感谢!

I have two tables, one is static database that i need to search in, the other is dynamic that i will be using to search the first database. Right now i have two separate queries. First on page load, values from second table are passed to first one as search term, and i am "capturing" the search result using cURL. This is very inefficient and probably really wrong way to do it, so i need help in fixing this issue. Currently page (html, front-end) takes 40 seconds to load.

Possible solutions: Turn it into function, but still makes so many calls out. Load table into memory and then run queries and unload cache once done. Use regexp to help speed up query? Possible join? But i am a noob so i can only imagine...

Search script:

require 'mysqlconnect.php';

    $id = NULL;
    if(isset($_GET['n'])) {     $id = mysql_real_escape_string($_GET['n']);     }
    if(isset($_POST['n'])) {    $id = mysql_real_escape_string($_POST['n']);    }

    if(!empty($id)){
        $getdata = "SELECT id, first_name, last_name, published_name,
                    department, telephone FROM $table WHERE id = '$id' LIMIT 1"; 

        $result = mysql_query($getdata) or die(mysql_error());
        $num_rows = mysql_num_rows($result);

        while($row = mysql_fetch_array($result, MYSQL_ASSOC))
        {
            echo <<<PRINTALL
            {$row[id]}~~::~~{$row[first_name]}~~::~~{$row[last_name]}~~::~~{$row[p_name]}~~::~~{$row[dept]}~~::~~{$row[ph]} 
PRINTALL;
        } 
    }

HTML Page Script:

require 'mysqlconnect.php';
    function get_data($url)
    {
      $ch = curl_init();
      $timeout = 5;
      curl_setopt($ch,CURLOPT_URL,$url);
      curl_setopt($ch,CURLOPT_RETURNTRANSFER,1);
      curl_setopt($ch,CURLOPT_CONNECTTIMEOUT,$timeout);
      $data = curl_exec($ch);
      curl_close($ch);
      return $data;
    }

    $getdata = "SELECT * FROM $table WHERE $table.mid != '1'ORDER BY $table.$sortbyme $o LIMIT $offset, $rowsPerPage";
    $result = mysql_query($getdata) or die(mysql_error());

    while($row = mysql_fetch_array($result, MYSQL_ASSOC))
    {
            $idurl = 'http://mydomain.com/dir/file.php?n='.$row['id'].'';
            $p_arr = explode('~~::~~',get_data($idurl));
            $p_str = implode(' ',$p_arr);

           //Use p_srt and p_arr if exists, otherwise just output rest of the
           //html code with second table values


    } 

As you can see, second table may or may not have valid id, hence no results but second table is quiet large, and all in all, i am reading and outputting 15k+ table cells. And as you can probably see from the code, i have tried paging but that solution doesn't fit my needs. I have to have all of the data on client side in single html page. So please advice.

Thanks!

EDIT

First table:

id_row    id          first_name   last_name    dept    telephone
1         aaa12345    joe          smith        ANS     800 555 5555
2         bbb67890    sarah        brown        ITL     800 848 8848

Second_table:

id_row    type        model        har               status    id         date         
1         ATX         Hybrion      88-85-5d-id-ss    y         aaa12345   2011/08/12
2         BTX         Savin        none              n         aaa12345   2010/04/05
3         Full        Hp           44-55-sd-qw-54    y         ashley a   2011/07/25
4         ATX         Delin        none              _         smith bon  2011/04/05

So the second table is the one that gets read and displayed, first is read and info displayed if ID is positive match. ID is only unique in the first one, second one has multi format input so it could or could not be ID as well as could be duplicate ID. Hope this gives better understanding of what i need. Thanks again!

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(1

拧巴小姐 2024-12-07 20:52:23

有几件事:

  1. 这里完全不需要 Curl。
  2. Order by 会大大减慢你的查询速度。
  3. 我会对 ID 进行 if is_numeric 检查。

当您在查询中限制为 1 时,为什么要使用 whilemysql_num_rows
$table 和其他这些东西在哪里设置?
缺少代码。

如果您向我们提供相关两个表的数据结构,我们可以帮助您进行查询,但就您现在的设置方式而言,我很惊讶它竟然还能正常工作。

您正在做的是,对于 $tablemid!=1 中的每一行,您正在执行对第二页的curl 调用,该调用将再次获取 ID 并进行查询。这真的非常糟糕,而且比需要的要复杂得多。让我们看看你的表结构。

基本上你可以做:

select first_name, last_name, published_name, department, telephone FROM $table1, $table2 WHERE $table1.id = $table2.id and $table2.mid != 1;

摆脱卷曲,摆脱爆炸/内爆。

A few things:

  1. Curl is completely unnecessary here.
  2. Order by will slow down your queries considerably.
  3. I'd throw in an if is_numeric check on the ID.

Why are you using while and mysql_num_rows when you're limiting to 1 in the query?
Where are $table and these other things being set?
There is code missing.

If you give us the data structure for the two tables in question we can help you with the queries, but the way you have this set up now, I'm surprised its even working at all.

What you're doing is, for each row in $table where mid!=1 you're executing a curl call to a 2nd page which takes the ID and queries again. This is really really bad, and much more convoluted than it needs to be. Lets see your table structures.

Basically you can do:

select first_name, last_name, published_name, department, telephone FROM $table1, $table2 WHERE $table1.id = $table2.id and $table2.mid != 1;

Get rid of the curl, get rid of the exploding/imploding.

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文