当前位置：文江博客话题详情

自动将数据从mysql导入到solr

发布于 2025-01-06 02:08:39 字数 131 浏览 0 评论 0原文

我想将值从 mysql 导入到 solr..我通过

使用 mysql 触发器调用 php 脚本来进行自动导入。但我读到这不是一个好方法..还有其他

自动导入数据的解决方案吗？

有人可以帮我吗lz...

原文

分享到QQ

分享到微博

如果你对这篇内容有疑问，欢迎到本站社区发帖提问参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。

发布评论

需要登录才能够评论，你可以免费注册一个本站的账号。

烦人精 2025-01-13 02:08:39

尽管正如其他响应中提到的那样，有一个内置的机制，数据导入处理程序 (DIH)，但我发现这个工具不是很灵活。我的意思是，如果我想在索引之前进行任何数据处理，我只能依赖 MySQL 函数，而我可以使用 PHP 函数。

我最终将自己的数据导入处理程序编写为 PHP 脚本，在其中执行初始查询，然后逐步执行结果并在插入 SOLR 索引时处理（和缓存）数据。它并不太复杂，看起来像这样（仅是演示性的）：

SELECT 
  book.id AS book_id,
  book.name AS book_name,
  GROUP_CONCAT(DISTINCT author.name) AS authors
FROM
  book
INNER JOIN
  link_book_author AS alink ON alink.book_id = book.id
INNER JOIN
  author ON author.id = alink.author_id
GROUP BY
  book.id;

$stmt = $dbo->prepare($sql);

$stmt->execute();

while ($row = $stmt->fetch(PDO::FETCH_OBJ)) {

    try {

        $document = new Apache_Solr_Document();

        $document->Id = $row->book_id;
        $document->BookName = $row->book_name;

        $document->Author = explode(',' $row->author);

        $this->getSearchEngineInstance()->addDocument($document);

    } catch (Exception $e) {

        error_log(sprintf('Unable to add document to index: (%s)', $e->getMessage());
    }
}

这只是您可以执行的操作的一个示例，在我的情况下，当我进行完全导入时，我还涉及缓存以提高性能。使用本机 DIH 无法做到的事情。

我用来通过PHP访问SOLR的API是solr-php-client，可能还有其他的在那里，所以谷歌周围。

Even though there is a built in mechanism for this very thing, Data Import Handler (DIH), as mentioned in the other responses, I found this tool not very flexible. What I mean by this is, if I wanted to do any data massaging before indexing I could only depend on MySQL functions, when I could have used PHP functions.

I ended up writing my own Data Import Handler as a PHP script, where it does the initial query, then steps through the results and massages (and caches) data upon insert into the SOLR index. It wasn't too complicated, and would look something like (demonstrative only):

SELECT 
  book.id AS book_id,
  book.name AS book_name,
  GROUP_CONCAT(DISTINCT author.name) AS authors
FROM
  book
INNER JOIN
  link_book_author AS alink ON alink.book_id = book.id
INNER JOIN
  author ON author.id = alink.author_id
GROUP BY
  book.id;

$stmt = $dbo->prepare($sql);

$stmt->execute();

while ($row = $stmt->fetch(PDO::FETCH_OBJ)) {

    try {

        $document = new Apache_Solr_Document();

        $document->Id = $row->book_id;
        $document->BookName = $row->book_name;

        $document->Author = explode(',' $row->author);

        $this->getSearchEngineInstance()->addDocument($document);

    } catch (Exception $e) {

        error_log(sprintf('Unable to add document to index: (%s)', $e->getMessage());
    }
}

This is just an example of what you can do, In my situation I also involve caching to increase performance when I do a full import. Something you cannot do using the native DIH.

The API I use to access SOLR through PHP is solr-php-client, there may be others out there, so google around.

回复收藏 0 原文