在我的例子中,新记录插入到数据库表中

发布于 2024-12-17 03:58:45 字数 2027 浏览 0 评论 0原文

我正在开发一个 Rails v2.3 应用程序,它是搜索项目信息的服务,其中项目信息存储在数据库中。

数据库中有一个现有的 projects 表,如下所示:

img src="https://i.sstatic.net/eOWyi.png" alt="在此处输入图像描述">

< 为了满足客户的要求,该表需要每天半夜插入新数据。

创建这些新记录的原因是为了让Rails应用程序除了按全名搜索之外,还能够按单个单词搜索项目。

例如,如果按单词搜索“门户”、汽车租赁门户位置跟踪门户记录均应由 Rails 应用程序找到。 这就是应用程序的数据库需要包含来自 project_name 的每个单词的所有记录。

因此,我的计划是通过拆分 project_name column(上述projects)中的值来生成这些新记录表)转换为单个单词,然后使用每个单词作为新记录project_name,同时保留表的其他列记录不变。

例如,在上表中,第一条记录有project_name汽车租赁门户”,我要做的就是将此字符串拆分为3单词并构造以下三个新记录以插入表中:

在此处输入图像描述

为了实现这一目标。我尝试创建一个 rake 任务,它从原始 projects 表中获取所有记录,并且对于每条记录,rake 任务 拆分字符串值将 project_name 列转换为Words,然后用Words构造新记录并插入到表中。我的 rake 任务 类似于下面的代码:

all_records = ActiveRecord::Base.execute("select * from projects;") 
all_records.each do |record|
     user_id = record[0]
     project_name=record[1]
     department = record[2]
     other = record[3]

     words=project_name.split()

     words.each do |word|
         sql = "insert into project values (#{user_id},#{word},#{department},#{other});"
         ActiveRecord::Base.execute(sql)
     end
end

rake 任务运行良好,它创建预期的新记录并插入到 projects 表中,< strong>但是问题是需要36小时才能完成!

这是可以理解的,因为原始表非常非常大,如果将字符串拆分为单词并创建新记录,就像创建一个大 3 倍的表(假设 project_name 的每个字符串有 3 个单词)。

我的问题:

  • 一些 Rails 专家能否建议我一些更有效的方法来实现我上面描述的新记录插入?

  • 或者在我的情况下有什么新方法可以启用单字搜索吗? (这就是不要使用我设计的将每个单词存储在数据库中的方式。)

I am developing a Rails v2.3 application which is a service to search projects' information where projects info are stored in database.

There is an existing projects table in the database like following:

enter image description here

For the sake of satisfying customer’s requirement, this table needs to insert new data in the mid-night everyday.

The reason of creating these new records is to make the Rails application be able to search projects by a single word besides searching by the full name.

For example, if search by word "portal", both Car rental portal and Position track portal records should be found by the Rails application. That's the app.'s database needs to have all the records of each single word from project_name.

So, my plan is to generate those new records by spliting the value in project_name column (of the above projects table) into single words and then use each single word as a new record's project_name while keep other columns of the record unchanged.

For example, in above table, the first record has project_name "Car rental portal", what I gonna do is to split this string into 3 words and construct the following three new records to be inserted into the table:

enter image description here

To achieve this. I tried to make a rake task which gets all records from the original projects table, and for each record, the rake task splits the string value of project_name column into words, then construct the new records with words and insert into the table. My rake task looks like the code below:

all_records = ActiveRecord::Base.execute("select * from projects;") 
all_records.each do |record|
     user_id = record[0]
     project_name=record[1]
     department = record[2]
     other = record[3]

     words=project_name.split()

     words.each do |word|
         sql = "insert into project values (#{user_id},#{word},#{department},#{other});"
         ActiveRecord::Base.execute(sql)
     end
end

The rake task works well, it creates the expected new records and inserted into the projects table, BUT the problem is it takes 36 hours to complete!

It is understandable since the origin table is very very large, if split the string to words and create the new record it's like create a 3 times larger table (suppose each string of project_name has 3 words).

My question:

  • Could some Rails experts suggest me some more efficient way to achieve the new record insertion thing I described above?

  • Or any new way to enable single word search in my case? (That's do not use the way I designed to have each single word store in the database.)

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(2

清音悠歌 2024-12-24 03:58:45

如果您这样做只是为了搜索目的,为什么不使用 Sunspot ?它支持全文搜索。

拆分项目名称对我来说听起来是一个非常糟糕的主意。

但如果你想花更少的时间,那么我鼓励你将这个单一任务分成更多的 rake 任务,这会做同样的事情,但对于其他项目集。

If you do this only for searching purpose, why don't you use Sunspot ? It supports full text search.

Splitting project name sounds like a really bad idea for me.

But if you want to take it less time, then I'd encourage you to split this single task into more rake tasks, that would do the same, but for other set of projects.

¢蛋碎的人ぎ生 2024-12-24 03:58:45

为了更快地导入,您需要使用 activerecord-import,它将加快您的执行速度数量级。

columns = [:title, :project_name, :department, :other]
values = all_records.inject([]) do |values_arr, record|

    user_id, project_name, department, other = record

    project_name.split.each do |name|
        values_arr << [user_id, name, department, other]
    end
    values_arr
end

class TempModel < ActiveRecord::Base; set_table_name "projects"; end
TempModel.import columns, values, :validate=>false

For faster importing, you want to use activerecord-import, it will speed up your execution by a couple orders of magnitude.

columns = [:title, :project_name, :department, :other]
values = all_records.inject([]) do |values_arr, record|

    user_id, project_name, department, other = record

    project_name.split.each do |name|
        values_arr << [user_id, name, department, other]
    end
    values_arr
end

class TempModel < ActiveRecord::Base; set_table_name "projects"; end
TempModel.import columns, values, :validate=>false
~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文