并发 goroutine 中的 Golang 和 Gorm 死锁批量插入

发布于 2025-01-17 21:12:55 字数 1806 浏览 0 评论 0原文

我试图使用 gorm ,Golang和MySQL插入许多记录。我的代码看起来像这样:

package main

import (
    "fmt"
    "sync"

    "gorm.io/driver/mysql"
    "gorm.io/gorm"
)

type Article struct {
    gorm.Model
    Code string `gorm:"size:255;uniqueIndex"`
}

func main() {
    db, err := gorm.Open(mysql.Open("root@tcp(127.0.0.1:3306)/q_test"), nil)
    if err != nil {
        panic(err)
    }

    db.AutoMigrate(&Article{})

    // err = db.Exec("TRUNCATE articles").Error
    err = db.Exec("DELETE FROM articles").Error
    if err != nil {
        panic(err)
    }

    // Build some articles
    n := 10000
    var articles []Article
    for i := 0; i < n; i++ {
        article := Article{Code: fmt.Sprintf("code_%d", i)}
        articles = append(articles, article)
    }

    // // Save articles
    // err = db.Create(&articles).Error
    // if err != nil {
    //  panic(err)
    // }

    // Save articles with goroutines
    chunkSize := 100
    var wg sync.WaitGroup
    wg.Add(n / chunkSize)
    for i := 0; i < n; i += chunkSize {
        go func(i int) {
            defer wg.Done()
            chunk := articles[i:(i + chunkSize)]
            err := db.Create(&chunk).Error
            if err != nil {
                panic(err)
            }
        }(i)
    }
    wg.Wait()
}

当我运行此代码有时(大约三分之一)时,我会遇到此错误:

panic: Error 1213: Deadlock found when trying to get lock; try restarting transaction

如果我在没有Goroutines的代码(注释行)的情况下运行代码,我将不会僵局。另外,我注意到,如果我删除代码字段上的唯一索引,则僵局将不再发生。而且,如果我用语句替换删除 druncate articles 僵局似乎不再发生。 我还使用PostgreSQL运行相同的代码,并且没有僵局。

知道为什么僵局仅在MySQL上的唯一索引以及如何避免僵局发生?

I'm trying to bulk insert many records using Gorm, Golang and MySQL. My code looks like this:

package main

import (
    "fmt"
    "sync"

    "gorm.io/driver/mysql"
    "gorm.io/gorm"
)

type Article struct {
    gorm.Model
    Code string `gorm:"size:255;uniqueIndex"`
}

func main() {
    db, err := gorm.Open(mysql.Open("root@tcp(127.0.0.1:3306)/q_test"), nil)
    if err != nil {
        panic(err)
    }

    db.AutoMigrate(&Article{})

    // err = db.Exec("TRUNCATE articles").Error
    err = db.Exec("DELETE FROM articles").Error
    if err != nil {
        panic(err)
    }

    // Build some articles
    n := 10000
    var articles []Article
    for i := 0; i < n; i++ {
        article := Article{Code: fmt.Sprintf("code_%d", i)}
        articles = append(articles, article)
    }

    // // Save articles
    // err = db.Create(&articles).Error
    // if err != nil {
    //  panic(err)
    // }

    // Save articles with goroutines
    chunkSize := 100
    var wg sync.WaitGroup
    wg.Add(n / chunkSize)
    for i := 0; i < n; i += chunkSize {
        go func(i int) {
            defer wg.Done()
            chunk := articles[i:(i + chunkSize)]
            err := db.Create(&chunk).Error
            if err != nil {
                panic(err)
            }
        }(i)
    }
    wg.Wait()
}

When I run this code sometimes (about one in three times) I get this error:

panic: Error 1213: Deadlock found when trying to get lock; try restarting transaction

If I run the code without goroutines (commented lines), I get no deadlock. Also, I've noticed that if I remove the unique index on the code field the deadlock doesn't happen anymore. And if I replace the DELETE FROM articles statement with TRUNCATE articles the deadlock doesn't seem to happen anymore.
I've also run the same code with Postgresql and it works, with no deadlocks.

Any idea why the deadlock happens only with the unique index on MySQL and how to avoid it?

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(1

醉态萌生 2025-01-24 21:12:55
  • 删除语句使用行锁执行,表中的每一行锁定以进行删除。
  • 截断表总是锁定表和页面,但并非每行。

资料来源: https://stackoverflow.com/a/a/20559931/18012302

我认为我想我SSQL需要时间进行删除查询。

尝试添加time.sleep查询删除后。

    err = db.Exec("DELETE FROM articles").Error
    if err != nil {
        panic(err)
    }
    time.Sleep(time.Second)
  • DELETE statement is executed using a row lock, each row in the table is locked for deletion.
  • TRUNCATE TABLE always locks the table and page but not each row.

source : https://stackoverflow.com/a/20559931/18012302

I think mysql need time to do DELETE query.

try add time.Sleep after query delete.

    err = db.Exec("DELETE FROM articles").Error
    if err != nil {
        panic(err)
    }
    time.Sleep(time.Second)
~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文