Beego: [orm] Transactions and concurrency

[pdf]

The way transactions are currently handled in the beego orm is sub-optimal. The reason the stdlib sql.Begin() returns the transaction is so that calls can be isolated to that particular connection/transaction. In beego, the transaction is assigned to orm.db, which means that with concurrency, any database calls that happen to occur while one piece of code has started a transaction will be bundled into that same transaction, and this is rarely what is wanted.

We can work around this by making a copy of the Ormer, and starting the transaction on that, but ideally, we would be able to request a transaction, execute queries against it, then commit and discard the transaction instance.

Example of currently broken behaviour:

var o orm.Ormer

type User struct {
    Id          int
    Name        string
}

func init() {
    // Set up connection, etc here
    orm.RegisterModel(new(User))
}

func main() {
    /*
      Here, we might want/expect only `Tester McTest` to be inserted as part of
      the transaction, but there is a race, so `Charlie Root` may also get
      bundled in the same transaction
    */
    go func() {
        user := new(User)
        user.Name = `Tester McTest`
        o.Begin()
        o.Insert(user)
        o.Commit()
    }()
    go func() {
        user := new(User)
        user.Name = `Charlie Root`
        o.Insert(user)
    }()
}

With a transaction returned:

var o orm.Ormer

type User struct {
    Id          int
    Name        string
}

func init() {
    // Set up connection, etc here
    orm.RegisterModel(new(User))
}

func main() {
    /*
      Here, we might want/expect only `Tester McTest` to be inserted as part of
      the transaction, and since we get back a transaction to work with, this will
      behave as expected.
    */
    go func() {
        user := new(User)
        user.Name = `Tester McTest`
        tx, _ := o.Begin()
        tx.Insert(user)
        tx.Commit()
    }()
    go func() {
        user := new(User)
        user.Name = `Charlie Root`
        o.Insert(user)
    }()
}

In this example, with a o.Begin() function that's been modified to return the transaction instead of assigning it to orm.db, the result would be consistently as expected.

I'm not sure how best to add this functionality though, because modifying the method return signature will break existing code. I suppose we would have to add a new function to handle this, something like NewTransaction? Do you have any preference for naming conventions here? It's a real shame though that beego orm.Begin() didn't mirror sql.Begin(), because now things will be confusing.

I'd also like to implement callbacks, but that's dependent on sane transaction handling.

[pdf]

@astaxie is this closed because it's been fixed?

[astaxie]

@pdf nop, sorry for the wrong close. I just clean all the issues now.

[mychaelgo]

Any work around for this issue?

[jaffcheng]

I'm facing the same issue, will this be fixed? I'm using the ormer globally since the golang stdlib sql.DB is designed to be long-lived and I assumed that the ormer will behave in the same way.

[leonth]

Is there any workaround for this? Not being able to use transactions properly is giving us major problems right now. We are not even sure whether it is possible to work around this by using raw db connections.

[michaelrios]

I am seeing this issue as well. One of my services is using an older version (v1.9.2), and the service where I am seeing this issue is v1.10.1

[dotSlashLu]

Simply don't use a global ormer, create a new one every time. If sharing transaction is intended, pass on this ormer.

    go func() {
        o := orm.NewOrm()
        user := new(User)
        user.Name = `Tester McTest`
        tx, _ := o.Begin()
        tx.Insert(user)
        tx.Commit()
    }()

[flycash]

@jianzhiyao will fix it

[jianzhiyao]

There should provide a closure function to fix that problem