Beego: How to implement the default value feature in ORM? Need your idea!

[flycash]

Here are serveral issues about default value. #3924, #4087 #3948
As I mentioned in #4087

Actually the default value is invalid for basic type. In this case, 
when we decide to generate SQL to insert into DB, we know status = 0, 
but we don't know the actual value is 0 or you don't set the value.

In other languages, 
for example, Java, 
we use boxed type Integer rather than int. 

So when status == null we think the value is not set. 

But golang doesn't have similar structure.

I have a short discussion with @tayoogunbiyi in #3924 . Maybe using pointer is the best solution, but it's not convinient for users.

Do you have any better solution?

[godcong]

The first is the determination of priority：

type User struct{
    age int `orm:"default(1)"`
}

u:=&User{
age:0,
}

db.Insert(a)

Which value do you want to used?
Or add a mandatory update field before the update

db.Columns("age").Insert(a)
db.Only("age").Insert(a)
db.Expect("age").Insert(a)

[flycash]

db.Columns("age").Insert(a)
db.Only("age").Insert(a)
db.Expect("age").Insert(a)

In this example, it means that we should use default value when executing Insert method?

[flycash]

Proposal 1:
We design some API like:

package bean

func Create(modle interface{}) interface{} {
   for field := range model {
     field.setDefault(xxxx)
   }
  return model
}

When users use this method, we will set the default value of model and then return. It's not in ORM module but in a new module named "bean". Users need to invoke this method to create an instance and then insert into DB:

m:= bean.Create(&M{})
m.XXX = xxx
...
orm.Insert(m)

The ORM is independent from this design. But we need to use reflection so it may not be high performance.

[flycash]

Proposal 2: Using pointer

If the type of a field is a basic type, we won't support default value feature.

But if they declare the field as pointer, we can support.

So in our design:

if field.isNotPointer {
  return ;
}
if field.Value == nil {
 setDefaultValue()
}

But users won't like this since they may want to use basic types.

[flycash]

Proposal 3: Using map
We can use map or bitmap:

type M struct {
   a A
}

type M_AutoGenerate struct {
    M
   dirty map[string]bool
}

func(m *M) SetA(xxx) {
  m.a = xxx
  m.dirty["a"] = true
}

When Insert is invoked, if the field is "clean", we will use default value.

This design can be used in Update method. We only update "dirty" field.

[godcong]

db.Columns("age").Insert(a)
db.Only("age").Insert(a)
db.Expect("age").Insert(a)

In this example, it means that we should use default value when executing Insert method?

That’s why I asked, which one do you want first
I think in general, orm:"default(1)" should have a lower priority
So that when we don't specify the field, it will used orm:"default(1)"
If i want to insert a value, Although it is 0. we can use:

db.Columns("age").Insert(a)
db.Only("age").Insert(a)

Let it update the specified field
This way we can update only the necessary fields each time instead of updating all fields each time

[tayoogunbiyi]

@flycash
Proposal 1 - I think it works and it's pretty straightforward. We simply need to change how to model instance is being initialized. The major problem ( as you mentioned ) is the use of reflection, which can be slow. We definitely don't want to add any bottleneck to a feature as crucial as inserting into the DB. However, we might just stick to this as we are already using reflection in quite a number of places in our code. The bottleneck may not be so significant and we won't really know till we benchmark it.

Proposal 2 - The user's experience is significantly impacted.

Proposal 3 - This is my favorite implementation as the user keeps using the models as before but setting values using the special SetField method.

[godcong]

@flycash
Proposal 1 - I think it works and it's pretty straightforward. We simply need to change how to model instance is being initialized. The major problem ( as you mentioned ) is the use of reflection, which can be slow. We definitely don't want to add any bottleneck to a feature as crucial as inserting into the DB. However, we might just stick to this as we are already using reflection in quite a number of places in our code. The bottleneck may not be so significant and we won't really know till we benchmark it.

Proposal 2 - The user's experience is significantly impacted.

Proposal 3 - This is my favorite implementation as the user keeps using the models as before but setting values using the special SetField method.

About Proposal 3
Why do you have to set them one by one instead of setting multiple at once?
Isn't it good to use ...?

[flycash]

About Proposal 3
Why do you have to set them one by one instead of setting multiple at once?
Isn't it good to use ...?

How to set multiple at once?

I prefer to:

m.SetA(a).Set(b)...

[flycash]

db.Columns("age").Insert(a)
db.Only("age").Insert(a)
db.Expect("age").Insert(a)

In this example, it means that we should use default value when executing Insert method?

That’s why I asked, which one do you want first
I think in general, orm:"default(1)" should have a lower priority
So that when we don't specify the field, it will used orm:"default(1)"
If i want to insert a value, Although it is 0. we can use:

db.Columns("age").Insert(a)
db.Only("age").Insert(a)

Let it update the specified field
This way we can update only the necessary fields each time instead of updating all fields each time

Ok, I think I get your point. This solution is good but still has two problems:

1. Users need to invoke serveral methods to tell us whether use default value;
2. It may be hard to implement this feature since current design of ORM module is very imcompatible with this feature.

[godcong]

db.Columns("age").Insert(a)
db.Only("age").Insert(a)
db.Expect("age").Insert(a)

In this example, it means that we should use default value when executing Insert method?

That’s why I asked, which one do you want first
I think in general, orm:"default(1)" should have a lower priority
So that when we don't specify the field, it will used orm:"default(1)"
If i want to insert a value, Although it is 0. we can use:

db.Columns("age").Insert(a)
db.Only("age").Insert(a)

Let it update the specified field
This way we can update only the necessary fields each time instead of updating all fields each time

Ok, I think I get your point. This solution is good but still has two problems:

1. Users need to invoke serveral methods to tell us whether use default value;
2. It may be hard to implement this feature since current design of ORM module is very imcompatible with this feature.

if Proposal 3 can be solved, this can used same program.
A tag like dirty to remark which value need use.
However, the handling of dirty may be a bit more complicated.
Hope we can find a simple and efficient way to solve this issue. :)

[tayoogunbiyi]

Using pointers seems like the easiest way

[flycash]

Using pointers seems like the easiest way

I wrote a demo to try this solution. But it looks not good. The codes look like:

    a := 123
    i := new(int)
        // i = &123 doesn't work...
    i = &a

And then if I want to use i, i need to do:

    b := (*i) * 13
    i = &b

I think the user experience is not good.

You can try with pointer to write some codes.

[godcong]

Why not define some interfaces, and then write some implementations of xorm and gorm, so that you don’t have to write orm yourself, :)

When go2 generics come out, I think the implementation of orm will change greatly.

[tayoogunbiyi]

What if we could have our own boxed version of each field.
For instance, in Django, the model fields use something like this

class Person(models.Model):
    first_name = models.CharField(max_length=30)
    last_name = models.CharField(max_length=30)

That way we've control over whether a field has been set or not, and can appropriately handle defaults. I feel we should settle this before going way too far in the ORM refactor as this may change a lot about how things work.

[flycash]

Why not define some interfaces, and then write some implementations of xorm and gorm, so that you don’t have to write orm yourself, :)

When go2 generics come out, I think the implementation of orm will change greatly.

That's my concern. So I prefer to support proposal1 which is out of ORM scope but it can resolve this issue.

[flycash]

So, I want to support proposal1 because it's the easiest way and current API is not necessary to be changed. The user experiences is fine. And this feature can be used in users business logic.

[AllenX2018]

It seems that gorm has the same problem and they solve this by using a pointer type or scanner/valuer.
reference: http://gorm.io/docs/create.html#Default-Values

[flycash]

Yes I know gorm. But I don't think it's a good solution. Using pointer cause bad user experience.
So I want to support those ways both and every proposal should be pluggable. The default implementation is that we use default value if the field is '', 0, false...