Yii2: Active record collection object

[dhiman252]

Hi Team,

I cannot find any collection object in the documentation for the active records.

Do we have any feature like the laravel framework offers : https://laravel.com/docs/5.1/collections ?

If not please refer to the link and if possible please add this feature in the next version release !!

Thanks

[SilverFire]

Yes, currently Yii has not collection object, we use arrays instead to be lightweight.

In my own development practice it's usually enough except some special cases, that can be solved locally without affecting overall application performance.

Could you describe your use case?

[adiramardiani]

Interesting :)
Is it possible to reference the link above (collections) as an enhancement for ArrayHelper ?

[SilverFire]

Could you elaborate your idea, please

[cebe]

@dhiman252 part of the functionality of the laravel collection is provided by DataProviders in Yii. Another part is directly part of the QueryInterface already. Please describe in detail what you need and why.

[dhiman252]

For the ease of use I would like this functionality incorporated in the framework, lets say i have table for user reviews and it contains the message and the points, now if i had to display the user rating average and all the reviews on a single page then i need to fire 2 SQL Queries one getting the reviews average and the other all reviews in an array
so if we have a collection object then 1 Query and ease of use
like
/* $reviewsAvg = $user->reviews->avg('points'); $reviewsCount = $user->reviews->count(); froreach($user->reviews as $review){ //display review } *
There are a lot of functions associated with the collections which makes the development its easy.

[cebe]

$reviewsAvg = $user->getReviews()->avg('points');
$reviewsCount = count($user->reviews);
froreach($user->reviews as $review){ 
    //display review
}

This already works in yii.

[Alex-Bond]

@cebe here is much more better example:
For example i need to query 1 time and show data in 2 columns, but not more that 10 records per column. First column contains only active elements (active=1) and second one showing not active (active=0).
In case of not using collections i need to filter it with if's and than show. If i use collections i can use filter() and using closure filter data as i want.
Or lets imagine that i need to show items 3 per row in bootstrap. According to best practice you need to create new row element for each 12 columns, so i need to create some $i and increase it every iteration and based on value print new row element.
In case of collections i can just use chunk() method and create loop inside loop.

This is only few of use cases. There are of different ways to use it.

P.S. I think this feature can be added to 2.1.* milestone because of simple integration.

[Razzwan]

I agree that it is a good idea.

More options use:

1. If you use the collection instead of an array, it is possible to implement the mechanism links are much more logical than done now. Instead of $object->link('string', $arrayOfObjects) we can use $object->relationName->link([$id1,$id2]). This is good becouse not use string constant. Of course this is only for linked relations
2. ActiveDataProvider - this is very good for return response but not good for use in code. That part of the functionality that exists in ActiveDataProvider also would not prevent. For example, for a particular copying objects when
3. Without collections voplne can do, and solutions will be simple. But these solutions will need to compose himself. A collection will have a ready solution in the documentation. I'm here because I see the advantage. In own castom decision, no matter how simple it may be, still it needs more time.

I would like to take up the implementation of this mechanism in 2.1 version.

[samdark]

It was discussed multiple times. While it sounds good there are doubts and cons:

1. At the time we were not able to come up with a good use case where collections are significantly better than arrays.
2. Extra overhead.
3. It was likely to break backward compatibility. Back then it was significant enough obstacle to drop proposal. Now, when 2.1 is in development, it's not an issue.

[juanmacias]

Sometimes you need business logic for a group of ARs.
This is an example: ShoppingCart and CartLine[].
You usually should create a ShoppingCart and Lines in memory. When you save the ShippingCart, everything is saved.

class CartLineCollection extends ActiveCollection
{
    /**
     * Rules for the collection
     * @return array
     */
    public function rules()
    {
        return [
            ["total", checkMaxTotal()], //makes a custom rule validation over the attribute total of each line
            ["sku", checkMaxItems()]
        ];
    }

    public function addLine($line)
    {
        if ($this->validateAdd($line)) {
            return parent::add($line);
        } else {
            return false;
        }
    }

    /**
     * Returns all lines with the specified TaxId
     * @param $tax_id
     * @return mixed
     */
    public function getLinesByTaxId($tax_id)
    {
        return parent::filter(["tax_id"=>$tax_id]);
    }

    /**
     * Return an array of totals by tax_id
     * @return mixed
     */
    public function getTotal()
    {
        return parent::sum("total")->groupby("tax_id");
    }

[samdark]

OK but that's standalone collection, not the one tied to AR. By "tied to AR" I mean when using Post::find()->all() you'll get a collection such as PostCollection instead of array of Post models.

[juanmacias]

Is not a standard collection, is a collection of ARs. When you call collection->save(), it makes a call to each element->save().
When you call validate, it calls each element->validate().

That's the only case I found, I used once. Although I still prefer Arrays.....

[dhiman252]

But what is needed is to reduce the common tasks efforts. A collection has common function of filteration, like we fetch active records and the relations via clousure now i want to filter those activic records which have returned null for the relation attribute.

[samdark]

@dhiman252 common function of filtration is provided via standard PHP array_filter(). The only difference is syntax:

$posts = array_filter(Post::find()->all(), function($post) {
    return $post->relation === null;
});

// vs 

$posts = Post::find()->all()->filter(function($post) {
    return $post->relation === null;
});

Not a big difference. Also the task itself is better to be solved on query level. Retrieving more than needed and filtering afterwards on PHP side is inefficient both CPU and memory-wise.

[lubosdz]

The functionality seems kind of specific and of rare use, though has attractive syntax.
Frameworks generally should stick to simple & generic solutions.
There are also unclear questions: _what's the memory & speed overhead ?_
Most of use cases above can be achieved with existing Yii objects, though sometimes in a clumsy way.

What about adding an extra option/method asObject(bool) that would return object list rather than array (default)?

Post::find()->asObject()->all();
Post::find()->asArray()->all();

[samdark]

You mean something like the following?

$postsCollection = Post::find()->asCollection(PostCollection::class)->all();

[lubosdz]

Well, yes, but is the argument PostCollection::class really necessary ?

[samdark]

Which collection are you going to fill then?

[lubosdz]

Post collection, of course.
What I meant was, that the argument PostCollection::class might be optional - if at all actually exposed to a user. May be just adding method to AR model would suffice (just example):

public static function collectionName()
{
    return "PostCollection"; // autogenerated by Gii
}

public static function asCollection($className = null)
{
    if(!$className){
        $className = self::collectionName();
    }
    // convert array to object list & bound & invoke events/methods
    return $className::toObjectList();
}

[samdark]

1. We don't want to change default behavior. Arrays are sufficient for majority of cases.
2. Generic collections doesn't make much sense except a bit nicer syntax.
3. Collections do make sense for interfaces and type hinting.

[samdark]

Basic implementation is done. One can either use it with simple collections as in tests or pass something more advanced as a class such as Laravel collections.

[klimov-paul]

If you use the collection instead of an array, it is possible to implement the mechanism links are much more logical than done now. Instead of $object->link('string', $arrayOfObjects) we can use $object->relationName->link([$id1,$id2]). This is good becouse not use string constant. Of course this is only for linked relations

For me it is better to put link() into a ActiveRelation (ActiveQuery) method.
Syntax could be:

$object->getRelationName()->link([$id1, $id2]);

I can't see how AR collection involved here.

ActiveDataProvider - this is very good for return response but not good for use in code. That part of the functionality that exists in ActiveDataProvider also would not prevent. For example, for a particular copying objects when

I do not understand this part. Why do you need to copy AR objects? And how it relates to the AR collection?

Without collections voplne can do, and solutions will be simple. But these solutions will need to compose himself. A collection will have a ready solution in the documentation. I'm here because I see the advantage. In own castom decision, no matter how simple it may be, still it needs more time.

I don't think the feature is general enough. Besides creating collection instance can be easily performed from the array of objects. Thus this feature can be composed into a separated extension outside the scope if 'yiisoft'. You could implemeted it already long ago.

[samdark]

@klimov-paul top level collections are indeed easy to implement. Relations aren't that easy, as for me.

[klimov-paul]

One more note about linking improvement: using collections may introduce you a better syntax for 'has-many' relations, but it will entrely unusable for 'has-one' relations.

[klimov-paul]

See https://github.com/yiisoft/yii2/pull/12304#issuecomment-242339116

For me the nearest thing to 'AR collections' meant here is yii\db\BatchQueryResult and yii\db\DataReader. Interacting with this functionality is necessary for the AR collection.

[Razzwan]

For me it is better to put link() into a ActiveRelation (ActiveQuery) method.
Syntax could be:

$object->relationName->link([$id1, $id2]);

Magic method __get($relation) returned a Collection object.

When I use Collection object in current project:
1. Link objects by ids

$object->relationName->link([$id1, $id2]);

2. Synchronization objects by ids

$object->relationName->sync([$id1, $id3]);

This code delete $id2 relation and add $id3.
3. Named filtering data objects:

$object->relationName->finteringPublic()

• return only public data.

$object->relationName->filteringForUserPerm($userPerm)

4. Mass saved/updated data $object->relationName->save() instead of

foreach ($object->relationName as $obj) {
    if (!$obj->validate()) {
        $object->addErrors('relationName', $obj->getErrors()); // this is simplified example
        if (!$obj->isNewRecord) {
            $obj->delete();
        }
    } else {
         $obj->save(false);   
    }
}

5. Mass load

$object->relationName->load(Yii::$app->getRequest()->getPost())

instead static method of ActiveRecord which less intuitive.

6. Copy data from one object to other.

$nonPayedDataObject->relationName = $payedDataObject->relationName->filteringNonPayedData()

[klimov-paul]

$object->relationName->link([$id1, $id2]);
Magic method __get($relation) returned a Collection object.

As I said at https://github.com/yiisoft/yii2/issues/10806#issuecomment-242342046 , it will work for 'has-many' relation, but any attempt to use such syntax against 'has-one' relation will cause PHP fatal error:

$post->comments->link([$id1, $id2]); // will work
$post->owner->link($userId); // PHP fatal error:: call to undefined method `link()` of `ActiveRecord`

[klimov-paul]

Copy data from one object to other. $nonPayedDataObject->relationName = $payedDataObject->relationName->filteringNonPayedData()

There was a conscious decision agains direct assignment of the relations via magic setters.
You can use populateRelation() for this purpose:

$nonPayedDataObject->populateRelation('relationName', array_filter($payedDataObject->relationName, fucntion() {...});

[Razzwan]

@klimov-paul

As I said at #10806 (comment) , it will work for 'has-many' relation, but any attempt to use such syntax against 'has-one' relation will cause PHP fatal error:...

What prevents to fix it? I agree that it must be fixed too.

[klimov-paul]

1. Mass saved/updated data $object->relationName->save() instead of

A horrible example: you propose to delete the records and insert the same one overa again.
The reason why there is sync fucntionality provide in the core - is it is not trivial: sometimes you need the algorithm you provided and sometimes - another one like: https://github.com/yii2tech/ar-linkmany/blob/master/LinkManyBehavior.php#L235

[klimov-paul]

What prevents to fix it? I agree that it must be fixed too.

Fix what?

[Razzwan]

@klimov-paul

There was a conscious decision agains direct assignment of the relations via magic setters.
You can use populateRelation() for this purpose:
$nonPayedDataObject->populateRelation('relationName', array_filter($payedDataObject->relationName, fucntion() {...});

Do you really believe that it's as simple as my example? You do not mind the presence of string constant in the code, which in the case of changing the name of relation all stop working?

[klimov-paul]

1. Named filtering data objects: $object->relationName->finteringPublic() - return only public data. $object->relationName->filteringForUserPerm($userPerm)

Navigation-type data processing is not the common for the remote databases. Introduction of it will encourage bad practice of selecting the entire list of records in the memory and processing them afterwards instead of use SQL filtering.

[Razzwan]

@klimov-paul

Navigation-type data processing is not the common for the remote databases. Introduction of it will encourage bad practice of selecting the entire list of records in the memory and processing them afterwards instead of use SQL filtering.

And who said that data filtering will not take place at the database level? Collection - class for work with data base.

[klimov-paul]

the presence of stringovoy konstynty in the code

String constants are use wide eough around the Yii. I can't see a crucial problem here. There are many examples which may cause just the same kind of trouble.

Besides as I already said here: https://github.com/yiisoft/yii2/issues/10806#issuecomment-242331322
We can improve ActiveRelation (ActiveQuery) to remove this problem:

$object->getRelationName()->populate([...]);

[klimov-paul]

And who said that data filtering will not take place at the database level? Collection - class for work with data base.

So what is wrong to create a separated ActiveQuery in the related AR class and introduce an internal method inside for the same purpose:

class RelationNameQuery extends \yii\db\ActiveQuery
{
    public function filteringPublic()
    {
        return $this->andWherePublic()->all();
    }
    ...
}

And what is wrong of general code:

$filteredRelatedData = $object->getRelationName()->andWhereActive()->all();

for the same purpose?

[klimov-paul]

Over this entire discussion I can't find any example, which indicates the feature is necessary or at least general.
All arguments provided relate to minor improvements and creating of the 'pretty' code for the rare use cases.

Note: Yii follows the principle 'simplicity over complexity', that is why Yii ActiveRecord is relatively simple. It can not compete with Doctrine and other major Database AR layer, lacking many features - this is by design. It is done in this way in order to keep it simple and robust.

That is why I am against of introduction of such functionality in the Yii core.
However as I said in the my initial comment: https://github.com/yiisoft/yii2/issues/10806#issuecomment-242331322 , you can create your own extension, which will provide this functionality for you, if you need it so badly.

[samdark]

@klimov-paul your suggestions about improving ActiveQuery are very interesting.

[klimov-paul]

your suggestions about improving ActiveQuery are very interesting.

Personally I do not consider usage of the string constants to be an issue - it is @Razzwan's concern.
Still this matter should be discussed at separated issue.

[samdark]

Closing the issue. Decision is not to create collections from ARs. It's not final so if you have strong arguments for returning collections from ARs, post these please.

For standalone collections, either implement your own or use any existing implementation such as Laravel's.

[juanmacias]

Although is closed, here is a real example.

This is a Kanban view.
There is two approach of doing this:

1. Call find() for each column. In this cases 5 times.
2. Call find() one, get all rows and filter in memory.

Also, I have to get a sum for each column.

What I have now is:
$opCol=new ActiveCollection(Opportunities::find()->somefilter()->all();
$total=$opCol->sum('annual_contract_value');
$column_rows=filter(["status_id"=>$status->id]);

I've more elaborated functions like filters by json values inside attributes.

But... there is a third solution in the AR, something like:

Opportunities::find()->transformby("status_id")->all();
will return an array of arrays with a key for each status_id.

[klimov-paul]

Same as here: https://github.com/yiisoft/yii2/issues/10806#issuecomment-242350062

Why do not introduce a special method inside ActiveQuery for your goal?

class OpportunityQuery extends \yii\db\ActiveQuery
{
    public function transformby($field)
    {
        $rows = $this->all();
        $result = [];
        foreach ($rows as $row) {
            $result[$field][] = $row;
            //...
        }
        return $result;
    }
    ...
}

[juanmacias]

Good idea!

On Aug 29, 2016 20:32, "Paul Klimov" [email protected] wrote:

Same as here: #10806 (comment)
https://github.com/yiisoft/yii2/issues/10806#issuecomment-242350062

Why do not introduce a special method inside ActiveQuery for your goal?

class OpportunityQuery extends \yii\dbActiveQuery{ public function transformby($field) { $rows = $this->indexBy($field)->all(); $result = []; foreach ($rows as $row) { //... } return $result; } ...}

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/yiisoft/yii2/issues/10806#issuecomment-243212257, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AEk1Zfdvunuw-NmXUDnYAxbZMm1Tu7NYks5qkyW9gaJpZM4HX-2v
.

[cebe]

see https://github.com/yiisoft/yii2-model-collection/issues/6