Yii2: Caching TagDependency TTL

[YellowPony]

Hello, TagDependency never expires and if default cache duration is set, tag can be deleted earlier than cached entry. I think simplest way to fix this is to add $duration property to TagDependency.

diff --git a/framework/caching/TagDependency.php b/framework/caching/TagDependency.php
index 9e484e9dd..53b621f0a 100644
--- a/framework/caching/TagDependency.php
+++ b/framework/caching/TagDependency.php
@@ -33,6 +33,11 @@ class TagDependency extends Dependency
      */
     public $tags = [];

+    /**
+     * @var int $duration default duration in seconds before the tag will expire.
+     */
+    public $duration;
+

     /**
      * Generates the data needed to determine if dependency has been changed.
@@ -93,7 +98,7 @@ class TagDependency extends Dependency
         foreach ($keys as $key) {
             $items[$key] = $time;
         }
-        $cache->multiSet($items);
+        $cache->multiSet($items, $this->duration);
         return $items;
     }

[samdark]

@yiisoft/reviewers would you please take a look at this one?

[schmunk42]

Should tag dependencies be able to expire?

[samdark]

No. It's not the job of the dependency.

[samdark]

Thanks for the right question, @schmunk42. @YellowPony won't be changed since it is like it is by design.

[YellowPony]

@schmunk42 why not? It is a cache entry, and it's life must be no longer than its related data (if supported by backend). Otherwise we need to build other system to track expired tags to flush them, if we want to keep our cache optimized ofc.

[schmunk42]

But tags itself can not expire, AFAIK they are not cache entries, it's rather a group or selector with which you can invalidate.

See https://www.yiiframework.com/doc/api/2.0/yii-caching-tagdependency - you can invalidate all cache entries with user-123 but not user-123 itself. Please correct me if I am wrong.

[YellowPony]

Hm..

and it's life must be no longer than its related data
But tags itself can not expire

I think we are partially wrong/right, this expiration can be developer's choice.
Like all items that stored in cache (whatever it is, but they are store in cache backend and use Cache component) can/must/should have TTL (unlimited or not). There are many cases when orphan tags stay in cache. And we have no simple way to prevent this situation.
May be we can class TTL addition like enhancement?

$tag = new TagDependency([
     'duration' => 100, // by default unlimited
     'tags' => ['a', 'b', 'c'],
]);

Offtopic:
As i write in first message

if default cache duration is set, tag can be deleted earlier than cached entry.

This statements is wrong. I check it, look into code, and found that $defaultDuration affects only single set method (found it later in property details).

[schmunk42]

There are many cases when orphan tags stay in cache.

Are tags itself really in the cache?

Can you provide a test case which fails in the current setup?

[YellowPony]

<?php

namespace app\commands;

use yii\caching\TagDependency;

class TestController extends \yii\console\Controller
{
    public function actionIndex()
    {
        $cache = \Yii::$app->cache; // \yii\redis\Cache

        $tag = new TagDependency([
            'tags' => ['A_' . rand(1, 10)],
        ]);

        $key = 'test';
        $data = $cache->get($key);
        if ($data === false) {
            $cache->set($key, ['test_data'], 10, $tag);
        }
    }
}

Record with key "0d4d0a98d6fded2013b1b5773556c29e" is "tag data" and its never expires.

I stop to use tags A_* by any reason, but "tag data" stay in cache untill redis flush.

[samdark]

Are tags itself really in the cache?

Yes, tags are there without expiration. And that's for a reason you cannot rely on native TTL:

1. You set a cache key A with tag tag1 and TTL till 30.03.2019.
2. You set a cache key B with tag tag1 and TTL till 29.03.2019.
3. You set a cache key C with tag tag1 and TTL till 28.03.2019.

Above you can do in totally separate processes so if you just set a TTL for tag1 to one of the values (except by accident it's value from 1.), tag will be invalidated before its associated items are.

[shushenghong]

hope this will be fixed, since tags cache will continuously generate, these will exhaust the cache space, see #17784 .
Or can we add a max_tags property, after max_tags reached, the oldest tags will be deleted.

[samdark]

Thought more about it. The problem of cache space pollution is a real one so yes, an option is needed.

[samdark]

@YellowPony, applying diff as in original post won't work because touchKeys is static. Also, as I said before, removing tags like that is not reliable.

[samdark]

@shushenghong there's no guarantee that oldest tags aren't valid.

[samdark]

Please share ideas on how to fix it in a reliable way.

[My6UoT9]

When the same tag gets cached again, would it be possible to update all references with the "highest" expiration?

[samdark]

What's the "highest" expiration?

[shushenghong]

I would suggest add a maxDependencyTags to Cache component, and use LRU algorithm to invalid tags cache.

abstract class Cache extends Component implements CacheInterface {
   /**
     * @var int max tag dependency tags num. After maximum tags reached, invalid tags cache with LRU algorithm. Default value is 0, meaning infinity.
     */ 
    public $maxDependencyTags = 0;

[My6UoT9]

What's the "highest" expiration?

I mean the expiration time which is furthest in future, e.g. the maximum.
I have no idea if its possible.

[samdark]

@My6UoT9 +1 year or so? That won't be different in result. Cache would be polluted with tag data.

@shushenghong that doesn't sound right design-wise. Tags is just one of the invalidation strategies and Cache knowing about it creates unwanted dependency.

[shushenghong]

@samdark the problem is tags dependency itself also used cache to determine whether related content changed or not, and these caches never expire. We need a solution to control the scale.

[shushenghong]

solutions:

1. do not use cache to determine whether tags related content changed.
2. use complex data structure like redis hash to save tags modified timestamp.
3. control the scale of tags modified timestamp cache, like max_tags_num, or tags timestamp cache ttl

[samdark]

Thanks for variants. 1,2 doesn't sound right. For example, I'll have to install Redis if I just want to cache data using memcached.

max_tags_num may be OK though.

[shushenghong]

I will try to code :)
Hope more advices.

[rob006]

We could pass null as duration for multiSet():

https://github.com/yiisoft/yii2/blob/ad83952197315072124027bf39263eabc1979e1a/framework/caching/TagDependency.php#L96

Then $defaultDuration from cache component will be used, so you can set expiry time for tags at component level.

[shushenghong]

We could pass null as duration for multiSet():

https://github.com/yiisoft/yii2/blob/ad83952197315072124027bf39263eabc1979e1a/framework/caching/TagDependency.php#L96

Then $defaultDuration from cache component will be used, so you can set expiry time for tags at component level.

sounds good :)