Carbon-fields: Caching

[timiwahalahti]

Since CF uses it's own way to get data from database (direct queries) and completely bypasses WP core default functions like get_post_meta and get_option, it does not take advantage of built-in caching.

I couldn't find any level of caching in CF own code, am I blind or is it really so that theres no cache?

[atanas-dev]

Hi @timiwahalahti ,

Currently, there is no cache when fetching meta values and this is something we are looking into implementing soon.

[dmhendricks]

If it helps, I have been using my own object caching wrapper for retrieving values. Something like:

$cache = new ObjectCache( 'object_cache' => array(
  'group' => 'my_plugin_cache',
  'expire_hours' => 72
));

And then to fetch a value:

$my_option = $cache->get_object( 'carbon_field_name', function() {
  return carbon_get_theme_option( 'carbon_field_name' );
});

This will return the result from the object cache (if found), else pull it from the database via the carbon_get command(s).

It's not perfect because if a returned value is empty it will check every time, but it has worked well for me with Redis object caching.

Of course, you need to make sure that you appropriately flush the cache when options are saved. Example for theme_options:

add_action( 'carbon_fields_theme_options_container_saved', array( $this, 'options_saved_event' ) );

public function options_saved_event() {
  // Clear the cache so that new values are returned
  $this->cache->flush();
}

Note that this code is alpha-quality, but you can create your own to suit your needs. It's just meant to be an example.

[dmhendricks]

@atanas-angelov-dev

As noted above, I have been managing my own caching with Carbon Fields (most of my sites use Redis or Memcached). However, it can be troublesome at times (particularly since you can't clear cache by group)... tracking the field names (yeah, I can loop through the container, but I'm not sure if that is available in uninstall.php unless I save them to the DB?), adding the blog_id if it is_multisite() and is not a network container, etc. I prefer not to flush the entire cache with each container save.

I was thinking that it would be nice if Carbon Fields handled some or all of it...

Container::make( 'theme_options', 'my_plugin_container', 'Theme Options' )
    ->set_cache_group( 'myplugin_cache_group' )
    ->set_cache_expire( 86400 ) // seconds; optional; defaults to 0
    ->add_fields( ...

If neither method is used, caching is not enabled. If one or both is set, perhaps something like this to flush the group?

// $site_id only relevant for network container type
carbon_flush_containertype_cache( $site_id = SITE_ID_CURRENT_SITE, 'my_plugin_container' );

It would pull the fields from from the container and flush each field key.

Automatically Flushing the Cache on Save

Ideally, it would be nice if we didn't have to hook into container_saved as well. Something like:

->set_flush_on_save( true )

Combining Methods

Perhaps they could be combined into one method:

/* $group_name_or_true is a string (cache group name) or boolean 'true'. If true, auto-set the group name.
 * $expire = number of seconds, default 0 (no expiry)
 * $flush_on_save = whether or not to flush on save, default true?
 */

->set_cache( $group_name_or_true, [$expire = 0], [$flush_on_save = true] )

Thoughts? I'm just thinking out loud...

[ju1ius]

Hi,
I just encountered an _N+1 queries_ issue, and I found this thread while searching for a solution.
I was trying to load 250 custom posts w/ 8 custom fields... query count was around 1800...
This is what I ended up doing in order to leverage the Wordpress built-in metadata cache:

<?php declare(strict_types=1);

namespace App\Carbon\Datasore;

use Carbon_Fields\Carbon_Fields;
use Carbon_Fields\Datastore\Post_Meta_Datastore;
use Carbon_Fields\Field\Field;
use Carbon_Fields\Toolset\Key_Toolset;

final class KeyValuePair
{
    public $key;
    public $value;

    public function __construct(string $key, string $value)
    {
        $this->key = $key;
        $this->value = $value;
    }
}

trait EagerLoadingMetaDatastore
{
    protected function get_storage_array(Field $field, $storage_key_patterns)
    {
        $storage = [];
        $meta = get_metadata($this->get_meta_type(), $this->get_object_id());
        if (!$meta) {
            return $storage; // new object
        }
        foreach ($storage_key_patterns as $storage_key => $type) {
            switch ($type) {
                case Key_Toolset::PATTERN_COMPARISON_EQUAL:
                    if (isset($meta[$storage_key])) {
                        $storage[] = new KeyValuePair($storage_key, $meta[$storage_key][0]);
                    }
                    break;
                case Key_Toolset::PATTERN_COMPARISON_STARTS_WITH:
                    foreach ($meta as $key => $value) {
                        if (strpos($key, $storage_key) === 0) {
                            $storage[] = new KeyValuePair($key, $meta[$key][0]);
                        }
                    }
                    break;
                default:
                    throw new \LogicException("Unknown storage key pattern type: {$type}");
                    break;
            }
        }

        $storage = apply_filters('carbon_fields_datastore_storage_array', $storage, $this, $storage_key_patterns);

        return $storage;
    }
}

final class EagerLoadingPostMetaDatastore extends Post_Meta_Datastore
{
    use EagerLoadingMetaDatastore;
}

add_action('carbon_fields_fields_registered', function() {
    $repo = Carbon_Fields::resolve('container_repository');
    foreach ($repo->get_containers() as $container) {
        $datastore = $container->get_datastore();
        if ($datastore instanceof Post_Meta_Datastore) {
            $container->set_datastore(new EagerLoadingPostMetaDatastore());
        }
    }
});

Query count down to 166 !

@atanas-angelov-dev, is there any particular reason why the datastores don't use the built-in get_metadata function ?

[dmhendricks]

This is good stuff. I'm wondering if you have time to do a PR? I would, but I'm not savvy enough to contribute such code. It appears that Atanas is assigned to WP Emerge now?

It would be nice to have caching support for all container types.

As much as I prefer free as beer and speech (who doesn't??), I kind of wish that htmlBurger would charge money for a premium version (he said, reluctantly). I would pay money for certain features, and this is one of them. The Gutenberg block thing mentioned in the blog is definitely another. I was stoked when they added the network container - these are all killer features.

I can't afford to hire a developer, but I'm sometimes okay at rallying the troops for money (and would also contribute what I can) if someone would get caching in CF core. I know that they are super-busy and I blame nobody - busy is good and CF is amazing! I would like to contribute to a solution with my limited means.

Money is a technicality. It is time that is precious. We never have enough time...

[ju1ius]

This is good stuff. I'm wondering if you have time to do a PR?

Sure, I could do that.
I'm just wondering if the «hit the database every time» behavior is actually done on purpose.
That's what I asked to @atanas-angelov-dev.

[emohamed]

Hey guys,

Optimization for the query count was initially planned, but we never dedicated the time to get it done.

@ju1ius thank you for the contribution. We're going to use your code as a starting point, and include an update in the next release.

@dmhendricks you raise some interesting questions. We considered offering paid version of Carbon Fields, but being a library rather than a plugin makes it rather hard to charge for the code. At this point we don't have plans to do that, but we might consider adding a paid tier in some post-Gutenberg release.