Suitecrm: Consider using JSON for Vardefs

[connorshea]

I wanted to propose that the vardef files, including editviewdefs.php, searchdefs.php, detailviewdefs.php, listviewdefs.php, etc. be (optionally?) changed to YAML files.

This idea is admittedly kind of crazy, and I don't know how likely it'd be to actually have it adopted, but I figured after the long conversation in Gitter about it I'd open an issue for it.

PHP:

<?php
$searchdefs [$module_name] = 
array (
  'layout' => 
  array (
    'basic_search' => 
    array (
      'name' => 
      array (
        'name' => 'name',
        'default' => true,
        'width' => '10%',
      ),
      'current_user_only' => 
      array (
        'name' => 'current_user_only',
        'label' => 'LBL_CURRENT_USER_FILTER',
        'type' => 'bool',
        'default' => true,
        'width' => '10%',
      ),
      'active' => 
      array (
        'name' => 'active',
        'label' => 'LBL_ACTIVE',
        'type' => 'bool',
        'default' => true,
        'width' => '10%',
      ),
    ),
    'advanced_search' => 
    array (
      'name' => 
      array (
        'name' => 'name',
        'default' => true,
        'width' => '10%',
      ),
      'assigned_user_id' => 
      array (
        'name' => 'assigned_user_id',
        'label' => 'LBL_ASSIGNED_TO',
        'type' => 'enum',
        'function' => 
        array (
          'name' => 'get_user_array',
          'params' => 
          array (
            0 => false,
          ),
        ),
        'default' => true,
        'width' => '10%',
      ),
    ),
  ),
  'templateMeta' => 
  array (
    'maxColumns' => '3',
    'maxColumnsBasic' => '4',
    'widths' => 
    array (
      'label' => '10',
      'field' => '50',
    ),
  ),
);

YAML:

layout:
  basic_search:
    name:
      name: name
      default: true
      width: 10%
    current_user_only:
      name: current_user_only
      label: LBL_CURRENT_USER_FILTER
      type: bool
      default: true
      width: 10%
    active:
      name: active
      label: LBL_ACTIVE
      type: bool
      default: true
      width: 10%
  advanced_search:
    name:
      name: name
      default: true
      width: 10%
    assigned_user_id:
      name: assigned_user_id
      label: LBL_ASSIGNED_TO
      type: enum
      function:
        name: get_user_array
        params:
          - false
      default: true
      width: 10%
    active:
      name: active
      label: LBL_ACTIVE
      type: bool
      default: true
      width: 10%
    location_address_city:
      type: varchar
      label: LBL_LOCATION_ADDRESS_CITY
      width: 10%
      default: true
      name: location_address_city
    location_address_state:
      type: varchar
      label: LBL_LOCATION_ADDRESS_STATE
      width: 10%
      default: true
      name: location_address_state
templateMeta:
  maxColumns: '3'
  maxColumnsBasic: '4'
  widths:
    label: '10'
    field: '50'

JSON version for funsies

{
  "layout": {
    "basic_search": {
      "name": {
        "name": "name",
        "default": true,
        "width": "10%"
      },
      "current_user_only": {
        "name": "current_user_only",
        "label": "LBL_CURRENT_USER_FILTER",
        "type": "bool",
        "default": true,
        "width": "10%"
      },
      "active": {
        "name": "active",
        "label": "LBL_ACTIVE",
        "type": "bool",
        "default": true,
        "width": "10%"
      }
    },
    "advanced_search": {
      "name": {
        "name": "name",
        "default": true,
        "width": "10%"
      },
      "assigned_user_id": {
        "name": "assigned_user_id",
        "label": "LBL_ASSIGNED_TO",
        "type": "enum",
        "function": {
          "name": "get_user_array",
          "params": {
            "0": false
          }
        },
        "default": true,
        "width": "10%"
      }
    }
  },
  "templateMeta": {
    "maxColumns": "3",
    "maxColumnsBasic": "4",
    "widths": {
      "label": "10",
      "field": "50"
    }
  }
}

You can generate YAML from a vardefs object with the following script, just replace the comment with your vardefs (after running composer require symfony/yaml):

<?php
require 'vendor/autoload.php';
use Symfony\Component\Yaml\Yaml;

$vardefs = array (
  // vardefs here
);

# Dump the PHP object as YAML, with a depth of up to 6 and 2-space indentation.
$yaml = Yaml::dump($vardefs, 6, 2); echo $yaml;

So it should be realistically possible to convert between the formats without too much pain (hopefully).

Pros:

• The Vardefs are much easier to read and write
• Less extra syntax
• No need for PHP code in files that are used exclusively for configuration (clear separation of concerns)

Cons:

• Migrating to this format will probably be a huge pain if you're doing anything abnormal here.
• Lots of code to rewrite.
• Maintenance burden, especially if YAML _and_ PHP are supported simultaneously.
• Getting Studio to output consistent YAML when changing a field/module might be difficult (not that it's really consistent currently)
• No way to use variables (do people actually use any sort of dynamic variables in these files?)

Honestly, this _probably_ isn't worth needing to constantly parse/dump the vardefs into/out of YAML when interacting with the CRM. But I thought it was an interesting enough idea to share. Feel free to close this issue if it's unlikely to ever happen :)

[pribeiro42]

Well... you do have a call to vardef manager at the bottom of most (all?) verdefs.php files:

VardefManager::createVardef( 'Accounts', 'Account', array( 'default', 'assignable', 'security_groups', 'company', ) );

[pgorod]

YAML, no, please. It's fragility due to the dependence on whitespace is just terrible.

If you really want to drop PHP syntax then go with TOML or JSON.

[Dillon-Brown]

My preference would probably be JSON, Interesting idea though :D.

[pgorod]

I can share some ideas from my experience with Hugo which we use on the Documentation site.

Hugo can read YAML, JSON or TOML configuration files and data files. It just decides based on the file extension. So everybody can use their favorite choice, and they can even mix styles if they want.

The code to read these isn't too hard - most languages already have the necessary parsers so you just tell PHP to load the stuff into an array. In our case the hard part, I believe, is _writing_ to these files, because I expect that code probably isn't abstracted into a writer class and is scattered in tons of places in SuiteCRM (especially in Studio and Module Builder).

[pribeiro42]

@pgorod I think the way it is now, its almost impossible to change suitecrm to read and write a bunch of formats.. the read code is also spread throughout the codebase.. just check how many (if file_exist(...) require ...) constructs we have...
The best course of action, i think, would be to extract all the reading and writing to appropriate functions, and then it would be easy to add the necessary format filters to the mix...
I think its a nice goal for, maybe SuiteCRM 9 or so... I guess its a huge work to put on SuiteCRM 8...

[pgorod]

I think most of the read code is done through a handful of places. Though I am sure there will be many little parts getting read from all over the place.

For example: the code in QR&R that grabs parts from the Extension directory and writes consolidated files in Ext directory. This is handling a lot of stuff, and it is all done in one place.

My claim is just that a good portion of the files could be made compatible with JSON with little work. If things were left backwards-compatible (which they need to be) then we could work incrementally - progressively support JSON in more places.

[PedroErnst]

JSON has a major, major con which is no comments. We don't always use comments in vardefs or viewdefs, but the inability to leave any comments at all seems very limiting.

[pgorod]

All things considered I am inclined to think that we should just leave vardefs as they are. PHP is not necessarily a bad syntax for this. And it's not _an additional_ syntax that one has to learn.

[ildarius]

I'd suggest looking into how Magento does it, they use XML, so there are closing and opening tags for being able to more easily navigate the structure, sometimes it's just a challenge to figure out inside which nested array you're working in.

3

It's definitely a lot more verbose but much more readable at the same time and just feels more user friendly imo