Osticket: Automation System :: RFC

[greezybacon]

Overview

Objective

There are many uses for a help desk. Trying to design a help desk software which will be common to all uses and should make all possible use cases available in the software presents a few major issues:

• the software configuration will likely be too complex to be at all intuitive,
• no matter how many features are in the software, some customization will likely always be required,
• features common to every help desk will likely need to work in slightly differing manners between different industries and use cases,
• not all help desk administrators know how or care to customize the help desk code base,
• customizing the code base makes upgrading it to the latest version difficult, perhaps nearly impossible.

To help administrators meet the needs and objectives of their end users, osTicket needs a simple and intuitive customization system, which:

• does not require coding skill
• does not make modifications to the code base
• seamlessly upgrades across future versions of osTicket
• is simple and intuitive
• allows administrators to share their customized features
• is pluggable via extensions to allow for more complex interaction among third-party systems

Feature Description

To meet this objective, this document serves as a proposal of a possible automation implementation. It does not describe how the feature would be implemented specifically, but attempts to lay out the functionality that could be provided and the corresponding components necessary to fulfill such a concept.

This document will be accompanied by a technical user interface implementation proposal and technical feature implementation proposal in order to describe the specific changes to the software to be made.

Requirements

Simple Task Automation

Automation actions are triggered from a few events triggered from tickets and other objects and the transitions in their life cycle.

For instance, a ticket might be escalated if it is not handled within a particular time frame. In this case, we would trigger off the last update from an agent and await that

{Ticket Activity: Update} -> 14 days -> [Action: Close]

Complex Task Automation

More complex tasks will involve multiple steps to be performed at various time periods with respect to an initial event and automation step. For instance, if a ticket is created for a customer with a particular SLA, the ticket might need to be escalated at some point in the future, based on certain time constraints and conditions. Once the ticket is escalated the first time, a second set of time constraints and conditions might be applied which were to be considered in the first phase.

User Interface Interaction

The automation system should also be able to interact with the user interface. For instance, automation steps should be allowed to provide warnings or errors to requested changes, as well add arbitrary follow-up messages to the web user interface or HTTP API.

Components

Schemes

Automation is divided into different sections which are intended to be for different purposes. One might think of the schemes as individual "features" or "workflows" for the help desk. When a new type of customization is needed, which is different in purpose from all other automations, then a new scheme is merited.

Schemes group together all the necessary pieces into a single place--like a folder--so that everything has a common place. The schemes in a help desk can also be imported, exported, and shared among other help desks. Since everything for the automation is contained in the scheme, importing a scheme or importing updates to a scheme does not affect other defined schemes.

Individual automation pieces are grouped in order to provide for the greatest reuse. The scheme allows for the user interface to divide the automation steps into logical groups with corresponding names. This allows for help desks with several complex automation schemes to be presented in a more approachable and simplistic interface.

Schemes can be enabled or disabled so that seasonal?

Schemes contain the following pieces:

• Variables: Placeholder values used throughout the scheme which are defined in one place. Having them in one place means updating happens in exactly one place and takes effect everywhere.
• Documents: Crafted templates such as pages and emails which can be used throughout the automation scheme. This allows for documents to be shared among several automation steps. Again, editing the documents happens in one place and takes effect everywhere.
• Triggers: Events which signal the automation system to do something. Triggers can be attached to virtually any action on any kind of object.
• Steps: The building blocks of the scheme. Steps combine a event signal with a condition, a delay, and an action.
• Blocks: Conditions and actions can be combined together to form a logical process. These defined process blocks can be used as a action, which allows them to be reused by multiple steps.

Steps

Step are the building blocks of the automation scheme. They are comprised of two major pieces: a trigger and an action. Steps have names and notes and can be sorted in the user interface to add clarity to how the automation scheme works.

Optionally, two conditions can be used to define whether or not the action should be performed when the event occurs:

• An immediate condition which is evaluated when the event happens. If the condition is not met, then no further part of the step is considered.
• A delayed condition which is evaluated after the delay. For instance, if the action is to be performed 4 days after the event, then the delayed condition will be evaluated after the 4 days elapses.

The user interface and the automation system should allow for complex conditions to be defined with AND, OR, NOT and possibly other combinatorial logic. Such will allow for sufficient complexity in describing conditions and would eliminate the need for multiple conditions or a list of conditions.

Triggers

A trigger is defined by a reference event, an optional condition, and a delay. So, when a certain event happens, an optional condition can be checked, and then, after a certain period of time, the associated action can be performed.

Actions

An action can be either a single, simple command such as sending a message or assigning a ticket. Or, more complex actions can be built with a block, and such a block can be executed as the action of a step.

Actions have a context defined in the step so that, when the action is executed, the step can define the details of the action. For instance, if a URL is to be requested for a web hook, then the step would likely need to define the URL to be passed to the web hook action.

A full tree of an automation trigger and block might look like:

|------------ TRIGGER ------------X---------------- ACTION ----------------|

ticket.created +---- immediately  +---- if user is new ----- send welcome
               |                  `---- if user in org ----- send msg to org
               |
               `---- after 3 days +---- if unanswered  +---- raise priority
                                  |                    `---- notify dept mgr
                                  `---- if ...

Blocks

Several actions and conditions can be logically grouped together similar to a logical procedure in an imperative programming language and used as a single "block" to be executed as the action of a step.

This model allows for reuse of steps to be used among multiple triggers. This disconnection is more useful for complex automation where actions may be performed for multiple reasons using differing delays. It also allows for the automation to vary based on placeholder variables defined in the block. When the block is used later, the placeholder variables can be defined.

Additionally, if a block is scheduled by a trigger, then the block can be canceled or rescheduled by another trigger. This allows a procedure, whether simple or complex, to be consistently referenced in step definitions.

Automation Documents

Many automation actions will make use of large amounts of carefully crafted text, graphics, and presentation design, for emails, web pages, error messages and warnings. This careful crafting should be self-contained and reusable--especially if automation pieces are to be shared among multiple help desks. Allowing documents or groups of documents to be shared means an administrator can craft an email template set and allow it to be imported into other help desks. Then, the help desk administrator could, after importing it, modify the respective automation tasks to use the new template(s). Together with the variable feature, the automation will allow for maximum ability for sharing and reuse.

Automation Variables

In order for automation schemes to be reused, as well as for grouped steps to be reused with multiple triggers, automation groups need to have a concept of variables. This will be especially useful for modular automation systems which are distributed to multiple help desk administrators who would like to use the system in slightly different ways -- perhaps with different timings, messages, drop-down references.

In Schemes

When an automation scheme is imported, the associated variables should be required by the user interface at the time of import. Of course, the values can be changed later, but defining all "required" variables at the time of import will allow for the administrator to determine and understand immediately how the automation will work on his or her help desk.

In Blocks

When a block is referenced from a trigger, any required or optional variables should be defined in the association. When used in a sentence, the user interface should present it like:

When a ticket is assigned to an agent, always notify the scrum master immediately with some context

where the blocked terms are (in order): the event, the condition, the action, and the delay. The block notify the scrum master is defined elsewhere can be executed with some context including things such as assignee, email template, and recipients.

In Documents

When a document is designed, it can have placeholder variables placed throughout the document. The value of the variables will be rendered when the document is rendered. The variables should be defined along with the document so the variables can be presented in a drop-down list in the document designer.

Custom Object State

The state of the object being automated and its history is important to track, because future automation triggers may need to base their automation steps on the history of the automations performed on the object. This feature will piggy-back on the variable feature by allowing objects to have values assigned into their state which can be used by the automation system. When certain events happen, the state of the object can be extended by the automation system. This feature will allow for more esoteric and automation-specific functions. An automation step might count the number of times that a ticket was reassigned and provide that number in the ticket object's state.

Because values assigned into the object's state are initially undefined, all usage of the object state variables will have to provide a default value. This default value will be used in the event that the value is not yet defined for the object.

This custom object state should be presentable to some users of the help desk will need to visually see and modify the custom state of objects used by the automation feature. This could be done by only showing the data to help desk "administrators" or by adding a new flag to agents, perhaps named something like "automation tester."

Challenges

Performance

The objective of the automation system is to allow simple customizations to the system to be tailored without any edits to the software codebase. The main trigger for automations is object-based events or scheduled time. However, this system needs to be implemented in such a way that the overall performance of the system is not adversely impacted--whether a system has only a few or no automation actions, or if it has hundreds.

If the automation is tied directly to the signal system, it would be possible to connect the automation system to any event triggered by any object. However, it would be easy to adversely affect the performance of the API and web user interface if each signaled event required extra time to determine if some automation step should be triggered from the signal.

One potential optimization might be to set up the automation system to use a deferred signal handler which will capture all the signals fired, potentially filtered for the signals of interest, and then perform the automation-system operations after the completion of the request. This will allow for a sophisticated automation system without delaying the user interface. However, it will prevent the automation system from interacting with the user interface, such as preventing changes to objects or the display of warnings or messages.

So in practice, the automation system will need a more complex, hybrid approach which allows for interaction with the user interface, but does not adversely impact the performance of the software in general.

Administrative User Interface

The user interface should be designed to allow an administrator to fully take advantage of all the concepts of the automation system, but should not require a book or special training in order to use. This concept is ultimately one of osTicket's greatest features. osTicket is simple to use and is administratively intuitive. The automation feature should not change this paradigm.

It should also make provision for adding and defining things inline so as to prevent an administrator from having to hop back and forth to define things in different places in order to define new automation steps.

Stop Processing Further on Match

The ticket filter feature (which will be replaced by the automation feature) has a feature to "stop processing further on match". This feature is primarily used to match and forbid ticket creation and then not process the other ticket filters (which might do things like send emails) or do other things which should not be done if the ticket were rejected.

Having this feature be part of the automation system could potentially be confusing and cause subtleties which could be difficult to troubleshoot. For instance, a single event on an object might trigger several steps in several different automation schemes. Intentional interruptions in one scheme should not affect other schemes.

However, there are cases, in which both the current automation scheme and also all other schemes should be halted for a particular event. For instance, when new objects or changes are rejected, and an action such as "Reject Change" should flag the event such that no step in any other scheme should be executed. This action should also be designed in such a way that any other actions which were previously scheduled should be canceled.

Then, to allow for the possibility for other schemes to interact with a rejected change, another delay should exist: "immediately, even if rejected", which will allow for actions to be executed on rejected changes. No other delays make sense because, since the change was rejected, there is no need for follow-up actions.

Note that the signaled event itself should be flagged as "rejected" in this case, in a standardized manner, which will allow other schemes executed after the one rejecting the change to detect that the change was rejected.

Execution Order

To simplify the automation administration process, the types of actions performed on the automated objects should have a defined priority. Then, the scheme should be executed in a manner that allows for the actions to work in a predictable manner without requiring the administrator to place the steps in the proper order. When a scheme is executed, this would be a possible execution strategy:

1. Only steps matching the event are considered
2. If the change is not already rejected
   
   
   
   1. Only "immediate" delays are considered
   
   
   2. All actions are queued and sorted by priority
   
   
   3. Actions are executed according to declared (internal) priority
   
   
3. Now, with potentially updated state, all delays other than "immediate" are considered
4. If the change is rejected,
   
   
   
   1. Only "immediate, even if rejected" is considered
   
   
   2. Again, all actions are queued and sorted by priority
   
   
   3. Actions are executed in sorted order
   
   
5. Otherwise, (if change is allowed),
   
   
   
   1. All non-immediate delays are considered
   
   
   2. Actions are executed in arbitrary order, because records will simply be created in the automation-state table.
   
   

When processing actions in the future, as part of the automation state table, after the trigger delay elapses, actions should be considered using this strategy:

1. All actions for the same scheme which should occur at the same time are considered together
2. If the action has a delayed condition,
   
   
   
   1. Delayed condition for each action is evaluated
   
   
   2. Action is discarded if condition evaluates to false
   
   
3. Actions are queued and sorted by priority
4. Actions are executed according to declared (internal) priority

Schemes in the system should be orderable so that schemes run in a particular order. This makes sense for schemes which will prevent certain kinds of changes.

Follow-up Automation Blocks

When an event triggers a block to be executed after a non-zero delay, it's possible that the time may need to be reset. For instance, considering the auto-close feature, which might auto-close a ticket 14 days after it is marked "resolved" with no follow-up from the end user, after the ticket is marked "resolved", a follow-up block will be scheduled in 14 days to close the ticket. Two things--maybe more--might happen:

1. The end user might respond
2. An agent might change the state of the ticket to something other than "resolved" or "closed"

Either of these events should cause the follow-up event to be canceled or rescheduled. So, the automation scheme created to handle this particular state machine should handle all the edge cases. If the end user responds, then the 14-day follow-up block should be canceled. After an agent responds, if the ticket is still marked "resolved" the previous follow-up action should be replaced with a new one. Or, the clock should simply be reset.

The custom object-state feature can make this somewhat simpler, because the ticket can be marked as "pending-auto-close", which will help the rest of blocks in the scheme to react only if the ticket is supposed to be auto-closed or not.

The problem is that multiple triggers could require the follow-up to be reset. For this reason, the blocks which perform the actions are separated from the triggers which schedule the actions to be performed. This allows for the same block to be referenced by multiple triggers, and allows auto-close action/block to be uniquely identified and clearly reset.

In this case, if a block is scheduled to be executed for an object, and it is already scheduled, then the block timer is always reset. And logically, there is a need to "cancel" the execution of another block from a step, so that an trigger action can cancel a previously-scheduled block.

Actions

Here are some of ideas for basic actions. The automation system should
implement a pluggable system to allow for plugins to extend the automation
system and define new actions. Actions implemented here are intended to be
simple steps which should be generic and reusable and might fit into a
larger automation scheme.

• Attach a note
• Attach a response
• Send an email
• Request a URL (with parameters)
• Change a value
  
  
  
  • Ticket priority
  
  
  • Custom data
  
  
  • Custom object state
  
  
• Transfer ticket
• (re)Assign ticket
• Reject a change
• Trigger an (custom) event

Transition

When transitioning from a previous version of osTicket, such as v1.14, an initial scheme should be created called "ticket filters". Then, all the current ticket filters should be converted to automation steps and blocks. For the imported items, the event will be ticket.created and the delay will be zero (0). Each ticket filter will be represented by its own block with the conditions and actions properly translated.

The execution order of the filters will be transitioned to the order of the steps in the scheme. However, since the order of the steps is not considered in the execution of the scheme, two schemes will be created. The first will contain and handle all actions which reject tickets. The second will contain all the ticket filters which perform other tasks.

The "stop processing further on match" is not transitioned, because the automation system handle it automatically.

Examples

Here are some examples which are meant to help understand and convey the design of the automation feature. There may be simpler or better ways to accomplish these tasks. Furthermore, you may have more specific needs for your automations. These examples are provided as an introductory mechanism and to test out the completeness of the automation concept rather than provide definitive answers to every administrator's requirements.

Auto-Close Automation Scheme

Many help desk administrators desire that, after a ticket is marked as resolved, the customer should have the opportunity to disagree. But if the customer makes no rebuttle to the status of the ticket, then it should be closed automatically after a certain time period.

This scheme focuses on automated closure, but the scheme could be easily extended to also whine to the end user to submit updates to the ticket or recommend for closure.

Custom Object State

name | data type | initial | notes
:--------------|:----------|:--------|:-----------------------------
pending_close | boolean | false | If ticket close is scheduled

Variables

name | data type | default | notes
:---------|:----------|:-----------|:-----------------------------------
Window | duration | 14 days | Time between last update and ticket close
Resolved | string | 'resolved' | Ticket state triggering the auto-close window
Closed | string | 'closed' | Ticket final state at closure

Steps

+-----------------------+                   +-------------------------+
| ticket.state          | -+-- +14 days --> | Execute [Close Ticket]  |
|     == 'resolved'     |  |                +-------------------------+
+-----------------------+  |                +-------------------------+
                           `- immediately > | Set ticket automation   |
                                            | state,                  |
                                            | `pending_close` = true  |
                                            +-------------------------+
+---------------------+                     +-------------------------+
| ticket.message.new  | ----- immediately > | Set ticket automation   |
+---------------------+                     | state,                  |
                                            | `pending_close` = false |
                                            +-------------------------+
+------------------------+                  +-----------------------+
| ticket.state           | -- immediately > | Cancel [Close Ticket] |
|     != 'resolved'      |                  +-----------------------+
+------------------------+      /-------------\
+------------------------+     /       If      \     [ YES ]
| ticket.response.new    | --- < `pending_close` > -- 14 days -+
+------------------------+     \    == true    /               |
                                \-------------/                |
                                                               V
                                           +------------------------+
                                           | Execute [Close Ticket] |
                                           +------------------------+

Or, as a table:

trigger | initial condition | delay | action / block
:--------------------|:-------------------------|:------------|:-----------------------------
ticket.state | == '%{Resolved}' | immediate | set {}.pending_close = true
ticket.state | == '%{Resolved}' | %{Window} | execute [Close Ticket]
ticket.message.new | none | immediate | set {}.pending_close = false
ticket.state | != '%{Resolved}' | immediate | cancel [Close Ticket]
ticket.response.new | {}.pending_close == true | %{Window} | execute [Close Ticket]

Blocks

Close Ticket

                /--------------\
+-------+      /        IF      \           +----------------------------+
| START | --> < {}.pending_close > -- Y --> | Set {}.state = '%{Closed}' |
+-------+      \    == true     /           +----------------------------+
                \--------------/

Idle Ticket Whining Scheme

Since end users create tickets and post messages with the expectation of responses, the help desk should whine to assignees about posting responses to tickets if an end users ticket remains idle for a certain time period.

Variables

name | data type | default | notes
:---------|:----------|:-----------|:---------------------------------------
Window | duration | 4 hours | Time between last message and reminder

Steps

trigger | initial condition | delay | action / block | context
:--------------------|:------------------|:-----------|:----------------|:--------
ticket.message.new | (none) | %{Window} | execute [Whine] | ticket
ticket.response.new | (none) | immediate | cancel [Whine] | (none)

Documents

Whine-Mail

Dear %{recipient.name},

Please respond to the end user on ticket %{ticket.url.agent}.

Cheers,

Blocks

Whine

[Send Email], with {
    document: "Whine-Mail"
    recipients: ticket.assignees,
    context: ticket,
}

[ntozier]

Sounds good! :)

[ghost]

Souds great indeed :)

[balojs]

nice @greezybacon 👍

[logmsf]

Woah very well detailed project. Looks really good. @greezybacon
Thanks for pointing this out to me @protich

In the actions, I would add: Change status

[joeldeteves]

I hope this also includes the automatic creation of scheduled/recurring tickets/tasks as well (a much requested feature in the forums, etc).

Excited for the possibility of seeing automation features in a future release!

[Richiricheh]

Huge fan of this project/ addition to osTicket. I would also suggest that what might be a good idea here is to add a portion of this code to the ticket creation page as well. What I mean by this is that a portion of this project will involve the creation of a "ticket template" this can easily be used/leveraged not only by an automation process for scheduled tasks etc. but also leveraged at the time of ticket creation. For example imagine a typical ticket one might create and lets assume that the ticket has 2-3 different sub tasks imagine selecting the ticket template at the time of ticket creation and simply entering in the customer/requestor's name and the rest gets filled out.

Looking ahead to what this automation project would leverage it would use a ticket template for system maintenance. As an organization you might automagically schedule/ create a ticket on the last Friday of every month. That Sys. maint. ticket (based on the template) would get created and then get addressed accordingly.