Openwhisk: Do we want Rules to be anonymous always, with no possibility to name them?

[markusthoemmes]

As part of the work on nuking the Activator and thus refactoring the way we fire triggers and match rules to them it became quite obvious that rules as they currently are implemented are just a name for the connection between a trigger and an action.

• They don't provide much value as they are 1:1 bindings anyway, which might as well be implemented as a connection between trigger and action, just without giving it a name
• They are activatable/deactivatable, but imho it makes more sense to have triggers activatable/deactivatable, as that would actually pause the stream of events to that trigger. To replace rules deactivating behaviour one might just remove the connection between triggers and actions.

Without rules, the steps to take on the CLI will roughly stay the same, but you don't need to name the connection and as we all know, "Naming things" is one of the hardest problems in IT.

Edit: The discussion has drifted away a bit from my initial thoughts. Title has been renamed by @rabbah for that purpose. Some of the comments might not fit the title anymore.

[domdom82]

I agree, tentatively. Actually I thought at the beginning we wanted rules to implement an 1:n relation between triggers and actions. Any particular reason for the 1:1 mapping? Anyway, even that could be done as a list or set of actions attached to a trigger.

[markusthoemmes]

Triggers to Actions are in fact 1:n. Every rule represents a 1:1 connection, summed up this is 1:n.

[perryibm]

@rabbah @sjfink In case you missed this thread

1. Make triggers rather than rules activable Any trigger could feed multiple actions so stopping the trigger would stop more than just stopping the rule.
2. Forget activability and use deletion (orthogonal to naming I am ambivalent on this one as it definitely simplifies to remove this concept. However.... Were there plans to persist information related to the rule such as user asking how many times this was fired? If so, deletion would erase that information and would not be the same as activation.
3. Don't name the rule Viewing this as a (directed) graph, it seems redundant to name the edge. I can see two reasons for naming. First is that you might eventually want multiple edges connecting the same trigger/action because there are different types of rules such as ones that throttle or perform filtering etc. (ie multigraph). Second as convenience if you find yourself having to talk about that edge. Currently, we use that for activability so the case here partly hinges on issue 2 and also on the possibility of future issues related to auditing and filtering. If the rule becomes unnamed, it doesn't change its role as an entity nor the obligation for permission checking and one would still need the ability to list rules.

[domdom82]

Different, yet related question: Do we care about referential integrity? I.e. if with this new model I have a couple of triggers T1, T2, T3 all referencing the same action A and I delete that action, will I need to update (i.e. remove) the reference to that action in all triggers?

Imho I would argue against that because it would force us to use relational structures on a noSQL db, i.e. traversing documents and finding occurrences of that now deleted action. I would simply not touch the triggers and log a warning message like "trigger T1 tried to fire nonexistent action A"

[sjfink]

Yes, I've been following -- guess it's time to jump in.

I'm opposed to removing Rules. All of @perryibm's points apply -- these are important features I don't want to lose.

Later if this debate is still alive when I get back from vacation, I can enumerate concrete use cases at length. Don't have time to write a detailed case with examples at the moment. Instead, let me state two abstract "principles":

1) I believe in event-driven programming as a powerful feature to build distributed systems without tight coupling between event producers and event consumers. The Rule is what allows this -- without the Rule, we throw away the concept of "event bus", and many good features of event-based programming.
Suggested Reading: Distributed Event-Based Systems (Muhl et al. 2006)

2) A fundamental principle of programming language design: if it's worth having a concept in the programming language, it's worth having a first-class object for the concept. I can/will come up lots of examples where having first-class Rules makes OpenWhisk programming easier.

Sorry for the relative brevity -- if this debate lives for a while, I'll come back with examples to justify these principles concretely.

[mbehrendt]

@sjfink can you please list concrete real-world mass-market use cases which we consider to be in scope for what we're trying to do with whisk? often, it's trade-off between simplicity and richness in the of concepts, and we need to consider both sides.

eg -- as markus stated, having to name a rule is at the moment introducing complexity that wouldn't be needed in most (maybe all) of today's use cases. in fact, the system would be more usable for the large majority of the use cases if we didn't have to assign a name, for instance. so in the context of thinking through concepts, we need to strongly factor in whether we're using a specific concept at the cost of complicating a mass market /simple use case, which is not viable. a simple use case always has to be as simple to be as simple to execute on as if a more complex concept didn't exist -- i.e. additional complexity must be optional. if that is not possible, we have to be open to sacrificing the more complex use case.

i'm not saying we have to keep or get rid of rules, but we can't just be asking to keep it without those considerations being addressed. however, i'm for removing rules if we can't identity strong enough use cases -- which i haven't seen so far.

[sjfink]

Real-world use case 1: DarkVision. I may want to add more image processors or remove image processors which are all driving off the single cloudant trigger, as the app is currently designed.

Real-world use case 2: Twitter Analytics: I may want to register and deregister multiple feed processors tied to one feed.

Real-world use case 3; Mobile Notifcations: Someone generates an event from their mobile phone. There are N categories of alerts (pager, phone, email, SMS) that I want to enable or disable for a stream of events.

Event-driven programming is fundamental to our whole model. Throwing away Rules throws away most of the benefit of event-driven programming as an architectural pattern.

[mbehrendt]

it seems like i can do all of that with a trigger-action mapping, that doesn't require a rule as an explicit, named entity?

[rabbah]

No free lunch. Rules are recorded in triggers is an implementation detail. You want to encode rules via the trigger API and not call them rules is broadening the complexity of the triggers API.

Why do we need triggers? I can argue that triggers are just echo/identity actions. Let's debate next why we need triggers?

[domdom82]

speaking pub/sub lingo a rule would be a named subscription to an event that got published via a topic we call trigger. action would be the consumer. there is some benefit to keeping a handle to the subscription. i.e. you can disable it temporarily without having to remove and add it again. afaik the topic does not know who has subscribed to it per se. it just publishes stuff (read: fires).

[rabbah]

@domdom82 "rule would be a named subscription" is at odds with @mbehrendt comment which suggests we should admit anonymous rules. The lack of specificity and without being very concrete and careful about terminology, this discussion will meander and lack focus.

The intent of this issues and the question posed by @markusthoemmes is borne out of the experience reimplementing the activator. It is more specific to the implementation of rules rather than an existential question of whether the whisk programming model should include rules. He should of course correct me if I'm wrong.

[sjfink]

I think this issue has degenerated into multiple independent threads being crossed:

1) How do we implement rule entities in the backend.
2) Do we need a Rules API?
3) Do we want to support anonymous entities as an option?
4) Do we want Rules to be anonymous always, with no possibility to name them?

I'd suggest we narrow the scope of this issue to topic 1) Implementation concerns.

If anyone wants to argue for API changes as per 2), 3), 4) -- let's break those out as separate discussion items.

[mbehrendt]

had a call with @sjfink on this -- summarizing here (steve, pls feel free to add / comment):

• we think anonymous rules make sense and we should support them
• we don't have a good real-world use case for named rules. however, will continue to support that for backwards compatibility and in case the need for them becomes higher-prio again.
• based on the previous bullet, the documented main stream flow for hooking up triggers and actions will not require to issue wsk rule create commands. Instead, we'll introduce a new command wsk trigger associate <triggerName> <actionName> and a corresponding disassociate command (can refine the verbs).
• we'll introduce the change above first on the cli and ui. Based on user adoption, we can gradually also change the api to support these verbs natively.

[rabbah]

In https://github.com/openwhisk/openwhisk/issues/833#issuecomment-230635305 @sjfink suggested splitting this issue into others (implementation vs named/anonymous ...) so I am retitling this issue according to the focus implied by comment https://github.com/openwhisk/openwhisk/issues/833#issuecomment-232882019.

[rabbah]

So if I want to create a cloudant feed T to invoke an action A and use that rule in two different scenarios (read apps) S1 and S2, and at some point I want to disable (pause) the rule for S1 but not S2, I do that how if the rule is anonymous?

It is a mistake to make the rule anonymous always and state that we support it named only for legacy (backward compatibility) reasons. It means you are making all rules unique. To get back the functionality of having named ruled you will create more than one trigger for the scenario I describe which means two feeds instead - so you traded two rules for two feeds. A feed is more heavyweight (to manage) than a rule. Alternatively you trade two rules for two actions A_S1 and A_S2 which now means no shared code and updating the same action code twice when there is a need to change A, and still have to create the rule for T to A_S1 and A_S2.

Now imagine more than 2 apps that want to consume the same feed but where you want to allow individual control over the rule.

This isn't a hypothetical scenario either. Dark vision uses one cloudant feed and two rules, one for Watson vision analysis and the other for thumbnail generation.

Further, what's the advantage of
wsk trigger associate triggerName actionName vs
wsk rule create triggerName actionName which we can add to create an anonymous rule and
wsk rule create ruleName triggerName actionName where the rule is created with a name.

And what about enable and disable for the rule? You suggest these become trigger associate and disassociate which is more like rule create and delete so now enable/disable which afford the ability to pause/resume a rule without having to recreate it become create/delete.

So the totality of augmenting the CLI commands for triggers and the API for triggerrs is to bloat and conflate concerns for what is otherwise a simple API for orthogonal concepts.

[rabbah]

Summary of subsequent discussion with @mbehrendt on this topic:

1. anonymous rule does not necessarily mean unnamed - the system may generate a name for the rule; by making the rule implicitly named, the trigger-action mapping should be unique (unless explicitly named).
2. rather than introduce two ways of creating rules (one via wsk rule create and another via wsk trigger associate) there will be only one way to create a rule (named or unnamed) and it will be via wsk rule create (with optional rule name); this avoids introducing new terminology and non-orthogonal mechanisms for doing the same thing.
3. to save a step for the user, creating a rule will enable it by default (similarly deleting a rule [that is not named] should disable it by default).
4. we may be able to implement these implicitly named rules via the CLI (or UI) - entirely on the client initially with light additive changes to the rules API (rules list should provide trigger/action details and possibly allow filtering by trigger name).
5. we will measure the popularity of unnamed vs named rules and learn from the data.
6. (somewhat related), rules should allow for parameters that may be merged with trigger payloads before passing to action invocations.