Eventing: Revert Importer naming decision.

[n3wscott]

Problem

Importer is an imperative name being used to describe a declarative concept of configuring a system to deliver events to an endpoint.

Solution

Revert sources-to-importers.md.

Our usage of source is consistent with how CloudEvents describes source.

The confusion importer was attempting to clear up is how to talk about the active (and optional) in-cluster component, called a receive adapter also called adapters by CloudEvents.

Background

A source as a concept is the installing of the declarative api in the cluster to request a target url or resource to be invoked by events from said source. The resource in kubernetes represents the particular configuration and filters required to allow this to happen.

Valid implementations of sources include:

• 1:1 source resource to active in-cluster data-plane component. IE: a deployment per resource.
• a single active in-cluster data plane component per source resource. IE: a multi-tenant implementation.
• No active in-cluster data-plane components. The source invokes the target directly, perhaps as an optimization that is allowed because of the overall cloud configuration.
• A proxy. IE: a trampline implementation.
• _Anything else_ as long as config results in a target receiving CloudEvents.

Additionally, a source controller may choose to migrate it's realization the resource config to various solutions depending on situation and scope.

[grantr]

Previous discussion at https://github.com/knative/eventing-contrib/issues/402

[deissnerk]

A source as a concept is the installing of the declarative api in the cluster to request a target url or resource to be invoked by events from said source.

In many cases this is not true, because the source actually refers to some intermediary like PubSub, Kafka or RabbitMQ. If the name wasn't taken, imho _ingress_ would be the perfect term.

There are also real sources in kn-eventing:
Any function or app inside the namespace/cluster that emits events, can be regarded as _source_. What if in the future we decide to somehow describe these sources with an API? Won't there be even more confusion?

[mattmoor]

I too think that the term Ingress (and frankly Egress) pair conceptually very well with events entering and leaving the system (it is fundamentally what those terms mean), and have a duality with Subscriptions and Topics in APIs like Pub/Sub.

That said, of the choices give, I think Source (and it's dual Sink) are my favorite.

[n3wscott]

In many cases this is not true, because the source actually refers to some intermediary like PubSub, Kafka or RabbitMQ. If the name wasn't taken, imho ingress would be the perfect term.

In the case of Knative with GCP, the _source_ is something like Google Cloud Storage, but it uses a Pub/Sub Topic and Subscription to bring the event into the cluster and convert it. This is the choice we use for general k8s clusters, but we could optimize this for GKE and skip the receive adapter if we had enough control over the ingress of the cluster. This is my point about picking a declarative name for the concept.

PubSub, Kafka and RabbitMQ are all transports, and it would make more sense for the user if they wanted events from MyApp from out of cluster, a controller knows those events are delivered (or setup to deliver) over Kafka for a given key. But if this is all declarative, we can choose another impl for the user if we have enough control over the environment...

[duglin]

I tend to think of "sources" as utility functions that do a couple of things:
1 - connect me up to some event stream/producer/whatever - it's like a subscribe()
2 - send those events to someplace - a "sink"
3 - if necessary it might do some adaptation of the event - e.g. converting it into a CloudEvent

There might be other things that go on, but from an end-user perspective I think they mostly just care that this "thingy" will connect them up to get new events (and manage that connection). The rest basically becomes impl details, or noise.

So, to me "Connection" (while long) isn't a bad term for it. "Source" is ok, but the concern I've had with that word is that it could imply to some people that it will "create the event producer", and while possible (e.g. in the cron case), it's usually just connecting up to an existing one.

"You need to create github connection to receive github events" - seems almost self-explanatory to me. Of course s/connection/subscription/ would too :-)

[matzew]

Honestly, we (our team) try to call them importer, but we end-up always talking "source"... I do appreciate this revert, to be very honest.

Fortunately none of the CRDs have been renamed, and yeah.... let's never talk about it again, like @n3wscott said!!!! :rofl:

[aslom]

@matzew I noticed the same thing - people call it source and rarely importer.

[aslom]

Just for reference here is CLoudEvents souce definition https://github.com/cloudevents/spec/blob/v0.3/spec.md#source

The "source" is the context in which the occurrence happened. 
In a distributed system it might consist of multiple Producers. 
If a source is not aware of CloudEvents, an external producer 
creates the CloudEvent on behalf of the source.

[bbrowning]

Obviously the name Importer never stuck for whatever reason. It's telling that we didn't update all the code, docs, etc to refer to Sources as Importers. So, I'm +1 for sticking with Source as that's the most widely used term for this in our community at this point.

[lionelvillard]

+1 for Source because

that's the most widely used term for this in our community at this point.

[sixolet]

"Source" meaning "the thing that produces events and eventually gets them to a sink" is great. That's what's meant by cloud events in the source field in the first place

"Source" as a piece of software like GithubEventSource is really confusing, because now you're confused between the source field in the event itself, github, the github importer, the type of the github importer...

The goal of sources to importers was to differentiate. We should not revert it, but rather achieve clarity that importers (or adapters, I don't care, either is a fine name) are a particular kind of software that achieves the goal of making events from a source (precise use of the word "source") available on the cluster.

Responses:

Importer is an imperative name being used to describe a declarative concept of configuring a system to deliver events to an endpoint.

It should be instead word used to describe a particular way of getting those events on the cluster.

Our usage of source is consistent with how CloudEvents describes source.

Previously, it was confusing because we were using "source" to describe both software to import events as well as where the events came from. Using "importer" for the software to import events frees us to use "source" consistently with CloudEvents.

A source as a concept is the installing of the declarative api in the cluster to request a target url or resource to be invoked by events from said source.

A source as a concept is a system that can produce events with a consistent source field. No more, no less.

The resource in kubernetes represents the particular configuration and filters required to allow this to happen.

An instance of a GetYerEventsFromGithub CRD is not Github. It's a piece of software that is getting events from Github. It's not the source, it's a conduit for events from the source. It might be reasonable to call the CRD itself (giving the user the power to get events from github) a "source" in some loose sense, because of what it enables the user to do (create instances of the CRD, thereby getting events from github). It's somewhat more sensible to call the CRD itself a "source" and an instance of it an "importer".

Valid implementations of sources include: ...

Some of these are importers and some of these are not. The whole point of making the name change was so that we had the ability to talk about importers, as a class, without implying we were talking about sources, as a class, and be more precise. If we call all of these "sources", we have no way of talking about "github" vs. "the software that implements the way of getting github events on the cluster" vs. "an instance of the software that implements the way of getting github events on the cluster".

Admittedly, differentiating that last two was weak and still is. We should fix that. Possibly by calling the first two "source", where we were previously calling the last two "source"? Anyway, it's pretty important to be able to talk about "an importer instance" somehow.

[nachocano]

+1 for Source because

• Probably nobody will change the existing code (nor documentation!) to use Importer. I think there are much more important things to do in the project.
• Talking about Importer as a concept but not backing that up in code, IMO will be confusing for new developers as they will only see Source in the code.
• Leaving existing code using Source and only new code start using Importer will also be confusing. IMO it's either all Source or all Importers, preferably the former.
• As mentioned in previous comments, Source it's the widely used term in the community at this point.

[alanconway]

Good move. Source may not be perfect, but in the absence of a perfect
replacement it's Good Enough. The rename is too much work with no really
significant payoff.

On Wed, Sep 11, 2019 at 12:35 PM Ignacio Cano [e-mail]
wrote:

>

• 1 for Source because

• Probably nobody will change the existing code (nor documentation!)
  to use Importer. I think there are much more important things to do in the
  project.

• Talking about Importer as a concept but not backing that up in code,
  IMO will be confusing for new developers as they will only see Source in
  the code.
• Leaving existing code using Source and only new code start using
  Importer will also be confusing. IMO it's either all Source or all
  Importers, preferably the former.
• As mentioned in previous comments, Source it's the widely used term
  in the community at this point.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/knative/eventing/issues/1857?email_source=notifications&email_token=AB3LUXSZF7PVRUQLDKT57GTQJEM53A5CNFSM4IVLGNKKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD6PDHOA#issuecomment-530461624,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AB3LUXVCSLJIL577BF5FLNTQJEM53ANCNFSM4IVLGNKA
.

[deissnerk]

@n3wscott

PubSub, Kafka and RabbitMQ are all transports, and it would make more sense for the user if they wanted events from MyApp from out of cluster, a controller knows those events are delivered (or setup to deliver) over Kafka for a given key. But if this is all declarative, we can choose another impl for the user if we have enough control over the environment...

If you want to hide the transport aspect, just get rid of _source/importer_ and use _trigger_. Some clever controllers can watch _triggers_ and decide when and how to create connections to event sources. I thought _source/importer_ is actually about all the transport stuff, so that _trigger_ can be just about events and (CloudEvent) sources.

[akashrv]

A while back when we were discussing things on paper (in issues and docs) it seemed obvious that importer is the technically correct name as it does the work on importing events into the cluster (knative eventing developer perspective).
Primary reason:

• However, the more I use the eventing system to build apps and work with customers, I think "source" makes more sense because the control plane object that I am creating is the representation of the source. Example: If I want to consume Cloud storage events, I will look into registry for CloudStorageSource, If I want to consume events from pubsub topic, I'll look into registry for PubSubSource etc.

Secondary points:

• Most of the code already refers to these are "source" and ROI of changing all of it to importer is not worth.
  
  
  
  • AWS EventSource in EventBridge. Please don't take this as "following AWS" or that "what AWS did is correct". But it is a fact (that cannot be ignored) that AWS customers using a GA product will be already trained to use the word "EventSource". There is no need to re-train them.
  
  

I understand that after reading cloud events spec with differentiation between source, producer etc it made total sense to not use "source" to be technically correct. But from a customer's (end user's) perspective, "source" makes more sense.

Big-NO to adapter or connectors, as they are too generic and are more difficult to understand than Source or Importer, purely from an end-user's perspective. Think this way: If I want to consume events from Cloud Storage, what registry do I search for? "Source registry" or "connector/adapter registry"? What is intuitive?

[deissnerk]

@akashrv I suppose for AWS EventBridge you could create a knative importer/source. If the name change was reverted back, you would then have an EventBridge source. From what I saw so far in their documentation, they only list actual event sources: services and applications that actually emit events. Regarding the GCP PubSub event source you mentioned, a topic is not a source. It is just the destination, the actual source or some intermediary sent the event to.

I think the term _source_ got introduced in knative, when it was restricted to messaging with channels. There it actually made sense, because it was the source of the messaging pipeline. If you now look at the event mesh with broker and trigger, it seems misplaced. Perhaps the importer that fetches events and pumps them into a broker, is just not the same thing as the source that is combined with a channel. There have already been ideas of propagating filters from triggers upstream. The importer for the event mesh should only fetch those events that are requested by some trigger.

[mattmoor]

Has any sort of user study been done? cc @steren 👀 🍿

_ducks_

[steren]

We can probably use user research to find a suitable name (like we did previously).

let me loop @tliberman

[vaikas]

Thanks all for the useful and thoughtful thoughts. There's really no consensus and nor will there be. We're currently in a little bit of a mixed worlds and hence at this point based on the actual feedback and experience with the difficulty of adopting the importer, I'm going to say that we'll just use Source until such a time that we can do a proper UX study and then re-visit this. I know some folks will be disappointed but we have larger issues to tackle at the moment, and I really don't want to spend that time churning through the code base. So, we're going back to referring to Sources and if @steren (and whoever else) can run a UX study for a more suitable name, then we can revisit this with actual data. Right now all empirical evidence says that importer is not meaningful and while Source is not perfect, we can live with it.
Decision: use Source over Importer.

[akashrv]

I would stick with WG Leads decision (@vaikas-google ) and move forward. Let's focus on the bigger problems we have (beta apis, more robust constructs that can support production workloads reliably, more documentation, the list goes on...) and re-visit this when @steren comes back with results from UX study.
If anyone feels otherwise, then please re-open the issue.
/close

[knative-prow-robot]

@akashrv: Closing this issue.

In response to this:

I would stick with WG Leads decision (Ville) and move forward. Let's focus on the bigger problems we have (beta apis, more robust constructs that can support production workloads reliably, more documentation, the list goes on...) and re-visit this when @steren comes back with results from UX study.
/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.