Syndesis: [UX Design] Create UX design for "Template" step

[dongniwang]

This is a...

[x] Feature request
[ ] Regression (a behavior that used to work and stopped working in a new release)
[ ] Bug report  
[ ] Documentation issue or request

The problem

See user story #3047

Expected behavior

Screenshot

Tasks involved

• [x] receive UX request / identify priority
• [x] verify user story and requirements [user story]
• [x] initial design
• [ ] design review
• [ ] design iteration
• [ ] design sign-off
• [ ] plan of action for implementation

[dongniwang]

@phantomjinx @gaughan - Here's the UX design for Template Step: https://redhat.invisionapp.com/share/GHNTV6PJK32.

cc: @amysueg @sjcox-rh @TovaCohen @gashcrumb

[phantomjinx]

Observations so far (established from test driven development on the backend. May, of course change as development moves forward):

• All 3 types of template have camel developed endpoints (so good for us!). They also, provide the ability to set the template content directly as a value in the camel headers. This avoids the need to construct server-based temporary files in order to server up file-based resource URIs. Therefore, it would be expected that the template would be transmitted from the UI to the server as a string-based value. This means that upload of template content would be handled on the UI-side regardless of whether it comes from file, URL or copy 'n paste;
• _How many Templates can users add?_ It is just one since the template is provided as a single resource URI. For example, see here;
• _Do users need to specify template type?_ At this early stage, my initial view is that yes they will. Mustache uses a syntax of "{{ ... }}" while Velocity and FreeMarker can basically both use "${ ... }" (note. Velocity can also use "$..."). Therefore, although it is possible to discern the template is a mustache syntax, it will be far more difficult or impossible to distinguish Velocity from FreeMarker;
• Validation may be conducted in the UI via the editor displayed. For example, codemirror has the ability to include 'modes'/parsers for all 3 languages: see here, here and here;

[dongniwang]

@phantomjinx - thanks so much for the comments! I'll make the updates based on the latest feedback.

[TovaCohen]

Page 2 - Choose a Step
For the description of the Template step, how's this:
Upload a FreeMarker, Mustache, or Velocity template to define a custom step.

After I add a Template step to an integration, suppose that I click "Add a Step". Am I correct in thinking that the "Template" step would not appear in the list of steps that I can add since I already added a Template step?

Page 3 - Upload Template, with "Upload a file" selected. The description under the title now says "Fill out the fields associated with the selected step." I would replace that with this: "An integration can have one custom step that you define by uploading a template."

Page 4 - with "Use a URL" selected.
The help text needs a tweak, to this:
"After uploading this template, Fuse Online does not automatically obtain any updates to it. You would have to upload an updated template that incorporates revisions."

That makes me wonder what I have to do when I want to update a template step. Would I have to do this:

1. Edit the integration that uses a template step to remove the template step.
2. Add a new template step to that integration.
3. Upload a new template for the template step.

Page 5 - Edit Template
This version of "Edit Template" appears after I upload a template file or after I enter a URL.
Therefore, I think the description under the title needs to be removed and not replaced. The text just above the input box makes it clear that you can edit the template.

Page 6 - Upload Template
Select "Copy and paste".
For the help text, I suggest:
"Copy and paste a template or enter a template manually."

Page 7 - Edit Template
This version appears after I select "Copy and paste".
I would remove the description under the title. Then just above the input field, I suggest using the same text as on page 6, that is:
"Copy and paste a template or enter a template manually."

Is validating the template optional?
What happens if I enter an invalid template?

Will I be able to upload an icon to represent my template step?

[dongniwang]

I'm working on updating the UX design, so the layout of the pages may change.

Page 2 - Choose a Step
For the description of the Template step, how's this:
Upload a FreeMarker, Mustache, or Velocity template to define a custom step.

I think this is correct, although I thought we're only supporting Mustanche for 7.2? Please confirm @gaughan @phantomjinx

After I add a Template step to an integration, suppose that I click "Add a Step". Am I correct in thinking that the "Template" step would not appear in the list of steps that I can add since I already added a Template step?

I think technically, user can add as many template steps as possible, but once a new template step is added, user also need to add a data mapping step before or after that new template step.

That makes me wonder what I have to do when I want to update a template step. Would I have to do this:
Edit the integration that uses a template step to remove the template step.
Add a new template step to that integration.
Upload a new template for the template step.

This is a question for both @gaughan and @phantomjinx - how would user edit a template step? It is always remove then upload a new one, OR we would be able to load the existing one in the simple text editor for user to edit?

Will I be able to upload an icon to represent my template step?

Since Template is one of the out-of-box steps, it will be represented as a generic step which uses the step icon we currently have for filter/data mapping/log.

[gaughan]

Yes, Mustache only for 7.2
The template could be used multiple times in a row but it is a bit of edge scenario I think.
Edit inline would be ideal.
Agree on standard icon.

[dongniwang]

Latest version of UX design is here: https://redhat.invisionapp.com/share/MKNX2N6SVE6. Please review and provide feedback!

@gaughan @phantomjinx @gashcrumb @TovaCohen

cc: @amysueg @sjcox-rh @michael-coker @seanforyou23

[gaughan]

LGTM, I added a question on the ability to detect the template language @phantomjinx

[phantomjinx]

@gaughan
Asked and answered although it's probably a more "time-will-tell" answer :-)

[gaughan]

:) ok!
In any case, for now we don't need it (the select template format section) until we introduce other template formats.

[phantomjinx]

@dongniwang
Would it be too much trouble to keep this version as the "long term where we want to be version" and then create a version 3a (or something) that is the "cut-down mustache-only 7.2 for-now" version, please?

[dongniwang]

@phantomjinx - Sure! I can definitely do that. I'll make the updates and put both v3 and v3a (7.2 only) in the UX design tracker.

[TovaCohen]

LGTMT. I left a comment to suggest that "Copy and paste" become "Create" in both the UI for the Template step and the UI for the API provider. And its hover help tip would be: "Copy and paste a template or enter text that defines a template."

One question - if a user adds more than one Template step to an integration, and each Template step uses a different template, is there anything in the UI that would help the user distinguish the template steps from one another? Maybe a user would never use two or more different templates in an integration so it is a non-issue. I don't know.

[gaughan]

+1 on the copy and paste.
Order in the IVP would distinguish I think.

[dongniwang]

Just uploaded v3.1 here: https://redhat.invisionapp.com/share/UYNXZ30MQ34. Both v3 and v3.1 can be found via the UX design tracker as well.

For 3.1 -

• Changed "Copy/paste" to "Create"
• Removed the choose template type section.

@TovaCohen - I think one way to help users understand the different template steps is by expanding the IVP to see the step numbers.

cc: @gaughan @phantomjinx

[dongniwang]

And I'll update the API Provider design to reflect this.

I left a comment to suggest that "Copy and paste" become "Create" in both the UI for the Template step and the UI for the API provider.

[phantomjinx]

So, implementing the UI, I have hit a problem, which may or may not be easily solved. Angular has checks in its API in place to protect against data being fetched from external urls.

The 'Upload Template' page allows choosing between 'file-upload', 'url-download' or 'create'. The end result of all of these is the template string content of the target. For 'url-download', this would normally be implemented using an http.get(). I have coded this by adding a method to the IntegrationSupportService:

downloadExternalData(url: string): Observable<any> {
    return this.apiHttpService
      .get<string>(url, { responseType: 'text'})
      .map(res => {
        console.log("Downloaded: " + res);
        return {
          url: url,
          data: res
        };
      });
  }

Now this code is barely a skeleton as testing it I get the following problems:

• Using an http address:

zone.js:2969 Mixed Content: The page at 'https://syndesis.birds-of-prey.phantomjinx.org.uk/integrations/i-LLn8FqIaeoZRjKB1H5Rz/-LLn7t2Hfwdnw4niZ4zy/edit/step-configure/1' was loaded over HTTPS, but requested an insecure XMLHttpRequest endpoint 'http://phantomjinx.co.uk/stuff/message.tmpl'. This request has been blocked; the content must be served over HTTPS.

• Using an https address:

Failed to load https://phantomjinx.co.uk/stuff/message.tmpl: Response to preflight request doesn't pass access control check: No 'Access-Control-Allow-Origin' header is present on the requested resource. Origin 'https://syndesis.birds-of-prey.phantomjinx.org.uk' is therefore not allowed access.

Now, whether both of these errors could be fixed or worked around, I wonder if it is worth it. Therefore my questions to consider are:

1. Is there a really simple easy fix to avoid these errors, ie. some clever javascript function/library?
2. If there is no easy fix, then is the 'url-download' option really necessary given it should be simple for the user to open the requisite url in a browser and copy n paste the content into the 'create' option?

@gashcrumb @dongniwang @gaughan
_* Please pull in anyone else they may be able to help._

[gashcrumb]

Yeah, this is a general problem with fetching a file via URL, it can't really be well implemented in javascript thanks to CORS restrictions.

I don't think we have a general purpose endpoint in the backend to fetch files from URLs for the frontend, but there's been talk of one previously as we've already run into this problem previously with the API connector upload stuff.

[gashcrumb]

@lburgazzoli @nicolaferraro ^^^^

[nicolaferraro]

We don't have a general purpose URL fetching service. We implemented it for Open API specifications (API consumer and provider): https://github.com/nicolaferraro/syndesis/blob/9a1b8e543a3eb7fddec9e876fede8c87ae0cf860/app/server/api-generator/src/main/java/io/syndesis/server/api/generator/swagger/util/SwaggerHelper.java#L222.

[phantomjinx]

So @nicolaferraro is suggesting that the template url is transmitted server-side then the content fetched using java, which is allowed. In fact, if it is done this way then it will be unnecessary to fetch the content since the url could be directly passed to the mustache-camel module and it would handle the fetching of the template directly.
The downside to this approach is there would be no preview of the content in the "Upload Template" page when URL is selected (see page 4).
Therefore, couple of choices:

• Remove URL upload altogether;
• Keep URL but remove 'Upload' button and 'Import Review' component, thereby transmitting the URL servers-side and letting the server components take care of getting the template;
• Create an endpoint to download content from a 3rd party external server, letting the server fetch the content and transmitting back to the client when the 'Upload' button is clicked.

My personal preference is option 2 since it keeps the URL option. Option 3 appears overkill and probably be a security hole.

WDYT?

@gaughan @dongniwang

[dongniwang]

Hey @phantomjinx - Just discussed the issue with @syndesisio/uxd and we're fine with dropping the "upload via URL" option altogether.

I'm assuming is very common to upload an API definition via URL (in the case of API Client Connector flow) but is less common to do so for uploading a template file? @gashcrumb @gaughan @nicolaferraro

[gaughan]

I'm good with both. I prefer option 2 (direct to server) if it doesn't add huge effort @phantomjinx ?

[phantomjinx]

@gaughan

I have had to remove the URL option since I need the content to display/validate before submitting to server. I will do a video of the current manifestation to indicate progress and see whether you think sufficient.

@dongniwang

[gaughan]

Ok, sounds good.