Syndesis: gmail connection is limited by expiration of oauth token
Creating a gmail connection requires adding an OAuth Client ID to the gmail API and registering syndesis accordingly (documented in the user guide). Publishing an integration involving gmail works as expected. However, subsequent republishings of the same integration or using the connection for other integrations frequently results in the published integration failing due to '401 unauthorized errors'.
The workaround is to remove the OAuth Client ID in the gmail API, recreate it and re-register the integration. The gmail documentation actually states:
This access token is passed to the Gmail API to grant your application access to user data for a limited time.
Is it possible that the token is quickly expiring and requires refreshing? If so, Is it possible to refresh the token automatically or lengthen the expiry time?
phantomjinx
All 43 comments
The token default expiration is one hour. I don't know if we can do something at syndesis level, probably we can use a refresh token mechanism in the camel component. Maybe @zregvart know more about the oauth mechanism in Syndesis.
oscerd
on 18 Sep 2018
@oscerd
I wonder if there is scope for creating a generic email connector that sends email using ssmtp. I use this very successfully in an irssi script to notfiy me, using email, of any irc messages mentioning me. It's a command line program so java would have to system.exec() but since we can control the construction of the container (using the assemble script) it should be doable. WDYT?
@heiko-braun @paoloantinori
phantomjinx
on 18 Sep 2018
I think is overkilling.
oscerd
on 18 Sep 2018
Because doing something like that requires a new camel component.
oscerd
on 18 Sep 2018
Hi @phantomjinx using ssmtp would make it not portable.
Currently Syndesis has direct dependencies only on the JVM and any lib that works on it that doesn't require specific service or binary available on the host AND OpenShift.
Other than that, Camel upstream has a quite large list of email components already, some of them might fix the issue described here already.
Having said so, Camel community loves contribution, and, outside the scope of Syndesis and this specific feature, that community might be interested in a ssmtp based component, albeit with the *nix platform limitation.
paoloantinori
on 18 Sep 2018
We currently don't have the functionality to refresh the token. Currently to tackle this we have two approaches.
The user can Reconnect from connection details:
Which kinda sucks, the user has to know when to reconnect and do it often, but it's a thing.
In the API custom client connectors we use the _refresh token_ received from the OAuth flow to get a new _access token_ when it is about to expire or on (some) HTTP errors:
So we could add the same functionality to the GMail connector.
We could do token refresh in the OAuth support in Syndesis, there is a cost to this, we would also have to push the newly refreshed _access token_ to the running integrations, and here we might have a bit of an effort. Some prior work was done in spring-cloud-kuberentes on refreshing Spring configuration on ConfigMap change. I kinda feel if we do it in the connector it's easier to develop and debug.
If this happens regularly after one hour this might be a bug we need to fix for the next release.
zregvart
on 18 Sep 2018
@oscerd @paoloantinori
Definitely overkill as I have now found the camel-mail component. Certainly far more portable. Any possibility I could explore conjuring up a connector for this at some point?
phantomjinx
on 18 Sep 2018
sure. pm, is in general open to have new connectors. they have their own most wanted list, but we have freedom to contribute those that you like.
paoloantinori
on 18 Sep 2018
So it looks like we aren't getting a refresh token in the original response, we need to specify a parameter "access_type=offline" and we should get the refresh token back. This should automatically work out of the box. I created an accessToken with this parameter through http request and I'm using it in a Camel test from more than an hour, so it should work. @zregvart can you point me at where we define custom fields for oauth?
oscerd
on 18 Sep 2018
@oscerd awesome :smile:
So we need custom query parameters added to the authorization URL?
Easy-peasy: what we basically need is to add a key-value to org.springframework.social.oauth2.OAuth2Parameters created here:
(something like parameters.putAll(additionalParameters);)
The additionalParameters map can be added to OAuth2CredentialProvider and OAuth2ConnectorProperties via tagged property of properties of Connector, we keep known tags as constants in Credentials class.
So something like:
diff --git a/app/server/credential/src/main/java/io/syndesis/server/credential/Credentials.java b/app/server/credential/src/main/java/io/syndesis/server/credential/Credentials.java
index 60f8a23..8ba9509 100644
--- a/app/server/credential/src/main/java/io/syndesis/server/credential/Credentials.java
+++ b/app/server/credential/src/main/java/io/syndesis/server/credential/Credentials.java
@@ -50,6 +50,8 @@
public static final String TOKEN_STRATEGY_TAG = "oauth-token-strategy";
+ public static final String ADDITIONAL_QUERY_PARAMETERS_TAG = "oauth-additional-query-parameters";
+
private final CredentialProviderLocator credentialProviderLocator;
private final DataManager dataManager;
diff --git a/app/server/credential/src/main/java/io/syndesis/server/credential/OAuth2ConnectorProperties.java b/app/server/credential/src/main/java/io/syndesis/server/credential/OAuth2ConnectorProperties.java
index 330b457..fc13a4c 100644
--- a/app/server/credential/src/main/java/io/syndesis/server/credential/OAuth2ConnectorProperties.java
+++ b/app/server/credential/src/main/java/io/syndesis/server/credential/OAuth2ConnectorProperties.java
@@ -15,11 +15,22 @@
*/
package io.syndesis.server.credential;
+import java.io.IOException;
+import java.util.Collections;
+import java.util.Map;
+
+import com.fasterxml.jackson.core.type.TypeReference;
+
import io.syndesis.common.model.connection.Connector;
+import io.syndesis.common.util.Json;
import org.springframework.social.oauth2.TokenStrategy;
public class OAuth2ConnectorProperties extends ConnectorSettings {
+
+ private static final TypeReference<Map<String, String>> MAP_TYPE = new TypeReference<Map<String,String>>() {
+ // type token pattern
+ };
private final String accessTokenUrl;
@@ -33,6 +44,8 @@
private final boolean useParameters;
+ private final Map<String, String> additionalQueryParameters;
+
public OAuth2ConnectorProperties(final Connector connector) {
super(connector);
@@ -43,6 +56,10 @@
.orElse(TokenStrategy.AUTHORIZATION_HEADER);
useParameters = optionalProperty(connector, Credentials.AUTHORIZE_USING_PARAMETERS_TAG).map(Boolean::valueOf).orElse(false);
scope = optionalProperty(connector, Credentials.SCOPE_TAG).orElse(null);
+
+ additionalQueryParameters = optionalProperty(connector, Credentials.ADDITIONAL_QUERY_PARAMETERS_TAG)
+ .map(params -> readJsonMap(params))
+ .orElse(Collections.emptyMap());
}
public String getAccessTokenUrl() {
@@ -69,4 +86,16 @@
return useParameters;
}
+ public Map<String, String> getAdditionalQueryParameters() {
+ return additionalQueryParameters;
+ }
+
+ private static Map<String, String> readJsonMap(String params) {
+ try {
+ return Json.reader().forType(MAP_TYPE).readValue(params);
+ } catch (IOException e) {
+ throw new ExceptionInInitializerError(e);
+ }
+ }
+
}
diff --git a/app/server/credential/src/main/java/io/syndesis/server/credential/OAuth2CredentialProvider.java b/app/server/credential/src/main/java/io/syndesis/server/credential/OAuth2CredentialProvider.java
index 363f4a4..549ce2d 100644
--- a/app/server/credential/src/main/java/io/syndesis/server/credential/OAuth2CredentialProvider.java
+++ b/app/server/credential/src/main/java/io/syndesis/server/credential/OAuth2CredentialProvider.java
@@ -16,6 +16,11 @@
package io.syndesis.server.credential;
import java.net.URI;
+import java.util.Collections;
+import java.util.List;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.stream.Collectors;
import io.syndesis.common.model.connection.Connection;
@@ -32,17 +37,19 @@
private final String id;
- public OAuth2CredentialProvider(final String id, final OAuth2ConnectionFactory<S> connectionFactory,
- final Applicator<AccessGrant> applicator) {
+ private final Map<String, List<String>> additionalQueryParameters;
+
+ public OAuth2CredentialProvider(final String id, final OAuth2ConnectionFactory<S> connectionFactory, final Applicator<AccessGrant> applicator,
+ final Map<String, String> additionalQueryParameters) {
this.id = id;
this.connectionFactory = connectionFactory;
this.applicator = applicator;
+ this.additionalQueryParameters = additionalQueryParameters.entrySet().stream().collect(Collectors.toMap(Entry::getKey, e -> Collections.singletonList(e.getValue())));
}
@Override
public AcquisitionMethod acquisitionMethod() {
- return new AcquisitionMethod.Builder().label(labelFor(id)).icon(iconFor(id)).type(Type.OAUTH2)
- .description(descriptionFor(id)).build();
+ return new AcquisitionMethod.Builder().label(labelFor(id)).icon(iconFor(id)).type(Type.OAUTH2).description(descriptionFor(id)).build();
}
@Override
@@ -56,8 +63,7 @@
public CredentialFlowState finish(final CredentialFlowState givenFlowState, final URI baseUrl) {
final OAuth2CredentialFlowState flowState = flowState(givenFlowState);
- final AccessGrant accessGrant = connectionFactory.getOAuthOperations().exchangeForAccess(flowState.getCode(),
- callbackUrlFor(baseUrl, EMPTY), null);
+ final AccessGrant accessGrant = connectionFactory.getOAuthOperations().exchangeForAccess(flowState.getCode(), callbackUrlFor(baseUrl, EMPTY), null);
return new OAuth2CredentialFlowState.Builder().createFrom(flowState).accessGrant(accessGrant).build();
}
@@ -69,10 +75,10 @@
@Override
public CredentialFlowState prepare(final String connectorId, final URI baseUrl, final URI returnUrl) {
- final OAuth2CredentialFlowState.Builder flowState = new OAuth2CredentialFlowState.Builder().returnUrl(returnUrl)
- .providerId(id);
+ final OAuth2CredentialFlowState.Builder flowState = new OAuth2CredentialFlowState.Builder().returnUrl(returnUrl).providerId(id);
final OAuth2Parameters parameters = new OAuth2Parameters();
+ parameters.putAll(additionalQueryParameters);
final String callbackUrl = callbackUrlFor(baseUrl, EMPTY);
parameters.setRedirectUri(callbackUrl);
diff --git a/app/server/credential/src/main/java/io/syndesis/server/credential/generic/OAuth2CredentialProviderFactory.java b/app/server/credential/src/main/java/io/syndesis/server/credential/generic/OAuth2CredentialProviderFactory.java
index 209411c..5dc9a5d 100644
--- a/app/server/credential/src/main/java/io/syndesis/server/credential/generic/OAuth2CredentialProviderFactory.java
+++ b/app/server/credential/src/main/java/io/syndesis/server/credential/generic/OAuth2CredentialProviderFactory.java
@@ -57,7 +57,7 @@
applicator.setClientIdProperty("clientId");
applicator.setClientSecretProperty("clientSecret");
- return new OAuth2CredentialProvider<>("oauth2", connectionFactory, applicator);
+ return new OAuth2CredentialProvider<>("oauth2", connectionFactory, applicator, oauth2Properties.getAdditionalQueryParameters());
}
@Override
The ugly bit is that now we have another JSON in JSON...
zregvart
on 18 Sep 2018
If you want to take a peek it's ok for me. The parameter is just access_type this seems to be the trick. Let me know
oscerd
on 18 Sep 2018
@oscerd I'll put this on my queue, if you don't robot your way to fix it before I think I can find the time towards the end of the week.
zregvart
on 18 Sep 2018
No problem. I have a lot work on connectors now. So take your time. If I finish before I'll help you on this.
oscerd
on 18 Sep 2018
I've pushed the changes from the diff above to https://github.com/zregvart/syndesis/tree/issue/3608, this was just a quick and dirty stab at it, probably doesn't work at all (no tests), might not compile at all.
Just for @oscerd if it makes sense for you to base on that, I think it needs more work and I'll probably squash/force push later in the week more changes on top of that...
zregvart
on 19 Sep 2018
I'll check it If I'll find some time in the week, thanks for the moment :-)
oscerd
on 19 Sep 2018
By the way I'm still wondering how Test and QE team could have miss something like this issue.
oscerd
on 19 Sep 2018
It would take a really long time for them to test long running issues like this, and I presume they would need some input from us on what credentials can expire (on what connectors/actions) and the timeout of each.
It's really fortunate that @phantomjinx found this.
cc @syndesisio/qe
zregvart
on 19 Sep 2018
Basically what @zregvart said. We should focus more on adding components into camel and test them there as here in syndesis-qe we mainly do rest and UI testing with only few performance and long running scenarios - both are very resource hungry.
mcada
on 19 Sep 2018
The problem still exists after the fix
Caused by: org.apache.camel.FailedToCreateRouteException: Failed to create route -LNKZ40AxnGkdY2TSh4T: Route(-LNKZ40AxnGkdY2TSh4T)[[From[google-mail-stream-0-0]] -... because of 401 Unauthorized
--
聽 | {
聽 | "code" : 401,
聽 | "errors" : [ {
聽 | "domain" : "global",
聽 | "location" : "Authorization",
聽 | "locationType" : "header",
聽 | "message" : "Invalid Credentials",
聽 | "reason" : "authError"
聽 | } ],
聽 | "message" : "Invalid Credentials"
聽 | }
聽 | at org.apache.camel.impl.RouteService.warmUp(RouteService.java:147) ~[camel-core-2.21.0.fuse-720021.jar!/:2.21.0.fuse-720021]
聽 | at org.apache.camel.impl.DefaultCamelContext.doWarmUpRoutes(DefaultCamelContext.java:3947) ~[camel-core-2.21.0.fuse-720021.jar!/:2.21.0.fuse-720021]
聽 | at org.apache.camel.impl.DefaultCamelContext.safelyStartRouteServices(DefaultCamelContext.java:3854) ~[camel-core-2.21.0.fuse-720021.jar!/:2.21.0.fuse-720021]
聽 | at org.apache.camel.impl.DefaultCamelContext.doStartOrResumeRoutes(DefaultCamelContext.java:3640) ~[camel-core-2.21.0.fuse-720021.jar!/:2.21.0.fuse-720021]
聽 | at org.apache.camel.impl.DefaultCamelContext.doStartCamel(DefaultCamelContext.java:3492) ~[camel-core-2.21.0.fuse-720021.jar!/:2.21.0.fuse-720021]
聽 | at org.apache.camel.impl.DefaultCamelContext.access$000(DefaultCamelContext.java:209) ~[camel-core-2.21.0.fuse-720021.jar!/:2.21.0.fuse-720021]
聽 | at org.apache.camel.impl.DefaultCamelContext$2.call(DefaultCamelContext.java:3251) ~[camel-core-2.21.0.fuse-720021.jar!/:2.21.0.fuse-720021]
聽 | at org.apache.camel.impl.DefaultCamelContext$2.call(DefaultCamelContext.java:3247) ~[camel-core-2.21.0.fuse-720021.jar!/:2.21.0.fuse-720021]
聽 | at org.apache.camel.impl.DefaultCamelContext.doWithDefinedClassLoader(DefaultCamelContext.java:3270) ~[camel-core-2.21.0.fuse-720021.jar!/:2.21.0.fuse-720021]
聽 | at org.apache.camel.impl.DefaultCamelContext.doStart(DefaultCamelContext.java:3247) ~[camel-core-2.21.0.fuse-720021.jar!/:2.21.0.fuse-720021]
聽 | at org.apache.camel.support.ServiceSupport.start(ServiceSupport.java:61) ~[camel-core-2.21.0.fuse-720021.jar!/:2.21.0.fuse-720021]
聽 | at org.apache.camel.impl.DefaultCamelContext.start(DefaultCamelContext.java:3163) ~[camel-core-2.21.0.fuse-720021.jar!/:2.21.0.fuse-720021]
聽 | at org.apache.camel.spring.SpringCamelContext.start(SpringCamelContext.java:133) ~[camel-spring-2.21.0.fuse-720021.jar!/:2.21.0.fuse-720021]
聽 | ... 24 common frames omitted
聽 | Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 401 Unauthorized
聽 | {
聽 | "code" : 401,
聽 | "errors" : [ {
聽 | "domain" : "global",
聽 | "location" : "Authorization",
聽 | "locationType" : "header",
聽 | "message" : "Invalid Credentials",
聽 | "reason" : "authError"
聽 | } ],
聽 | "message" : "Invalid Credentials"
聽 | }
聽 | at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:146) ~[google-api-client-1.22.0.jar!/:1.22.0]
聽 | at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113) ~[google-api-client-1.22.0.jar!/:1.22.0]
聽 | at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40) ~[google-api-client-1.22.0.jar!/:1.22.0]
聽 | at com.google.api.client.googleapis.services.AbstractGoogleClientRequest$1.interceptResponse(AbstractGoogleClientRequest.java:321) ~[google-api-client-1.22.0.jar!/:1.22.0]
聽 | at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:1065) ~[google-http-client-1.22.0.jar!/:1.22.0]
聽 | at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:419) ~[google-api-client-1.22.0.jar!/:1.22.0]
聽 | at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:352) ~[google-api-client-1.22.0.jar!/:1.22.0]
聽 | at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:469) ~[google-api-client-1.22.0.jar!/:1.22.0]
聽 | at org.apache.camel.component.google.mail.stream.GoogleMailStreamEndpoint.createConsumer(GoogleMailStreamEndpoint.java:75) ~[camel-google-mail-2.21.0.fuse-720021.jar!/:2.21.0.fuse-720021]
聽 | at io.syndesis.integration.component.proxy.ComponentProxyEndpoint.createConsumer(ComponentProxyEndpoint.java:66) ~[integration-component-proxy-1.5-SNAPSHOT.jar!/:1.5-SNAPSHOT]
聽 | at org.apache.camel.impl.EventDrivenConsumerRoute.addServices(EventDrivenConsumerRoute.java:69) ~[camel-core-2.21.0.fuse-720021.jar!/:2.21.0.fuse-720021]
聽 | at org.apache.camel.impl.DefaultRoute.onStartingServices(DefaultRoute.java:103) ~[camel-core-2.21.0.fuse-720021.jar!/:2.21.0.fuse-720021]
聽 | at org.apache.camel.impl.RouteService.doWarmUp(RouteService.java:172) ~[camel-core-2.21.0.fuse-720021.jar!/:2.21.0.fuse-720021]
聽 | at org.apache.camel.impl.RouteService.warmUp(RouteService.java:145) ~[camel-core-2.21.0.fuse-720021.jar!/:2.21.0.fuse-720021]
聽 | ... 36 common frames omitted
After an hour of oauth authentication
oscerd
on 26 Sep 2018
It looks like the access_type is not submitted
oscerd
on 26 Sep 2018
@oscerd can you check if the connection has the refreshToken configured property?
Either from the /api/v1/connections or via SQL:
SELECT c.path, c.value
FROM jsondb c JOIN jsondb p ON c.path like rtrim(p.path, 'connector/id')||'/configuredProperties/%'
WHERE p.path like '/connections/%/connector/id/' AND p.value = '`gmail';
If there's no refreshToken property and the authorization URL did contain access_type=offline then you need to delete your developer project/OAuth app and recreate it (I deleted the whole project to get it to work).
From what I understood, it's either time based or the refresh token can be issued only on the first time the same OAuth application tries to authorize. You might also need to remove the permission via https://myaccount.google.com/permissions.
zregvart
on 26 Sep 2018
Let me have a look
oscerd
on 26 Sep 2018
@oscerd if you don't see access_type=offline in the first request to authorize, then doublecheck that the newest Connector metadata is loaded.
Something like:
SELECT path, value FROM jsondb WHERE path LIKE '/connectors/:gmail/configuredProperties/%';
There should be a row with:
path | value
-------------------------------------------------------------------------+---------------------------------------------------------------------------
/connectors/:gmail/configuredProperties/additionalOAuthQueryParameters/ | `{"access_type": "offline"}
I'm not 100% sure if we need a migration script for updating Connector metadata, my understanding is that it should be overwritten in JSONDB, but I could be wrong on that.
zregvart
on 26 Sep 2018
I'm checking
oscerd
on 26 Sep 2018
It's not there:
syndesis=# SELECT path, value FROM jsondb WHERE path LIKE '/connectors/:gmail/configuredProperties/%';
path | value
-------------------------------------------------------------+---------------------------------------------------------------------------
/connectors/:gmail/configuredProperties/applicationName/ | `gmail-syndesis
/connectors/:gmail/configuredProperties/authenticationType/ | `oauth2
/connectors/:gmail/configuredProperties/authorizationUrl/ | `https://accounts.google.com/o/oauth2/v2/auth
/connectors/:gmail/configuredProperties/clientId/ | `xxx
/connectors/:gmail/configuredProperties/clientSecret/ | `xxxx
/connectors/:gmail/configuredProperties/googleScopes/ | `https://mail.google.com/
/connectors/:gmail/configuredProperties/tokenUrl/ | `https://accounts.google.com/o/oauth2/token
(7 rows)
I'm polishing the db and redeploying the server.
oscerd
on 26 Sep 2018
After that
syndesis=# SELECT path, value FROM jsondb WHERE path LIKE '/connectors/:gmail/configuredProperties/%';
path | value
-------------------------------------------------------------------------+---------------------------------------------------------------------------
/connectors/:gmail/configuredProperties/additionalOAuthQueryParameters/ | `{"access_type": "offline"}
/connectors/:gmail/configuredProperties/applicationName/ | `gmail-syndesis
/connectors/:gmail/configuredProperties/authenticationType/ | `oauth2
/connectors/:gmail/configuredProperties/authorizationUrl/ | `https://accounts.google.com/o/oauth2/v2/auth
/connectors/:gmail/configuredProperties/clientId/ | `xxxx
/connectors/:gmail/configuredProperties/clientSecret/ | `xxxx
/connectors/:gmail/configuredProperties/googleScopes/ | `https://mail.google.com/
/connectors/:gmail/configuredProperties/tokenUrl/ | `https://www.googleapis.com/oauth2/v4/token
(8 rows)
but there isn't refreshToken set on the connection:
syndesis=# SELECT c.path, c.value FROM jsondb c JOIN jsondb p ON c.path like rtrim(p.path, 'connector/id')||'/configuredProperties/%' WHERE p.path like '/connections/%/connector/id/' AND p.value = '`gmail';
path | value
---------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
/connections/:i-LNKfvkEU7QH1KV6a0Hbz/configuredProperties/clientSecret/ | `禄ENC:4425ab8610ca90d3dcc87d56ab1a46db9bfdc5b029d2746a561b808642fb04b7720b60555c76201e9c7db4c6250b2bf2
/connections/:i-LNKfvkEU7QH1KV6a0Hbz/configuredProperties/clientId/ | `xxxx
/connections/:i-LNKfvkEU7QH1KV6a0Hbz/configuredProperties/accessToken/ | `禄ENC:c21f33bd1623ed24d05c1ed5615dbf0d827ebd533777e3095fe73a7d8ef9b3054543c11e5909eb48afc6059af9ae649ee946b54320141be8e2abdfdd08d3c2c9ae56b4981b788618160d9c64df2e54346238890a92541c7c8b8da2695a42f2cbdfbc9403e29ae5b0bd60a00d8164c71aa1d9f9d08dd1f9b1b18095ad861a6086bf76036d0ae87d93fbb67fdad4503b6722299778ee141981e2f7a128961f7157
I'll try with a fresh project
oscerd
on 26 Sep 2018
I'll try with a fresh project
Yeah, super surprising that it will be issued only for the first try and also there's a lot of confusion around this. I don't know what exactly helped in my case, I ended up deleting the project and it worked after that. The token endpoint simply returned null refresh token value regardless of auth_type=offline being specified on the authorization URL.
zregvart
on 26 Sep 2018
It seems we have the refresh token set this time, I'm running an integration and I'll wait more than an hour to check. I'll let you know.
oscerd
on 26 Sep 2018
If the workflow requires to have a fresh application on the first oauth flow attempt we may need to document this stuff very well.
oscerd
on 26 Sep 2018
@TovaCohen this is a Google OAuth issue. Our first release that contained the GMail connector didn't request the refresh token. This caused this issue, i.e. after the access token expired your integration stopped working. Now we have it fixed so we request the issuance of the refresh token. But it appears that the OAuth application definition on Google's side needs to be re-done, as it will only issue the refresh token on the first authorization and not on the subsequent, regardless of if we ask for the refresh token to be issued. Not sure if we need to emphasise this in our documentation, but it can be something users will stumble upon.
Also cc @syndesisio/qe to make sure that OAuth application in Google developer console is redefined before any tests to make sure that the refresh token is given.
zregvart
on 26 Sep 2018
It works fine.
oscerd
on 26 Sep 2018
I think this requires two doc tasks:
Update and re-publish the 7.1 release notes to add a Fuse Online known issue:
Published integrations that connect to Gmail stop executing when their Gmail API access token expires. To enable the integration to execute, re-register your Fuse Online environment as a Gmail API client and enter the new client ID and secret ID in the Fuse Online Settings page. (and that would be a link to the instructions for doing that) In the next release, it is expected that Fuse Online will obtain a new access token when needed and you will not need to re-register.In the 7.2 release notes, in the Important Notes section for Fuse Online, add this:
If you have any running integrations that connect to Gmail, be sure to:- Register your Fuse Online 7.2 environment as a Gmail API client.
- Stop the running integration.
- Re-publish the integration.
Is this right?
Is there anything else that the doc needs to do?
TovaCohen
on 26 Sep 2018
@zregvart @TovaCohen
If this has been sorted then please disregard ...
In my original identifying of this issue, I found that the properties cached in my Integration for the gmail connection Client and Secret IDs were not actually updated when the new values were inserted in Settings > OAuth Application Management. Therefore, when I stopped the running integration and then re-published, I still had the original token and obviously the same 401 Authorization failure. My habit quickly became removing my integration in total, updating the gmail properties and rebuilding the integration.
This may have been fixed, so guys please advise accordingly. However, this may certainly require some confirmation testing in both 7.1 and 7.2, I would think and if still the case the documentation would have to reflect this.
phantomjinx
on 26 Sep 2018
If you use the oauth flow without specify the new parameter added by Zoran the accessToken will expire anyway after 60 minutes, so the issue is still in the same context but it become something different along the way. By the way it seems to be ok now. I'm running different integrations with the same oauth configuration from more than 3 hours.
oscerd
on 26 Sep 2018
@TovaCohen I think these are the steps required for 7.2:
- Register your Fuse Online 7.2 environment as a Gmail API client making sure to create a new set of OAuth Credentials in Google API Console
- Reconnect the existing GMail Connection
- Stop the running integration.
- Re-publish the integration.
I'm not 100% sure if just stopping and re-publishing will set new (reconnected) connection details, from what I understand @phantomjinx is saying that it won't.
And I'm not 100% sure (perhaps @syndesisio/qe can verify) if just creating new set of credentials in the Google API console is enough, I had success only after I created a whole new project in the Google API console.
zregvart
on 26 Sep 2018
@zregvart
I can confirm from my experience that deleting the oauth credentials in the google API console and creating new ones was sufficient. (As long as the new credentials were applied as I described above)
phantomjinx
on 26 Sep 2018
Thanks Zoran and Paul. We have time to get the 7.2 release notes text exactly right. I created this FUSEDOC issue to make sure it gets done: https://issues.jboss.org/browse/FUSEDOC-2869.
For 7.1, I'll add the following today, and publish it on the customer portal unless I hear otherwise. @gaughan can you confirm that I should do this?
I will the following note here: [https://access.redhat.com/documentation/en-us/red_hat_fuse/7.1/html/release_notes/onlinedistrib#OnlineDistrib-Important] (Important Notes).
Published integrations that connect to Gmail stop executing when their Gmail API access token expires. To enable the integration to execute, re-register your Fuse Online environment as a Gmail API client and enter the new client ID and secret ID in the Fuse Online Settings page. (and that would be a link to the instructions for doing that) In the next release, it is expected that Fuse Online will obtain a new access token when needed and you will not need to re-register.
TovaCohen
on 26 Sep 2018
@TovaCohen agreed, thanks
gaughan
on 26 Sep 2018
Thanks Gary! The note is published on the portal:
https://access.redhat.com/documentation/en-us/red_hat_fuse/7.1/html/release_notes/onlinedistrib#OnlineDistrib-Important
TovaCohen
on 26 Sep 2018
Related issues
honghuac
路
4Comments
seanforyou23
路
3Comments
blaggacao
路
5Comments
tplevko
路
5Comments
gaughan
路
5Comments