Serverless: Deploy many micro/nano services to one API Gateway

[rafaljanicki]

This is a Feature Proposal

Description

I'd like to deploy many microservices to one API Gateway, but currently I'm not able to as the template is creating a new Gateway for every service. Would it be possible to change the behaviour? I may create a Pull Request for this if you agree

Topic on the forum: http://forum.serverless.com/t/multiple-services-behind-a-single-api-gateway/191

[schabert]

👍👍👍
My template in the forum is a example how a API gateway could shared with cross stack references between multiple services

[rafaljanicki]

Any update?

[sarenjus]

For us this is mandatory feature to move from 0.5.6 to 1.*. We are having many services and cloudfront in front of them. We are managing cloudfront distributions manually (without cloudformation or other automated solution) and we are not going to manage all those different origins manually. That would be a huge amount of manual work for each environment we are having.

Hopefully this will get more priority soon!

[antti-terho]

We are also using the Serverless 0.5.6 at the moment with multiple micro services under one API and it would be nice to have this kind on functionality to the new version of Serverless too.

[kerryhatcher]

Seems like @Christoph-Schabert has the right idea. We hit a wall with a project at work over this.

[axaubet]

Is not under discussion? Which is the process to get this issue under consideration? Should be possible for us to add this feature throught a plugin and later check if it could be part of the core?.

[pmuens]

This issue is on our radar and we'd like to support something in the core.

Would be great if more feedback / discussion about different implementation proposals could be provided so that a solution can be build which will serve as many different use-cases / configurations as possible.

[kerryhatcher]

This would also alleviate the issue noted in the current docs by allowing a project to be broken up: https://serverless.com/framework/docs/providers/aws/guide/installation#organization

Currently, every service will create a separate REST API on AWS API Gateway. Due to a limitation with AWS API Gateway, you can only have a custom domain per one REST API. If you plan on making a large REST API, please make note of this limitation. Also, a fix is in the works and is a top priority.

[kerryhatcher]

My employer needed a solution asap and there is a higher corporate mandate to use terraform so our solution in the short term was to write a plugin to generate Terraform JSON instead of CloudFormations JSON. Since this issue is directly related to CloudFormation, not the underlying services, it has solved our problems in the short term.

[axaubet]

@pmuens We have lost my configuration proposal for solving that point as we closed the duplicated threat. Should we add this here somewhere?.

@kerryhatcher So you have built a serverless plugin that instead of generating cloudformation templates is generating Terraform files? Cool!. From your explanation i understand that you have already solved that use case of having a single API Gateway. Please could you share your approach for this in terms of serverless.yml file configuration?. Thanks!

[pmuens]

We have lost my configuration proposal for solving that point as we closed the duplicated threat. Should we add this here somewhere?.

Yes, sure. I'll just past it here (taken from https://github.com/serverless/serverless/issues/3460#issue-221224159):

# We could add and extra http event attribute called isolation

functions:
 sum:
  handler: com.gsngames.seam.services.calculator.lambda.SumOperation
  events:
    - http:
        path: calc/sum
        method: post
        **isolation: stage**

---

Thanks for the explanation and details about your solution @kerryhatcher. I agree, this should only be a short term solution.

[kerryhatcher]

@axaubet I'm trying to get permission to open source our plugin, but there is a lot of bureaucracy to get rolling (and for good reason, don't get me wrong).

The basic concept is that you can create terraform files in JSON instead of HCL. So the idea is to create an object off of this.serverless.service called this.serverless.service.TF and then process through this.serverless.service.functions to build all of the required objects then eventually dump this.serverless.service.TF as JSON to a file and boom you can deploy via TF.

[axaubet]

@kerryhatcher. Thanks a lot!.
@pmuens. Thanks for updating the thread.

As resume of what i understand we should be able to setup:

Use Case 1 (Every Service gets deployed to a different API Gateway).
This could be the standard behavior so if we are not specifying anything we get this.

Use Case 2 (Service functions are deployed to an stage API Gateway)
By specifying an extra attribute in each function we can link them to a "general" stage gateway. If we a re deploying the first function with this setup then the gateway is created if it's just another one then we reuse the already existing one. The only thing that the developer has to control is that all path published to the stage gateway are unique.

Use Case 3 (Service functions are deployed to a group API Gateway)
Non of the above use cases covers what we want. Imagine we want to have inside the same stage different gateways grouping services.

Maybe by covering the Use Case 3 we are already killing Use Case 2. And we just need an attribute at the function level to specify the API Gateway group the function belongs to.

Comments? Thoughts?

[HyperBrain]

I tought about Use case 2. In my opinion it is not possible to control the integrity of deployed endpoints, as they are uploaded to APIG from different deployments. The APIG deployment resource "moves" the endpoints uploaded to APIG into a stage (and afterwards they are persisted). The problem here is, that you MUST guarantee that all endpoints that are not part of the current service are the version you expect them to be.
Without a complex dependency management between services, deploying to one APIG / API stage would not work. I am not sure if that is worth it.
Furthermore it might be difficult to recover on errors, like there is one of the endpoints missing.

[axaubet]

@HyperBrain Thanks for your feedback!. From my point of view this coherence problem is out of the scope. I think this issue is already present as you cannot guarantee coherence between services, each developer MUST manage this some way for example we control this with our CI/CD system plus semantic versioning. With the new proposal what we are giving are new tools to setup things in a different way knowing that you MUST find the right usage pattern to avoid issues like the one you are relating. The point is that right know with the current options the constraints are too big and we are forced to a single pattern.

[HyperBrain]

@axaubet Our solution here is to keep services small, so that they are also microservices from an API point of view, but combining them unter the same base paths via custom domain mappings. E.g. the mappings are:

api.mydomain.com/myserviceroot/v1/myapp/myservice1/....
api.mydomain.com/myserviceroot/v1/myapp/myservice2/....

I think you even can position the micro-service root on arbitryry levels. With that you still keep the isolation between micro-services. Even if that approach generates multiple APIG apis they are mapped to the consumers in a meaningful way.

Maybe this would be a 4th case for your feature, and could also be configurable through the YAML as soon as CF supports setting the SSL certificates correctly.

[axaubet]

@HyperBrain I completly agree in keeping services small. How do you achieve this custom domain mapping? using a custom balancer/proxy? .

As you said the key is to be able to expose the API in a meaningful way for the customer so they don't need extrta services for discovering the location for the different services.

[mykemd]

Hey guys,

I am using this feature for ApiGateway
http://docs.aws.amazon.com/apigateway/latest/developerguide/api-gateway-create-api-as-simple-proxy-for-http.html

You can create your Master Api as HTTP proxy to your "microservice" deployments done by Serverless - sort of the same mechanism you would do with NGINX or HAProxy with microservices deployed within containers or on private instances.

Your Master API can have structure
/api
/serviceA/*
/serviceB/*
...

you can deploy it as serverless resource or with custom cloud formation project per stage, per version etc...: that will allow you to create cloud front distribution with one custom domain for all your Serverless services.

[axaubet]

@mykemd so we are creating an APIG in front of our microservices, that have theor own APIG, correct?.

Althought its a working solution, for us won't be cost efffective. The APIG is an expensive resouce in AWS and with this setup we are doubling this cost.

[axaubet]

We have build a solution (alpha version) that adjusts the CF template to adjust the setup to have a single gateway for different services. Basically we are talking directly to AWS api's to check if the needed gateway exists an avoid creating a new one and readjusts all the other CF entries to point to this located gateway.

Finally for simplicity we are using just a custom var that specifices the behavior.

[darren-rogan]

Hey @axaubet could you share some that in a GIST. We are hitting the same issue with API Gateway and work prefer not to reinvent the wheel. Thanks

[axaubet]

Iep.

What we have developed it's a plugin that by setting a custom general var specificied the isolation level for the service.

We have 2 possible values "service" or "stage" if stage is selected the plugin removes the creation of the API gateway from the cloudformation template and does it (if needed) directly by calling to AWS so we get an API Gateway that doesn't belong to an specific service. With the id of the Gateway we modify the cloudformation template of the service so every function points to it correctly.

It's a bit tricky but works perfectly.

[spinify-matt]

@eahefnawy any reason this was added and removed? We are super keen to get it resolved - how can we get this done? Many thanks.

[spinify-matt]

Any update ? How do we solve? Many thanks.

[alexbielen]

Any updates on this?

[spinify-matt]

Any idea on the release date of 1.17 ?

[spinify-matt]

@pmuens anything we can do to make it happen sooner?

[pmuens]

Hey @spinify-matt thanks for getting back 👍 !

The scheduled release date is the July 5, 2017.

Really helpful would be if we could kick-start different implementation proposals.

• How do you guys think we can implement that?
• How should the DX for this look like?
• How would you use this feature with your existing Serverless services?

[HyperBrain]

To keep things non-breaking the very simplistic and working approach would be:

Add a new optional property awsApiGatewayApiId under the provider or service node.
If the property is set, skip generation of the ApiGateway CF resource and use the API id instead for all references. (The property must accept CF references like Ref, Fn::GetAtt, Fn::ImportValue as well as a plain string id, to support different use cases).

One technical detail that has to be clarified though, is if the ApiGateway root resource id that is also needed for APIG resources, should be replaced automatically by a Fn::GetAtt: [ <provider.awsApiGatewayApiId>, 'rootResource' ].

I think that is the minimal viable implementation that bears the least risk of breaking anything.
The AWS APIG will then receive multiple APIG deployments from different services. Of course the API must not be deleted from the outside. This can easily be prohibited, by creating the API in a different stack manually, and using a Fn::ImportValue reference to that stack as awsApiGatewayApiId reference.

[kerryhatcher]

@pmuens I'll second @HyperBrain 's idea. Allowing a serverless project to use an existing API gateway would work just fine. For our project, it allows us to have each micro-service in a separate repo and then have them all available via the same domain name. In turn, we will then be able to have a CICD pipeline for each one separately.

[pmuens]

Nice! Thanks for the writeup @HyperBrain 💯
Thanks for confirming that this would solve the problem @kerryhatcher 😊.

I like this proposal since it's simple, yet flexible and non-breaking. We can extend this with more features later on, but this is a good starting point 👍.

/cc @eahefnawy 🔝

[rafaljanicki]

I think that @HyperBrain's idea would definitely solve the issue and wouldn't add much to the process (only creation of the root API which can be solved by a CF template). Thanks!

[jasonj3]

Do we think this is still on track for the 1.17 release?

Thanks :)

[pmuens]

Hey @jasonj3 👋

Thanks for commenting. We're currently working on this and a WIP PR should be out soon!

[pmuens]

Hey everyone!

Just wanted to give you a quick update that we just PRed a WIP solution for this. It's not 100% there yet, but at least smth. we can discuss about (it's based on @HyperBrain proposal :pray:).

Feel free to chime in / make suggestions.

The PR can be found at #3878.

[kerryhatcher]

@pmuens I like how the config works. Solves the issue for us.

[pmuens]

@pmuens I like how the config works. Solves the issue for us.

Thanks for the feedback @kerryhatcher 👍 Great to hear that it solves your issue.

[mgurnani]

Hello,
I would like to use this feature and test it out for you.. :)
I have seen the PR #3878 .
How do I take the source for this change? Pull request on which branch ?

[mgurnani]

Ok I pulled the branch "shared-api-gateway-support" and added my Rest api id in provider:

  apiGatewayRestApiId:
      Ref: 7vpmbgy8yb

This is under my resources:

resources:
    Resources:
        user-register-api:
             Type: AWS::ApiGateway::RestApi
             Properties:
               Name: register

When I do a deploy there is an error :

"The CloudFormation template is invalid: Template format error: Unresolved resource dependencies [7vpmbgy8yb] in the Resources block of the template"

I do have a gateway deployed with the given id.

[eahefnawy]

@mgurnani should be:

apiGatewayRestApiId:
      Ref: user-register-api

also, I'm not sure if that's a valid logical ID?! I think dashes are not permitted in CF logical IDs

[mgurnani]

ok. CF Id's can not have dashes. I thought the name was just a placeholder - not being used.

I kept the apiGatewayRestApiId same and changed the Resources section. My yml now reads:

provider:
  name: aws
  runtime: java8
  ..........
  apiGatewayRestApiId:
      Ref: 7vpmbgy8yb
.......
resources:
    Resources:
        7vpmbgy8yb:
             Type: AWS::ApiGateway::RestApi
             Properties:
               Name: register

It still gives the same error on deploy:
"The CloudFormation template is invalid: Template format error: Unresolved resource dependencies [7vpmbgy8yb] in the Resources block of the template".

Just to be clear - my API GW has an API named "prod" deployed which has the id "7vpmbgy8yb".

[spinify-matt]

Any update here on a release date?

[pmuens]

Any update here on a release date?

Hey @spinify-matt thanks for commenting 👍

The PR should be in a state where it can be finalized and merged soon. However we still have some issues we need to resolve before we can merge it. You can read more about it in the PR. Feel free to chime in and give feedback for a possible solution!

[mhenag]

Great to see this feature. We've been manually managing our apigw side of things for the last 18 months as our api was a reverse proxy to multiple services.
Thanks all.

[Kjuib]

We are really looking forward to this feature. It will really help as we have multiple teams working on difference services and want them all under the same API gateway.

[stormit-vn]

I am also looking for a solution on this issue. Our project has 4 different stages (DEV, SIT, UAT, and PROD), so it would be better if we just create a single API Gateway with different stage, then we can add custom domain for each stage

Anyway, is there anyone has a temporary workaround to make this works?

[HyperBrain]

@stormit-vn The serverless-aws-alias plugin let's you deploy a single "serverless stage" into different aliases, which match your DEV, SIT, UAT and PROD stages. It will automatically create APIG stages in one APIG API and also tag the deployed functions with aliases, that are called from the deployed APIG stages.
The next upcoming big feature for the plugin is alias promotion, where you can then seamlessly "move" a quality stage to another like DEV -> SIT without deploying anything.

[stormit-vn]

@HyperBrain Thank you for your suggestion, now I have able to deploy serverless to different stages of APIG, but there is still an issue that I need to solve that is the naming convention. Looks the serverless named the resources by adding stage suffix without any option to defined a fixed name

[HyperBrain]

@stormit-vn Yes, I know. A solution is, to name the serverless "stage" in some neutral way, as you do not really use them with a specific semantic meaning. Or do you mean something else?

If this gets us deeper in a separate discussion, we could open an issue for the discussion over at the plugin's repo so that we do not clutter this thread. What do you think?

[stormit-vn]

@HyperBrain Yes, I also tried to name the stage without using environment name to avoid confusing such as my service name is dnd-api, then I name the stage is api, then update the serverless name to dnd, see the serverless.yml below

service: dnd
frameworkVersion: ">=1.1.0 <2.0.0"
provider:
  name: aws
  stage: api
  alias: dev
  runtime: nodejs6.10
  region: us-east-1

then if I try to deploy the serverless with specific alia (i.e. dev), the plugin does not allow me to do that and it returns an error that required master alias to be deployed first Error: You have to deploy the master alias at least once with "serverless deploy". If I try to deploy the master alias, then the first alias will be api not my expectation that is dev must be default alias. Is there any way to change that behavior? (we can open another plugin issue for deeply discussion then).

Thanks

[HyperBrain]

@stormit-vn Yes, currently the master alias is the name of the Serverless stage - this will be used automatically, if you do not specify an alias on the command line. I think it might be possible to add some service variable to set the master alias to something different than the Serverless stage. However this will be a new feature for the plugin.
I also see value in this, so feel free to add a feature request in the repo. I will elaborate a proper non-breaking solution then and phase it in into the next feature release.

[brcline]

Any update?

[pmuens]

@brcline thanks commenting 👍

The PR which currently covers that is https://github.com/serverless/serverless/pull/3934. We still have some problems over there which needs to be solved (see this thread here and the corresponding thread in the PR). Would be nice to get some feedback on those so that we can get this into mergeable state soon!

[alexdebrie]

Hey all, I wrote a post on how to deploy multiple services to the same API Gateway with the use of a plugin. Check it out and let me know if that's a workable solution to this problem.

[brcline]

@alexdebrie 404 not found on the link you provided

[alexdebrie]

@brcline Thanks for reporting. It's working for me. Can you try clearing cache when you're on serverless.com?

[stgogm]

@alexdebrie I got a 404 as well. Even after clearing the cache :(

[alexdebrie]

@stgogm Sorry to hear that. We've had some issues with Service Workers on the site.

Are you able to access it if you go to the blog home and then click on the article? It's currently second in the list: https://serverless.com/blog/.

[stgogm]

@alexdebrie I don't think I see it on the list. However, I see this one https://serverless.com/blog/serverless-webtasks/ but I also get a 404 :/

Edit: It works if I do a refresh.

[brcline]

Only works for me on Edge. :(

[RafalWilinski]

I don't have any problems accessing this link on Safari or Chrome.

[stgogm]

I was able to see the article on Opera and Vivaldi, but not in Chrome or Firefox. It looks like a caching or balancing issue.

I'm using Ubuntu btw.

[tommedema]

Shouldn't this issue be about having multiple services share the same AWS resource, rather than only API gateways? For example a Cognito User Pool and Identity Pool likely has to be re-used by multiple separate services.

[keshavkaul]

@tommedema I concur. We should think in terms of sharing of resources by the services.

[alexdebrie]

@tommedema @keshavkaul There are other ways to share resources across services, such as using the CloudFormation Output variable syntax, or just copy-pasting the ARNs across services (avoid this if possible 😄 ).

This issue is focused on API Gateway specifically since it's non intuitive how to stitch multiple services together on the same domain.

[danbrauer]

Hey folks, a question: It seems like the seleverless-domain-manager plugin @alexdebrie posts about is a nice solution, and it's even been included in SLS's own documentation. Is there any thought to just incorporating this plugin into SLS proper, and letting that be the official solution to this issue?

If not, how come? I see that https://github.com/serverless/serverless/pull/4247 is active and is another route for solving this, but I am not knowledgeable enough to judge why its approach would be better than the plugin’s.

From what I can tell, while the plugin does not give you a single API Gateway, it does allow you to have one domain for all the Gateways you deploy, which to the end user is the same.

Thank you!

[HyperBrain]

@danbrauer Nice thoughts. From my side (as owner of serverless-aws-alias) I like your idea, mainly because of https://github.com/HyperBrain/serverless-aws-alias/issues/79#issuecomment-343273602 and the chance that the implementation in #4247 will be incompatible with some more plugins.

But as sls-maintainer, I'd like to hear some more thoughts of the people here, who might know, what problem is solved with #4247 but not with the serverless-domain-manager.

Any thoughts guys?

[alexdebrie]

@danbrauer I think that's an excellent idea. It could be an optional way for users to set up custom domains, including using base path mappings to put multiple services on the same domain. I don't think it conflicts with other methods of solving the micro/nano service problem.

I'd imagine usage like:

# serverless.yml

provider:
  domain:
    domainName: serverless.foo.com
    stage: ci
    basePath: api
    certificateName: *.foo.com
    createRoute53Record: true

Curious to hear thoughts from @aoskotsky-amplify & @majones-amplify about this. Are you good with us bringing that functionality into the Framework core?

[tommedema]

I highly discourage incorporating serverless-domain-manager into core, because it does not use cloudformation. It relies on AWS SDK calls which are much more likely to result in a corrupted state because it does not guarantee cloudformation rollbacks etc. to work as expected.

It also relies on commands whereas really this is a configuration issue and all actions should run based on config, not based on remembering to run a certain command manually at a certain point in time. This again relies on state, which cloudformation is made for.

If this plugin can be re-implemented by modifying the cloudformation template before passing it to AWS, it might be a better idea.

[HyperBrain]

@tommedema Yeah, that's actually a really good point. SLS core should only rely on CF to setup and manage it's deployments, so that everything stays consistent.

[alexdebrie]

@tommedema @HyperBrain I disagree somewhat on this. The core functionality of serverless-domain-manager does use CloudFormation.

There are two aspects of the plugin that do use direct SDK calls: (1) creating or deleting the domain, and (2) handling the Route53 record.

For the first, I'm not concerned that this isn't in CloudFormation. It's a one-time creation process that doesn't really fit in CloudFormation. If a user hasn't created the domain before trying to add to its CloudFormation, we can provide that information in the error printed to the console.

The Route53 record is something I would like to be handled in CloudFormation. I haven't looked closely enough to understand why it's managed via SDK call right now.

[badunk]

@HyperBrain In regards to domain manager vs sharing an API gateway - there is a default soft limit of 60 APIG per account (we had it increased after talking to support).

We write many microservices and for each we stage them accordingly (dev, stage, prod) and we may invite developers to make changes and deploy to their own stages.

That quickly adds up to 60 API gateways in our account.

[aoskotsky-amplify]

@alexdebrie I'm fine with getting the domain manager functionality added to the core framework.

I believe we used SDK calls for Route53 and the API Gateway Custom Domain because Cloudformation didn't support it at the time. It probably makes sense to move those to Cloudformation now either by updating the plugin or into the core framework.

There was recently a feature request https://github.com/amplify-education/serverless-domain-manager/pull/90 though that wouldn't work if we used a single Cloudformation stack for this but I don't know how common that case is.

Also I'm a bit confused how this will work with multiple API Gateways sharing the same Custom Domain. Which Cloudformation stack would have the definitions of the domain and Route53 records if multiple ones need to use the same domain? This was one of the reasons we made a separate sls create_domain command.

[SteveSmith21]

Hi guys, I've recently opened this issue rich is about being able to load multiple website with one serverless app.

https://github.com/serverless/examples/issues/246

Could this be solved by what you're proposing and should I then close my issue since it's basically a duplicate then?

[matt-moser]

It looks like discussion here didn't notice that a solution is available in v1.26.0? #4247 included in that release provides the ability to reuse an existing API Gateway.

[dijonkitchen]

Any updates or should it be closed?

[mikecann]

If this is fixed then maybe the docs should be updated? https://serverless.com/framework/docs/providers/aws/guide/services/

[rugwirobaker]

Has this been fixed?? The issue is still open.

[fedebalderas]

https://serverless.com/framework/docs/providers/aws/events/apigateway/#share-api-gateway-and-api-resources

[0x6C38]

This works as far as I can tell. The documentation seems to be wrong though, declaring the API Gateway as a resource isn't necessary. The default gateway resource is called ApiGatewayRestApi and can be exported like so:

  Outputs:
    apiGatewayRestApiId:
      Value:
        Ref: ApiGatewayRestApi
      Export:
        Name: apiGateway-restApiId
    apiGatewayRestApiRootResourceId:
      Value:
         Fn::GetAtt:
          - ApiGatewayRestApi
          - RootResourceId 
      Export:
        Name: apiGateway-rootResourceId

[pmuens]

Closing this for now since there are a couple of solutions mentioned in this thread. Feel free to continue the discussion...

[Mumashika]

Could you please also update documentation since it points to this issue

[tranvanthuc]

https://serverless-stack.com/chapters/api-gateway-domains-across-services.html
I follow this tutorial and it works fine. You can try it.
Hope can help you

[yhrabovskyinomi]

folks, just wondering if there any way to enable x-ray in Servereless for this main/single api-gateway? because this relates to ApiGateway:Stage resource which created while creating lambda, I can not push x-ray.
Also

tracing:
    apiGateway: true

not working for this use case