Vapor: Vapor 3 async changes before alpha

[Joannis]

Discussion topics:

• Stream notifications for errors
• Stream close notification
• Cascading notifications throughout streams (up the stream and/or down the stream)
• Multiple future types
• Lightweight versus easy to use futures
• Timing out promises efficiently
• Helpers for managing tasks on the event loop

These topics explained from top to bottom:

Error stream notifications

Currently, stream notifications are handled by registering a closure. You can set a closure on the receiving/draining end of the stream, the result. But anywhere else will break the error chain unless explicitly passed on. This also prevents errors from cascading back. An error in TCP will trigger later in the process of handling the HTTP request, but the other way around doesn't work (Database errors cascading to the HTTP layer). That is still a useful feature, for example, for throwing 500 pages by default.

let requests = socket.stream(to: HTTPParser)
let responses = request.stream(to: myApp)
responses.stream(to: socket)

responses.catch { error in
  print("500 error")
}

// At this moment, the socket does not find any errors, nor do the parser serializer

Stream close notifications

Closing streams is handled the same way, currently. We should decide if we want to keep that this way.

// WebSocket.swift
socket.onClose = {
  websocket.sendCloseMessage()
}

// WebSocket+SSL.swift
socket.onClose = {
  tls.deinitialize()
}

// Chat.swift
socket.onClose = {
  friends.notifyOfflineStatus()
}

Cascading notifications

As discussed in the error stream notification, cascading these errors can interrupt a process causing no additional unnecessary work to be executed. But this must be implemented with care. Interrupting in the middle of a registration form (adding the database entry, sending the email and 2 other ops) can prevent some ops from executing whilst others did execute.

There should be sensible helpers for that.

drop.get {
  return database.all(User.self, in: "SELECT * FROM users")
}

class ErrorMiddleware: Middleware {
  func handle(request: Request, chainingTo responder: Responder) -> Future<Response> {
    let promise = Promise<Response>()

    return responder.respond(to: request).do(promise.complete).catch {
        promise.complete(Response(status: 500))
    }

    return promise.future
  }
}

Multiple future types

As of right now, futures are really easy to use and sensible. They conform to FutureType allowing extensibility and more (generic) integration. One such example is the conformance of Response to FutureType, effectively making Future(Response(...)) unnecessary bloat.

Pseudocode Problem

let users = database.all(User.self, in: "SELECT * FROM users LIMIT 100") // [User].count == 100

If User is a struct of 10KB, [User] will be copied around 2 times with an overhead. Once to the future, once from the future to the handler. This copies around 2MB of extra data.

Timing out promises

Promises can be timed out whilst blocking, currently. But you cannot easily dispatch a task like this without manually interfacing with the DispatchQueue in a worker's eventloop. Having to block for waiting is not sensible, and against the async design. If an HTTP request is waiting for >30 and especially >60 seconds, you're pretty much guaranteed to have timed out. And if any client-side operation done by Vapor (such as calling stripe) times out after X seconds, we shouldn't keep the promise unfulfilled in memory.

Use case

// Throws a timeout after 5 seconds of waiting for a result
return websocket.connect(to: "ws://localhost:8080/").map(timeout: .seconds(5)) { websocket in
  ...
}

Helpers for managing tasks

Finally, playing into this last topic. Many people are using websockets or other features that require cron-jobs. Some people may want to send a notification every minute for new chat messages, regardless of circumstances. Or send a ping to the client every 15 seconds to keep the connection open. They're very common use cases and should be covered from the start, especially with async.

Use case

websocket.worker.every(.seconds(15)) {
  // keep the connection alive
  websocket.ping()
}

[vzsg]

So, are you embracing or _reimplementing_ RX? :)

[Joannis]

@vzsg We could look into that. My main 2 problems with it are the lack of control and the lack of focus on Linux. this is a good discussion to have

[TofPlay]

Pleaseeee not Rx!!! 🙏 😰

[kaqu]

I am experimetning with some framework for extensive future usage and task parallelism right now. If I would have some great concept to show I will link it here. But, according to previous comment, please do not use RX 😄
To take those requirements and do some progress in this topic discussion - Would you like to use some kind of future with streams or it should be separated concepts? It could be (I hope so) interesting approach to treat them equally. If so, how would you describe future of stream (future of promise is rather well known)? I was thinking about that for a while, and I hope it could lead to some interesting concept. If you think around, this looks like interesting abstraction. Since if you are opening a stream you are interested in future of that stream represented by subsequent parts of data, while stream is like multiple time fulfilling promise. Transformations or adding conditions for handling changes or results is still good abstraction while future of future is still a future right? 😄 What do you think about that?

[twof]

Out of curiosity, why not Rx? @kaqu @TofPlay

[Joannis]

@kaqu A stream's future could be triggered for every event @kaqu , so yes. It's worth considering. It's actually something I'm really interested in, too. It's also not an unreasonable approach and would simplify and streamline the async API a lot more than it already is. @tanner0101 , what do you think?

@TofPlay and @kaqu , what are your reasons against RX? I didn't look extensively into it myself.

@vzsg , @TofPlay @twof and @kaqu , thanks for contributing to the conversation! This is a really important subject, and opinions are super important here. Are there any concepts or implementations in RxSwift that are good and should be looked into?

[jdmcd]

I think there are good concepts in RxSwift that are worth implementing, especially when it comes to things like promises and concurrency. However, in my experience, the required technical knowledge re-work and potential technical debt that can come with introducing a beginner to reactive models in incredibly (and unnecessarily) high.

That being said, I know the point of the framework isn't to make it a beginner framework, but at the same time I think it's _really_ critically important to consider if the complexity that is being added is for the sake of being complex and saying, "Oh yeah, Vapor implements RxSwift" or if it's actually providing useful, relevant and necessary features.

[calebkleveter]

Could you make it so that you could opt out of RX, or would that defeat the whole purpose? I have never used RX before (or really know what it means), so I would like to be educated on the subject.

[kaqu]

@Joannis @twof - not RXSwift for 2 reasons: first and I think could be some kind of important - external dependency. I know that it is some kind of special, but still, I would think at least twice before adding that kind of dependecy, especially to some core things. On the other hand, I think that there can be more suitable tools or better solutions for that specific requirements.
There is one more thing that I don't know about, but maybe someone knows something more - it is performance. Is RX Swift fast enough to be used as core of server solution? It should ba at least lightning fast! 😁
I feel that RX as a concept is really good and agree with @mcdappdev . It is not that it cannot suit that needs, especially with stream handling, but I hope that there is better solution 😄 Before adding that I would at least think about some strong alternative.
Merge of stream, promise and long lasting concurrent task handling in same or similar way or ideally with same interface in form of some future object could be interesting deal to think as alternative.

[jdmcd]

100% agree with @kaqu! Also important point that I had forgotten about which is that adding a dependency this large and important is a decision that should not be taken lightly.

[Joannis]

It is definitely not taken lightly! Hence why we're really into developing our own async library right now.

I think @kaqu is coming with really interesting and good points. Especially streams as a future type.

[jdmcd]

Agreed! Glad that it's being seriously considered, just another thing I love about the Vapor team :)

[TofPlay]

Sorry for my english. It's not my first language but I will try to explain my point. If you start with Rx you will have to change everything to be consistent with the Rx approach. That's what's happening with iOS apps when people trying to use Rx. The application become complex for... nothing. Concurrency? If you master GCD and closure you have all you need. Performances and ressources? Rx it's an overhead and even a big overhead from what I could see. For me Rx it's more like a syntax sugar. I think that the vapor core must remain as light as possible with a minimum of dependencies.

[tanner0101]

TBH I have no idea what RxSwift is (and that's even after spending 10 minutes reading their GitHub page).

We're using promises in Vapor 3 as a crutch for Swift 4's lack of coroutines (async/await read this). Once Swift supports coroutines natively, all of the promises will go away. Streams will likely stick around, but those are rarely seen in the public facing API.

I don't really see a reason to build a complex async design pattern into Vapor when we're expecting to get coroutines from the language soon.

Functions using promises....

func makeResponse(for req: Request) -> Future<Response>

will convert to something like this when Swift gets async/await.

func makeResponse(for req: Request) async throws -> Response

Code using promises...

app.get("foo") { req in
    return req.authenticatedUser().map { user in
         return user.name
    }
}

will look like this.

app.get("foo") { req in
    let user = try await req.authenticatedUser()
    return user.name
}

The reason we decided to use promises is that we believe Swift will support async/await sooner than later (promises provide a relatively straightforward migration to async/await. mostly just deleting closures and the word Future). We also have reason to believe the highly anticipated Swift Server API's Transport module will be promise based. If/when that module is released, we will be in a prime position to adopt it and swap out our promise implementation for theirs.

However, that being said, if someone does like RxSwift they should be free to implement that in their Vapor projects. Maybe they could contribute that back as a vapor/community package. Idk if that would be possible though as I don't really understand what it does.

[jdmcd]

I agree with all of Tanner’s points

[kaqu]

@tanner0101 - good point. Async/await would be great deal, but it can take a few years before being implemented unless it will not be implemented at all. Swift had in example built in curring functions but it has no more (I really miss that thing... 😢 ). I agree with you that if it will be for sure implemented it would be nice deal to stick with basic promises and do not invent some complex system, just for easy and quick swap. But do you think that when it will become available, it will had to be used in it's pure form? Maybe even if async/await will be shipped some good abstraction on top of it would be nice, wouldn't it? I do not say that it have to be a great idea, but I think it would be nice to think about.
I do not know a lot about mentioned Swift Server API's Transport module, but if it is key to be used in way presented there, maybe... do as similar as there?
Anyway... there is still case of streams, and streams are not going to change soon 😄

[RuiAAPeres]

disclaimer 1: I am ReactiveCocoa core member and being doing FRP since 2013.
disclaimer 2: Being RxSwift or ReactiveCocoa, for the sake of the argument, it's pretty much the same.

There are some good points in this thread like:

(...) adding a dependency this large and important is a decision that should not be taken lightly.

On the other hand, there are some misconceptions of what RxSwift does/is, like:

especially when it comes to things like promises and concurrency

Promises is a small subset of what RxSwift has/provides.

Could you make it so that you could opt out of RX

It would be difficult, because Rx is a framework (you don't call it, it calls you). If you want to opt-out, you need to abstract the whole API you touch, which by itself would be a PITA (I know, I tried...).

Especially streams as a future type.

This is my main issue with the whole thread. You can't model a stream of events with a Promise/Future. Simply because a Promise/Future once it's fulfilled it's done (aka completed). A stream might have an infinite number of next values, at an extreme, there is no end (or the idea of ending). A quick read on wikipedia, although the behaviour of a Promise/Future shows/tells you that, clarifies it:

Notably, a future may be defined without specifying which specific promise will set its value, and different possible promises may set the value of a given future, though this can be done only once for a given future

If you want to conceptually create something that does that, it's not a Promise/Future, it's something else (Observable, or SignalProducer/Signal).

Concurrency? If you master GCD and closure you have all you need. Performances and ressources? Rx it's an overhead and even a big overhead from what I could see.

I would be very careful on spreading this sort of information/claims in a public forum.

For me Rx it's more like a syntax sugar.

This tells me that you have either zero experience with RxSwift, or never checked the source code (or both).

On a more neutral stand:

Is RX Swift fast enough to be used as core of server solution? It should ba at least lightning fast!

You can check some of those here.

Hence why we're really into developing our own async library right now.

I wish you good luck, in all honesty. As the current maintainer of RxJava puts it: "you must be crazy to do your own FRP implementation" ( I am just paraphrasing him).

If the idea is to use Promises/Futures (again, remember the idea of the promise once fulfilled it's done) you could potentially use something like PromiseKit, which has been around for some years now.

[jdmcd]

@RuiAAPeres I think you make some fantastic points, and I agree that Rx aside, the most important part of the discussion is using a Promise type for a stream.

I also want to clarify that I don't think any of us were intentionally trying to bash or rip on Rx, obviously the work that you all are doing over there is incredibly impressive and important for so many difference architectures. We just want to make sure that the right choices are made for this particular framework and what it's trying to accomplish so that backpedalling doesn't have to happen in the future. Once again, though, I'm really glad to hear some input from a member of a different community, especially one as prevalent as Rx.

[TofPlay]

This tells me that you have either zero experience with RxSwift, or never checked the source code (or both).

You don't know me. 😄
I'm curious. When I met a new framework my first action is to study it and play with. Just to know what is it and see if it can be useful on my projects and if it's fit with my needs and my objectives. When the framework, like RxSwift, can have a big impact on the project I take a look to the source code to know what I put on my projects. I maintain what I said in my previous post.

[kaqu]

There is another one big question for all external frameworks (if we like to use some) - does it supports linux? I feel like most of Swift frameworks are iOS oriented. It is not something bad, but I am trying to say that we must think about it before using any of those. I was looking about that in RXSwift implementation and it looks like it supports Linux.
@RuiAAPeres - good points. You are right that using "future" with stream will not be standard or actual future pattern. Although stream and promise can have a enough common things to try unify handling of them. Stream differs from future in that abstraction only by one thing - mid states emitted by stream. But stream same as promise have end - can be closed.

I can imagine some kind of api like this to handle both (naming should be different though 😄) :

protocol Future : class {
    associatedtype IntermedieteType
    associatedtype FinalType
    associatedtype ErrorType


    func handleIntermediates(_ handler: @escaping (IntermedieteType)->())
    func handleCompletion(_ handler: @escaping (IntermedieteType)->())
    func handleError(_ handler: @escaping (IntermedieteType)->())
}

Promise IntermedieteType could be kind of progress or Void whereas in stream FinalType could be a Void or something more useable.

[RuiAAPeres]

There is another one big question for all external frameworks (if we like to use some) - does it supports linux?

Both ReactiveCocoa/ReactiveSwift and RxSwift do support Linux.

Looking at your proposal, or at least the general idea:

protocol Future : class {
    associatedtype IntermedieteType
    associatedtype FinalType
    associatedtype ErrorType


    func handleIntermediates(_ handler: @escaping (IntermedieteType)->())
    func handleCompletion(_ handler: @escaping (IntermedieteType)->())
    func handleError(_ handler: @escaping (IntermedieteType)->())
}

You are pretty much moving towards a FRP implementation (example1, example2), but without the functional parts to a given degree (higher-order operators):

Intermediete -> Value
Final -> Completed
Error -> Failed

Looking at the arguments:

1. Adding a big dependency.
2. The cognitive burden for beginners.
3. Performance.

And my counterpoint:

1. Cannot say much: it is what it is.
2. The cognitive burden will be the same, if you continue with your proposal (differentiation between next, error and completed).
3. I think this is a non-issue, until proven otherwise. Saying it's an "overhead" without numbers, it's pure speculation and spread of misinformation.

[mluisbrown]

Once again, though, I'm really glad to hear some input from a member of a different community, especially one as prevalent as Rx.

@mcdappdev just an FYI that @RuiAAPeres is not a member of the RxSwift team. He is a core contributor to ReactiveSwift which is a competitor library. In this instance though he is arguing in favour of FRP in general, regardless of which library you end up using 😄

[Joannis]

@RuiAAPeres you're right on the point. I have a few primary opinions regarding this:

My primary reason against this async library:

Writing our own Async library is an extra burden. We need to put a lot of effort into optimisation and careful API design before it's good enough. Developing our own async ecosystem will also be _yet_ another competitor to those already established libs. That's could be bad for everyone involved.

But we do have some reasons to develop this...

We should write async based on protocols, so people can use their own APIs/flavours. There's no one-fits all solution, ever. And especially not here. A big dependency like RxSwift and ReactiveCocoa should be possible. We shouldn't depend on a big library at such an essential level for Vapor. Especially since we'll have to choose one over the other. There are many iOS devs using it, and using those dependencies on both sides is great for them.

Vapor should provide a solid foundation for async APIs, but as minimal as necessary. Internally, we do need to use async functionalities. And people need to be able to interact with said code.

The cognitive burden needs to be limited where possible. We already are doing some work there.
One such example is making Response conform to the FutureType protocol and returning itself. It's not "clean", but it does ensure that the API is easy to get into for beginners.

We could also leave out Async entirely, but there's a few reasons not to

We're developing in Vapor and Swift for a reason. We're not here to _just_ make backends. We're here to write type-safe, readable and performant backends. These are the biggest traits for Swift in my opinion.

For framework developers it's not just a matter of making it easy and/or performant. We're not just here to help you write code and be productive. We also need to ensure the code you are writing is of a high quality. For Vapor, like any ecosystem, the job of the maintainers is to ensure that the APIs are created are being used properly and effectively. That they're easy to understand and reusable. To ensure quality.

You might have a small app, and we'll try to make it simple to get started. You might get more requirements, and you'll need more features. Then you grow bigger, and your application is meeting limits. At this point you'll be noticing the performance hog that is synchronous programming. All whilst your application is maintainable.

This, to me, is the goal of a framework. To meet the needs of all these situations. And people that depend on Vapor and get bigger. Or just people that have a bigger business and want to start using Vapor. Both of these will need async sooner or later.

Async's overhead

In all these cases, async will have a toll on your application. Depending on the situation, Async will have (I estimate) between 0.5 and 10% overhead. Futures need to store their completed result for example. This needs to be allocated and deallocated. There's a lot extra going on.

For small applications, this will make async very unattractive. Async is an overhead that you don't need. It costs extra time to understand, and it can complicate the API a bit more, making the entry barrier a bit higher. The entry barrier is probably the only valid point I made here, because small applications aren't going to notice the performance overhead, even if it's 20%. Once you get bigger, and get tens to hundreds of concurrent users, you'll start feeling the impact of async a bit more. 200 users would now be 210-220. But let's take a moment to look at what a synchronous codebase would feel. Synchronous code would make 200 users feel like 200 users. One difference, though. Whilst you're processing 200 Database calls and API calls to services like mailgun and stripe, you're having to wait for the result of the database, mailgun and stripe. Assuming 50ms per call to JSON APIs and 5ms for a database call, your request now has 105ms of overhead. At a 4-core machine, this limits your application to 4000ms / 105ms in requests per second, excluding the HTTP and other processing cost. Limiting your application to ~38 requests/second for a 200 requests/second application.

[calebkleveter]

Wow, interesting stuff @Joannis. But now I am confused. From reading your comment, it sounds like:

1. After adding async to Vapor, it will no longer the the framework we know and love

2. It will make the application both slower and faster 😕

   Once you get bigger, and get tens to hundreds of concurrent users, you'll start feeling the impact of async a bit more. 200 users would now be 210-220.

3. It will be bad for the OS community to have another async library available, but Vapor needs one that is specifically designed for it.

   Developing our own async ecosystem will also be yet another competitor to those already established libs. That's could be bad for everyone involved.

I am pretty sure that 2 - 3 of these points are incorrect, so could you expand a bit, or did I miss something that you already said?

[Joannis]

1. That statement really depends on what you "know and love". We're adding codable support, which removes a huge amount of bloat from Fluent, JSON etc. We're changing a lot of things that differ it from Vapor 2, which we believe are for the better. Routing is still the same/similar.
2. It'll have more performance overhead, but that pays itself back when you have concurrent clients (which is when performance really matters). So the overhead is there, but it's much faster. Besides that, vapor 3 is among the go frameworks in performance, so no worries!
3. It'll be bad for the async ecosystem. My conclusion Is that vapor 3 async needs to be minimal and protocol oriented. That way it won't be (as) segregated.

More specifically about point 1: The primary change you'll notice that isn't for the better of everyone is async. Async is going to be impacting operations such as database queries and API calls primarily and almost exclusively. The internals are async, but you almost certainly won't use that directly.

[jdmcd]

@mluisbrown Thanks for clearing that up! @RuiAAPeres My points still hold true for ReactiveSwift as well :)

[tanner0101]

After adding async to Vapor, it will no longer the the framework we know and love

@calebkleveter Not true. Vapor 3 kicks ass. You will love it! :)

It will make the application both slower and faster

@calebkleveter Concurrency > speed. Would you rather respond to 100 requests in 1ms or respond to 1,000,000 requests in 1.1ms?

This is my main issue with the whole thread. You can't model a stream of events with a Promise/Future.

@RuiAAPeres Totally agree. Not sure where that idea came from.

I wish you good luck, in all honesty. As the current maintainer of RxJava puts it: "you must be crazy to do your own FRP implementation" ( I am just paraphrasing him).
If the idea is to use Promises/Futures (again, remember the idea of the promise once fulfilled it's done) you could potentially use something like PromiseKit, which has been around for some years now.

@RuiAAPeres The existing promise implementations have more overhead than we need for Vapor 3 (from what i can tell at least, I'd be more than happy to be proven wrong here). Since we're using one event loop (serial dispatch queue) per connection, we don't need to introduce any extraneous dispatch calls to ensure internal or external thread safety. There's a framework wide contract that you must complete a promise from the thread it was created on. The same contract doesn't apply to iOS where threads are cheap and abundant. Subtle differences like that have an enormous impact on performance as a server (for example, promises creating dispatch queues would murder performance). Server-side Swift needs a promise implementation that caters to its unique needs and that's what we're trying to create. (Again, I do believe the Swift Server APIs group is going to release an Apple-backed promise implementation soon. Moreover, from what I have heard it will look similar to what we've created. If/when they release that, Vapor will obviously switch over.)

As for the reactive pattern, my hope is that Vapor 3 allows people to _add_ reactive on top. I assume this must be possible if people do reactive iOS using UIKit which is closure based. Maybe we would commit to an async pattern like reactive in the future, or release a spin off framework that features it, if we see a lot of people liking it. But we still, as a community, have a lot of learning to do in terms of using async/nonblocking swift on the server. Things that work great on iOS fall flat on their face when applied in the context of a server.

That said, I really hope @RuiAAPeres and all other reactive gurus will help us forge that path and make SSS great!

[calebkleveter]

What if we used something like Hydra?

I thought I would just throw it out there, though it might be a bit late at this point.

[Joannis]

The main two goals we set for Vapor 3 with regards to async:

• Tasks must stay within the same DispatchQueue. Especially promises, streams etc
• DispatchQueues are wrapped in a Worker that can carry a context.

However, Vapor 3's current async library is mostly protocol oriented. It would be possible to link Hydra and other reactive/async libraries to the Vapor async library by means of extensions.

[0xTim]

Hydra has also never been tested on Linux (the allTests it is referring to in LinuxMain.swift in the test case doesn't exist) which would make me uneasy about taking it and making it so fundamental to Vapor

[Joannis]

Should an error in a stream in addition to emitting an error notification also notify closing? Or should an error infer that the stream is closed (keeping them separate)?

[tanner0101]

@Joannis the stream should remain open even if it receives an error. if you want the stream to close, add a call to .close() in the error handler.

[Joannis]

I'll explicitly document that intention, then.

[Joannis]

Since the alphas hit we're starting to improve Async on all sides. Everyone's feedback is greatly appreciated. The discussions and feedback in this thread and on the alpha has helped us a lot. Thanks for your feedback!

I'm closing this issue since the initial discussion I prompted is now partially implemented and partially outdated.