I think that Middleware as it is right now is very complex and confusing to new users. I propose that we deprecate the current method of creating Middleware and instead have Middleware be a struct that is generic over the type of your state and provides functions such as map, filter and flatMap.
Here is what I've written for ReactiveReSwift
Here's an example of how you'd use it:
let middleware = Middleware<AppState>().sideEffect { getState, dispatch, action in // you can only dispatch new actions from `sideEffect`
print(action)
dispatch(PopupAction(description: "new action"))
}.map { getState, action in // `map` cannot return `nil`
return action
}.flatMap { getState, action in // `flatMap` can return `nil` to drop the action before it reaches the store
if action is PopupAction {
show(action.description)
return nil
} else {
return action
}
}.filter { getState, action in
return action is AppAction
}
If you create multiple Middleware instead of using transformative functions you can then use Middleware(firstMiddleware, secondMiddleware, [...]) to instantiate a single Middleware from multiple.
@Qata there are certainly some interesting ideas in here! Thanks for sharing. One big downside I'm seeing is that the middleware now needs to carry a generic type argument that matches the one of the store. This makes sharing middleware between different projects difficult. This is one of the main reasons we decided not to make the middleware generic.
I'll take a closer look as I go on vacation during the end of the year :)
@Qata I also renamed the issue so that the title is closer to your suggested changes :)
You can still share this Middleware between projects with a generic type, all that's required is that the type it's generic over conforms to StateType.
The current method of creating middleware is not friendly at all and there's no way I'd be able to type it from memory, it was a barrier to entry for me when I first started with ReSwft and I'd like to see it made better.
I also want to point out that my implementation using a generic type is incidental and many benefits could still be reaped even if you change the GetState function to () -> StateType and require casting of the state.
@Qata that makes sense. By removing the generic type argument we could indeed share Middleware between different projects. I agree that the API looks a lot more intuitive than what we currently have, especially to Swift developers.
IMO I'd like to move forward using this, if it's OK to drop the generic type argument for Middleware sharing. @agentk @DivineDominion what are your thoughts?
@Qata if we use this new API, could you see if the current Middleware in the tests can be implemented in terms of this new API? I strongly assume so, but it would be an important aspect to show people how to migrate from the old to the new API.
@Ben-G Yeah the current Middleware tests just need to be converted (see here for how I did so for ReactiveReSwift)
Okay so I started work not long ago and I've converted the project to use the new Middleware on my fork. The only problem is that the ability to return Any is incompatible with the assurance that the next function will be passed an Action.
To go ahead with these changes we'd need to remove the ability to return from the dispatch(_:) function (which to be honest I do not like as a feature).
The alternative is modifying the code so that you can still return Any, I do have an idea about how this can be done but it just rubs me the wrong way.
@Ben-G @agentk @DivineDominion
@Qata I understand that it's not typical for Swift to work with the Any type instead of a well defined one. However, it's important for a Redux-like library as this allows to use Middleware to enhance the store in ways that is not possible without an arbitrary return type. In Redux all async middleware relies on this by returning some sort of promise or signal from the dispatch function.
Some of the ideas in Redux just fundamentally require dynamic typing. So for now I would be in favor of keeping the Any return type.
ReSwift is its own library with its own documentation, and Swift is a very different language to JavaScript. Adhering to Redux as a blueprint rather than a guideline will end up with sub optimal code that is unfamiliar and unintuitive to Swift developers.
I am torn. And I cannot add much practical value to the discussion, only opinion and principle-driven argument.
Thunks are convenient, but at what price? Will replacing thunks with subscribers that asynchronously dispatch results make things so much worse? Does introducing thunks make things so much better?
The most sophisticated middleware I ever wrote is a wrapper of actions into undoable actions before continuing the dispatch process. It does not performing unexpected side-effects or branch off at all. When I saw what thunks can do, I was flabberghasted -- and immediately resisted their design. Lately, I tried to utilize thunks, but found I still didn't need thm. So I cannot really weigh the pros and cons. (Or, put differently, don't see the point of utilizing this feature :))
I have avoided the return value of dispatch like the plague because I'm very afraid of mixing commands with queries I like to separate concerns on all levels of granularity. It's not obvious what the return value of a command reading "dispatch event" would be. In terms of 20th century C-programming, a success boolean could be expected, but nope :)
Now the Any return type is very honest: you, the reader, have to go hunt for info in the app's code. With vanilla ReSwift, you can ignore this altogether because it merely returns the action that's being dispatched -- and which is known at call-site anyway. (I find this to be completely useless, in fact.)
I use vanilla ReSwift as an example of modeling data and information flow in an application. With this approach, you can design bottlenecks of input (events, which are handled by well-defined reducers) and output (state updates, transformed by subscribers for UI and network requests etc.) that make maintenance easier. ReSwift is just supplying a few convenient types, but you could realize the same design without ReSwift through proper separation of processes and architecture. I cannot come up with any good reason to make this worse.
If I had to design this library from scratch, I'd write dispatch to return Void by default.
I also don't like that middlewares can affect the return type. Thunks and sagas look magical at first sight, and that's the very problem: they utilize a simple mechanism of the store and affect the outcome. That makes reasoning about what's happening harder. You have to know that dispatching a ThunkAction will do something different. It'll be swallowed by a middleware.
Essentially, you're putting a long-running async process on a global queue. You may return a Promise and work with that. The sole benefit I can see of handing such a closure to store.dispatch is that you don't have to worry about who keeps strong references to the closure. The process can access the current state of the store -- at different points in time! So calling it early on may result in a while calling it when a long-running background fetch finishes results in b. That, in turn, defeats the whole purpose of uni-directionality. In a small place in your app, you hijack the store's functionality without _intending_ to reach the reducers or subscribers. Then you may ignore where current state info is _intended_ to come from (subscriptions) and dispatch success/failure actions back to the happy land of ReSwift.
All this I will avoid. It defeats the purpose of uni-directional data flow. It's a "secret weapon" people may pick up but it's not part of my agenda.
So the sole purpose of the return value of Store.dispatch is to be changed by middlewares. It's a design compromise favoring flexibility. It's pure forward-compatibility, because thunks and sagas aren't even part of ReSwift itself.
As a compromise, maybe it pays off to find a flexible common base where a non-returning DispatchFunctionType = (Action) -> Void is the default but can be exchanged with a (Action) -> Any variant.
| Keep it | Remove it |
| --- | --- |
| Breaking API change. | Easier to understand API. |
| Discardable return values didn't harm my apps at all. I just ignored that feature | Prevents weird code paths. Opinionated and educational: If you want to write "crappy" code, you have to excert more effort to bend ReSwift to your will. |
| 3rd party libraries can do more fun async stuff with dispatch. | Void is true to the principle of uni-directional flow. It's not just there to satisfy 3rd party library requirements. |
I wouldn't have added this feature in the beginning, but now that it's here, I don't see pressing reasons why we have to remove it. Then again, from a principle-based standpoint, "this may be useful for 3rd party libs" is not the best pro-argument either.
@Qata @DivineDominion I thought about this more over the weekend, and your arguments definitely make sense. As far as I know there are very little and only experimental middlewares out there that rely on the dispatch return value. In general I'm more than happy to move away from Redux-style APIs where it makes sense; but I also like the extensibility that Redux provides. Though in this particular case the potential usage of the dispatch return value is very limited.
I think it makes sense to remove this API for now. I would like to revisit a potential new API if/when we explore other ways of allowing plugins to provide APIs for dispatching + waiting for async response. Even though this feature should be used as seldom as possible, it is necessary in some cases.
For now we have the AsyncActionCreator API for that (which we likely want to revisit as well). As long as we provide one built-in solution I think it's a good idea to move forward with this approach.
@agentk what are your thoughts?
I ended up having a long conversation with some frontend devs - a couple just learning Redux/React and a one with a lot of experience with Redux.
The 'loudest' question the new devs had is - Why do we need so many f*%ing packages on top of Redux? Why do we need redux-thunk, redux-promise and redux-saga? Arn't we just logging events?
And they are completely right.
The problem as @DivineDominion pointed out, is that people are using Redux as a command bus. The whole purpose of UDSF (used here to refer to the pattern of Unidirectional State Flow as opposed to libraries) is to derive State from a transaction log of events. Which are intended to be taken for granted that they are a record of things that have happened in the past, and as such should never change (unless you actually have a time machine).
Plugins for Redux have been adding functionality around UDSF, such as our AsyncActionCreators because the outcomes of those Commands have direct influences on the State with the Events they generate. The two have become so blurred in the Redux world that there is no clear separation anymore in almost all JS projects that use Redux. They are all just a tradeoff of useful functionality and clear separations.
The only time I have ever seen it useful for middleware to modify actions or dispatch to return an action, is when ReSwift is treated as a CommandBus.
I guess I'm asking if anybody has good usage examples of dispatch return values and middleware action interference within strict UDSF?
It would make usage a lot clearer if Middleware was renamed to CommandBus and separated out of the Store.
TL;DR IMO dispatch should return Void, and I would be in favour of the Middleware from ReactiveReSwift if it was renamed to CommandBus and separated out into it's own Store.
As I said, I'm all for returning Void from dispatch. Regarding strict UDSF middlewares, what about action transformations? I'm using middlewares to map some events to another event type (the undoable-stuff from above) and to print an event log. I can put both into the reducer as a "preamble," but I do see value in injecting middleware to e.g. add filters to incoming events before they reach the reducer. A subscriber cannot log the event itself, only the result. In general, transformations like map and filter are good candidates for middleware.
Sorry for slow responses, traveling for the holidays.
TL;DR IMO dispatch should return Void, and I would be in favour of the Middleware from ReactiveReSwift if it was renamed to CommandBus and separated out into it's own Store.
I think we all agree on Void return type 馃憤 What are you referring to when you say separating the Middleware into it's own store?
IMO we would be fine with changing the existing Middleware to the one with a Void return type?
I'll open a separate PR to turn dispatch to Void.
I think I'm a bit confused as to why the generic middleware has been rejected, I could just be missing something.
@Qata: You can still share this Middleware between projects with a generic type, all that's required is that the type it's generic over conforms to StateType.
@Ben-G: @Qata that makes sense. By removing the generic type argument we could indeed share Middleware between different projects. I agree that the API looks a lot more intuitive than what we currently have, especially to Swift developers.
Does @Qata's comment above here not make one able to leave the generic type in? Requirements for a given middleware can be given via protocol conformance. I've built an example here using a bit of a fake, function-style middleware
// Middleware implementation
protocol HasString {
var string: String
}
func myShareableMiddleware<T: StateType>(state: T) where T: HasString {
print("string: \(state.string)")
}
// ReSwift Store
class Store<T: StateType> {
typealias Middleware = (T) -> Void
let middleware: [Middleware]
init(middleware: [Middleware]) {
self.middleware = middleware
}
}
// Usage:
struct FakeState: StateType {
var string: String
}
extension FakeState: HasString {}
_ = TestStore<FakeState>(
middleware: [
myShareableMiddleware,
]
)
In the example here, myShareableMiddleware can be shared as long as the user's StateType conforming object also conforms to HasString
This gives us strict compile type assurances of what the middleware needs from the state, while still allowing sharing.
Middleware that doesn't need anything in the state can omit the where T: XYZ portion from its definition to work with all stores.
Note: I'm not sure initially how this would work with struct-style middleware
@Ben-G I think @mjarvis brings up some good points, what do you think?
@Qata @mjarvis I think you are right that the issue of sharing could be solved by adding a generic type argument that constrains to StateType. My original concern was that many middleware implementations probably don't care about the state type at all (e.g. a middleware that just wants to log a message for every action) - so it felt that requiring to specify the state type is at least confusing. But I haven't spent a long time thinking about this.
I'd like to try this out in a playground to see whether or not the generic type is an actual issue.
@mjarvis @Qata I briefly looked into this today. The problem with the struct solution is that it requires a concrete type upon instantiation, a protocol, such as StateType cannot be used:

This means that my original concern is still valid with the current API + generics - we would need to tie a Middleware to a specific state type when defining it. @mjarvis your solution works since you're defining the function, but are not invoking it, and thus you don't need to provide a specific type.
This means we either need to go without the generic type argument or need to change the API to not require instantiating the Middleware type...
Yes, Swift doesn't support using protocols for generics in that fashion.
We'd have to figure out a way such that middleware isn't assigned to anything until its being used in a project, which the struct-based middleware doesn't really support. We'd have to be functional in order to get the generics.
To be completely honest, I'm not a huge fan of the struct-based middleware described here. I don't think it fits in with the (mostly) functional style of the rest of ReSwift. It also creates an odd setup for shared middleware, where the provider framework gives the user an actual instance of the middleware to use (as opposed to an interface for them to instance themselves).
I do see the advantages of splitting up different types of middleware for simple cases, though I also find it a bit more confusing for middleware that needs to do multiple steps.
Simplified, all that this is doing, is providing smaller interfaces to (GetState, DispatchFunction, Action) -> [Action], which is very close to the original middleware.
Can we do that in a different manner to make things easier for users? IE better documentation showing examples of the different middleware types (IE: Map example, filter example, etc)
@Qata Any thoughts on this?
The recreation of Middleware as a struct is intentionally close to the behaviour of the old Middleware, just more explicit and cleaner. I also don't understand the criticism that it doesn't fit in with being functional, whereas the whole point is that it's using functional concepts like map and filter.
It uses functional concepts [map, filter,etc] (which I like), but its not functional in that the base middleware is a stored variable (initialized struct), instead of being a function itself.
For instance, composing middleware is composition of multiple instances of Middleware, instead of composition of functions themselves.
I'm not saying this is a bad idea, generally speaking. I just think, personally, that I'd prefer having Generics for the state on middleware over these map/reduce/filter separations.
Unrelated to the above conversation:
How does filter work when crossing multiple middleware? Does it filter just for this middleware? or will it stop all other actions from reaching the store entirely? It feels to me like these functions should affect only my middleware.
The way this is structured, it makes me want to do filter({ ... }).sideEffect({ ... }) in my middleware, but looking closely I'm now aware that that will drop other actions from reaching other middleware / reducers.
It seems to me that this restricts a single middleware to only using one function, as its not really chaining effects WITHIN the middleware, its actually chaining Middleware itself.
Maybe I'm completely mis-understanding how this works. Heres an example:
let middleware = Middleware().filter { getState, action in
return action is AppAction
}.map { getState, action in // `map` cannot return `nil`
return action
}
is this equivalent to this?
let middleware = Middleware().filter { getState, action in
return action is AppAction
}
let middleware2 = Middleware().map { getState, action in // `map` cannot return `nil`
return action
}
let combined = Middleware(middleware, middleware2)
P.S: Sorry if the amount of feedback I'm giving here is overwhelming.
In this case Middleware is behaving as a Monad, the value itself isn't a function but it provides many functions to work with the value. This is no different from Optional or Array, both are values. The difference here is that instead of the Middleware containing a list of items or a possible item, it contains a composed function. It's very similar to the Reader Monad from Haskell, which is actually where my inspiration came from.
You make a good point about filter. I might actually remove the initialiser that lets you combine multiple Middleware since it works by combining them left to right, so middleware2 wouldn't receive any actions that aren't AppAction. Instead of using a initialiser, using an explicit concat like middleware1.concat(middleware2) would show this intent with much more clarity and allow you to think carefully about how you want your Middleware to be structured.
Having multiple Middleware that all run independently of each other wouldn't be feasible, since you'd have to deal with all of them each making an action and I don't see how you'd pick which is the "correct" action. They need to be chained so that all values produced are intentional.
@Ben-G What do you think of removing Middleware(_ middleware: Middleware...) in favour of using explicit calls to .concat(_)?
.filter().map().sideEffect()) to make up one middleware, but in actuality, each function is a separate middleware. It makes me want to use it incorrectly, without understanding that filter for instance will completely drop other actions from reaching the reducers, not just filter for my remaining work. I think I'd prefer this if there were no chaining at all.What if we could do something like allowing only a single function per-middleware? This makes it immediately obvious that each action needs to be stand-alone, and its affect is global, not just local to the middleware.
The immediate difference here is no function chaining (but personally I think its better without, one can declare a helper that lets people who understand have full control)
let middlewareOne = Middleware.filter { $0 is AppAction }
let middlewareTwo = Middleware.sideEffect { getState, dispatch, action in ... }
Using this sort of manner, we can get the benefit of individual functions (sideEffect, map, etc), while still getting generic state types, supporting protocols.
I do apologize, as this is fairly long, but it is a working example of generic middleware, with helper functions available for easier creation of the middleware:
// ReSwift Internals
public protocol StateType {}
public protocol Action {}
public class Store<S: StateType> {
public typealias DispatchFunction = (Action) -> Void
public typealias GetState = () -> S
public typealias Middleware = (@escaping GetState, @escaping DispatchFunction) -> (@escaping DispatchFunction) -> DispatchFunction
public var middleware: [Middleware] = []
}
public struct MiddlewareHelpers<S: StateType> {
public static func sideEffect(_ effect: @escaping (Store<S>.GetState, Store<S>.DispatchFunction, Action) -> Void) -> Store<S>.Middleware {
return { getState, dispatch in
return { next in
return { action in
effect(getState, dispatch, action)
next(action)
}
}
}
}
}
// Example state / actions for testing
struct ActionOne: Action {}
struct ActionTwo: Action {}
struct MockState: StateType, HasSome {
var some: Int
}
protocol HasSome {
var some: Int { get }
}
// Generic middleware that runs a side effect
func middlewareOne<S: StateType>() -> Store<S>.Middleware where S: HasSome { // or <S: StateType & HasSome> works as well
return MiddlewareHelpers.sideEffect { getState, dispatch, action in
if getState().some == 5 {
dispatch(ActionOne())
}
}
}
// Adding generic middleware to store with more specific state type
let store = Store<MockState>()
store.middleware.append(middlewareOne())
// Testing the usage of the middleware:
let getState = { MockState(some: 5) }
let dispatch = { (action: Action) -> Void in
print("dispatch")
dump(action)
}
let reducer = { (action: Action) -> Void in
print("reducer")
dump(action)
}
middlewareOne()(getState, dispatch)(reducer)(ActionTwo())
Disclaimer: I'm still probably not getting what you folks use Middlewares for. You may want to tell me to shut up. I'd be fine with that :)
@Qata's approach reads nice but I don't see the point of chaining Middlewares to perform tasks like map and filter. To me, a Middleware is a _service._ It looks like here's a missed opportunity to extract a new concept.
Again, I don't know what you want to achieve. As a mere beholder, I'd favor a split into Middleware and SideEffect where a Middleware is a function that you can hook into the process to mess with the data flow -- and where some may emit SideEffects that are handled orthogonal to the action reducing sequence, no matter if they are consuming the incoming action or not.
When you start executing a SideEffect, it's only accidental that ReSwift will have to handle the result in the end. A simple side effect may be a network request dispatching the result as an action again. So passing the dispatch function (or the store instance?) to a SideEffect as parameter makes sense. Plugging it into the return action-chain, not so much.
Which makes me ponder the use of Middleware again.
A good use case is a logging middleware. It's pluggable and doesn't mess with your app. A DropEveryThirdActionMiddleware is not only pointless but if you plug it in, it completely alters what your app will be doing. (I think this is weird but I still would not want to restricting the power of Middlewares to keep flexibility high.)
I don't agree with the use of thunk/saga for reasons stated in the Gitter chat some time ago. But thunks/sagas and Middlewares that do a lot of stuff indicate to me that it's too painful to write simple and focused StoreSubscribers.
I'm testing an approach to performing file changes in my app. It doesn't scale beautifully, but it kinda works:
PendingFileChange queue each that's part of the app stateCompletingFileRenaming and similarThis is a lot of boilerplate for a rather simple side effect.
It seems what you crazy people would want to do:
RenamingFileAction to a SideEffect instance)@DivineDominion Here are some examples of real middleware I'm using: (simplified)
NetworkRequestStart action, runs a side effect (alamofire), dispatches NetworkRequestComplete or NetworkRequestFail appropriately.NetworkRequestStart action. Checks state to see if request has already been successfully ran. If so, drops the action. (since the data exists in state already)(1) is a side effect, but doesn't do any work back to the store
(2) is a side effect, and requires dispatch access to submit results
(3) drops certain actions (relies on being executed before the networking middleware) [unwritten, might be integrated with the networking middleware]
Nice idea, @Qata, however I'd rather keep ReSwift dead-simple as well as inlined to the original redux implementation - in this way newcomers can easily use redux docs as an additional source. I'm totally agree that middlewares are hard to understand but when I tried to implement the same idea on my own I quickly get to the exact state where the ReSwift is :) I believe that a nice guide (step by step Middleware impl) gonna help users to get a deeper understanding of the underlying motivations behind the Middleware and why it looks so strange. What do you think?
@dimazen I don't agree. Having to teach newcomers how to use the current implementation isn't "dead-simple", it's an obvious flaw in the design. Instead of having it look strange, you can instead stop having it look strange and that's what I'm trying to achieve.
I went and implemented a Middleware based on the tried and tested formula of the Reader monad from Haskell, it's not like this paradigm hasn't existed for a while to solve problems very similar to this one.
@Qata Thanks for the explanation. Now your motivation looks clearer to me. Also I wasn't aware of the Reader monad (shame on me - gonna check it).
Going back to my learning process - it wasn't that easy to understand how to use middlewares, I totally agree that it is a bad sign. So your point does make a lot of sense to me.
馃帀 So, thanks to https://gitter.im/ReSwift/public?at=58be2a93872fc8ce62c73302 I figured out a way to get the things I was looking for while maintaining @Qata's separation of map / sideEffect, etc.
Heres an example implementation:
struct AppState: StateType, HasFoo, HasBar {
var foo: String = "Foo"
var bar: String = "Bar"
}
protocol HasFoo {
var foo: String { get }
}
protocol HasBar {
var bar: String { get }
}
func fooMiddleware<S: StateType & HasFoo>() -> Middleware<S> {
return Middleware<S>().sideEffect { getState, dispatch, action in
let state = getState()
print(state.foo)
}
}
func barMiddleware<S: StateType & HasBar>() -> Middleware<S> {
return Middleware<S>().sideEffect { getState, dispatch, action in
let state = getState()
print(state.bar)
}
}
let store = Store<AppState>(
middleware: [
fooMiddleware,
barMiddleware,
]
)
And heres a playground with this working:
https://gist.github.com/mjarvis/d501b49ecd231d0ca8aa1df778f12b59
This is nearly identical to @Qata's implementation. I've removed the concat & init(first, rest) as middleware is now combined how it used to be during store init, and I made the state type generic. Otherwise the effect functions are literal copy-paste from #188 .
So something I noticed while working on my last comment is that this middleware is missing something very important that I use a lot: The ability to hook middleware after the reducers.
For example, in the existing implementation we can do this:
next(action)
// ...
dispatch?(OtherAction())
This allows one to have conditionals in middleware based on the change in state.
(I currently do some form of this in 4 of the 11 middleware I'm using at the moment)
Never thought about this. What are the 4 use cases?
Also, the change looks cool. 馃憤 Have to look at it in more depth later!
@DivineDominion Never thought about this. What are the 4 use cases?
@mjarvis Are there any suggestions you have to @Qata's approach that would enable your use cases? I'm thinking we should cut a new release (4.0.0) soon and I was wondering if the middleware changes could make it into it or not.
I think, in the short term, if you want to cut a new release, the only change to middleware should be to make it state-generic. I think while this idea has merit, it requires some more work to flesh out the edge case usages of middleware.
Generic State in Middleware: https://github.com/ReSwift/ReSwift/pull/226
I seem to be having an issue with that compiling in Xcode 8.2, though... Not sure why, it locks up the compiler & indexing.
@mjarvis Wanted to ask you for an example - but you beat me to the punch!
I wrote tests for my middleware the other day, and ...
let result = filterSearchMiddleware(irrelevantDispatch, irrelevantGetState)(nextDouble.next)(action)
... it isn't pretty :)
What do we need to do to move this forward? It totally disappeared from my radar. As far as I'm concerned, if it compiles and is easier to understand and easier to write than triple nested curried functions, let's merge #226 into a dev branch and play with it.
Most helpful comment
ReSwift is its own library with its own documentation, and Swift is a very different language to JavaScript. Adhering to Redux as a blueprint rather than a guideline will end up with sub optimal code that is unfamiliar and unintuitive to Swift developers.