Syndesis: Implement a Redux architecture in Syndesis UI

Created on 17 Nov 2017 · 10Comments · Source: syndesisio/syndesis

In order to make state management easier in Syndesis UI, while reducing the code boilerplate existing on each component to handle state and digest data back to the data layers on each UI interaction, embracing an architectural pattern such as Redux seems the way to go. This would allow for leaner component and service implementations, less code and a centralized source of truth for all things that impact the overall application state.

After reviewing the entire codebase, I finally managed to come up with a plan for implementing Redux-driven stores in the application, leveraging the excellent @ngrx library (another monorepo, yay!), which has become a de-facto standard in the Angular arena. My major concerns here were, bearing in mind that this is a live application that we still need to grow and maintain in the interim, and that most of those changes tap into architectural calls, how to make everything work while we keep building other features and no feature toggles support is available. The idea would be to build the foundation for Redux in a way that operates in parallel to the rest of the app and then go switching the state implementation for each group of component groups, one group at a time. I figured out the steps required and put them together in an issues list named _The Path to a Redux Architecture_, (yes, it's a cheesy name and I'll burn in hell for the rest of my life because of this).

The Path to a Redux Architecture (15 issues)

Build our own HttpClient wrapper + tests
Strip WebSocket Client implementation out from store/entity/events.service to its own service provider + tests
Strip EventSource Client implementation out from store/entity/events.service to its own service provider + tests
Implement EventServiceFactory wrapping our new WebSocketService and EventSourceService and expose it as an Observable events emitter from @syndesis/ui/core (read below about _@syndesis/ui packages_)
Implement support for @ngrx v4.* plus Side Effect Events modeller and Store Dev Tools.
Port ACTIONS state management to a local store, managed by a set of typed Actions handled by reducers.
Wire all events emitted by EventService and all HTTP calls required by ACTIONS into streams piped to middleware Effects triggering Redux actions.
Hook up all ACTIONS components with data selectors pointing to the Store. Bind UI events to action dispatchers.
Repeat the above steps with CONNECTIONS
Repeat the above steps with CONNECTORS
Repeat the above steps with ENTITIES
Repeat the above steps with INTEGRATIONS
Repeat the above steps with OAuth USER GRAPH
Phase out Restangular in favor of our own HttpClient wrapper
Phase out all entities in src/app/store in favor of our new implementations for Effects-driven HTTP calls

While this is pretty ideal, problem is that Redux requires A LOT of new files to be created per each entity, and in order to swap references later seamlessly we need a better project layout in regards of filesystem design, folder names and locations, etc. I also need a better tooling to track down errors and debug stuff and this makes a good case to centralize settings ina centralized service so we can strip URLs and stuff out from services and components. So I put together all the steps required and defined a set of _packages_ that will summarize the application architecture, just by embracing the industry conventions for large Angular applications. For the record, this is not a big application, but it will likely become one upon time. Once this architecture is implemented, the different packages will allow for better refactoring operations and will help repurposing our components' data implementation more easily. The resulting to-do list looks as follows:

The Path to an Angular 5/6 Architecture (12 Issues)

Upgrade Angular to v5.x and Angular CLI to v1.5.x
Create @syndesis/ui/* alias as a symlink to ui/src/app/* to better manage code refactorings and module reshuffling based on façades (this is done from tsconfig.json).
Wrap aplication models, Redux constructs (when available) and abstract service classes into separate contexts exposed from @syndesis/ui/platform
Wrap common-use components, directives and pipes in a new SharedModule exposed from @syndesis/ui/shared
Wrap application-wide service providers mapped to platform abstract service classes into new CoreModule exposed from @syndesis/ui/core
Move connections, customizations, dashboard, integrations, settings (and templates?) to @syndesis/ui/web
Remove hardcoded references to services/URLs/ENV variable and expose it from a centralized settings service in exposed from @syndesis/ui/core
Implement custom global application error handler wired to Angular's ErrorHandler provider
Wrap vendor libraries in @syndesis/ui/vendor abstracting their APIs through our own façade (mostly logging operations).
Introduce yarn _DEBUG MODE_ command and log messages in console when triggered. Do not log anything otherwise
Rename files and folders according to Angular's Official Styleguide
OPTIONAL - Transition from Angular CLI to Webpack v3 + Rollup w/ AOT compilation support.

Summary

The idea here is to port the application to a proper Angular 5/6 environment first and then implement Redux afterwards, as a follow up EPIC.

Just to summarize, this is the final set of packages proposed:

@syndesis/ui/platform: Application model types, abstract services, Redux entities (when available), and routing guards.
@syndesis/ui/core: Actual implementations of platform abstract services as injectable providers.
@syndesis/ui/shared: Application shared components, directives and pipes, consumed from all routed modules, lazy loaded or not. Additionally, we will expose here all static functions such as application helpers and TypeScript decorators.
@syndesis/ui/web: All our UI components, grouped by contexts exposing a routed feature module each.
@syndesis/ui/vendor: All our 3rd party libraries, plus our own façade services for them,
@syndesis/ui/debug: Initially a no-op NgModule, but a future home for debug components and directives, and feature toggles.

cadiscussion catechdebt grouui

Source

deeleman

All 10 comments

@deeleman thanks a lot for this wonderful plan. I firmly believe we should not hesitate to make brave changes likes this if the result is a more flexible, maintainable and scalable (in size) application.

However, I'm really no Angular expert and have not seen more than an introductory presentation on Redux. So I'd like to defer the review of this roadmap to @gashcrumb , @kahboom and @seanforyou23 (and also about the benefits of this move).

As you have stated and assumed, the important thing for me is, that it is an incremental process that we can tackle in parallel the evolution of the application itself. If we come to the conclusion that these steps make sense (as it sounds to me), then the next question is _when_ to start.

The next big milestone for use is the TP3 release which will happen mid-January. My question is, if this migration is _not_ finished until then, is still possible to have a (technical preview level) stable UI ?

If this is at risk, we then should start with the beginning with TP4, but of course, I would like to start immediately to minimize the risk to bring it in before it is too late in the game.

rhuss on 17 Nov 2017

You did not disappoint :-) So 2 epics, support Angular 5/6 and then migrate to Redux architecture. Maybe actually what we should consider is breaking these up even further and see what we can align with the TP releases.

I'm +1 for all of this, but yeah the key bit is that we also have to continue to add features to the UI, so we need to be mindful that we can't disrupt new feature development.

In terms of supporting a move to Angular 5/6 these bullet items aren't necessarily in order, are they?

gashcrumb on 17 Nov 2017

🎉1

Create @syndesis/ui/* alias as a symlink to ui/src/app/* to better manage code refactorings and module reshuffling based on façades (this is done from tsconfig.json).

Good thing I moved off of Windows, I think this should be okay :-)

Wrap vendor libraries in @syndesis/ui/vendor abstracting their APIs through our own façade (mostly logging operations).

I definitely like the idea behind this one. Would we do this for third-party components or just APIs? The main one I have in mind is the dynamic forms library, we use the API from that. And then there's ng-patternfly which has components that we use and some API services, would we facade that stuff too?

Introduce yarn DEBUG MODE command and log messages in console when triggered. Do not log anything otherwise

+1 and I'm actually not a fan of our current logger, I kinda want something that's as easy to use as console.log

gashcrumb on 17 Nov 2017

😄1

The next big milestone for use is the TP3 release which will happen mid-January. My question is, if this migration is not finished until then, is still possible to have a (technical preview level) stable UI?

The answer is YES, definitely. In a nutshell, the issues depicted in the "Angular 5 migration" entail basically moving code files around and changing folder names and import lines to abstract its content so we can move things easier in the future. I can give a rough estimate of 2 weeks to get this done, stripping the migration to Webpack v3 our since it is not really necessary for this.

I would like to start immediately to minimize the risk to bring it in too late to the game.

Agreed, mostly because Angular versioning is an issue here due given its current roadmap. While support for our current version is guaranteed till Oct 2018, ensuring that we are in good state i regards of our dependencies for the next TP seems quite relevant.

Maybe actually what we should consider is breaking these up even further and see what we can align with the TP releases. [...] In terms of supporting a move to Angular 5/6 these bullet items aren't necessarily in order, are they?

Indeed. In fact, we can turn the Angular v5.x + CLI v1.5 migration into a standalone epic by itself. The directory structure is only required for a later stage. It is not an actual requirement for Angular 5, but more of a foundation for any future refactor (such as implementing Redux) and as such can be implemented later (the new CLI is required for it tho).

the key bit is that we also have to continue to add features to the UI, so we need to be mindful that we can't disrupt new feature development.

100% agreed. Work on Syndesis is focused mostly on components these days and in regards of components the only change required according to the proposed architecture is to move the component folders from ui/src/app to ui/src/app/web, which can be done in no time with barely no impact for the rest of the team.

The roadmap proposed does not entail refactoring the components or other code constructs (at least for the time being), just to repurpose the directory layout and replace file paths by abstracted source tokens, as the Angular core team does with Angular itself. The important part comes with @syndesis/ui/platform. Basically it will expose all the services currently located in store/entity for the components to consume. But those services will not be implemented there though, being mere abstract classes which will be later on implemented at @syndesis/ui/core by means of Angular's dynamic dependency injection machinery. This allows us to toggle and replace services (as a sort of feature toggle) while developing new ones and only swap those when we're done. The rest of the application is unaware of that change since it still consumes dependencies from @syndesis/ui/platform, which is the single source of truth for models and services.

This allows to easily parallelize application development and maintenance at the same time, releasing small chunks as we go.

Create /* alias as a symlink to ui/src/app/* to better manage code refactorings and module reshuffling based on façades (this is done from tsconfig.json).

We can actually do this now, but our current CLI version is a bit finicky when it comes to assign aliases. Basically we define an alias here and we're good to go.

Would we do this for third-party components or just APIs? The main one I have in mind is the dynamic forms library, we use the API from that. And then there's ng-patternfly which has components that we use and some API services, would we facade that stuff too?

Ideally, third party components should be distributed as npm packages, so there's not much we need to do in that end. For those components which come as raw files, this would be the place to be. The idea of a vendor package is to wrap 3rd party libraries in our own services and import those at once by means of a wrapper Angular module, so its internal implementation is opaque to the rest of the app. Then we can have 3rd party libraries (such as loggers or error handlers, UI trackers, analytics software, etc) fully abstracted therein. This way, if for whatever reason we decide to ditch the typescript-logging library or whatever, we will not have to update each and every of the 62 matches found for log.debugc('blah blah') in order to update the import line or the method signature. We will define our own opinionated logger API, implement whatever logger we fancy internally, and then reimplement any other in the future in a transparent way for the rest of the app. The final goal is to claim full ownership of all the APIs used by our own application, including those that rely on 3rd party libraries.

Introduce yarn DEBUG MODE command and log messages in console when triggered. Do not log anything otherwise

In fact I noticed we pollute the console with too many messages and this hinders debugging. My idea was to toggle console logging on and off depending on what are we after. The DEBUG MODE has other uses though, such as bootstrapping the app with certain ENV variables set somehow so our app reacts to that accordingly and shows certain messages or toggle certain features required for debugging, such as WebSocket emulators and stuff. I have not any particular idea in mind ATM, but I want to have the opportunity to toggle features from the CLI for debugging purposes.

deeleman on 17 Nov 2017

Just to summarize, this is the final set of packages proposed. I'll add this same information to the issue description for future readers:

@syndesis/ui/platform: Application model types, abstract services, Redux entities (when available), and routing guards.
@syndesis/ui/core: Actual implementations of platform abstract services as injectable providers.
@syndesis/ui/shared: Application shared components, directives and pipes, consumed from all routed modules, lazy loaded or not. Additionally, we will expose here all static functions such as application helpers and TypeScript decorators.
@syndesis/ui/web: All our UI components, grouped by contexts exposing a routed feature module each.
@syndesis/ui/vendor: All our 3rd party libraries, plus our own façade services for them,
@syndesis/ui/debug: Initially a no-op NgModule, but a future home for debug components and directives, and feature toggles.

deeleman on 17 Nov 2017

Definitely would like to get going on this straightway. We've next week in this current sprint and we'd like to get a lot of the API and tech extension stuff in place, not sure if it's safe to start on directory changes now or wait until the start of the next sprint. Seems like it'd be a quick thing, but then with the monorepo PRs are already a lot slower so I'm concerned about having some frightening merge conflicts to figure out potentially :-)

gashcrumb on 17 Nov 2017

also @kahboom feel free to chime in :-)

gashcrumb on 17 Nov 2017

We've next week in this current sprint and we'd like to get a lot of the API and tech extension stuff in place, not sure if it's safe to start on directory changes now or wait until the start of the next sprint.

I'd rather wait for next sprint for sure. In the meantime I can properly split down each todo list or epic into GH issues. I might night some guidance on labeling them tho.

Aside from that I would promote the Angular and CLI packages upgrade to their very own epic or milestone. Whatever course of direction we take, we will need the Angular packages to be up to date, no matter what. And I'd suggest to do this this week. Updating angular and the CLI to the next version should be transparent to the rest of the codebase.

deeleman on 20 Nov 2017

I'd rather wait for next sprint for sure. In the meantime I can properly split down each todo list or epic into GH issues. I might night some guidance on labeling them tho.

Perfect, +1 thanks!