Shadow-cljs: Support target for React Native

[DogLooksGood]

Hi, @thheller. Thanks for all your hard work on this wonderful project!

I test the usability with following setup. I can have HMR and REPL, source map is not support yet.
https://github.com/DogLooksGood/shadow-cljs-with-react-native-note

I think it will be awesome to have a react-native (or javascript-core ?) target.
The goal should be use without shim.

List the important steps here:

1. auto set :devtools to http://host-ip:9630
2. provide a client like shadow.cljs.devtools.client.browser, in this client, don't use js/document.
3. client auto connect in develop build.
4. some work to support source map.

[thheller]

I only played with react-native briefly but so far I'm thinking about

{:target :react-native
 :output-to "index.js"
 :entry some.ns}

Need to specify one entry/main namespace that should do the AppRegistry call. The REPL client and hot reloader could be injected before into that.

We can probably get away without the :devtools-url if I make the connect slightly smarter and try all NetworkIntefaces instead of only localhost.

Source maps will probably not work until metro supports it, see https://github.com/facebook/metro/issues/104.

Could maybe try to bypass metro completely and live in our own world instead. I think figwheel-bridge.js does something similar.

[DogLooksGood]

1. I think REPL client is needed and hot loader is not, since React Native has its own.

2. If there's a :output-to without output-dir, where are the rest compiled .cljs file?

3. I think the source map patch in re-natal is this: https://github.com/drapanjanas/re-natal/pull/102/files, No need to patch in shadow-cljs

[thheller]

@DogLooksGood source maps work with figwheel since the files are not processed by the react-native packager. no idea what that specific PR did but it definitely didn't "enable" anything.

:output-to since the release build will only be one file, where the rest ends up shouldn't matter as long as they can be loaded somehow. still figuring that out.

[idokutela]

Hi there,
I've been playing on my own with shadow-cljs and react-native. I thought you might be interested in my experiences. My goals are probably a little different to your example - first, I need the bare-bones react native, because I often need to interact with native modules, and second, I have existing react native projects I want to add some clojurescript magic too. After trying various things, I settled on the following:

• I made a little repo that patches the dependencies rum has to work in react native, as well as defines a module react-native/core which makes all the react-native components directly usable from rum.

• then, whenever I want to use clojurescript in a react-native project, I

  1. clone the repo into the root of the project,
  2. make a folder for my clojurescript source,

  3. Make shadow-cljs.edn as follows

     {:resource-paths ["cljs-src" "rum-native"]
     :dependencies [[rum "0.11.2"]]
     :bulds {:app {:target :npm-module :output-dir "lib"}}}
     

     The nice thing about this is that shadow-cljs automatically shadows rum's dependencies, so that it uses the monkeypatches in rum-native.

     I can then depend on the generated js files in lib. If I want to, I can use the utility to register the app that's in react-native/core, but if I already have a bunch of legacy javascript and I just want to add some stuff from the clojurescript repo, I can depend on it as needed.

I'm generally pleased with how nicely everything works together. In particular, shadow-cljs allows one to be minimally invasive, and just change things as one needs. Also, react-native's own hot reload is perfectly sufficient for smooth development.

However, there are a few pain points:

1. Unsurprisingly, the only build target I can get to work is :npm-module. This is ok, but it would be really nice if one could modify, say, the node-module build target to build a single output for use,
2. I have no idea how to get a repl to work: I imagine one ought to be able to monkeypatch the clojurescript browser repl to work, but I wouldn't even know where to start.

From my point of view there are two things a react-native target could do to make things nice:

1. most importantly, monkeypatch the repl so that one can use it inside the emulator,
2. bundle things similarly to node-library.

Having an :entry that automagically registers the component is not necessary, and could be outright annoying if one has some legacy code to interact with.

Anyway, those are my 2c.

[thheller]

The problem with :npm-module is that the files will not be loaded as they are but instead the react-native bundler will be in charge of loading them. This messes with things in breaking ways so it is pretty much impossible to guarantee that the REPL will actually work. This is also why source maps don't work.

re-natal works by using a figwheel-bridge.js which is basically a shim file that hides the actual CLJS output from metro. Instead metro will only load the bridge which will then load the CLJS files on its own. From what I have seen so far this seems to be the most reliable approach to get everything we need. It isn't perfect though as interop with JS gets harder this way.

In the meantime I could probably write a simplified REPL client that is not the browser client. But I have no idea what networking capabilities are available in react-native? Can I use WebSocket? Can I use npm "net" or "ws"?

[idokutela]

Thanks again for your quick response!

I was aware re-natal used figwheel bridge, and that was why sourcemaps worked, but I didn't realise that the metro bundler also broke any possiblilty of the repl working. I had naively thought that as long as one had websockets around, one could get the repl to run :/

Anyway, react native has websockets – https://facebook.github.io/react-native/docs/network.html#websocket-support and most other things you have in the browser (except the dom, of course). What it doesn't have is all the stuff one might have in node (fs, net, ...). One can always use the bridge to access native capabilities, but generally it's a pain best avoided. A rule of thumb might be: if you can do something in a webworker, you can probably do it in react native.

I actually quite like the idea of the original poster: defining a :javascript-core target that makes as few assumptions as possible about the environment, so that one can have a hope of using the code directly in any environment where javascript is embedded. Basically, javascript-core should generate node-like modules, but expect as little as possible outside standard ES-2015 (or 6), have some sort of simple repl possible, and make the outside dependencies clear so they can be polyfilled as necessary (WebSockets, presumably. Anything else?).

From my side: please don't spend too much time on this unless you wish to. Hot reload means one can build the UI components fairly ergonomically, and for everything else, one can just compile to a non-react-native target and enjoy the repl there.

[DogLooksGood]

React Native hot reloader will work without figwheel_bridge.js. The client for browser not work because it uses dom.

And the reason why source map not work may relative to the path, not sure for this. but the command enable-source-map in re-natal just simply patch the packager.

[thheller]

I did some basic work trying to create a target for :react-native which is available as of 2.2.30.

You can either use the :npm-module target with a new :runtime :react-native to inject the necessary things for the REPL. You must however load those manually since :npm-module has no single defined entry. Meaning that you need to require("./out/shadow.cljs.devtools.client.react_native") AND require("./out/your.app").

{:target :npm-module
 :runtime :react-native
 :entries [your.app]
 :output-dir "out"}

The goal was to create a dedicated :target :react-native and it sort of works but is nowhere near user-friendly.

{:target :react-native
  :entry your.app
  :output-to "out/index.js"}

Since this has a single entry the REPL code is automatically added. You require("./out/index") which would be the same as require("./out/your.app") only that it also does the REPL require.

I kinda got frustrated by create-react-native-app and expo so I stopped there. expo lets me change the package.json main to directly use out/index.js. create-react-native-app somehow ignores that and always tries to load a ./App.js which sort of wanted to avoid.

Anyways I'll probably look into this more once the frustration wears off.

[idokutela]

Hi there,

thanks for keeping looking at this!

I completely sympathise with your frustration – I constantly feel like I fight similar things when using react native, especially with clojurescript : a build tool on top of a framework on top of a build tool on top of a build tool. What could go wrong?! That's actually one of the main reasons I first looked at shadow-cljs: it was appealing to be able cut out some of the stuff hidden behind the shadows, and at least just build the clojurescript directly.

As far as I'm concerned, it would be super if you built to two files: one which contained the compiled and concatenated source, and another with the repl. It's really no annoyance to add a dev index.js to require both, and that way one can choose _not_ require the repl when one doesn't need it. Bundling all the non-repl code into one file is nice, though, because the metro bundler is slow, and the standard npm-module output leads to a _very_ slow initial metro bundle when one first runs react-native...

Indeed, I've been spending the last couple of days trying to figure out a decent workflow with shadow-cljs, and it seems actually to be pretty nice to have two targets: one building to react-native and running in the emulator, another building to node and shimming out the react-native deps, and doing most of the development there. The constant react-native rebuild and rebundle is annoying, and if you're running emulators you get your own frustrations: it turns out much quicker to do much of the work in a "headless" environment, and only when you actually do ui work, fire up the full react-native machine.

I actually came across a very weird issue today (I'm _almost_ certain it's not shadow's fault, but I might as well mention it, in case you think it is): when requiring react-native inside of clojurescript, everything works fine, except the "InputAccessoryView" disappears (in other words, (= nil (.-InputAccessoryView (js/require "react-native"))). If one makes a javascript file, and does the equivalent require require('react-native').InputAccessoryView, it's there! I assume it's some sort of shadowing going on, but I haven't been able to track it down. InputAccessoryView only got added pretty recently, and perhaps the require inside the compiled clojurescript is finding a different react-native. As I said, I don't think it has anything to do with shadow, but more likely with the metro bundler, but it's weird. Fortunately for me, I have no current need for InputAccessoryView, so no urgency in tracking down the cause…

Anyway, I'll have a look at what you've done when I next sit down to code. Thanks again: I love shadow.

[idokutela]

By the way, does shadow-cljs understand (js/require "xxx") when building to prod and preparing things for the clojure compiler? I found myself forced to use that, instead of (:require "xxx"), because one can't just require "react-native" inside node, and so had to conditionally require depending on the environment. My hack for now, is here: https://github.com/idokutela/reagent-native/blob/master/reagent_native/core.cljs -- if you have a look, you'll see I went to a little effort to make sure that things accessed are not mangled by clojure on prod. However, I haven't had a chance yet to make sure that works...

[thheller]

js/require is just a JS function call to shadow. It won't add anything to the build but when only targetting node or react-native that is not really a problem.

Not sure why require('react-native').InputAccessoryView doesn't work in JS. I still don't have the slightest clue what metro actually does since it never writes anything to disk or I can't find where to.

Emitting just a single file for development was actually the first thing I tried. It is very easy to do BUT the metro bundler really doesn't like one huge file. It took something like 3 minutes for it to load with every compile.

It seems to me that the best option really is to bypass the metro bundler as much as possible and do the same tricks re-natal does. That at least is fast and allows us to use the shadow reloading.

[idokutela]

Ah, ok. In that case lots of small files :) At least one then only has _one_ slow build (on my machine, about four minutes), but after that the incremental builds are fast.

[thheller]

Quick note: If you add a .babelrc to your :output-dir things get significantly faster with metro. For some reason this didn't work when using a single file though.

[idokutela]

Ah, thanks for the tip!

[thheller]

After I got completely frustrated with the expo and the react-native packager (again) I started investigating an alternative. The idea was to run react-native bundle ... to create an "offline" bundle. shadow-cljs then consumes this bundle and adds the CLJS code to it. This generated file is then served so that the Expo App can load it.

This works perfectly for my basic test app. REPL and live-reload both work as expected and flawlessy. Since the react-native packager or expo tools don't run at all they can't interfere with anything either. You can still require any npm package and those will be bundled by the react-native tool.

Just need to add source map support and some HTTP methods the expo/rn app seems to call (eg. /symbolicate). /logs was is sort of supported already.

My app is pretty limited though so it might totally blow up when you do actual stuff.

(ns demo.rn
  (:require
    ["expo" :as expo]
    ["react" :as react :rename {createElement $}]
    ["react-native" :as rn :refer (Text View)]
    ["create-react-class" :as crc]))

(js/console.log "foo10")

(defn render-root []
  ($ View nil
    ($ Text nil "Hello World from CLJS! 1")
    ($ Text nil "Hello World from CLJS! 2")
    ($ Text nil "Hello World from CLJS! 3")))

(defonce root-ref (atom nil))

(defn ^:dev/after-load re-render []
  (.forceUpdate @root-ref))

(defn init []
  (js/console.log "rn-init called")
  (let [Root
        (crc
          #js {:componentDidMount
               (fn []
                 (this-as this
                   (reset! root-ref this)))
               :render #(render-root)})]

    (expo/registerRootComponent Root)))

Config currently looks like this:

  {:target :expo
   :init-fn demo.rn/init
   :output-dir "out/TestCRNA/public"
   :devtools
   {:autoload true
    :http-port 19000
    :http-root "out/TestCRNA/public"
    :http-handler shadow.expo/ring}}

Basically I point the Expo app at exp://my-ip:19000 just like the exp or create-react-native-app scripts would.

Since I know nothing about either react-native or Expo I hacked something together with registerRootComponent. The re-render part could probably be nicer.

Is there perhaps an Expo CLJS demo app somewhere I could use for testing?

I'll publish what I have now later after cleaning up some hard coded IPs and such.

[tiensonqin]

I'm very excited on this!
I've never thought about consuming the react native bundle directly in CLJS, really awesome.

I tried to create an shadow-cljs + rn example using :target :react-native yesterday, it just works,
and as you said, add a .babelrc to the output-dir is much faster.

Such a pity that I didn't do too much rn development for a very long time (since lymchat), so I can't
provide any Expo CLJS demo apps.

I'm bootstrapping a new project now, the web app was using shadow-cljs since the start, and soon I can work on Expo with shadow-cljs.

Looking forward to it, really cool!

[DogLooksGood]

If use the offline bundle, is that we have to create a new bundle whenever we change our npm dependencies?

[thheller]

shadow-cljs runs react-native bundle whenever the JS deps change so you don't have to do anything manually.

Still need to figure out if its possible to do the Remote Debugging thing in Chrome somehow via shadow-cljs too. Seems pretty important but so far that seems to be the only thing missing.

Removed some more boilerplate code. This looks reasonable now.

(ns demo.rn
  (:require
    ["react" :as react :rename {createElement $}]
    ["react-native" :as rn :refer (Text View)]
    [shadow.expo :as expo]
    ))

(defn render-root []
  ($ View nil
    ($ Text nil "Hello World from CLJS! 1")
    ($ Text nil "Hello World from CLJS! 2")
    ($ Text nil "Hello World from CLJS! 3")))

(defn ^:dev/after-load start []
  (expo/render-root render-root))

(defn init []
  (start))

[thheller]

Alright the basics are working and I made a sample project.

https://github.com/shadow-cljs/expo-test

Only watch or compile work. release is not yet supported. Assets probably don't work correctly yet.

Anyways please try this and report if it actually works. Only tested this on my PC so it might not work at all anywhere else.

[idokutela]

That looks pretty awesome! Nice way to turn the build process around.

I wonder if one can do something similar with "vanilla" react-native (ie, without expo)? From my very basic understanding of how react works under the hood, it ought to, but I wouldn't know how to make it do so. It would be nice if it could: in my projects, I need some native modules, so expo is out of the question.

I think, though, that finding a solution that works well with create-react-app and expo is an excellent target: expo eases a lot of pain in the app development process, and create-react-app is, by now, the standard way to start a new project. If the solution that comes out also works with the vanilla react-native-cli, all the better.

[thheller]

but I wouldn't know how to make it do so.

Same here. In theory I see no reason why it wouldn't work. Just need a way to run bundle the iOS/Android app with the JS we created instead of what the metro packager created. expo just made it easy to tell the app which JS to use in a remote way that didn't require touching the app at all.

The main problem seems to be that all the react-native tools are pretty hard-coded towards the metro packager and there does not seem to be a way to change that. It might be possible to achieve this via their transformer functionality but I didn't look into that yet.

[idokutela]

Yes: I slapped together a node webserver this morning, and tried to figure out what a react-native dev app requests, but it wasn't as simple as I had hoped. If I have a spare afternoon, I'll burrow into the react-native code, and see if I can reverse engineer what's going on. To be honest, even if I figure it out, I hate doing this sort of reverse-engineering. One is very dependent on api that may change at any time. Even if you get everything working, you just need a version bump, and you're back to square one...

In the mean time, I'll do most of my cljs dev work building to node, and only build for react-native when I need to.

[thheller]

My thinking was that the RN app will just load some JS and we just need to replace which one it loads. I thought the react-native app would just bundle the JS + assets somehow and don't talk to a remote server at all but that might be completely wrong.

I share your concern and I definitely do not want to do this work for every other react-native/expo release. The :target :react-native is still available and doesn't do any of these expo-related hacks and doesn't require any either.

It might still be best to go with the figwheel-bridge.js way of completely loading the CLJS code separately.

[idokutela]

Yes: in theory, it should be a simple case of telling the react-native app to load a different piece of js + assets. I think it might even work when building for "release" -- one could just swap the bundle. What I meant with the server was when running react-native in "dev": it then starts a web-server that serves up the js to be used, and any diffs each time the code changes. Ideally one could just swap out where that server serves its assets from, but, as happens with all projects that have had some time, by now there's a lot of extra stuff that server does, and with just a short time playing around, I couldn't figure out quite how to do so.

I think the figwheel-bridge way of doing things is perfectly valid. My only problem with it is that it puts yet another tool between your code and what's actually run. That means that sometimes you get figwheel handling errors, sometimes you get the react-native bridge handling errors, if you mess with native code, sometimes there are errors there. Keeping abreast of what is actually a problem, and what is just a build problem, can become quite complicated (at least for me).

[AndrewMorsillo]

I see there is an "expo" target now. Is there any documentation on using this?

https://github.com/thheller/shadow-cljs/blob/master/src/main/shadow/build/targets/expo.clj

[thheller]

@AndrewMorsillo no that target was an experiment and is broken. Do not use it.

I don't use react-native myself so I haven't worked on this in a while. You can use :target :npm-module easily but it may require some manual setup to integrate with react-native from the JS side given that things like create-react-native-app are hardcoded to look for a App.js and so on.

I'll try to write some docs to show the basic setup.

[AndrewMorsillo]

@thheller ok thanks. The reason I ask is that I am interested in learning cljs by prototyping a react native app. I used the expo Leiningen template which works but isn't configured for connecting to development tools nREPL (Calva in vscode in my case). I've had a difficult time figuring out how to configure Leiningen/figwheel to start a cljs nRepl with cider. In fact I still haven't figured it out. Shadow-cljs seems much easier to use than lein so I was hoping to find a configuration/template for use with expo.

[thheller]

@AndrewMorsillo I created a demo app for fulcro+expo here: https://github.com/thheller/fulcro-expo

I'll probably create an example app for reagent as well and write a blog post and some docs.

[AndrewMorsillo]

Thank you very much for creating that. That will be a great place for me to start exploring.

[thheller]

Finished the reagent demo as well. https://github.com/thheller/reagent-expo

I'll try to write proper docs this weekend.

[attentive]

I am extremely appreciative of the work you're doing in this direction.

I have a fairly mature, non-expo Re-Frame React Native app built with re-natal that I'd love to migrate to shadow-cljs for so many reasons, but I'm a bit wary of heading down that road as I can tell the level of tooling knowledge is well over my head.

Do you have plans, or a recommended approach for getting these demos running in an "ejected" state?

[thheller]

@attentive "ejected" doesn't change anything regarding the shadow-cljs integration. You can find a demo for pure react-native + re-frame here: https://github.com/PEZ/rn-rf-shadow

[Wiltzu]

I'm working on a React Native template with Shadow CLJS, that could replace re-renal in our projects. I have managed to change expo/render-root to work with an ejected React Native app and release builds are working fine. However, I still have a couple of development time issues, that I couldn't figured out. I also managed to reproduce all these issues with this demo https://github.com/PEZ/rn-rf-shadow.

• It seems that :devtools/preloads are not working with :react-native target. Those namespaces are not included in builds and are missing from output directory.
• I wasn't able to get cljs-devtools to work. I also tried installing it manually, but I still got an error: NOT installing CLJS DevTools 0.9.10 under advanced build.
• When developing, Android app is generating a new REPL runtime every time the app is reloaded and previous runtimes are left running, but are not active. Reloads with iOS are working fine and only one runtime is active all the time. Is there any way to close those runtimes or select runtime in IDE like Cursive? (shadow/repl-next :app) is only working in command line.

[thheller]

@Wiltzu can you elaborate on the " a new REPL runtime every time the app is reloaded" part? You are supposed to completely disable all built-in reload functionality from react-native/metro. It should not be reloading anything ever. shadow-cljs does its own reloading and would just cause issues if something else was reloading as well.

Picking a REPL session is tricky since the repl-next functionality is not possible with nREPL.

:preloads will be fixed in the next release.

cljs-devtools I don't know. I guess it has issues detecing the code properly since it will be processed by metro. I have some ideas on how to skip that processing entirely but not sure thats actually worth since things seem to be working fine now.

[svdo]

@Wiltzu I was able to get cljs-devtools working by disabling the advanced-build check. In my app entry point, I manually load the cljs-devoots:

(devtools/set-pref! :disable-advanced-mode-check true)
(devtools/install!)

(I'm not using the :preloads.)

[Wiltzu]

@thheller Thank you for quick response! I have live and hot reload disabled from React Native. I meant manual reloading, that is usually required, when code is broken. After that I could no longer use nREPL, because the first runtime is no longer active, but it still gets selected. All the requests will timeout after that. Could it be possible to delete runtimes manually or with some condition, when those are not usable anymore? Switching runtimes would also be nice, when developing both Android and iOS apps at the same time.

@svdo thanks that seems to be a good workaround! This could be then included in projects own preload file.

I would also like to share this. It can be used as a render-root function in ejected React Native app to get shadow-cljs reload working. Could this be included in shadow-cljs library as well?

(ns shadow.react-native
  (:require
    ["react-native" :as rn]
    ["create-react-class" :as crc]))

(defonce root-ref (atom nil))
(defonce root-component-ref (atom nil))

(defn render-root [app-key root]
    (let [first-call? (nil? @root-ref)]
      (reset! root-ref root)

      (if-not first-call?
        (when-let [root @root-component-ref]
          (.forceUpdate ^js root))
        (let [Root
              (crc
                #js {:componentDidMount
                     (fn []
                       (this-as this
                         (reset! root-component-ref this)))
                     :componentWillUnmount
                     (fn []
                       (reset! root-component-ref nil))
                     :render
                     (fn []
                       (let [body @root-ref]
                         (if (fn? body)
                           (body)
                           body)))})]

          (.registerComponent rn/AppRegistry app-key (fn [] Root))))))

It can be used like this (shadow.react-native/render-root "MyAppName" (reagent.core/as-element [app-root])

[Wiltzu]

@thheller I found an other issue with the websocket that is used to reload codes. It selects IP with some logic and if I have a iOS device connected with an USB cable, it starts using 169.254.*.* IP, that iOS device has created locally to my computer. Then, websocket connection fails with Android device. Probably, there should be a way to configure this IP manually? 192.168.*.* IP works nicely with both platforms, if they are in the same network with computer.

[thheller]

@Wiltzu yeah the IP detection unfortunately is a bit unreliable. You can configure it manually by setting :local-ip "123.123.123.123" in the build config or if you prefer to keep it out of the build config via

shadow-cljs watch app --config-merge '{:local-ip "123.123.123.123"}'

I'm sort of hesitant to include the render-root function since it may not always apply and some react packages require setting the root differently (eg. react-native-navigation). But given that I included the basic shadow.expo helper I can probably do the same for pure react-native.

[Wiltzu]

@thheller thanks! Nice to know that changing IP is possible, when it's required.

I found another issue that is related to reloading JS files with watch. Basically, I have core namespace, that loads views from screens and it uses multiple custom_views from different namespaces (core -> screens -> custom_views). So, when I changed custom_views only screens gets loaded, but not core. It results views not been re-rendered. Is it so that only the closest dependencies are reloaded? Is there some configuration to change it or what should I do?

[thheller]

Why does core need to be reloaded when custom_views changes?

If you want you can set (ns ^:dev/always app.core) to force it to always be recompiled/reloaded but unless there are side-effecting macros called in core that should never be necessary?

[Wiltzu]

@theller the problem wasn't the thing that core wasn't reloading. It was related to react-navigation library and its navigator definitions that aren't changed, when some screen or custom view changes. I have to make the following changes to root component to make it reload navigator definitions every time. Otherwise, it won't work.

(def cnt (atom 0))
(defn reloading-app-root []
  @cnt
  [:> (create-app-container-component)])

(defn start
  {:dev/after-load true}
  []
  (swap! cnt inc))

[svdo]

Just a "FYI": when using the local-ip setting I can indeed connect with the REPL to the Android app. Inside that app, I also have a WebView that is running JS compiled by shadow-cljs. On iOS I can also connect to that with a REPL, but on Android I cannot.

[NuclearKev]

@Wiltzu I'm attempting to get a project working without expo and I can't seem to get past the document is undefined error... I tried to use shadow/cljs/devtools/client/browser.cljs but doesn't appear to be working at all. Any insight?

[theller]

Please note that @theller and @thheller are different persons!

[NuclearKev]

@Wiltzu Never mind, I seem to have fixed it. I'm not entirely sure how I did it though...

[PrestanceDesign]

Interresting, I have exactly the same issues as @Wiltzu.

When reloading manually mobile app, the repl return timeout because the websocket or something changes.
And the hot reloading doesn't trigger for files/namespaces in subfolder like main/myapp/components/search.cljs, main/myapp/navigation/navigation.cljs...modifying theses files doesn't show change as the main app.cljs do for eg.

Do you have find solutions @Wiltzu ?