Umbrel: Running Umbrel on AWS EC2 Instance

Created on 22 Nov 2020  ·  22Comments  ·  Source: getumbrel/umbrel

Hi,

I set up umbrel on an AWS EC2 spot instance as an experiment and here are a few issues I encountered that happened not to be mentioned and/or needed to be locally patched for now. All in all, it was indeed a great improvement over my last experience setting up my own bitcoind and lnd a year or so ago, so kudos to you all for making this _nearly_ a 1-click install!

--joe

P.S. For anyone else trying this with spot (t2.small) instances on AWS, I recommend you set it up as a permanent single spot instance and assign an Elastic IP (free external static IP) to the instance's network interface (you also need to set up the associated security group to open all the necessary ports). This should only cost ~$4-5/mo, plus the storage (I used a 1TB magnetic disk, which I believe comes out to about $15-16/mo). Total, ~$20/mo!!! I looked at hosting alternatives, including hosted RasPi's, but nothing comes close to this price because of the economical spot instances and spinning HDD versus SSD; and, sure, it's only 1 vCPU (1 core, 1 thread), slow spinning disk, likely swapping, but I'm not sure you need anything more...and as an added bonus, I'm running my urbit ship there, too.

$ cat /proc/version
Linux version 4.14.203-156.332.amzn2.x86_64 (mockbuild@ip-10-0-1-132) (gcc version 7.3.1 20180712 (Red Hat 7.3.1-10) (GCC)) #1 SMP Fri Oct 30 19:19:33 UTC 2020
  • fswatch wasn't built for yum so I needed to build it myself, and it subsequently failed _check_dependencies()_ in the start script. I didn't figure out why at the time so I commented out the check to bypass it. I realized while writing this just now it was because the start script runs under _root_, and _root_ doesn't have _/usr/local/bin_ in its PATH, where fswatch was installed. I created a link in _/usr/bin_ and it worked. Just a note in case anyone else runs into this.
  • The docker daemon needed to be started, sudo systemctl start docker.
  • The accommodations for CORS in the start script doesn't work on AWS because the commands ip and hostname return the internal address/name. The way to get the external ip and name is to use the service at the reserved address http://169.254.169.254 (you can google that URL if you want to know more about it): 
DEVICE_IP=`curl http://169.254.169.254/latest/meta-data/public-ipv4`
DEVICE_HOSTNAME=`curl http://169.254.169.254/latest/meta-data/public-hostname`

I also fiddled with the following few lines as follows to get it to work, e.g., getting rid of the .local. I'm not a CORS expert so I'm not sure how to set these properly, but the following did avoid the CORS issue for both a regular browser using the standard address and a TOR browser using the .onion address (I used brave).

DEVICE_HOSTS="http://${DEVICE_IP}:8081,http://${DEVICE_HOSTNAME}:8081,https://${DEVICE_HOSTNAME}"
if [[ -f "${UMBREL_ROOT}/tor/data/web/hostname" ]]; then
    hidden_service_url=$(cat "${UMBREL_ROOT}/tor/data/web/hostname")
    DEVICE_HOSTS="${DEVICE_HOSTS},http://${hidden_service_url}"
fi
export DEVICE_HOSTS=$DEVICE_HOSTS
export DEVICE_HOSTNAME="${DEVICE_HOSTNAME}"
  • I'm running another HTTP server on port 80, urbit, and I am unable to change it for various (bad) reasons, so I am not able to run nginx there. In order to change the nginx port address and everything that seemed to depend on it, I needed to make the following changes (presented below as a diff):
$ pwd
/home/ec2-user/umbrel

$ diff docker-compose.yml docker-compose.yml-ORIG
33c33
<                     - "8081:8081"
---
>                     - "80:80"

$ diff nginx/conf.d/default.conf nginx/conf.d/default.conf-ORIG
2,3c2,3
<     listen         8081 default_server;
<     listen         [::]:8081 default_server;
---
>     listen         80 default_server;
>     listen         [::]:80 default_server;

$ diff tor/torrc tor/torrc-ORIG
7c7
< HiddenServicePort 8081 10.11.0.2:8081
---
> HiddenServicePort 80 10.11.0.2:80
picture jlatone
👍2

All 22 comments

Thank you, @jlatone, for taking the time to provide a thorough breakdown of the steps you took to get it working on AWS!

Could you make doubly sure if fswatch is indeed working correctly? It's a key component of the backup system and the update system, so I'd recommend a quick test to verify that it's up and running fine:

  1. Take a note of the last backup timestamp on the settings page.
  2. In your Umbrel installation directory, run:
    touch lnd/data/chain/bitcoin/mainnet/channel.backup
  3. After ~30 seconds, refresh the settings page and verify if the timestamp has updated to the current time.

If there's no change in the timestamp, it means that there's probably a problem with fswatch.

To avoid CORS issues, you can change the hostname of your EC2 instance to its elastic IP (without http or https), and then restart Umbrel. That way Umbrel will be accessible at http(s):// and you wouldn't have to worry about an OTA update breaking your install by overwriting the start script changes.

The problem with encountering CORS issue on the onion address is a separate issue (#304) which happens on the first run of Umbrel, and is fixed once and for all with a restart.

Regarding port change in nginx, a fix is in the works (#256) but it might not be ready for the next release. So your install will be broken by the next OTA update and you'd probably have to apply the same changes again and restart Umbrel manually. :(

On a side note, I'm curious if you are sure about 1TB magnetic disk costing $15-$16/mo on AWS? AWS's pricing page for older generation EBS shows $0.05 per GB, which would be $50/mo for a 1TB drive. We're running a staging server on AWS too so I'd love to know if there's a neat trick you used to get inexpensive storage.

mayankchhabra picture mayankchhabra on 23 Nov 2020

Thx, @mayankchhabra.

  1. I verified fswatch is working correctly, according to your test.
  2. I will change the hostname today and revert to your original code (I think I can run urbit on another port, now the port 80 constraint is no longer necessary).
  3. AWS pricing is always a mystery, until you get the bill. I saw the $0.015 price for "Cold HDD (sc1) Volumes" on this page, https://aws.amazon.com/ebs/pricing/. Now that I look at it, "Magnetic (standard)" is not the same (see pic) so thanks for that, and I will try switching/moving over to it (it's 80 MB/s for a 1 terabyte volume versus 250).
    image

--joe

jlatone picture jlatone on 23 Nov 2020

I installed it again on a newly configures instance (because of the storage cost issue you pointed out, and it was easier to just redo it from scratch than copy data over since there was nothing so save).

It took ~5min this time. 👍

I still have the CORS issue. I don't have a public DNS name and I'm not quite sure what to do other than this:

[ec2-user@latone umbrel]$ diff scripts/start scripts/start-orig
57,59c57
< AWS_IP=`curl http://169.254.169.254/latest/meta-data/public-ipv4`
< AWS_HOSTNAME=`curl http://169.254.169.254/latest/meta-data/public-hostname`
< DEVICE_HOSTS="http://${DEVICE_IP},http://${DEVICE_HOSTNAME}.local,https://${DEVICE_HOSTNAME}.local,http://${DEVICE_HOSTNAME},https://${DEVICE_HOSTNAME},http://${AWS_IP},https://${AWS_IP},http://${AWS_HOSTNAME},https://${AWS_HOSTNAME}"
---
> DEVICE_HOSTS="http://${DEVICE_IP},http://${DEVICE_HOSTNAME}.local,https://${DEVICE_HOSTNAME}.local,http://${DEVICE_HOSTNAME},https://${DEVICE_HOSTNAME}"

--joe

jlatone picture jlatone on 23 Nov 2020

@jlatone did you try setting the hostname to the elastic IP?

So if your elastic IP is 51.52.53.54, here's what you can do:

sudo hostnamectl set-hostname 51.52.53.54

# Restart Umbrel
sudo path-to-umbrel/scripts/stop
sudo path-to-umbrel/scripts/start

Then try accessing http://51.52.53.54.

mayankchhabra picture mayankchhabra on 30 Nov 2020

Nevermind...the suggestion you gave above fixed the problem I had described, now I'm stuck here:
image

21:35:35.488604 IP c-71-198-134-183.hsd1.ca.comcast.net.51619 > ip-172-31-2-215.ec2.internal.http: Flags [P.], seq 27555:28299, ack 7068, win 2048, options [nop,nop,TS val 2244529979 ecr 2973435150], length 744: HTTP: GET /api/v1/lnd/info/status HTTP/1.1
21:35:35.507070 IP ip-172-31-2-215.ec2.internal.http > c-71-198-134-183.hsd1.ca.comcast.net.51619: Flags [P.], seq 7068:7259, ack 28299, win 669, options [nop,nop,TS val 2973436497 ecr 2244529979], length 191: HTTP: HTTP/1.1 304 Not Modified
21:35:35.589843 IP c-71-198-134-183.hsd1.ca.comcast.net.51619 > ip-172-31-2-215.ec2.internal.http: Flags [.], ack 7259, win 2045, options [nop,nop,TS val 2244530081 ecr 2973436497], length 0
21:35:38.106243 IP c-71-198-134-183.hsd1.ca.comcast.net.51619 > ip-172-31-2-215.ec2.internal.http: Flags [P.], seq 28299:29043, ack 7259, win 2048, options [nop,nop,TS val 2244532586 ecr 2973436497], length 744: HTTP: GET /api/v1/lnd/info/status HTTP/1.1
21:35:38.121972 IP ip-172-31-2-215.ec2.internal.http > c-71-198-134-183.hsd1.ca.comcast.net.51619: Flags [P.], seq 7259:7450, ack 29043, win 681, options [nop,nop,TS val 2973439112 ecr 2244532586], length 191: HTTP: HTTP/1.1 304 Not Modified
21:35:38.205006 IP c-71-198-134-183.hsd1.ca.comcast.net.51619 > ip-172-31-2-215.ec2.internal.http: Flags [.], ack 7450, win 2045, options [nop,nop,TS val 2244532683 ecr 2973439112], length 0
Jaxo picture Jaxo on 30 Nov 2020

Also, how long does it typically take a home user to fully replicate the bitcoin blockchain? I've been running now for ~10days, and it's only 40GB, which I gather represents a little more than 10%, despite the dashboard saying it's >50%, which I gather is measured in years not size? --joe

Jaxo picture Jaxo on 30 Nov 2020

@Jaxo Is it still stuck at "Loading LND..."? If so, can you please try to have a look at LND's logs and see if you can spot an error:

# In Umbrel's directory, run

docker-compose logs --tail=100 lnd

how long does it typically take a home user to fully replicate the bitcoin blockchain?

It usually takes 3-4 days with and SSD and ~4 weeks with an HDD for a home user with RPi 4 4GB/8GB RAM.

which I gather is measured in years not size?

Correct, we should change that logic on our end (#243), so you're right — technically it's only synced a little over 10%.

mayankchhabra picture mayankchhabra on 1 Dec 2020

(oops, sry for changing id's, @jaxo = @jlatone)

I restarted and tried connecting through both my .com & .onion addresses, and both are stuck at "Loading LND..." and here's the lnd log, which looks fine.

$ sudo docker-compose logs --tail=100 lnd
Attaching to lnd
lnd                  | 2020-12-01 16:50:56.379 [INF] LTND: Version: 0.11.1-beta commit=v0.11.1-beta, build=production, logging=default
lnd                  | 2020-12-01 16:50:56.380 [INF] LTND: Active chain: Bitcoin (network=mainnet)
lnd                  | 2020-12-01 16:50:56.380 [INF] LTND: Opening the main database, this might take a few minutes...
lnd                  | 2020-12-01 16:50:56.380 [INF] LTND: Opening bbolt database, sync_freelist=false
lnd                  | 2020-12-01 16:50:56.499 [INF] CHDB: Checking for schema update: latest_version=17, db_version=17
lnd                  | 2020-12-01 16:50:56.500 [INF] LTND: Database now open (time_to_open=119.110449ms)!

When I tried opening the dashboard in a browser with a cleared cache, it gets stuck on "Loading Middleware..."

Here's the log of all services, perhaps you can quickly spot the issue: 

$ sudo docker-compose logs --tail=100
Attaching to neutrino-switcher, electrs, lnd, nginx, middleware, bitcoin, manager, dashboard, frontail, tor
electrs              | Config { log: StdErrLog { verbosity: Trace, quiet: false, show_level: true, timestamp: Off, modules: [], writer: "stderr", color_choice: Never }, network_type: bitcoin, db_path: "/data/db/mainnet", daemon_dir: "/data/.bitcoin", blocks_dir: "/data/.bitcoin/blocks", daemon_rpc_addr: V4(10.11.1.1:8332), electrum_rpc_addr: V4(0.0.0.0:50001), monitoring_addr: V4(127.0.0.1:4224), jsonrpc_import: false, index_batch_size: 100, bulk_index_threads: 1, tx_cache_size: 10485760, txid_limit: 100, server_banner: "Umbrel v0.2.15", blocktxids_cache_size: 10485760 }
electrs              | DEBUG - Server listening on 127.0.0.1:4224
electrs              | DEBUG - Running accept thread
electrs              | WARN - reconnecting to bitcoind: {"code":-28,"message":"Loading block index..."}
electrs              | WARN - reconnecting to bitcoind: {"code":-28,"message":"Loading block index..."}
electrs              | WARN - reconnecting to bitcoind: {"code":-28,"message":"Loading block index..."}
electrs              | WARN - reconnecting to bitcoind: {"code":-28,"message":"Loading block index..."}
electrs              | WARN - reconnecting to bitcoind: {"code":-28,"message":"Loading block index..."}
electrs              | WARN - reconnecting to bitcoind: {"code":-28,"message":"Loading block index..."}
electrs              | WARN - reconnecting to bitcoind: {"code":-28,"message":"Rewinding blocks..."}
electrs              | WARN - reconnecting to bitcoind: {"code":-28,"message":"Verifying blocks..."}
electrs              | WARN - reconnecting to bitcoind: {"code":-28,"message":"Verifying blocks..."}
electrs              | WARN - reconnecting to bitcoind: {"code":-28,"message":"Verifying blocks..."}
electrs              | WARN - reconnecting to bitcoind: {"code":-28,"message":"Verifying blocks..."}
electrs              | WARN - reconnecting to bitcoind: {"code":-28,"message":"Verifying blocks..."}
electrs              | WARN - reconnecting to bitcoind: {"code":-28,"message":"Verifying blocks..."}
electrs              | WARN - reconnecting to bitcoind: {"code":-28,"message":"Verifying blocks..."}
electrs              | WARN - reconnecting to bitcoind: {"code":-28,"message":"Verifying blocks..."}
electrs              | WARN - reconnecting to bitcoind: {"code":-28,"message":"Verifying blocks..."}
electrs              | WARN - reconnecting to bitcoind: {"code":-28,"message":"Verifying blocks..."}
electrs              | WARN - reconnecting to bitcoind: {"code":-28,"message":"Verifying blocks..."}
electrs              | WARN - reconnecting to bitcoind: {"code":-28,"message":"Verifying blocks..."}
electrs              | WARN - reconnecting to bitcoind: {"code":-28,"message":"Verifying blocks..."}
electrs              | WARN - reconnecting to bitcoind: {"code":-28,"message":"Verifying blocks..."}
electrs              | WARN - reconnecting to bitcoind: {"code":-28,"message":"Verifying blocks..."}
electrs              | WARN - reconnecting to bitcoind: {"code":-28,"message":"Verifying blocks..."}
electrs              | WARN - reconnecting to bitcoind: {"code":-28,"message":"Verifying blocks..."}
electrs              | WARN - reconnecting to bitcoind: {"code":-28,"message":"Verifying blocks..."}
electrs              | WARN - reconnecting to bitcoind: {"code":-28,"message":"Verifying blocks..."}
electrs              | WARN - reconnecting to bitcoind: {"code":-28,"message":"Verifying blocks..."}
electrs              | WARN - reconnecting to bitcoind: {"code":-28,"message":"Verifying blocks..."}
electrs              | WARN - reconnecting to bitcoind: {"code":-28,"message":"Verifying blocks..."}
electrs              | WARN - reconnecting to bitcoind: {"code":-28,"message":"Verifying blocks..."}
electrs              | WARN - reconnecting to bitcoind: {"code":-28,"message":"Verifying blocks..."}
electrs              | WARN - reconnecting to bitcoind: {"code":-28,"message":"Verifying blocks..."}
electrs              | WARN - reconnecting to bitcoind: {"code":-28,"message":"Verifying blocks..."}
electrs              | WARN - reconnecting to bitcoind: {"code":-28,"message":"Verifying blocks..."}
electrs              | WARN - reconnecting to bitcoind: {"code":-28,"message":"Verifying blocks..."}
electrs              | WARN - reconnecting to bitcoind: {"code":-28,"message":"Verifying blocks..."}
electrs              | WARN - reconnecting to bitcoind: {"code":-28,"message":"Verifying blocks..."}
electrs              | WARN - reconnecting to bitcoind: {"code":-28,"message":"Verifying blocks..."}
electrs              | WARN - reconnecting to bitcoind: {"code":-28,"message":"Verifying blocks..."}
electrs              | WARN - reconnecting to bitcoind: {"code":-28,"message":"Verifying blocks..."}
electrs              | WARN - reconnecting to bitcoind: {"code":-28,"message":"Verifying blocks..."}
electrs              | WARN - reconnecting to bitcoind: {"code":-28,"message":"Verifying blocks..."}
electrs              | WARN - reconnecting to bitcoind: {"code":-28,"message":"Verifying blocks..."}
electrs              | WARN - reconnecting to bitcoind: {"code":-28,"message":"Verifying blocks..."}
electrs              | WARN - reconnecting to bitcoind: {"code":-28,"message":"Verifying blocks..."}
electrs              | WARN - reconnecting to bitcoind: {"code":-28,"message":"Verifying blocks..."}
electrs              | WARN - reconnecting to bitcoind: {"code":-28,"message":"Verifying blocks..."}
electrs              | WARN - reconnecting to bitcoind: {"code":-28,"message":"Verifying blocks..."}
electrs              | WARN - reconnecting to bitcoind: {"code":-28,"message":"Loading wallet..."}
electrs              | INFO - NetworkInfo { version: 200100, subversion: "/Satoshi:0.20.1/", relayfee: 0.00001 }
electrs              | INFO - BlockchainInfo { chain: "main", blocks: 361058, headers: 659505, verificationprogress: 0.1229525765195864, bestblockhash: "00000000000000000199981d19bb3e83286cdb7ceb023bb2533e226209173a59", pruned: false, initialblockdownload: true }
electrs              | WARN - wait until IBD is over: headers=659505 blocks=361059 progress=0.1229540978341984
electrs              | WARN - wait until IBD is over: headers=659505 blocks=361060 progress=0.122958016978165
electrs              | WARN - wait until IBD is over: headers=659505 blocks=361061 progress=0.1229599786645994
electrs              | WARN - wait until IBD is over: headers=659505 blocks=361062 progress=0.1229627670432779
electrs              | WARN - wait until IBD is over: headers=659505 blocks=361063 progress=0.1229651803917981
electrs              | WARN - wait until IBD is over: headers=659505 blocks=361064 progress=0.1229676170547496
electrs              | WARN - wait until IBD is over: headers=659505 blocks=361066 progress=0.1229710254535921
electrs              | WARN - wait until IBD is over: headers=659505 blocks=361067 progress=0.1229713004309995
electrs              | WARN - wait until IBD is over: headers=659505 blocks=361068 progress=0.1229745752364496
electrs              | WARN - wait until IBD is over: headers=659505 blocks=361069 progress=0.1229760937268666
electrs              | WARN - wait until IBD is over: headers=659505 blocks=361070 progress=0.1229797469786818
electrs              | WARN - wait until IBD is over: headers=659505 blocks=361072 progress=0.1229831759251193
electrs              | WARN - wait until IBD is over: headers=659505 blocks=361073 progress=0.1229848952744754
electrs              | WARN - wait until IBD is over: headers=659505 blocks=361074 progress=0.122987303777357
electrs              | WARN - wait until IBD is over: headers=659505 blocks=361075 progress=0.12298856996337
electrs              | WARN - wait until IBD is over: headers=659505 blocks=361076 progress=0.1229888643204602
electrs              | WARN - wait until IBD is over: headers=659505 blocks=361077 progress=0.1229903478268778
electrs              | WARN - wait until IBD is over: headers=659505 blocks=361078 progress=0.1229908142290988
electrs              | WARN - wait until IBD is over: headers=659505 blocks=361079 progress=0.1229945127688068
electrs              | WARN - wait until IBD is over: headers=659505 blocks=361080 progress=0.1229959436755241
electrs              | WARN - wait until IBD is over: headers=659505 blocks=361081 progress=0.1229979100768453
electrs              | WARN - wait until IBD is over: headers=659505 blocks=361082 progress=0.1229987197622384
electrs              | WARN - wait until IBD is over: headers=659505 blocks=361083 progress=0.1230024480663459
lnd                  | 2020-12-01 16:50:56.379 [INF] LTND: Version: 0.11.1-beta commit=v0.11.1-beta, build=production, logging=default
lnd                  | 2020-12-01 16:50:56.380 [INF] LTND: Active chain: Bitcoin (network=mainnet)
lnd                  | 2020-12-01 16:50:56.380 [INF] LTND: Opening the main database, this might take a few minutes...
lnd                  | 2020-12-01 16:50:56.380 [INF] LTND: Opening bbolt database, sync_freelist=false
lnd                  | 2020-12-01 16:50:56.499 [INF] CHDB: Checking for schema update: latest_version=17, db_version=17
lnd                  | 2020-12-01 16:50:56.500 [INF] LTND: Database now open (time_to_open=119.110449ms)!
dashboard            | Starting up http-server, serving /dist
dashboard            | Available on:
dashboard            |   http://127.0.0.1: 3004
dashboard            |   http://10.11.0.3: 3004
dashboard            | Hit CTRL-C to stop the server
dashboard            | [2020-12-01T16:51:37.990Z]  "GET /" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.183 Safari/537.36"
dashboard            | (node:1) [DEP0066] DeprecationWarning: OutgoingMessage.prototype._headers is deprecated
dashboard            | [2020-12-01T16:51:38.517Z]  "GET /css/app.a30b2401.css" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.183 Safari/537.36"
dashboard            | [2020-12-01T16:51:39.166Z]  "GET /css/chunk-vendors.a7bbc042.css" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.183 Safari/537.36"
dashboard            | [2020-12-01T16:51:41.157Z]  "GET /js/app.dafeb21a.js" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.183 Safari/537.36"
dashboard            | [2020-12-01T16:51:41.497Z]  "GET /js/chunk-vendors.0cb9af06.js" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.183 Safari/537.36"
dashboard            | [2020-12-01T16:51:43.790Z]  "GET /img/logo.5a50986c.svg" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.183 Safari/537.36"
dashboard            | [2020-12-01T16:51:47.115Z]  "GET /favicon.png" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.183 Safari/537.36"
dashboard            | [2020-12-01T16:54:49.514Z]  "GET /config/getuser?index=0" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:76.0) Gecko/20100101 Firefox/76.0"
dashboard            | [2020-12-01T17:14:42.823Z]  "GET /" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.183 Safari/537.36 OPR/72.0.3815.320"
dashboard            | [2020-12-01T17:14:43.206Z]  "GET /css/app.a30b2401.css" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.183 Safari/537.36 OPR/72.0.3815.320"
dashboard            | [2020-12-01T17:14:43.455Z]  "GET /js/app.dafeb21a.js" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.183 Safari/537.36 OPR/72.0.3815.320"
dashboard            | [2020-12-01T17:14:43.462Z]  "GET /css/chunk-vendors.a7bbc042.css" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.183 Safari/537.36 OPR/72.0.3815.320"
dashboard            | [2020-12-01T17:14:43.561Z]  "GET /js/chunk-vendors.0cb9af06.js" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.183 Safari/537.36 OPR/72.0.3815.320"
dashboard            | [2020-12-01T17:14:47.628Z]  "GET /" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.183 Safari/537.36 OPR/72.0.3815.320"
dashboard            | [2020-12-01T17:14:47.836Z]  "GET /img/logo.5a50986c.svg" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.183 Safari/537.36 OPR/72.0.3815.320"
dashboard            | [2020-12-01T17:14:48.042Z]  "GET /img/logo.5a50986c.svg" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.183 Safari/537.36 OPR/72.0.3815.320"
dashboard            | [2020-12-01T17:14:48.141Z]  "GET /favicon.png" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.183 Safari/537.36 OPR/72.0.3815.320"
frontail             | tail: error reading '/var/log/syslog': Is a directory
frontail             | tail: /var/log/syslog: cannot follow end of this type of file
manager              | umbrel-manager
manager              | ::ffff:10.11.0.2 - - [Tue, 01 Dec 2020 17:00:39 GMT] "GET /v1/system/update-status HTTP/1.0" 304 - "http://ec2-54-144-105-150.compute-1.amazonaws.com/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.183 Safari/537.36"
manager              | umbrel-manager
manager              | ::ffff:10.11.0.2 - - [Tue, 01 Dec 2020 17:00:45 GMT] "GET /v1/system/update-status HTTP/1.0" 304 - "http://aw6iajzu7jhf4qcjahddudoot4hvkfjxpmdilo3xlgpbqssdhcjkk6yd.onion/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.183 Safari/537.36"
manager              | umbrel-manager
manager              | ::ffff:10.11.0.2 - - [Tue, 01 Dec 2020 17:01:39 GMT] "GET /v1/system/update-status HTTP/1.0" 304 - "http://ec2-54-144-105-150.compute-1.amazonaws.com/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.183 Safari/537.36"
manager              | umbrel-manager
manager              | ::ffff:10.11.0.2 - - [Tue, 01 Dec 2020 17:01:43 GMT] "GET /v1/system/update-status HTTP/1.0" 304 - "http://aw6iajzu7jhf4qcjahddudoot4hvkfjxpmdilo3xlgpbqssdhcjkk6yd.onion/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.183 Safari/537.36"
manager              | umbrel-manager
manager              | ::ffff:10.11.0.2 - - [Tue, 01 Dec 2020 17:02:39 GMT] "GET /v1/system/update-status HTTP/1.0" 304 - "http://ec2-54-144-105-150.compute-1.amazonaws.com/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.183 Safari/537.36"
manager              | umbrel-manager
manager              | ::ffff:10.11.0.2 - - [Tue, 01 Dec 2020 17:02:45 GMT] "GET /v1/system/update-status HTTP/1.0" 304 - "http://aw6iajzu7jhf4qcjahddudoot4hvkfjxpmdilo3xlgpbqssdhcjkk6yd.onion/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.183 Safari/537.36"
manager              | umbrel-manager
manager              | ::ffff:10.11.0.2 - - [Tue, 01 Dec 2020 17:03:39 GMT] "GET /v1/system/update-status HTTP/1.0" 304 - "http://ec2-54-144-105-150.compute-1.amazonaws.com/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.183 Safari/537.36"
manager              | umbrel-manager
manager              | ::ffff:10.11.0.2 - - [Tue, 01 Dec 2020 17:03:44 GMT] "GET /v1/system/update-status HTTP/1.0" 304 - "http://aw6iajzu7jhf4qcjahddudoot4hvkfjxpmdilo3xlgpbqssdhcjkk6yd.onion/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.183 Safari/537.36"
manager              | umbrel-manager
manager              | ::ffff:10.11.0.2 - - [Tue, 01 Dec 2020 17:04:40 GMT] "GET /v1/system/update-status HTTP/1.0" 304 - "http://ec2-54-144-105-150.compute-1.amazonaws.com/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.183 Safari/537.36"
manager              | umbrel-manager
manager              | ::ffff:10.11.0.2 - - [Tue, 01 Dec 2020 17:04:44 GMT] "GET /v1/system/update-status HTTP/1.0" 304 - "http://aw6iajzu7jhf4qcjahddudoot4hvkfjxpmdilo3xlgpbqssdhcjkk6yd.onion/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.183 Safari/537.36"
manager              | umbrel-manager
manager              | ::ffff:10.11.0.2 - - [Tue, 01 Dec 2020 17:05:39 GMT] "GET /v1/system/update-status HTTP/1.0" 304 - "http://ec2-54-144-105-150.compute-1.amazonaws.com/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.183 Safari/537.36"
manager              | umbrel-manager
manager              | ::ffff:10.11.0.2 - - [Tue, 01 Dec 2020 17:05:44 GMT] "GET /v1/system/update-status HTTP/1.0" 304 - "http://aw6iajzu7jhf4qcjahddudoot4hvkfjxpmdilo3xlgpbqssdhcjkk6yd.onion/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.183 Safari/537.36"
manager              | umbrel-manager
manager              | ::ffff:10.11.0.2 - - [Tue, 01 Dec 2020 17:06:39 GMT] "GET /v1/system/update-status HTTP/1.0" 304 - "http://ec2-54-144-105-150.compute-1.amazonaws.com/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.183 Safari/537.36"
manager              | umbrel-manager
manager              | ::ffff:10.11.0.2 - - [Tue, 01 Dec 2020 17:06:44 GMT] "GET /v1/system/update-status HTTP/1.0" 304 - "http://aw6iajzu7jhf4qcjahddudoot4hvkfjxpmdilo3xlgpbqssdhcjkk6yd.onion/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.183 Safari/537.36"
manager              | umbrel-manager
manager              | ::ffff:10.11.0.2 - - [Tue, 01 Dec 2020 17:07:39 GMT] "GET /v1/system/update-status HTTP/1.0" 304 - "http://ec2-54-144-105-150.compute-1.amazonaws.com/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.183 Safari/537.36"
manager              | umbrel-manager
manager              | ::ffff:10.11.0.2 - - [Tue, 01 Dec 2020 17:07:44 GMT] "GET /v1/system/update-status HTTP/1.0" 304 - "http://aw6iajzu7jhf4qcjahddudoot4hvkfjxpmdilo3xlgpbqssdhcjkk6yd.onion/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.183 Safari/537.36"
manager              | umbrel-manager
manager              | ::ffff:10.11.0.2 - - [Tue, 01 Dec 2020 17:08:39 GMT] "GET /v1/system/update-status HTTP/1.0" 304 - "http://ec2-54-144-105-150.compute-1.amazonaws.com/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.183 Safari/537.36"
manager              | umbrel-manager
manager              | ::ffff:10.11.0.2 - - [Tue, 01 Dec 2020 17:08:45 GMT] "GET /v1/system/update-status HTTP/1.0" 304 - "http://aw6iajzu7jhf4qcjahddudoot4hvkfjxpmdilo3xlgpbqssdhcjkk6yd.onion/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.183 Safari/537.36"
manager              | umbrel-manager
middleware           | umbrel-middleware
middleware           | ::ffff:10.11.0.2 - - [Tue, 01 Dec 2020 17:04:11 GMT] "GET /v1/lnd/info/status HTTP/1.0" 304 - "http://ec2-54-144-105-150.compute-1.amazonaws.com/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.183 Safari/537.36"
middleware           | umbrel-middleware
middleware           | ::ffff:10.11.0.2 - - [Tue, 01 Dec 2020 17:04:12 GMT] "GET /v1/lnd/info/status HTTP/1.0" 304 - "http://aw6iajzu7jhf4qcjahddudoot4hvkfjxpmdilo3xlgpbqssdhcjkk6yd.onion/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.183 Safari/537.36"
middleware           | umbrel-middleware
middleware           | ::ffff:10.11.0.2 - - [Tue, 01 Dec 2020 17:04:13 GMT] "GET /v1/lnd/info/status HTTP/1.0" 304 - "http://ec2-54-144-105-150.compute-1.amazonaws.com/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.183 Safari/537.36"
middleware           | umbrel-middleware
middleware           | ::ffff:10.11.0.2 - - [Tue, 01 Dec 2020 17:04:14 GMT] "GET /v1/lnd/info/status HTTP/1.0" 304 - "http://aw6iajzu7jhf4qcjahddudoot4hvkfjxpmdilo3xlgpbqssdhcjkk6yd.onion/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.183 Safari/537.36"
middleware           | umbrel-middleware
middleware           | ::ffff:10.11.0.2 - - [Tue, 01 Dec 2020 17:04:15 GMT] "GET /v1/lnd/info/status HTTP/1.0" 304 - "http://ec2-54-144-105-150.compute-1.amazonaws.com/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.183 Safari/537.36"
middleware           | umbrel-middleware
middleware           | ::ffff:10.11.0.2 - - [Tue, 01 Dec 2020 17:04:16 GMT] "GET /v1/lnd/info/status HTTP/1.0" 304 - "http://aw6iajzu7jhf4qcjahddudoot4hvkfjxpmdilo3xlgpbqssdhcjkk6yd.onion/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.183 Safari/537.36"
middleware           | umbrel-middleware
middleware           | ::ffff:10.11.0.2 - - [Tue, 01 Dec 2020 17:04:17 GMT] "GET /v1/lnd/info/status HTTP/1.0" 304 - "http://ec2-54-144-105-150.compute-1.amazonaws.com/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.183 Safari/537.36"
middleware           | umbrel-middleware
middleware           | ::ffff:10.11.0.2 - - [Tue, 01 Dec 2020 17:04:18 GMT] "GET /v1/lnd/info/status HTTP/1.0" 304 - "http://aw6iajzu7jhf4qcjahddudoot4hvkfjxpmdilo3xlgpbqssdhcjkk6yd.onion/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.183 Safari/537.36"
middleware           | umbrel-middleware
middleware           | ::ffff:10.11.0.2 - - [Tue, 01 Dec 2020 17:04:19 GMT] "GET /v1/lnd/info/status HTTP/1.0" 304 - "http://ec2-54-144-105-150.compute-1.amazonaws.com/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.183 Safari/537.36"
middleware           | umbrel-middleware
middleware           | ::ffff:10.11.0.2 - - [Tue, 01 Dec 2020 17:04:20 GMT] "GET /v1/lnd/info/status HTTP/1.0" 304 - "http://aw6iajzu7jhf4qcjahddudoot4hvkfjxpmdilo3xlgpbqssdhcjkk6yd.onion/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.183 Safari/537.36"
middleware           | umbrel-middleware
middleware           | ::ffff:10.11.0.2 - - [Tue, 01 Dec 2020 17:04:21 GMT] "GET /v1/lnd/info/status HTTP/1.0" 304 - "http://ec2-54-144-105-150.compute-1.amazonaws.com/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.183 Safari/537.36"
middleware           | umbrel-middleware
middleware           | ::ffff:10.11.0.2 - - [Tue, 01 Dec 2020 17:04:22 GMT] "GET /v1/lnd/info/status HTTP/1.0" 304 - "http://aw6iajzu7jhf4qcjahddudoot4hvkfjxpmdilo3xlgpbqssdhcjkk6yd.onion/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.183 Safari/537.36"
middleware           | umbrel-middleware
middleware           | ::ffff:10.11.0.2 - - [Tue, 01 Dec 2020 17:04:23 GMT] "GET /v1/lnd/info/status HTTP/1.0" 304 - "http://ec2-54-144-105-150.compute-1.amazonaws.com/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.183 Safari/537.36"
middleware           | umbrel-middleware
middleware           | ::ffff:10.11.0.2 - - [Tue, 01 Dec 2020 17:04:24 GMT] "GET /v1/lnd/info/status HTTP/1.0" 304 - "http://aw6iajzu7jhf4qcjahddudoot4hvkfjxpmdilo3xlgpbqssdhcjkk6yd.onion/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.183 Safari/537.36"
middleware           | umbrel-middleware
middleware           | ::ffff:10.11.0.2 - - [Tue, 01 Dec 2020 17:04:25 GMT] "GET /v1/lnd/info/status HTTP/1.0" 304 - "http://ec2-54-144-105-150.compute-1.amazonaws.com/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.183 Safari/537.36"
middleware           | umbrel-middleware
middleware           | ::ffff:10.11.0.2 - - [Tue, 01 Dec 2020 17:04:26 GMT] "GET /v1/lnd/info/status HTTP/1.0" 304 - "http://aw6iajzu7jhf4qcjahddudoot4hvkfjxpmdilo3xlgpbqssdhcjkk6yd.onion/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.183 Safari/537.36"
middleware           | umbrel-middleware
middleware           | ::ffff:10.11.0.2 - - [Tue, 01 Dec 2020 17:04:27 GMT] "GET /v1/lnd/info/status HTTP/1.0" 304 - "http://ec2-54-144-105-150.compute-1.amazonaws.com/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.183 Safari/537.36"
middleware           | umbrel-middleware
middleware           | ::ffff:10.11.0.2 - - [Tue, 01 Dec 2020 17:04:28 GMT] "GET /v1/lnd/info/status HTTP/1.0" 304 - "http://aw6iajzu7jhf4qcjahddudoot4hvkfjxpmdilo3xlgpbqssdhcjkk6yd.onion/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.183 Safari/537.36"
middleware           | umbrel-middleware
middleware           | ::ffff:10.11.0.2 - - [Tue, 01 Dec 2020 17:04:29 GMT] "GET /v1/lnd/info/status HTTP/1.0" 304 - "http://ec2-54-144-105-150.compute-1.amazonaws.com/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.183 Safari/537.36"
middleware           | umbrel-middleware
middleware           | ::ffff:10.11.0.2 - - [Tue, 01 Dec 2020 17:04:30 GMT] "GET /v1/lnd/info/status HTTP/1.0" 304 - "http://aw6iajzu7jhf4qcjahddudoot4hvkfjxpmdilo3xlgpbqssdhcjkk6yd.onion/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.183 Safari/537.36"
middleware           | umbrel-middleware
middleware           | ::ffff:10.11.0.2 - - [Tue, 01 Dec 2020 17:04:31 GMT] "GET /v1/lnd/info/status HTTP/1.0" 304 - "http://ec2-54-144-105-150.compute-1.amazonaws.com/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.183 Safari/537.36"
middleware           | umbrel-middleware
middleware           | ::ffff:10.11.0.2 - - [Tue, 01 Dec 2020 17:04:32 GMT] "GET /v1/lnd/info/status HTTP/1.0" 304 - "http://aw6iajzu7jhf4qcjahddudoot4hvkfjxpmdilo3xlgpbqssdhcjkk6yd.onion/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.183 Safari/537.36"
middleware           | umbrel-middleware
middleware           | ::ffff:10.11.0.2 - - [Tue, 01 Dec 2020 17:04:33 GMT] "GET /v1/lnd/info/status HTTP/1.0" 304 - "http://ec2-54-144-105-150.compute-1.amazonaws.com/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.183 Safari/537.36"
middleware           | umbrel-middleware
middleware           | ::ffff:10.11.0.2 - - [Tue, 01 Dec 2020 17:04:34 GMT] "GET /v1/lnd/info/status HTTP/1.0" 304 - "http://aw6iajzu7jhf4qcjahddudoot4hvkfjxpmdilo3xlgpbqssdhcjkk6yd.onion/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.183 Safari/537.36"
middleware           | umbrel-middleware
middleware           | ::ffff:10.11.0.2 - - [Tue, 01 Dec 2020 17:04:35 GMT] "GET /v1/lnd/info/status HTTP/1.0" 304 - "http://ec2-54-144-105-150.compute-1.amazonaws.com/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.183 Safari/537.36"
middleware           | umbrel-middleware
middleware           | ::ffff:10.11.0.2 - - [Tue, 01 Dec 2020 17:04:36 GMT] "GET /v1/lnd/info/status HTTP/1.0" 304 - "http://aw6iajzu7jhf4qcjahddudoot4hvkfjxpmdilo3xlgpbqssdhcjkk6yd.onion/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.183 Safari/537.36"
middleware           | umbrel-middleware
middleware           | ::ffff:10.11.0.2 - - [Tue, 01 Dec 2020 17:04:37 GMT] "GET /v1/lnd/info/status HTTP/1.0" 304 - "http://ec2-54-144-105-150.compute-1.amazonaws.com/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.183 Safari/537.36"
middleware           | umbrel-middleware
middleware           | ::ffff:10.11.0.2 - - [Tue, 01 Dec 2020 17:04:38 GMT] "GET /v1/lnd/info/status HTTP/1.0" 304 - "http://aw6iajzu7jhf4qcjahddudoot4hvkfjxpmdilo3xlgpbqssdhcjkk6yd.onion/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.183 Safari/537.36"
middleware           | umbrel-middleware
middleware           | ::ffff:10.11.0.2 - - [Tue, 01 Dec 2020 17:04:39 GMT] "GET /v1/lnd/info/status HTTP/1.0" 304 - "http://ec2-54-144-105-150.compute-1.amazonaws.com/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.183 Safari/537.36"
middleware           | umbrel-middleware
manager              | ::ffff:10.11.0.2 - - [Tue, 01 Dec 2020 17:09:39 GMT] "GET /v1/system/update-status HTTP/1.0" 304 - "http://ec2-54-144-105-150.compute-1.amazonaws.com/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.183 Safari/537.36"
manager              | umbrel-manager
manager              | ::ffff:10.11.0.2 - - [Tue, 01 Dec 2020 17:09:44 GMT] "GET /v1/system/update-status HTTP/1.0" 304 - "http://aw6iajzu7jhf4qcjahddudoot4hvkfjxpmdilo3xlgpbqssdhcjkk6yd.onion/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.183 Safari/537.36"
manager              | umbrel-manager
manager              | ::ffff:10.11.0.2 - - [Tue, 01 Dec 2020 17:10:39 GMT] "GET /v1/system/update-status HTTP/1.0" 304 - "http://ec2-54-144-105-150.compute-1.amazonaws.com/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.183 Safari/537.36"
manager              | umbrel-manager
manager              | ::ffff:10.11.0.2 - - [Tue, 01 Dec 2020 17:10:44 GMT] "GET /v1/system/update-status HTTP/1.0" 304 - "http://aw6iajzu7jhf4qcjahddudoot4hvkfjxpmdilo3xlgpbqssdhcjkk6yd.onion/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.183 Safari/537.36"
manager              | umbrel-manager
manager              | ::ffff:10.11.0.2 - - [Tue, 01 Dec 2020 17:11:39 GMT] "GET /v1/system/update-status HTTP/1.0" 304 - "http://ec2-54-144-105-150.compute-1.amazonaws.com/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.183 Safari/537.36"
manager              | umbrel-manager
manager              | ::ffff:10.11.0.2 - - [Tue, 01 Dec 2020 17:11:45 GMT] "GET /v1/system/update-status HTTP/1.0" 304 - "http://aw6iajzu7jhf4qcjahddudoot4hvkfjxpmdilo3xlgpbqssdhcjkk6yd.onion/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.183 Safari/537.36"
manager              | umbrel-manager
manager              | ::ffff:10.11.0.2 - - [Tue, 01 Dec 2020 17:12:39 GMT] "GET /v1/system/update-status HTTP/1.0" 304 - "http://ec2-54-144-105-150.compute-1.amazonaws.com/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.183 Safari/537.36"
manager              | umbrel-manager
manager              | ::ffff:10.11.0.2 - - [Tue, 01 Dec 2020 17:12:45 GMT] "GET /v1/system/update-status HTTP/1.0" 304 - "http://aw6iajzu7jhf4qcjahddudoot4hvkfjxpmdilo3xlgpbqssdhcjkk6yd.onion/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.183 Safari/537.36"
manager              | umbrel-manager
manager              | ::ffff:10.11.0.2 - - [Tue, 01 Dec 2020 17:13:39 GMT] "GET /v1/system/update-status HTTP/1.0" 304 - "http://ec2-54-144-105-150.compute-1.amazonaws.com/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.183 Safari/537.36"
manager              | umbrel-manager
manager              | ::ffff:10.11.0.2 - - [Tue, 01 Dec 2020 17:13:45 GMT] "GET /v1/system/update-status HTTP/1.0" 304 - "http://aw6iajzu7jhf4qcjahddudoot4hvkfjxpmdilo3xlgpbqssdhcjkk6yd.onion/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.183 Safari/537.36"
manager              | umbrel-manager
manager              | ::ffff:10.11.0.2 - - [Tue, 01 Dec 2020 17:14:39 GMT] "GET /v1/system/update-status HTTP/1.0" 304 - "http://ec2-54-144-105-150.compute-1.amazonaws.com/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.183 Safari/537.36"
manager              | umbrel-manager
manager              | ::ffff:10.11.0.2 - - [Tue, 01 Dec 2020 17:14:44 GMT] "GET /v1/system/update-status HTTP/1.0" 304 - "http://aw6iajzu7jhf4qcjahddudoot4hvkfjxpmdilo3xlgpbqssdhcjkk6yd.onion/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.183 Safari/537.36"
manager              | umbrel-manager
manager              | ::ffff:10.11.0.2 - - [Tue, 01 Dec 2020 17:14:47 GMT] "GET /v1/system/update-status HTTP/1.0" - - "http://ec2-54-144-105-150.compute-1.amazonaws.com/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.183 Safari/537.36 OPR/72.0.3815.320"
manager              | umbrel-manager
manager              | ::ffff:10.11.0.2 - - [Tue, 01 Dec 2020 17:14:48 GMT] "GET /v1/system/update-status HTTP/1.0" 200 91 "http://ec2-54-144-105-150.compute-1.amazonaws.com/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.183 Safari/537.36 OPR/72.0.3815.320"
manager              | umbrel-manager
manager              | ::ffff:10.11.0.2 - - [Tue, 01 Dec 2020 17:14:49 GMT] "GET /ping HTTP/1.0" 200 34 "http://ec2-54-144-105-150.compute-1.amazonaws.com/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.183 Safari/537.36 OPR/72.0.3815.320"
manager              | umbrel-manager
bitcoin              | 2020-12-01T16:51:11Z LoadBlockIndexDB: last block file = 284
bitcoin              | 2020-12-01T16:51:11Z LoadBlockIndexDB: last block file info: CBlockFileInfo(blocks=30, size=14932462, heights=361015...361410, time=2015-06-15...2015-06-18)
bitcoin              | 2020-12-01T16:51:11Z Checking all blk files are present...
bitcoin              | 2020-12-01T16:51:13Z Opening LevelDB in /data/.bitcoin/chainstate
bitcoin              | 2020-12-01T16:51:13Z Opened LevelDB successfully
bitcoin              | 2020-12-01T16:51:14Z Using obfuscation key for /data/.bitcoin/chainstate: 4d8a5881e599d642
bitcoin              | 2020-12-01T16:51:14Z Loaded best chain: hashBestChain=0000000000000000144b77936d3b194c15e00e6b4634bce4b414a3c39bc3a8bc height=361056 date=2015-06-15T12:46:11Z progress=0.122951
bitcoin              | 2020-12-01T16:51:14Z init message: Rewinding blocks...
bitcoin              | 2020-12-01T16:51:15Z FlushStateToDisk: write coins cache to disk (0 coins, 0kB) started
bitcoin              | 2020-12-01T16:51:15Z FlushStateToDisk: write coins cache to disk (0 coins, 0kB) completed (0.00s)
bitcoin              | 2020-12-01T16:51:15Z init message: Verifying blocks...
bitcoin              | 2020-12-01T16:51:15Z Verifying last 6 blocks at level 3
bitcoin              | 2020-12-01T16:51:15Z [0%]...[16%]...[33%]...[50%]...[66%]...[83%]...[99%]...[DONE].
bitcoin              | 2020-12-01T16:53:07Z No coin database inconsistencies in last 6 blocks (5415 transactions)
bitcoin              | 2020-12-01T16:53:07Z  block index          131118ms
bitcoin              | 2020-12-01T16:53:07Z Opening LevelDB in /data/.bitcoin/indexes/txindex
middleware           | ::ffff:10.11.0.2 - - [Tue, 01 Dec 2020 17:04:41 GMT] "GET /v1/lnd/info/status HTTP/1.0" 304 - "http://aw6iajzu7jhf4qcjahddudoot4hvkfjxpmdilo3xlgpbqssdhcjkk6yd.onion/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.183 Safari/537.36"
middleware           | umbrel-middleware
middleware           | ::ffff:10.11.0.2 - - [Tue, 01 Dec 2020 17:04:41 GMT] "GET /v1/lnd/info/status HTTP/1.0" 304 - "http://ec2-54-144-105-150.compute-1.amazonaws.com/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.183 Safari/537.36"
middleware           | umbrel-middleware
middleware           | ::ffff:10.11.0.2 - - [Tue, 01 Dec 2020 17:04:42 GMT] "GET /v1/lnd/info/status HTTP/1.0" 304 - "http://aw6iajzu7jhf4qcjahddudoot4hvkfjxpmdilo3xlgpbqssdhcjkk6yd.onion/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.183 Safari/537.36"
middleware           | umbrel-middleware
middleware           | ::ffff:10.11.0.2 - - [Tue, 01 Dec 2020 17:04:43 GMT] "GET /v1/lnd/info/status HTTP/1.0" 304 - "http://ec2-54-144-105-150.compute-1.amazonaws.com/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.183 Safari/537.36"
middleware           | umbrel-middleware
bitcoin              | 2020-12-01T16:53:15Z Opened LevelDB successfully
bitcoin              | 2020-12-01T16:53:15Z Using obfuscation key for /data/.bitcoin/indexes/txindex: 0000000000000000
bitcoin              | 2020-12-01T16:53:15Z txindex thread start
bitcoin              | 2020-12-01T16:53:15Z txindex is enabled at height 361056
bitcoin              | 2020-12-01T16:53:15Z txindex thread exit
bitcoin              | 2020-12-01T16:53:15Z Opening LevelDB in /data/.bitcoin/indexes/blockfilter/basic/db
bitcoin              | 2020-12-01T16:53:21Z Opened LevelDB successfully
bitcoin              | 2020-12-01T16:53:21Z Using obfuscation key for /data/.bitcoin/indexes/blockfilter/basic/db: 0000000000000000
bitcoin              | 2020-12-01T16:53:21Z init message: Loading wallet...
bitcoin              | 2020-12-01T16:53:21Z BerkeleyEnvironment::Open: LogDir=/data/.bitcoin/database ErrorFile=/data/.bitcoin/db.log
bitcoin              | 2020-12-01T16:53:21Z basic block filter index thread start
bitcoin              | 2020-12-01T16:53:21Z basic block filter index is enabled at height 361056
bitcoin              | 2020-12-01T16:53:21Z basic block filter index thread exit
bitcoin              | 2020-12-01T16:53:21Z [default wallet] Wallet File Version = 169900
bitcoin              | 2020-12-01T16:53:21Z [default wallet] Keys: 2001 plaintext, 0 encrypted, 2001 w/ metadata, 2001 total. Unknown wallet records: 0
bitcoin              | 2020-12-01T16:53:21Z [default wallet] Wallet completed loading in             749ms
bitcoin              | 2020-12-01T16:53:21Z [default wallet] setKeyPool.size() = 2000
bitcoin              | 2020-12-01T16:53:21Z [default wallet] mapWallet.size() = 0
bitcoin              | 2020-12-01T16:53:21Z [default wallet] m_address_book.size() = 0
bitcoin              | 2020-12-01T16:53:21Z block tree size = 659506
bitcoin              | 2020-12-01T16:53:21Z nBestHeight = 361056
bitcoin              | 2020-12-01T16:53:21Z Bound to 10.11.1.1:8333
bitcoin              | 2020-12-01T16:53:21Z init message: Loading P2P addresses...
bitcoin              | 2020-12-01T16:53:21Z torcontrol thread start
bitcoin              | 2020-12-01T16:53:22Z Loaded 63031 addresses from peers.dat  285ms
bitcoin              | 2020-12-01T16:53:22Z init message: Starting network threads...
bitcoin              | 2020-12-01T16:53:22Z init message: Done loading
bitcoin              | 2020-12-01T16:53:22Z msghand thread start
bitcoin              | 2020-12-01T16:53:22Z opencon thread start
bitcoin              | 2020-12-01T16:53:22Z addcon thread start
bitcoin              | 2020-12-01T16:53:22Z dnsseed thread start
bitcoin              | 2020-12-01T16:53:22Z net thread start
bitcoin              | 2020-12-01T16:53:33Z Loading addresses from DNS seed seed.bitcoinstats.com
bitcoin              | 2020-12-01T16:53:33Z Loading addresses from DNS seed dnsseed.bitcoin.dashjr.org
bitcoin              | 2020-12-01T16:53:33Z Loading addresses from DNS seed seed.bitcoin.jonasschnelli.ch
bitcoin              | 2020-12-01T16:53:36Z UpdateTip: new best=00000000000000000e5fe90d3ac76ad38cba8f8d3f3c5b6a61ddc1f69267823d height=361057 version=0x00000003 log2_work=82.93965 tx=72155002 date='2015-06-15T12:46:52Z' progress=0.122951 cache=0.7MiB(5143txo)
bitcoin              | 2020-12-01T16:53:44Z Loading addresses from DNS seed seed.btc.petertodd.org
bitcoin              | 2020-12-01T16:53:44Z Loading addresses from DNS seed dnsseed.emzy.de
bitcoin              | 2020-12-01T16:53:44Z Loading addresses from DNS seed seed.bitcoin.sprovoost.nl
bitcoin              | 2020-12-01T16:53:55Z Loading addresses from DNS seed dnsseed.bluematt.me
bitcoin              | 2020-12-01T16:53:55Z Loading addresses from DNS seed seed.bitcoin.sipa.be
bitcoin              | 2020-12-01T16:53:55Z 0 addresses found from DNS seeds
bitcoin              | 2020-12-01T16:53:55Z dnsseed thread exit
bitcoin              | 2020-12-01T16:54:49Z UpdateTip: new best=00000000000000000199981d19bb3e83286cdb7ceb023bb2533e226209173a59 height=361058 version=0x00000003 log2_work=82.939683 tx=72156060 date='2015-06-15T12:55:34Z' progress=0.122953 cache=1.2MiB(9216txo)
bitcoin              | 2020-12-01T16:54:49Z socket send error Broken pipe (32)
bitcoin              | 2020-12-01T16:55:51Z UpdateTip: new best=00000000000000000360e990bbcfa448dec7f3b47afce43f134a8e77f26e1f19 height=361059 version=0x00000003 log2_work=82.939716 tx=72156978 date='2015-06-15T13:04:42Z' progress=0.122954 cache=1.7MiB(12581txo)
bitcoin              | 2020-12-01T16:55:51Z socket send error Broken pipe (32)
bitcoin              | 2020-12-01T16:56:23Z UpdateTip: new best=00000000000000000f8b9b9dbe3af13a8ef6755f2403bc820d1cfc7dd981d9a4 height=361060 version=0x00000002 log2_work=82.93975 tx=72159291 date='2015-06-15T13:50:59Z' progress=0.122958 cache=2.5MiB(18507txo)
bitcoin              | 2020-12-01T16:57:19Z UpdateTip: new best=000000000000000015562326ddf2998366e905f019c097a68b7436a1e6b09031 height=361061 version=0x00000003 log2_work=82.939783 tx=72160465 date='2015-06-15T13:48:47Z' progress=0.122960 cache=3.0MiB(22246txo)
bitcoin              | 2020-12-01T16:58:21Z Pre-allocating up to position 0x1200000 in rev00282.dat
bitcoin              | 2020-12-01T16:58:22Z UpdateTip: new best=00000000000000000cc262ab35d706c7a9e61f07d13cd5c0277f193aa3110fe0 height=361062 version=0x00000003 log2_work=82.939816 tx=72162127 date='2015-06-15T14:01:16Z' progress=0.122963 cache=3.5MiB(26898txo)
bitcoin              | 2020-12-01T16:58:22Z socket send error Broken pipe (32)
bitcoin              | 2020-12-01T16:59:08Z UpdateTip: new best=0000000000000000083923252050cf1730e2577d1767f36bd14c5cb8128c7907 height=361063 version=0x00000003 log2_work=82.939849 tx=72163562 date='2015-06-15T14:18:00Z' progress=0.122965 cache=4.0MiB(30538txo)
bitcoin              | 2020-12-01T17:00:12Z UpdateTip: new best=000000000000000006a0fedef5c03782d1c785118832447d91d3d75c93e82c6f height=361064 version=0x00000003 log2_work=82.939882 tx=72165018 date='2015-06-15T14:34:32Z' progress=0.122968 cache=4.6MiB(34189txo)
bitcoin              | 2020-12-01T17:02:19Z UpdateTip: new best=00000000000000000d5c07be1fd17ce6a91695e6092e0f5f0d4dfb556a7c423f height=361065 version=0x00000003 log2_work=82.939916 tx=72166478 date='2015-06-15T14:46:42Z' progress=0.122970 cache=5.2MiB(38652txo)
bitcoin              | 2020-12-01T17:03:38Z UpdateTip: new best=000000000000000001213e3d9af899fc8ecbcf0146fe63b030db136acbdac25f height=361066 version=0x00000003 log2_work=82.939949 tx=72167102 date='2015-06-15T14:48:20Z' progress=0.122971 cache=6.2MiB(46632txo)
bitcoin              | 2020-12-01T17:03:38Z socket send error Broken pipe (32)
bitcoin              | 2020-12-01T17:03:42Z UpdateTip: new best=0000000000000000149b7c177da31fb8e6fbb0acbd74d61a9b2154b596a17107 height=361067 version=0x00000003 log2_work=82.939982 tx=72167265 date='2015-06-15T14:49:17Z' progress=0.122971 cache=6.2MiB(46870txo)
bitcoin              | 2020-12-01T17:04:34Z UpdateTip: new best=00000000000000000407e82afc5105e0f8ec0d5ab00136b10ec5972208ff8433 height=361068 version=0x00000003 log2_work=82.940015 tx=72169208 date='2015-06-15T15:09:28Z' progress=0.122975 cache=6.7MiB(50915txo)
bitcoin              | 2020-12-01T17:05:13Z UpdateTip: new best=000000000000000015eaba9b3aeff977fae06179b23f55b26e924970f1ba159b height=361069 version=0x00000002 log2_work=82.940048 tx=72170115 date='2015-06-15T15:15:32Z' progress=0.122976 cache=6.9MiB(52783txo)
bitcoin              | 2020-12-01T17:05:50Z UpdateTip: new best=00000000000000000d8dd5d3d69130d3a8c771b861f2145e582cf69570bfebce height=361070 version=0x00000002 log2_work=82.940082 tx=72172274 date='2015-06-15T15:54:38Z' progress=0.122980 cache=7.5MiB(57326txo)
bitcoin              | 2020-12-01T17:05:50Z New outbound peer connected: version: 70015, blocks=659505, peer=9 (full-relay)
bitcoin              | 2020-12-01T17:05:50Z UpdateTip: new best=000000000000000001a7b49cf354265046324db8921e50e12c2c7dba6c138bb1 height=361071 version=0x00000002 log2_work=82.940115 tx=72172275 date='2015-06-15T15:48:01Z' progress=0.122980 cache=7.5MiB(57327txo)
bitcoin              | 2020-12-01T17:07:03Z UpdateTip: new best=00000000000000000cc09c2cdcace44742f7cd95e37012b8e6c4d07cc0524e20 height=361072 version=0x00000003 log2_work=82.940148 tx=72174316 date='2015-06-15T16:12:10Z' progress=0.122983 cache=8.6MiB(62712txo)
bitcoin              | 2020-12-01T17:07:30Z UpdateTip: new best=00000000000000000ec3181fd1aaeaf9731902c9fe357302727c6d5202d78e9c height=361073 version=0x00000003 log2_work=82.940181 tx=72175336 date='2015-06-15T16:11:06Z' progress=0.122985 cache=8.9MiB(64812txo)
bitcoin              | 2020-12-01T17:07:51Z UpdateTip: new best=00000000000000000fd4dc1c29f2b69299b0178e6fcc4463b7eae7b1407084f9 height=361074 version=0x00000003 log2_work=82.940214 tx=72176758 date='2015-06-15T16:19:06Z' progress=0.122987 cache=9.2MiB(67318txo)
bitcoin              | 2020-12-01T17:07:51Z New outbound peer connected: version: 70015, blocks=659506, peer=11 (full-relay)
bitcoin              | 2020-12-01T17:08:40Z UpdateTip: new best=000000000000000003d7a61542ef0694375e58b377fbb62e2f58d70d1986626c height=361075 version=0x00000003 log2_work=82.940247 tx=72177521 date='2015-06-15T16:25:15Z' progress=0.122989 cache=9.8MiB(72607txo)
bitcoin              | 2020-12-01T17:08:48Z UpdateTip: new best=000000000000000001492bc6aadc1b1ef35e9a60e57b424c3ee5da71a7566b05 height=361076 version=0x00000002 log2_work=82.940281 tx=72177697 date='2015-06-15T16:28:32Z' progress=0.122989 cache=9.9MiB(72904txo)
bitcoin              | 2020-12-01T17:08:48Z New outbound peer connected: version: 70015, blocks=659506, peer=12 (full-relay)
bitcoin              | 2020-12-01T17:09:16Z UpdateTip: new best=00000000000000000dccb8a3d10e7827ea09919f38903b5681820d547e2bda67 height=361077 version=0x00000003 log2_work=82.940314 tx=72178579 date='2015-06-15T16:35:35Z' progress=0.122990 cache=10.1MiB(74474txo)
bitcoin              | 2020-12-01T17:09:29Z UpdateTip: new best=0000000000000000056c6b26e7ae6f9c80d6cbbede87e3fadfeac509a162a88b height=361078 version=0x00000003 log2_work=82.940347 tx=72178858 date='2015-06-15T16:36:57Z' progress=0.122991 cache=10.1MiB(75100txo)
bitcoin              | 2020-12-01T17:09:29Z socket send error Broken pipe (32)
bitcoin              | 2020-12-01T17:10:39Z Pre-allocating up to position 0x300000 in rev00283.dat
bitcoin              | 2020-12-01T17:10:39Z UpdateTip: new best=00000000000000000e4ca54408afffaf4281bfb60643b43cfaee6488acd731d8 height=361079 version=0x00000003 log2_work=82.94038 tx=72181057 date='2015-06-15T17:04:38Z' progress=0.122995 cache=10.7MiB(79465txo)
bitcoin              | 2020-12-01T17:10:39Z socket send error Broken pipe (32)
bitcoin              | 2020-12-01T17:10:39Z socket send error Broken pipe (32)
bitcoin              | 2020-12-01T17:12:18Z UpdateTip: new best=00000000000000000fb80913350585003b204477e08355ed36a1a3fe3e4334c5 height=361080 version=0x00000003 log2_work=82.940413 tx=72181937 date='2015-06-15T17:11:10Z' progress=0.122996 cache=11.1MiB(82575txo)
bitcoin              | 2020-12-01T17:12:18Z socket send error Broken pipe (32)
bitcoin              | 2020-12-01T17:13:17Z UpdateTip: new best=00000000000000000bd5264956ab37a2b7938d41d12b5659c1842c19fb4188d4 height=361081 version=0x00000003 log2_work=82.940447 tx=72183115 date='2015-06-15T17:21:41Z' progress=0.122998 cache=11.4MiB(85299txo)
bitcoin              | 2020-12-01T17:13:17Z socket send error Broken pipe (32)
bitcoin              | 2020-12-01T17:13:51Z UpdateTip: new best=00000000000000000e423d10eca202b97fb495bfe499541603bd72ff5c5fc0ef height=361082 version=0x00000003 log2_work=82.94048 tx=72183604 date='2015-06-15T17:24:46Z' progress=0.122999 cache=11.6MiB(87228txo)
bitcoin              | 2020-12-01T17:14:50Z UpdateTip: new best=000000000000000014582f5bf6900e755f7adb7b3511d4d76c1df67f20b59922 height=361083 version=0x00000002 log2_work=82.940513 tx=72185816 date='2015-06-15T17:56:48Z' progress=0.123002 cache=12.1MiB(91315txo)
bitcoin              | 2020-12-01T17:14:50Z socket send error Broken pipe (32)
tor                  | Dec 01 16:50:49.268 [notice] Tor 0.4.1.9 running on Linux with Libevent 2.1.11-stable, OpenSSL 1.1.1g, Zlib 1.2.11, Liblzma N/A, and Libzstd N/A.
tor                  | Dec 01 16:50:49.268 [notice] Tor can't help you if you use it wrong! Learn how to be safe at https://www.torproject.org/download/download#warning
tor                  | Dec 01 16:50:49.280 [notice] Read configuration file "/etc/tor/torrc".
tor                  | Dec 01 16:50:49.290 [warn] You have a ControlPort set to accept connections from a non-local address.  This means that programs not running on your computer can reconfigure your Tor.  That's pretty bad, since the controller protocol isn't encrypted!  Maybe you should just listen on 127.0.0.1 and use a tool like stunnel or ssh to encrypt remote connections to your control port.
tor                  | Dec 01 16:50:49.361 [notice] You configured a non-loopback address '10.11.5.1:29050' for SocksPort. This allows everybody on your local network to use your machine as a proxy. Make sure this is what you wanted.
tor                  | Dec 01 16:50:49.361 [warn] You have a ControlPort set to accept connections from a non-local address.  This means that programs not running on your computer can reconfigure your Tor.  That's pretty bad, since the controller protocol isn't encrypted!  Maybe you should just listen on 127.0.0.1 and use a tool like stunnel or ssh to encrypt remote connections to your control port.
tor                  | Dec 01 16:50:49.361 [notice] Opening Socks listener on 10.11.5.1:29050
tor                  | Dec 01 16:50:49.361 [notice] Opened Socks listener on 10.11.5.1:29050
tor                  | Dec 01 16:50:49.361 [notice] Opening Control listener on 10.11.5.1:29051
tor                  | Dec 01 16:50:49.361 [notice] Opened Control listener on 10.11.5.1:29051
tor                  | Dec 01 16:50:49.000 [notice] Parsing GEOIP IPv4 file /usr/share/tor/geoip.
tor                  | Dec 01 16:50:49.000 [notice] Parsing GEOIP IPv6 file /usr/share/tor/geoip6.
tor                  | Dec 01 16:50:49.000 [notice] Bootstrapped 0% (starting): Starting
tor                  | Dec 01 16:50:49.000 [notice] Starting with guard context "default"
tor                  | Dec 01 16:50:51.000 [notice] Bootstrapped 5% (conn): Connecting to a relay
tor                  | Dec 01 16:50:51.000 [notice] Bootstrapped 10% (conn_done): Connected to a relay
tor                  | Dec 01 16:50:51.000 [notice] Bootstrapped 14% (handshake): Handshaking with a relay
tor                  | Dec 01 16:50:51.000 [notice] Bootstrapped 15% (handshake_done): Handshake with a relay done
tor                  | Dec 01 16:50:51.000 [notice] Bootstrapped 20% (onehop_create): Establishing an encrypted directory connection
neutrino-switcher    | Checking if synced...
tor                  | Dec 01 16:50:51.000 [notice] Bootstrapped 25% (requesting_status): Asking for networkstatus consensus
tor                  | Dec 01 16:50:51.000 [notice] Bootstrapped 30% (loading_status): Loading networkstatus consensus
tor                  | Dec 01 16:50:51.000 [notice] I learned some more directory information, but not enough to build a circuit: We have no usable consensus.
tor                  | Dec 01 16:50:51.000 [notice] Bootstrapped 40% (loading_keys): Loading authority key certs
tor                  | Dec 01 16:50:52.000 [notice] The current consensus has no exit nodes. Tor can only build internal paths, such as paths to onion services.
tor                  | Dec 01 16:50:52.000 [notice] Bootstrapped 45% (requesting_descriptors): Asking for relay descriptors
tor                  | Dec 01 16:50:52.000 [notice] I learned some more directory information, but not enough to build a circuit: We need more microdescriptors: we have 0/6912, and can only build 0% of likely paths. (We have 0% of guards bw, 0% of midpoint bw, and 0% of end bw (no exits in consensus, using mid) = 0% of path bw.)
tor                  | Dec 01 16:50:52.000 [notice] Bootstrapped 50% (loading_descriptors): Loading relay descriptors
tor                  | Dec 01 16:50:53.000 [notice] The current consensus contains exit nodes. Tor can build exit and internal paths.
tor                  | Dec 01 16:50:56.000 [notice] Bootstrapped 57% (loading_descriptors): Loading relay descriptors
tor                  | Dec 01 16:50:56.000 [notice] Bootstrapped 63% (loading_descriptors): Loading relay descriptors
tor                  | Dec 01 16:50:56.000 [notice] Bootstrapped 68% (loading_descriptors): Loading relay descriptors
tor                  | Dec 01 16:50:57.000 [notice] Bootstrapped 75% (enough_dirinfo): Loaded enough directory info to build circuits
tor                  | Dec 01 16:50:57.000 [notice] Bootstrapped 80% (ap_conn): Connecting to a relay to build circuits
tor                  | Dec 01 16:50:57.000 [notice] Bootstrapped 85% (ap_conn_done): Connected to a relay to build circuits
tor                  | Dec 01 16:50:57.000 [notice] Bootstrapped 89% (ap_handshake): Finishing handshake with a relay to build circuits
tor                  | Dec 01 16:50:58.000 [notice] Bootstrapped 90% (ap_handshake_done): Handshake finished with a relay to build circuits
tor                  | Dec 01 16:50:58.000 [notice] Bootstrapped 95% (circuit_create): Establishing a Tor circuit
tor                  | Dec 01 16:50:58.000 [notice] Bootstrapped 100% (done): Done
neutrino-switcher    | If set to neutrino then lets check bitcoind
neutrino-switcher    | Error: from bitcoind
neutrino-switcher    | {
neutrino-switcher    |   "code": -28,
neutrino-switcher    |   "message": "Loading block index..."
neutrino-switcher    | }
nginx                | nginx: [warn] 1337 worker_connections exceed open file resource limit: 1024
Jaxo picture Jaxo on 1 Dec 2020

Everything looks normal from the logs. I think the EC2 instance is most likely running out of RAM and CPU juice as the block validation is getting harder and harder with bigger blocks. You could also try to create a 4GB swap on the disk and see if that helps.

Even if you get that working, I'm not sure if the instance will ever be able to catch up with the chain.

If you're up for more adventure, you could try doing the sync on a much more powerful instance + SSD, and then migrate to a less powerful instance + HDD:

  • Spin a much more powerful spot instance (like t3.xlarge) with an SSD (first please make sure if AWS allows you to restore a snapshot of an SSD volume on an HDD volume).
  • Do a fresh install of Umbrel on the SSD.
  • Wait for it to sync the chain (should take about ~12-18 hours or so).
  • After the sync is complete, stop Umbrel.
  • Detach the SSD from the instance.
  • Terminate the instance.
  • Create a snapshot of the SSD volume.
  • Discard the SSD after snapshot is successful.
  • Create an HDD volume from the SSD's snapshot.
  • Launch a less powerful AWS spot instance (t2.small), attach the HDD volume and create a 4GB swapfile.
  • Run Umbrel from the HDD.
  • (Hopefully) tada! 100% synced blockchain, t2.small can now slowly and steadily keep up with new blocks.
mayankchhabra picture mayankchhabra on 1 Dec 2020
👍1

Thanks for looking at it, and the suggestion! I may try it with the larger instance & SSD--you're right, it'll never catch up as is-- or I'll simply write this off as a failed economical-cloud experiment and just go buy a Pi and run it in-house.

Jaxo picture Jaxo on 1 Dec 2020

Awesome! Nothing beats the experience of running Umbrel on self-owned hardware.

Thanks for sharing your experience and tips, @Jaxo! Closing this issue now, and I've renamed the title for better visibility and SEO.

mayankchhabra picture mayankchhabra on 2 Dec 2020

Btw, @mayankchhabra, since I had to try it...your suggestion about 1st syncing with a t3/ssd and then snapshotting to less expensive instance/disk worked quite well! --joe

Jaxo picture Jaxo on 10 Dec 2020

562 will also help for people with shared systems too.

junderw picture junderw on 23 Feb 2021

Hi Folks, this is a bit off-topic, but in the log above there is a warning from Tor:
You have a ControlPort set to accept connections from a non-local address. This means that programs not running on your computer can reconfigure your Tor. That's pretty bad, since the controller protocol isn't encrypted! Maybe you should just listen on 127.0.0.1 and use a tool like stunnel or ssh to encrypt remote connections to your control port

I have several Umbrel instances running and one of them is running in a Linux environment in the cloud. I am a bit concerned about this error. At the moment, this instance is only running as a test system, so it's not that bad yet.

zernaIO picture zernaIO on 14 Mar 2021

Admittedly, @zernaIO, I haven't looked into any potential security issues with Umbrel and the most secure way to configure one with a public IP (my AWS instance is for experimentation).

Jaxo picture Jaxo on 15 Mar 2021

In AWS its actually quite easy, you can run it there without any issues, if you setup correctly security group and not allow everything towards your EC2 instance.

You should be then able to access Umbrel via Tor URL :)

downet picture downet on 15 Mar 2021
👍1

@zernaIO The Tor error is a warning for people running tor on host OS. tor is running in a container and the container is only exposing that port to a local address, so that other containers (bitcoind etc) can access it.

Since containers think they are their own OS, Tor thinks "OH NO MY CONTROL PORT IS ACCESIBLE FROM THE INTERNET!!!" but the "internet" in this case, unknown to the container, just means "the docker process of the host machine"

junderw picture junderw on 15 Mar 2021
👍1

Thanks, @downet, I had hoped it was that straightforward. So, I gather if I only allow port 22 inbound and "all" outbound on my EC2 instance, I'm good (assuming I'm only going to access it through tor)?

And thanks, @junderw, for the tor warning message clarification.

Jaxo picture Jaxo on 15 Mar 2021

Hi @downet and @junderw , thank you for this quick and quality explanation!

zernaIO picture zernaIO on 15 Mar 2021

@Jaxo Yep, incoming just keep 22 (best would be to change the port to something else like 2222 -> 22 if possible) and outgoing keep open.

Also good practice is try to check if any port from netstat -ntlp is opened or not (you should not be able to connect there if it's closed.

downet picture downet on 15 Mar 2021
👍1

I've also just got this working nice last night with spot instances - I chose Ubuntu and I installed all what I needed as a base, before creating an AMI to use for it.
I'm using an Elastic IP so added the following user data script:

```

cloud-config

apt_upgrade: true
runcmd:

  • hostnamectl set-hostname
  • sudo /home/ubuntu/scripts/start
    ```

@downet - Just to confirm what you're saying. I only need to allow inbound on port 22 for SSH?

chearmstrong picture chearmstrong on 18 May 2021

Just to confirm what you're saying. I only need to allow inbound on port 22 for SSH?

Correct, @chearmstrong, if you're only going to use tor to connect to it.

jlatone picture jlatone on 18 May 2021
👍1
Was this page helpful?
0 / 5 - 0 ratings

Related issues

mayankchhabra picture mayankchhabra  ·  4Comments
jaonoctus picture jaonoctus  ·  3Comments
rstmsn picture rstmsn  ·  5Comments
esackbauer picture esackbauer  ·  9Comments
junderw picture junderw  ·  3Comments