Raspiblitz: LNBits 502 Bad Gateway on fresh install on 1.6.0

[philbertw4]

I just successfully upgraded to version 1.6.0 from 1.5.x. I had never enabled LNBits in my prior setup, but after upgrading to 1.6.0 I figured to give LNBits a try. So I simply followed the instructions in the menu:

1. Services -- > LNBits (marked x to enable, pressed enter)
2. From what I could tell the setup process for LNBits completed smoothly
3. I was then shown the dialog with the address to access LNBits on my local network (dialog also included onion address to access over tor)

Visiting the address, I was then greeted with a 502 Bad Gateway error served up by nginx. Is there something I should do to help diagnose further for you?

[frennkie]

There were some last minute attempts to fix some things..

Please check sudo systemctl status lnbits

[philbertw4]

Thanks for the reply. Here is the output:

admin@raspberrypi:~ $ sudo systemctl status lnbits
● lnbits.service - lnbits
   Loaded: loaded (/etc/systemd/system/lnbits.service; enabled; vendor preset: enabled)
   Active: activating (auto-restart) (Result: exit-code) since Fri 2020-08-07 00:44:32 BST; 24s ago
  Process: 28886 ExecStart=/bin/sh -c cd /home/lnbits/lnbits && pipenv run gunicorn -b 127.0.0.1:5000 lnbits:app -k gevent (code=exited, status=1/FAILURE)
 Main PID: 28886 (code=exited, status=1/FAILURE)
admin@raspberrypi:~ $ 

I do not know yet if it is related, but I seem to remember that before the update to 1.6.0 my bash prompt was admin@<mynodesname>:~ $ whereas now it is the more generic admin@raspberrypi. This makes me think that somehow the hostname did not survive the update? Edit: probably not hostname issue, since it looks like it is purposeful that the blitz keeps hostname at the default/generic raspberrypi (from https://github.com/rootzoll/raspiblitz/issues/819)

[rootzoll]

When you go MAINMENU > REPAIR > SOFTWARE you see a lot of debug info ... can you copy paste the LNbits part here?

[philbertw4]

Ok, during the re-install of LNBits (I had uninstalled it via the menu after opening this issue), I was able to capture the following output which seems to indicated a couple errors:

make sure symlink to central app-data directory exists
# OK - macaroons written to /home/lnbits/lnbits/.env
# installing application dependencies
Creating a virtualenv for this project…
Using /usr/bin/python3.7m (3.7.3) to create virtualenv…
⠋Running virtualenv with interpreter /usr/bin/python3.7m
Using base prefix '/usr'
/usr/lib/python3/dist-packages/virtualenv.py:1090: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
  import imp
New python executable in /home/lnbits/.local/share/virtualenvs/lnbits-xqYMrTZ5/bin/python3.7m
Also creating executable in /home/lnbits/.local/share/virtualenvs/lnbits-xqYMrTZ5/bin/python
Installing setuptools, pkg_resources, pip, wheel...done.

Virtualenv location: /home/lnbits/.local/share/virtualenvs/lnbits-xqYMrTZ5
Installing dependencies from Pipfile.lock (2952f4)…
An error occurred while installing chardet==3.0.4! Will try again.
An error occurred while installing gevent==20.6.2! Will try again.
  🐍   ▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉ 31/31 — 00:02:09
Installing initially–failed dependencies…
Success installing chardet==3.0.4!▉▉▉ 0/2 — 00:00:00
Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
Collecting gevent==20.6.2
  Using cached gevent-20.6.2.tar.gz (5.8 MB)
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'error'

  ERROR: Command errored out with exit status 1:
   command: /home/lnbits/.local/share/virtualenvs/lnbits-xqYMrTZ5/bin/python3.7m /home/lnbits/.local/share/virtualenvs/lnbits-xqYMrTZ5/lib/python3.7/site-packages/pip/_vendor/pep517/_in_process.py get_requires_for_build_wheel /tmp/tmp8q1bm821
       cwd: /tmp/pip-install-pwy5cfev/gevent
  Complete output (62 lines):
  Compiling src/gevent/resolver/cares.pyx because it depends on /tmp/pip-build-env-b1ry45wf/overlay/lib/python3.7/site-packages/Cython/Includes/libc/string.pxd.
  [1/1] Cythonizing src/gevent/resolver/cares.pyx
  Traceback (most recent call last):
    File "/home/lnbits/.local/share/virtualenvs/lnbits-xqYMrTZ5/lib/python3.7/site-packages/pip/_vendor/pep517/_in_process.py", line 280, in <module>
      main()
    File "/home/lnbits/.local/share/virtualenvs/lnbits-xqYMrTZ5/lib/python3.7/site-packages/pip/_vendor/pep517/_in_process.py", line 263, in main
      json_out['return_val'] = hook(**hook_input['kwargs'])
    File "/home/lnbits/.local/share/virtualenvs/lnbits-xqYMrTZ5/lib/python3.7/site-packages/pip/_vendor/pep517/_in_process.py", line 114, in get_requires_for_build_wheel
      return hook(config_settings)
    File "/tmp/pip-build-env-b1ry45wf/overlay/lib/python3.7/site-packages/setuptools/build_meta.py", line 147, in get_requires_for_build_wheel
      config_settings, requirements=['wheel'])
    File "/tmp/pip-build-env-b1ry45wf/overlay/lib/python3.7/site-packages/setuptools/build_meta.py", line 127, in _get_build_requires
      self.run_setup()
    File "/tmp/pip-build-env-b1ry45wf/overlay/lib/python3.7/site-packages/setuptools/build_meta.py", line 249, in run_setup
      self).run_setup(setup_script=setup_script)
    File "/tmp/pip-build-env-b1ry45wf/overlay/lib/python3.7/site-packages/setuptools/build_meta.py", line 142, in run_setup
      exec(compile(code, __file__, 'exec'), locals())
    File "setup.py", line 47, in <module>
      from _setupares import ARES
    File "/tmp/pip-install-pwy5cfev/gevent/_setupares.py", line 129, in <module>
      ARES = cythonize1(ARES)
    File "/tmp/pip-install-pwy5cfev/gevent/_setuputils.py", line 260, in cythonize1
      common_utility_include_dir=COMMON_UTILITY_INCLUDE_DIR,
    File "/tmp/pip-build-env-b1ry45wf/overlay/lib/python3.7/site-packages/Cython/Build/Dependencies.py", line 1110, in cythonize
      cythonize_one(*args)
    File "/tmp/pip-build-env-b1ry45wf/overlay/lib/python3.7/site-packages/Cython/Build/Dependencies.py", line 1256, in cythonize_one
      result = compile_single(pyx_file, options, full_module_name=full_module_name)
    File "/tmp/pip-build-env-b1ry45wf/overlay/lib/python3.7/site-packages/Cython/Compiler/Main.py", line 575, in compile_single
      return run_pipeline(source, options, full_module_name)
    File "/tmp/pip-build-env-b1ry45wf/overlay/lib/python3.7/site-packages/Cython/Compiler/Main.py", line 494, in run_pipeline
      pipeline = Pipeline.create_pyx_pipeline(context, options, result)
    File "/tmp/pip-build-env-b1ry45wf/overlay/lib/python3.7/site-packages/Cython/Compiler/Pipeline.py", line 252, in create_pyx_pipeline
      create_pipeline(context, mode, exclude_classes=exclude_classes),
    File "/tmp/pip-build-env-b1ry45wf/overlay/lib/python3.7/site-packages/Cython/Compiler/Pipeline.py", line 144, in create_pipeline
      from .ParseTreeTransforms import WithTransform, NormalizeTree, PostParse, PxdPostParse
    File "/tmp/pip-build-env-b1ry45wf/overlay/lib/python3.7/site-packages/Cython/Compiler/ParseTreeTransforms.py", line 1632, in <module>
      class AnalyseDeclarationsTransform(EnvTransform):
    File "/tmp/pip-build-env-b1ry45wf/overlay/lib/python3.7/site-packages/Cython/Compiler/ParseTreeTransforms.py", line 1640, in AnalyseDeclarationsTransform
      """, level='c_class', pipeline=[NormalizeTree(None)])
    File "/tmp/pip-build-env-b1ry45wf/overlay/lib/python3.7/site-packages/Cython/Compiler/TreeFragment.py", line 235, in __init__
      mod = t = parse_from_strings(name, fmt_code, fmt_pxds, level=level, initial_pos=initial_pos)
    File "/tmp/pip-build-env-b1ry45wf/overlay/lib/python3.7/site-packages/Cython/Compiler/TreeFragment.py", line 75, in parse_from_strings
      scope = scope, context = context, initial_pos = initial_pos)
    File "Cython/Compiler/Scanning.py", line 306, in Cython.Compiler.Scanning.PyrexScanner.__init__
    File "Cython/Compiler/Scanning.py", line 35, in Cython.Compiler.Scanning.get_lexicon
    File "/tmp/pip-build-env-b1ry45wf/overlay/lib/python3.7/site-packages/Cython/Compiler/Lexicon.py", line 147, in make_lexicon
      (Eof, Method('eof_action'))
    File "/tmp/pip-build-env-b1ry45wf/overlay/lib/python3.7/site-packages/Cython/Plex/Lexicons.py", line 129, in __init__
      nfa, default_initial_state, spec, token_number)
    File "/tmp/pip-build-env-b1ry45wf/overlay/lib/python3.7/site-packages/Cython/Plex/Lexicons.py", line 162, in add_token_to_machine
      match_bol=1, nocase=0)
    File "/tmp/pip-build-env-b1ry45wf/overlay/lib/python3.7/site-packages/Cython/Plex/Regexps.py", line 301, in build_machine
      re.build_machine(m, s1, s2, match_bol, nocase)
    File "/tmp/pip-build-env-b1ry45wf/overlay/lib/python3.7/site-packages/Cython/Plex/Regexps.py", line 340, in build_machine
      initial_state = self.build_opt(m, initial_state, BOL)
    File "/tmp/pip-build-env-b1ry45wf/overlay/lib/python3.7/site-packages/Cython/Plex/Regexps.py", line 136, in build_opt
      initial_state.link_to(s)
    File "Cython/Plex/Machines.py", line 86, in Cython.Plex.Machines.Node.link_to
    File "Cython/Plex/Machines.py", line 82, in Cython.Plex.Machines.Node.add_transition
    File "Cython/Plex/Transitions.py", line 60, in Cython.Plex.Transitions.TransitionMap.add
    File "Cython/Plex/Machines.py", line 124, in Cython.Plex.Machines.Node.__hash__
  OverflowError: Python int too large to convert to C ssize_t
  ----------------------------------------
ERROR: Command errored out with exit status 1: /home/lnbits/.local/share/virtualenvs/lnbits-xqYMrTZ5/bin/python3.7m /home/lnbits/.local/share/virtualenvs/lnbits-xqYMrTZ5/lib/python3.7/site-packages/pip/_vendor/pep517/_in_process.py get_requires_for_build_wheel /tmp/tmp8q1bm821 Check the logs for full command output.

Below is the output from the LNBits portion of the debug dump you requested:

*** LAST 20 LNbits LOGS ***
sudo journalctl -u lnbits -b --no-pager -n20
-- Logs begin at Sun 2020-08-09 12:57:07 BST, end at Sun 2020-08-09 21:38:46 BST. --
Aug 09 21:38:18 raspberrypi sh[8239]:     import gevent
Aug 09 21:38:18 raspberrypi sh[8239]: ModuleNotFoundError: No module named 'gevent'
Aug 09 21:38:18 raspberrypi sh[8239]: During handling of the above exception, another exception occurred:
Aug 09 21:38:18 raspberrypi sh[8239]: Traceback (most recent call last):
Aug 09 21:38:18 raspberrypi sh[8239]:   File "/home/lnbits/.local/share/virtualenvs/lnbits-xqYMrTZ5/lib/python3.7/site-packages/gunicorn/util.py", line 99, in load_class
Aug 09 21:38:18 raspberrypi sh[8239]:     mod = importlib.import_module('.'.join(components))
Aug 09 21:38:18 raspberrypi sh[8239]:   File "/home/lnbits/.local/share/virtualenvs/lnbits-xqYMrTZ5/lib/python3.7/importlib/__init__.py", line 127, in import_module
Aug 09 21:38:18 raspberrypi sh[8239]:     return _bootstrap._gcd_import(name[level:], package, level)
Aug 09 21:38:18 raspberrypi sh[8239]:   File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
Aug 09 21:38:18 raspberrypi sh[8239]:   File "<frozen importlib._bootstrap>", line 983, in _find_and_load
Aug 09 21:38:18 raspberrypi sh[8239]:   File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
Aug 09 21:38:18 raspberrypi sh[8239]:   File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
Aug 09 21:38:18 raspberrypi sh[8239]:   File "<frozen importlib._bootstrap_external>", line 728, in exec_module
Aug 09 21:38:18 raspberrypi sh[8239]:   File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
Aug 09 21:38:18 raspberrypi sh[8239]:   File "/home/lnbits/.local/share/virtualenvs/lnbits-xqYMrTZ5/lib/python3.7/site-packages/gunicorn/workers/ggevent.py", line 16, in <module>
Aug 09 21:38:18 raspberrypi sh[8239]:     raise RuntimeError("gevent worker requires gevent 1.4 or higher")
Aug 09 21:38:18 raspberrypi sh[8239]: RuntimeError: gevent worker requires gevent 1.4 or higher
Aug 09 21:38:18 raspberrypi sh[8239]: ]
Aug 09 21:38:18 raspberrypi systemd[1]: lnbits.service: Main process exited, code=exited, status=1/FAILURE
Aug 09 21:38:18 raspberrypi systemd[1]: lnbits.service: Failed with result 'exit-code'.

Looks like the install of lnbits failed for this gevent library dependency it has?

[rootzoll]

OK something is off with the dependencies in the LNbits "Pipfile.lock" (gevent is missing) .. will check for v1.6.1 service release.

[rootzoll]

@philbertw4 can you try to the re-install lnbits ... the raspiblitz tag got updated

[philbertw4]

Ok, just tried it now. Unfortunately it is stalling at step 15/29. I'll give it some more time. Here's the output so far:

# get the github code
Cloning into 'lnbits'...
remote: Enumerating objects: 121, done.
remote: Counting objects: 100% (121/121), done.
remote: Compressing objects: 100% (104/104), done.
remote: Total 4401 (delta 70), reused 50 (delta 13), pack-reused 4280
Receiving objects: 100% (4401/4401), 11.45 MiB | 829.00 KiB/s, done.
Resolving deltas: 100% (2690/2690), done.
# checking out tag 'raspiblitz'
Note: checking out 'tags/raspiblitz'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:

  git checkout -b <new-branch-name>

HEAD is now at dd2a282 Fixed migration bug
# preparing env file
make sure symlink to central app-data directory exists
# OK - macaroons written to /home/lnbits/lnbits/.env
# installing application dependencies
Creating a virtualenv for this project…
Using /usr/bin/python3.7m (3.7.3) to create virtualenv…
⠋Running virtualenv with interpreter /usr/bin/python3.7m
Using base prefix '/usr'
/usr/lib/python3/dist-packages/virtualenv.py:1090: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
  import imp
New python executable in /home/lnbits/.local/share/virtualenvs/lnbits-xqYMrTZ5/bin/python3.7m
Also creating executable in /home/lnbits/.local/share/virtualenvs/lnbits-xqYMrTZ5/bin/python
Installing setuptools, pkg_resources, pip, wheel...done.

Virtualenv location: /home/lnbits/.local/share/virtualenvs/lnbits-xqYMrTZ5
Installing dependencies from Pipfile.lock (2952f4)…
  🐍   ▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉ 15/29 — 00:00:0

[philbertw4]

Ok, after some minutes (I thought it was stalled permanently and was about to Ctrl+C) it did successfully install, and I verified that it is working. There is an SSL error in the browser since the certificate is self-signed, but in general it now seems to work. Thanks!

[rootzoll]

Yeah the self-signed cert is for basic HTTPS .. with IP2TOR&LetsEncrypt you can get a valid HTTPS cert and even make LNbits reachable from the outside whiloe running behind TOR.

Closing this issue.

[philbertw4]

Sorry for re-opening this @rootzoll . Using the raspiblitz menus, I uninstalled LNBits (a few days ago) and then re-installed it (today). On the re-install, there was another error pertaining to this gevent npm module (I know nothing about npm), and now I am back to the 502 gateway issue. So, unfortunately we didn't seem to quite fix this yet. Debug output below.

*** LAST 20 LNbits LOGS ***
sudo journalctl -u lnbits -b --no-pager -n20
-- Logs begin at Mon 2020-08-17 11:46:57 BST, end at Mon 2020-08-17 20:35:04 BST. --
Aug 17 20:34:38 raspberrypi sh[22988]:     import gevent
Aug 17 20:34:38 raspberrypi sh[22988]: ModuleNotFoundError: No module named 'gevent'
Aug 17 20:34:38 raspberrypi sh[22988]: During handling of the above exception, another exception occurred:
Aug 17 20:34:38 raspberrypi sh[22988]: Traceback (most recent call last):
Aug 17 20:34:38 raspberrypi sh[22988]:   File "/home/lnbits/.local/share/virtualenvs/lnbits-xqYMrTZ5/lib/python3.7/site-packages/gunicorn/util.py", line 99, in load_class
Aug 17 20:34:38 raspberrypi sh[22988]:     mod = importlib.import_module('.'.join(components))
Aug 17 20:34:38 raspberrypi sh[22988]:   File "/home/lnbits/.local/share/virtualenvs/lnbits-xqYMrTZ5/lib/python3.7/importlib/__init__.py", line 127, in import_module
Aug 17 20:34:38 raspberrypi sh[22988]:     return _bootstrap._gcd_import(name[level:], package, level)
Aug 17 20:34:38 raspberrypi sh[22988]:   File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
Aug 17 20:34:38 raspberrypi sh[22988]:   File "<frozen importlib._bootstrap>", line 983, in _find_and_load
Aug 17 20:34:38 raspberrypi sh[22988]:   File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
Aug 17 20:34:38 raspberrypi sh[22988]:   File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
Aug 17 20:34:38 raspberrypi sh[22988]:   File "<frozen importlib._bootstrap_external>", line 728, in exec_module
Aug 17 20:34:38 raspberrypi sh[22988]:   File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
Aug 17 20:34:38 raspberrypi sh[22988]:   File "/home/lnbits/.local/share/virtualenvs/lnbits-xqYMrTZ5/lib/python3.7/site-packages/gunicorn/workers/ggevent.py", line 16, in <module>
Aug 17 20:34:38 raspberrypi sh[22988]:     raise RuntimeError("gevent worker requires gevent 1.4 or higher")
Aug 17 20:34:38 raspberrypi sh[22988]: RuntimeError: gevent worker requires gevent 1.4 or higher
Aug 17 20:34:38 raspberrypi sh[22988]: ]
Aug 17 20:34:38 raspberrypi systemd[1]: lnbits.service: Main process exited, code=exited, status=1/FAILURE
Aug 17 20:34:38 raspberrypi systemd[1]: lnbits.service: Failed with result 'exit-code'.

[ChuckNorrison]

hey, pelase update your raspiblitz via menu, this gevent error got fixed

[philbertw4]

@ChuckNorrison thanks. I updated my github fork for branch 1.6 and patched my blitz to sync with that (via the menu like you suggested). LNBits now works!