Fastlane: Rare crash on startup in fastlane_folder.rb: in `join': no implicit conversion of nil into String

Created on 23 Jun 2018 · 121Comments · Source: fastlane/fastlane

New Issue Checklist

[x] Updated fastlane to the latest version
[x] I read the Contribution Guidelines
[x] I read docs.fastlane.tools
[x] I searched for existing GitHub issues

Issue Description

We currently run fastlane-2.96.0 on a build server (running buildbot-1.2.0), and occasionally notice a build failure due to one of fastlane's actions failing. The actual action itself seems to be irrelevant (cert, sigh, etc.), however it seems to happen more often when the build server machine is under a heavy load.

Rerunning the build again shortly after the build fails has so far always succeeded, so this is more or less a small annoyance than anything problematic.

We also haven't seen this happen before 2.96.0.

I am aware we aren't using the latest version of fastlane right now, but I decided to submit this report before I forget about it. :) I'll update to 2.98.0 right after this and post a reply here if I see it happen again after the update.

I'll also mention that fastlane suggests I run "gem cleanup" to speed up the launch, but we already do this after every fastlane update. The build machine simultaneously runs multiple compiler processes across multiple VMs, so an occasional slow start is not at all unexpected for our use case.

Complete output when running fastlane, including the stack trace and command used

bash -c 'env FASTLANE_PASSWORD=$(< fastlane/xyz.key.txt) fastlane cert'

 in dir /Volumes/Buildbot/buildbot-osx/CrystalClash-iOS/build (timeout 1200 secs)

 watching logfiles {}

 argv: ['bash', '-c', 'env FASTLANE_PASSWORD=$(< fastlane/xyz.key.txt) fastlane cert']

 environment:

  Apple_PubSub_Socket_Render=/private/tmp/com.apple.launchd.i0vQVsLZRk/Render

  DISPLAY=/private/tmp/com.apple.launchd.htpenpkJin/org.macosforge.xquartz:0

  EDITOR=vi

  HOME=/Users/falken

  LC_CTYPE=UTF-8

  LOGNAME=falken

  PATH=/Users/falken/.fastlane/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/X11/bin

  PWD=/Volumes/Buildbot/buildbot-osx/CrystalClash-iOS/build

  SECURITYSESSIONID=186a6

  SHELL=/bin/bash

  SHLVL=1

  SSH_AUTH_SOCK=/private/tmp/com.apple.launchd.GtMWyocuk4/Listeners

  TERM=xterm-256color

  TERM_PROGRAM=Apple_Terminal

  TERM_PROGRAM_VERSION=404

  TERM_SESSION_ID=21E43212-25DB-484D-9BE5-BD234D17C8A1

  TMPDIR=/var/folders/_y/bk85gnj92fj_mlk9x1c8nm0w0000gn/T/

  USER=falken

  XPC_FLAGS=0x0

  XPC_SERVICE_NAME=0

  _=/usr/local/bin/buildbot-worker

  __CF_USER_TEXT_ENCODING=0x1F5:0x0:0x0

 using PTY: False

/Users/falken/.fastlane/bin/bundle/lib/ruby/gems/2.2.0/gems/fastlane-2.96.0/fastlane_core/lib/fastlane_core/fastlane_folder.rb:49:in join': no implicit conversion of nil into String (TypeError)
    from /Users/falken/.fastlane/bin/bundle/lib/ruby/gems/2.2.0/gems/fastlane-2.96.0/fastlane_core/lib/fastlane_core/fastlane_folder.rb:49:infastfile_path'

    from /Users/falken/.fastlane/bin/bundle/lib/ruby/gems/2.2.0/gems/fastlane-2.96.0/fastlane/lib/fastlane/cli_tools_distributor.rb:167:in available_lanes'
    from /Users/falken/.fastlane/bin/bundle/lib/ruby/gems/2.2.0/gems/fastlane-2.96.0/fastlane/lib/fastlane/cli_tools_distributor.rb:74:intake_off'

    from /Users/falken/.fastlane/bin/bundle/lib/ruby/gems/2.2.0/gems/fastlane-2.96.0/bin/fastlane:20:in <top (required)>'
    from /Users/falken/.fastlane/bin/bundle/bin/fastlane:22:inload'

    from /Users/falken/.fastlane/bin/bundle/bin/fastlane:22:in `'

program finished with exit code 1

elapsedTime=8.027540

Environment

🚫 fastlane environment 🚫

Stack

| Key | Value |
| --------------------------- | ------------------------------------------- |
| OS | 10.13.5 |
| Ruby | 2.2.4 |
| Bundler? | false |
| Git | git version 2.15.2 (Apple Git-101.1) |
| Installation Source | ~/.fastlane/bin/bundle/bin/fastlane |
| Host | Mac OS X 10.13.5 (17F77) |
| Ruby Lib Dir | ~/.fastlane/bin/bundle/lib |
| OpenSSL Version | OpenSSL 1.0.2g 1 Mar 2016 |
| Is contained | false |
| Is homebrew | true |
| Is installed via Fabric.app | false |
| Xcode Path | /Applications/Xcode.app/Contents/Developer/ |
| Xcode Version | 9.4.1 |

System Locale

| Variable | Value | |
| -------- | ----------- | - |
| LANG | en_US.UTF-8 | ✅ |
| LC_ALL | en_US.UTF-8 | ✅ |
| LANGUAGE | en_US.UTF-8 | ✅ |

fastlane files:

No Fastfile found

fastlane gems

| Gem | Version | Update-Status |
| -------- | ------- | ------------------ |
| fastlane | 2.96.0 | 🚫 Update available |

Loaded fastlane plugins:

No plugins Loaded

Loaded gems

| Gem | Version |
| ------------------------- | ------------ |
| slack-notifier | 2.3.2 |
| atomos | 0.1.2 |
| CFPropertyList | 2.3.5 |
| claide | 1.0.2 |
| colored2 | 3.1.2 |
| nanaimo | 0.2.4 |
| xcodeproj | 1.5.7 |
| rouge | 1.11.1 |
| xcpretty | 0.2.6 |
| terminal-notifier | 1.7.1 |
| unicode-display_width | 1.1.3 |
| terminal-table | 1.7.3 |
| plist | 3.2.0 |
| public_suffix | 2.0.5 |
| addressable | 2.5.1 |
| multipart-post | 2.0.0 |
| word_wrap | 1.0.0 |
| tty-screen | 0.6.4 |
| tty-cursor | 0.5.0 |
| tty-spinner | 0.8.0 |
| babosa | 1.0.2 |
| colored | 1.2 |
| highline | 1.7.8 |
| commander-fastlane | 4.4.6 |
| excon | 0.55.0 |
| faraday | 0.12.1 |
| unf_ext | 0.0.7.4 |
| unf | 0.1.4 |
| domain_name | 0.5.20170404 |
| http-cookie | 1.0.3 |
| faraday-cookie_jar | 0.0.6 |
| fastimage | 2.1.0 |
| gh_inspector | 1.1.3 |
| json | 1.8.1 |
| mini_magick | 4.5.1 |
| multi_json | 1.12.1 |
| multi_xml | 0.6.0 |
| rubyzip | 1.2.1 |
| security | 0.1.3 |
| xcpretty-travis-formatter | 0.0.4 |
| dotenv | 2.2.0 |
| bundler | 1.14.6 |
| faraday_middleware | 0.11.0.1 |
| naturally | 2.1.0 |
| simctl | 1.6.4 |
| uber | 0.0.15 |
| declarative | 0.0.9 |
| declarative-option | 0.1.0 |
| representable | 3.0.4 |
| retriable | 2.1.0 |
| mime-types-data | 3.2016.0521 |
| mime-types | 3.1 |
| little-plugger | 1.1.4 |
| logging | 2.2.2 |
| jwt | 1.5.6 |
| memoist | 0.15.0 |
| os | 0.9.6 |
| signet | 0.7.3 |
| googleauth | 0.5.1 |
| httpclient | 2.8.3 |
| google-api-client | 0.21.2 |
| emoji_regex | 0.1.1 |
| libxml-ruby | 3.0.0 |

generated on: 2018-06-23

auto-closed waiting-for-reply bug ci

Source

falken42

👍4

Most helpful comment

Good news: I think I got it: #13918

I could reproduce the error pretty consistently locally and on Azure Pipelines. Disabling the update made it go away in 100% of the cases, opting out of Analytics did as well. So I looked into the Analytics a bit and discovered that this specific data was not used any more - so I could simply remove the code. #13918 is the PR doing that, and with the next release the problem should be gone.

janpio on 15 Dec 2018

👍2

All 121 comments

@Falken42 This is super weird, TBH 😬 The lines is causing confusion is... https://github.com/fastlane/fastlane/blob/master/fastlane/lib/fastlane/cli_tools_distributor.rb#L74

Its not seeing cert as a valid tool name so its trying to show available lanes which isn't a thing for you because you don't have any. And it this is happening just randomly it sounds like it might just be something with your machine and its heavy load 😔

Can you try downgrading to something pre-2.96.0 to see if you do actually stop getting this random error again? 🤔

joshdholtz on 3 Jul 2018

I have the very same issue form some time now... I thought new versions of fastlane will solve it, but so far no luck.

I have this issue when I run my builds in CircleCI. As @Falken42 has mentioned - it's a random thing, so I cannot determine exactly what's happening and what's causing the problem. Sometimes I have to rerun workflow multiple times, to pass, but sometimes it works on the first run!

In my CircleCI's workflow I have two jobs speficied, and this always fails for me when I run the first one. If it completes I'm sure the second job will complete as well!
This job calls only one fastlane actions and it is:

bundle exec fastlane scan

I have also two ENV variables set:

SCAN_DEVICE: "iPhone 6"
SCAN_SCHEME: "home24 - Staging"

cojoj on 5 Jul 2018

@joshdholtz I went back and did a deep dive through our buildbot logs, and it turns out this actually has happened with previous versions of fastlane. My original statement that we hadn't seen this before 2.96.0 was incorrect, sorry about that.

Log from fastlane-2.85.0:



fastlane sigh manage -e -f

 in dir /Volumes/Buildbot/buildbot-osx/CrystalClash-iOS/build (timeout 1200 secs)

 watching logfiles {}

 argv: ['fastlane', 'sigh', 'manage', '-e', '-f']

 environment:

  Apple_PubSub_Socket_Render=/private/tmp/com.apple.launchd.2j8KZClksG/Render

  DISPLAY=/private/tmp/com.apple.launchd.rAO2EqRAku/org.macosforge.xquartz:0

  EDITOR=vi

  HOME=/Users/falken

  LC_CTYPE=UTF-8

  LOGNAME=falken

  PATH=/Users/falken/.fastlane/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/X11/bin

  PWD=/Volumes/Buildbot/buildbot-osx/CrystalClash-iOS/build

  SECURITYSESSIONID=186a8

  SHELL=/bin/bash

  SHLVL=1

  SSH_AUTH_SOCK=/private/tmp/com.apple.launchd.I7zyoYjKSk/Listeners

  TERM=xterm-256color

  TERM_PROGRAM=Apple_Terminal

  TERM_PROGRAM_VERSION=400

  TERM_SESSION_ID=21E43212-25DB-484D-9BE5-BD234D17C8A1

  TMPDIR=/var/folders/_y/bk85gnj92fj_mlk9x1c8nm0w0000gn/T/

  USER=falken

  XPC_FLAGS=0x0

  XPC_SERVICE_NAME=0

  _=/usr/local/bin/buildbot-worker

  __CF_USER_TEXT_ENCODING=0x1F5:0x0:0x0

 using PTY: False

/Users/falken/.fastlane/bin/bundle/lib/ruby/gems/2.2.0/gems/fastlane-2.85.0/fastlane_core/lib/fastlane_core/fastlane_folder.rb:52:in join': no implicit conversion of nil into String (TypeError)
    from /Users/falken/.fastlane/bin/bundle/lib/ruby/gems/2.2.0/gems/fastlane-2.85.0/fastlane_core/lib/fastlane_core/fastlane_folder.rb:52:infastfile_path'

    from /Users/falken/.fastlane/bin/bundle/lib/ruby/gems/2.2.0/gems/fastlane-2.85.0/fastlane/lib/fastlane/cli_tools_distributor.rb:157:in available_lanes'
    from /Users/falken/.fastlane/bin/bundle/lib/ruby/gems/2.2.0/gems/fastlane-2.85.0/fastlane/lib/fastlane/cli_tools_distributor.rb:64:intake_off'

    from /Users/falken/.fastlane/bin/bundle/lib/ruby/gems/2.2.0/gems/fastlane-2.85.0/bin/fastlane:20:in <top (required)>'
    from /Users/falken/.fastlane/bin/bundle/bin/fastlane:22:inload'

    from /Users/falken/.fastlane/bin/bundle/bin/fastlane:22:in `'

program finished with exit code 1

elapsedTime=5.721645

Two more data points: the very first fastlane version we began using was 2.41.0, and the last version that didn't exhibit this error was with 2.64.1.

It might be purely coincidental, but it seems like we started seeing this once fastlane began to suggest we run "gem cleanup". At least, every time fastlane fails, the suggestion to run gem cleanup is always in the log.

I've currently updated fastlane to 2.98.0 and haven't seen it repro here yet. It is possible for us to downgrade, but I can't downgrade too far (there are newer deliver and supply features that we depend on to deploy our apps). If there is a specific target version you'd like me to test, let me know and I can see if we can use it.

falken42 on 5 Jul 2018

We just had this happen with 2.98.0:



fastlane sigh manage -e -f

 in dir /Volumes/Buildbot/buildbot-osx/Sandbox-iOS/build (timeout 1200 secs)

 watching logfiles {}

 argv: ['fastlane', 'sigh', 'manage', '-e', '-f']

 environment:

  Apple_PubSub_Socket_Render=/private/tmp/com.apple.launchd.i0vQVsLZRk/Render

  DISPLAY=/private/tmp/com.apple.launchd.htpenpkJin/org.macosforge.xquartz:0

  EDITOR=vi

  HOME=/Users/falken

  LC_CTYPE=UTF-8

  LOGNAME=falken

  PATH=/Users/falken/.fastlane/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/X11/bin

  PWD=/Volumes/Buildbot/buildbot-osx/Sandbox-iOS/build

  SECURITYSESSIONID=186a6

  SHELL=/bin/bash

  SHLVL=1

  SSH_AUTH_SOCK=/private/tmp/com.apple.launchd.GtMWyocuk4/Listeners

  TERM=xterm-256color

  TERM_PROGRAM=Apple_Terminal

  TERM_PROGRAM_VERSION=404

  TERM_SESSION_ID=21E43212-25DB-484D-9BE5-BD234D17C8A1

  TMPDIR=/var/folders/_y/bk85gnj92fj_mlk9x1c8nm0w0000gn/T/

  USER=falken

  XPC_FLAGS=0x0

  XPC_SERVICE_NAME=0

  _=/usr/local/bin/buildbot-worker

  __CF_USER_TEXT_ENCODING=0x1F5:0x0:0x0

 using PTY: False

/Users/falken/.fastlane/bin/bundle/lib/ruby/gems/2.2.0/gems/fastlane-2.98.0/fastlane_core/lib/fastlane_core/fastlane_folder.rb:49:in join': no implicit conversion of nil into String (TypeError)
    from /Users/falken/.fastlane/bin/bundle/lib/ruby/gems/2.2.0/gems/fastlane-2.98.0/fastlane_core/lib/fastlane_core/fastlane_folder.rb:49:infastfile_path'

    from /Users/falken/.fastlane/bin/bundle/lib/ruby/gems/2.2.0/gems/fastlane-2.98.0/fastlane/lib/fastlane/cli_tools_distributor.rb:167:in available_lanes'
    from /Users/falken/.fastlane/bin/bundle/lib/ruby/gems/2.2.0/gems/fastlane-2.98.0/fastlane/lib/fastlane/cli_tools_distributor.rb:74:intake_off'

    from /Users/falken/.fastlane/bin/bundle/lib/ruby/gems/2.2.0/gems/fastlane-2.98.0/bin/fastlane:20:in <top (required)>'
    from /Users/falken/.fastlane/bin/bundle/bin/fastlane:22:inload'

    from /Users/falken/.fastlane/bin/bundle/bin/fastlane:22:in `'

program finished with exit code 1

elapsedTime=5.697607

One thing that's different this time though is fastlane isn't suggesting to run "gem cleanup", it just simply failed.

falken42 on 7 Jul 2018

There hasn't been any activity on this issue recently. Due to the high number of incoming GitHub notifications, we have to clean some of the old issues, as many of them have already been resolved with the latest updates.

Please make sure to update to the latest fastlane version and check if that solves the issue. Let us know if that works for you by adding a comment :+1:

fastlane-bot on 6 Aug 2018

We've updated to 2.100.1 about a week ago, but I haven't seen a repro yet.

falken42 on 7 Aug 2018

2.102.0

+ fastlane deliver --skip_screenshots
[11:48:58]: [33mGet started using a Gemfile for fastlane https://docs.fastlane.tools/getting-started/ios/setup/#use-a-gemfile[0m
/usr/local/lib/ruby/gems/2.5.0/gems/fastlane-2.102.0/fastlane_core/lib/fastlane_core/fastlane_folder.rb:49:in `join': no implicit conversion of nil into String (TypeError)
    from /usr/local/lib/ruby/gems/2.5.0/gems/fastlane-2.102.0/fastlane_core/lib/fastlane_core/fastlane_folder.rb:49:in `fastfile_path'
    from /usr/local/lib/ruby/gems/2.5.0/gems/fastlane-2.102.0/fastlane/lib/fastlane/cli_tools_distributor.rb:167:in `available_lanes'
    from /usr/local/lib/ruby/gems/2.5.0/gems/fastlane-2.102.0/fastlane/lib/fastlane/cli_tools_distributor.rb:74:in `take_off'
    from /usr/local/lib/ruby/gems/2.5.0/gems/fastlane-2.102.0/bin/fastlane:20:in `<top (required)>'
    from /usr/local/bin/fastlane:23:in `load'
    from /usr/local/bin/fastlane:23:in `<main>'

eriadam on 27 Aug 2018

I also ran into this on 2.102.0. It was not an intermittent failure.

Uninstalled fastlane & commander-fastlane, then reinstalled and it started working again.

jamezilla on 28 Aug 2018

I honestly wasn't expecting to see this happen again.

We've moved to GitLab CI (instead of buildbot), and our macOS builder is no longer under the same heavy load as it was when I first posted this report.

But yet, our pipeline just randomly failed today:

$ bash -c 'env FASTLANE_PASSWORD=`vault kv get -field=password secret/ci/apple-developer` fastlane sigh --development'
[14:02:09]: Get started using a Gemfile for fastlane https://docs.fastlane.tools/getting-started/ios/setup/#use-a-gemfile
/Users/falken/.fastlane/bin/bundle/lib/ruby/gems/2.2.0/gems/fastlane-2.101.1/fastlane_core/lib/fastlane_core/fastlane_folder.rb:49:in `join': no implicit conversion of nil into String (TypeError)
    from /Users/falken/.fastlane/bin/bundle/lib/ruby/gems/2.2.0/gems/fastlane-2.101.1/fastlane_core/lib/fastlane_core/fastlane_folder.rb:49:in `fastfile_path'
    from /Users/falken/.fastlane/bin/bundle/lib/ruby/gems/2.2.0/gems/fastlane-2.101.1/fastlane/lib/fastlane/cli_tools_distributor.rb:167:in `available_lanes'
    from /Users/falken/.fastlane/bin/bundle/lib/ruby/gems/2.2.0/gems/fastlane-2.101.1/fastlane/lib/fastlane/cli_tools_distributor.rb:74:in `take_off'
    from /Users/falken/.fastlane/bin/bundle/lib/ruby/gems/2.2.0/gems/fastlane-2.101.1/bin/fastlane:20:in `<top (required)>'
    from /Users/falken/.fastlane/bin/bundle/bin/fastlane:22:in `load'
    from /Users/falken/.fastlane/bin/bundle/bin/fastlane:22:in `<main>'
ERROR: Job failed: exit status 1

As usual, retrying the pipeline worked without any problems.

falken42 on 28 Aug 2018

Please make sure to update to the latest fastlane version and check if that solves the issue. Let us know if that works for you by adding a comment :+1:

fastlane-bot on 27 Sep 2018

This issue will be auto-closed because there hasn't been any activity for a few months. Feel free to open a new one if you still experience this problem :+1:

fastlane-bot on 5 Oct 2018

Still happening with fastlane-2.105.2.

eriadam on 12 Oct 2018

Also repro'd again with fastlane-2.105.2 here as well.

It's pretty rare, happening once out of every 70-80 or so builds for us. Retrying the pipeline immediately afterwards is always successful.

$ bash -c 'env FASTLANE_PASSWORD=`vault kv get -field=password secret/ci/apple-developer` fastlane sigh --development'
[12:54:56]: Get started using a Gemfile for fastlane https://docs.fastlane.tools/getting-started/ios/setup/#use-a-gemfile
/Users/falken/.fastlane/bin/bundle/lib/ruby/gems/2.2.0/gems/fastlane-2.105.2/fastlane_core/lib/fastlane_core/fastlane_folder.rb:50:in `join': no implicit conversion of nil into String (TypeError)
    from /Users/falken/.fastlane/bin/bundle/lib/ruby/gems/2.2.0/gems/fastlane-2.105.2/fastlane_core/lib/fastlane_core/fastlane_folder.rb:50:in `fastfile_path'
    from /Users/falken/.fastlane/bin/bundle/lib/ruby/gems/2.2.0/gems/fastlane-2.105.2/fastlane/lib/fastlane/cli_tools_distributor.rb:167:in `available_lanes'
    from /Users/falken/.fastlane/bin/bundle/lib/ruby/gems/2.2.0/gems/fastlane-2.105.2/fastlane/lib/fastlane/cli_tools_distributor.rb:74:in `take_off'
    from /Users/falken/.fastlane/bin/bundle/lib/ruby/gems/2.2.0/gems/fastlane-2.105.2/bin/fastlane:20:in `<top (required)>'
    from /Users/falken/.fastlane/bin/bundle/bin/fastlane:22:in `load'
    from /Users/falken/.fastlane/bin/bundle/bin/fastlane:22:in `<main>'
ERROR: Job failed: exit status 1

I'll update to 2.107.0 now and report back if it happens again.

falken42 on 17 Oct 2018

Seeing this happen intermittently on multiple recent versions, including 2.107.0.

ronocod on 23 Oct 2018

For us this is now happening on a daily bases with 2.107.0.

eriadam on 25 Oct 2018

Another occurence with 2.108.0 on Azure Devops Pipelines (CI): https://github.com/fastlane/fastlane/issues/13680

janpio on 14 Nov 2018

👍2

Here someone investigated a bit what is going on: https://github.com/fastlane/fastlane/issues/13586#issuecomment-435394634
Seems self.path is randomly mutating?!

janpio on 14 Nov 2018

Hi, I'm getting the same(?) error every time with the latest fastlane/deliver.

INFO [2018-11-16 11:48:43.70]: Making sure the latest version on App Store Connect matches '4.13.0' from the ipa file...
INFO [2018-11-16 11:48:46.67]: '4.13.0' is the latest version on App Store Connect
ERROR [2018-11-16 11:48:48.86]: #<TypeError: no implicit conversion of nil into String>
ERROR [2018-11-16 11:48:48.86]: /path/to/ruby/2.5.0/gems/fastlane-2.108.0/deliver/lib/deliver/html_generator.rb:37:in `initialize'
/path/to/ruby/2.5.0/gems/fastlane-2.108.0/deliver/lib/deliver/html_generator.rb:37:in `new'
/path/to/ruby/2.5.0//gems/fastlane-2.108.0/deliver/lib/deliver/html_generator.rb:37:in `render_relative_path'
...

[2018-11-16 11:48:48.86]: Could not render HTML preview. Do you still want to continue?

Executing deliver did succeed when I continued.

mathaeus on 16 Nov 2018

I searched further and actually found this issue:
https://github.com/fastlane/fastlane/issues/13512

You can probably solve it by adding force true to your Deliverfile

Not for such errors ;)

mathaeus on 16 Nov 2018

Nah, yours is from completely different code @mathaeus - see the code files and lines in the stracktrace, totally different from all the similar ones posted here. Please open a new issue with your problem and provide a bit more info, then we can take a look at it. Thanks.

janpio on 16 Nov 2018

@janpio I already found and linked the relevant issue, see above.

see the code files and lines in the stracktrace, totally different from all the similar ones posted here

I was aware of that, I thought they were connected since the error message was the same, but yeah I'll try to do better with searching for an existing issue that fits ;)

mathaeus on 16 Nov 2018

❤1

Another "random" problem people are encountering could be caused by this as the actual error:
https://github.com/fastlane/fastlane/issues/13512, see https://github.com/fastlane/fastlane/issues/13512#issuecomment-439506736

janpio on 16 Nov 2018

Still happening with 2.108.0 on CI: https://bitrise.io

Run Fastlane
$ bundle "exec" "fastlane" "fastlane" "android" "test"
bundler: failed to load command: fastlane (/usr/local/bin/fastlane)
TypeError: no implicit conversion of nil into String
  /usr/local/lib/ruby/gems/2.5.0/gems/fastlane-2.108.0/fastlane_core/lib/fastlane_core/fastlane_folder.rb:53:in `join'
  /usr/local/lib/ruby/gems/2.5.0/gems/fastlane-2.108.0/fastlane_core/lib/fastlane_core/fastlane_folder.rb:53:in `fastfile_path'
  /usr/local/lib/ruby/gems/2.5.0/gems/fastlane-2.108.0/fastlane/lib/fastlane/cli_tools_distributor.rb:167:in `available_lanes'
  /usr/local/lib/ruby/gems/2.5.0/gems/fastlane-2.108.0/fastlane/lib/fastlane/cli_tools_distributor.rb:74:in `take_off'
  /usr/local/lib/ruby/gems/2.5.0/gems/fastlane-2.108.0/bin/fastlane:23:in `<top (required)>'
  /usr/local/bin/fastlane:23:in `load'
  /usr/local/bin/fastlane:23:in `<top (required)>'

cesarferreira on 30 Nov 2018

Hi,

Still happening on our side with 2.109.1. We deliver everyday at least 4 builds, and our success rate is below 50%. I would not say this is a rare issue, unless our setup expose it more often (bitrise setup).

dloic on 5 Dec 2018

@dloic What is the quickest way to reproduce this on Bitrise? Locally and on my environments I can only very rarely reproduce this, which makes it really hard to debug.

janpio on 5 Dec 2018

@janpio That's a good question :) . I will describe as close as possible our setup:

we have two set of scheduled builds every days (2 in // the morning, 2 in // later in the day)
we use fastlane 2.109.1, but we've seen the issue since a few months with previous fastlane versions
our Fastfile is stored in fastlane/Fastfile
we currently use Xcode 10 stack, but seen the issue with previous stacks
our workflow is closed to (i'm hidding some internal parts):
- init: activate ssh, clone repo, recreate user scheme, bitrise cache pull, pods and certificate/profil installer
- build & test: Xcode test (2.0.0 is bitrise version) and Xcode Archive (2.4.6)
- Upload to testflight (2.1.3), slack
- close: deploy to bitrise, cache push

We're using elite machines, but we've seen the issue also with smaller machines.

I can provide more log info if needed, let me know! Anything that can help you will help us :)

dloic on 5 Dec 2018

I was hoping more for a "fork this repo, click these 2 buttons in bitrise, wait for failure" ;)

janpio on 5 Dec 2018

Unfortunatly I cannot share more. I don't think this is related to repo. But if I can provide something else, let me know (like log or other).

dloic on 5 Dec 2018

Nah, you misunderstood me. I have no experience with Bitrise, and am looking for the quickest way to get a working setup on there that will let me replicate the problem. I don't want to read through all their docs etc :/

janpio on 5 Dec 2018

This happens for me locally as well (although it seems rarer).

By reading the related code, it seems that self.path is returning different values in the same run?

Maybe caching self.path in a local variable and using it consistently inside self.fastfile_path would fix the issue, but it seems that there's an underneath issue: why would self.path return something and then nil?

marcelofabri on 6 Dec 2018

@janpio : to be honest bitrise is really simple, if you have any repo with an iOS project, doing the setup is just a few clicks ;)

@marcelofabri I agree that this value is not subject to change, so probably computing it once will save us from concurrent changes.

dloic on 6 Dec 2018

Caching would only fix the issue if the call that is being cached returns the valid value. As we don't understand what is causing an invalid value to be returned, that just might make the bug less frequent - but it would still be there, right?

Ok, recap time:

Line 50 or 53 here are crashing because one of the parameters of File.join() are nil which can't be converted to a String:
https://github.com/fastlane/fastlane/blob/cd88359fa9b6772f91a31a290f8f3c790a716dc0/fastlane_core/lib/fastlane_core/fastlane_folder.rb#L46-L56

As the one is literally a string in both cases, it has to be self.path. But self.path is defined here:
https://github.com/fastlane/fastlane/blob/cd88359fa9b6772f91a31a290f8f3c790a716dc0/fastlane_core/lib/fastlane_core/fastlane_folder.rb#L8-L16

This just goes through a list of folders and files and returns the first matching one. Returning nil here doesn't make sense if it worked before. The folders and files do exist. (Here someone observe the return value mutating)

So: Whut!?

janpio on 6 Dec 2018

Can Dir.getwd return a wrong value in some scenarii?

dloic on 6 Dec 2018

None that I can think of.

I quickly created a new branch based on current 2.109.1 that contains a simple puts in self.path: https://github.com/fastlane/fastlane/tree/janpio-path_nil (changes visible here: https://github.com/fastlane/fastlane/pull/13840/files)

You can use this by creating a Gemfile in your project that uses fastlane. Some general instructions: https://docs.fastlane.tools/getting-started/ios/setup/#use-a-gemfile Instead of just gem "fastlane" you have to use gem 'fastlane', :git => 'https://github.com/fastlane/fastlane.git', :branch => 'janpio-path_nil'. Remember to use bundle exec fastlane ... instead of just fastlane ... to run your lane.

This will spam your output with each time self.path is called:

DEBUG: self.path = './fastlane/', Dir.getwd = 'C:/Projects/Fastlane/fastlane-e2e-tests/android'
DEBUG: self.path = './fastlane/', Dir.getwd = 'C:/Projects/Fastlane/fastlane-e2e-tests/android'
DEBUG: self.path = './fastlane/', Dir.getwd = 'C:/Projects/Fastlane/fastlane-e2e-tests/android'
[17:45:23]: ------------------------------
[17:45:23]: --- Step: default_platform ---
[17:45:23]: ------------------------------
DEBUG: self.path = './', Dir.getwd = 'C:/Projects/Fastlane/fastlane-e2e-tests/android/fastlane'
DEBUG: self.path = './fastlane/', Dir.getwd = 'C:/Projects/Fastlane/fastlane-e2e-tests/android'

Here we also see why we can't just cache the value - it might be different depending on which folder (Dir.getwd) fastlane operates at the time. The second to last line here is inside the fastlane dir for example, while all the others are in the project root.

If you discover something interesting, please post your output!
(Or let us know if this addition makes the bug go away or hide...)

janpio on 6 Dec 2018

@janpio here's what I got:

bundle exec fastlane scan
[✔] 🚀 
DEBUG: self.path = './fastlane/', Dir.getwd = 'project-folder'
DEBUG: self.path = './fastlane/', Dir.getwd = 'project-folder/fastlane'
DEBUG: self.path = '', Dir.getwd = 'project-folder'
bundler: failed to load command: fastlane (/Users/marcelofabri/.rbenv/versions/2.5.3/lib/ruby/gems/2.5.0/bin/fastlane)
TypeError: no implicit conversion of nil into String
  /Users/marcelofabri/.rbenv/versions/2.5.3/lib/ruby/gems/2.5.0/bundler/gems/fastlane-89a336f38651/fastlane_core/lib/fastlane_core/fastlane_folder.rb:51:in `join'
  /Users/marcelofabri/.rbenv/versions/2.5.3/lib/ruby/gems/2.5.0/bundler/gems/fastlane-89a336f38651/fastlane_core/lib/fastlane_core/fastlane_folder.rb:51:in `fastfile_path'
  /Users/marcelofabri/.rbenv/versions/2.5.3/lib/ruby/gems/2.5.0/bundler/gems/fastlane-89a336f38651/fastlane/lib/fastlane/cli_tools_distributor.rb:167:in `available_lanes'
  /Users/marcelofabri/.rbenv/versions/2.5.3/lib/ruby/gems/2.5.0/bundler/gems/fastlane-89a336f38651/fastlane/lib/fastlane/cli_tools_distributor.rb:74:in `take_off'
  /Users/marcelofabri/.rbenv/versions/2.5.3/lib/ruby/gems/2.5.0/bundler/gems/fastlane-89a336f38651/bin/fastlane:23:in `<top (required)>'
  /Users/marcelofabri/.rbenv/versions/2.5.3/lib/ruby/gems/2.5.0/bin/fastlane:23:in `load'
  /Users/marcelofabri/.rbenv/versions/2.5.3/lib/ruby/gems/2.5.0/bin/fastlane:23:in `<top (required)>'
make: *** [test] Error 1

marcelofabri on 6 Dec 2018

❤1

@janpio thanks for the dedicated branch. We'll setup our CI to use it and will let you know the result.

dloic on 6 Dec 2018

@marcelofabri Now we're talking. Console log debugging via a branch on a Github repo... something I don't do every day ;)

I added a few puts inside cli_tools_distributor.rb and fastlane_folder.rb. Because what I see in your output doesn't really make sense, does it? Why can the path become nil when we are in the same folder as before?
Please run run bundle install to install 2.109.1.foo1 and run it again 🌪

janpio on 6 Dec 2018

😄1

@janpio more logs!

bundle exec fastlane scan
take_off
[✔] 🚀 
take_off: after require
take_off: after lang
fastlane_folder: 1
fastlane_folder: 2: #{value}
fastlane_folder: 3: #{value}
fastlane_folder: 4: #{value}
fastlane_folder: 5: #{value}
fastlane_folder: 6: #{value}
fastlane_folder: 7: #{value}
DEBUG: self.path = './fastlane/', Dir.getwd = 'project-folder'
take_off: after dotenv
take_off: after updatechecker
take_off: after tool_name (= #{tool_name}), before if ...
fastlane_folder: 1
fastlane_folder: 2: #{value}
fastlane_folder: 3: #{value}
fastlane_folder: 4: #{value}
fastlane_folder: 5: #{value}
fastlane_folder: 6: #{value}
fastlane_folder: 7: #{value}
DEBUG: self.path = './fastlane/', Dir.getwd = 'project-folder/fastlane'
fastlane_folder: 1
fastlane_folder: 2: #{value}
fastlane_folder: 3: #{value}
fastlane_folder: 4: #{value}
fastlane_folder: 5: #{value}
fastlane_folder: 6: #{value}
fastlane_folder: 7: #{value}
DEBUG: self.path = './', Dir.getwd = 'project-folder/fastlane'
fastlane_folder: 1
fastlane_folder: 2: #{value}
fastlane_folder: 3: #{value}
fastlane_folder: 4: #{value}
fastlane_folder: 5: #{value}
fastlane_folder: 6: #{value}
fastlane_folder: 7: #{value}
DEBUG: self.path = '', Dir.getwd = 'project-folder'
bundler: failed to load command: fastlane (/Users/marcelofabri/.rbenv/versions/2.5.3/lib/ruby/gems/2.5.0/bin/fastlane)
TypeError: no implicit conversion of nil into String
  /Users/marcelofabri/.rbenv/versions/2.5.3/lib/ruby/gems/2.5.0/bundler/gems/fastlane-ad44e90daa08/fastlane_core/lib/fastlane_core/fastlane_folder.rb:61:in `join'
  /Users/marcelofabri/.rbenv/versions/2.5.3/lib/ruby/gems/2.5.0/bundler/gems/fastlane-ad44e90daa08/fastlane_core/lib/fastlane_core/fastlane_folder.rb:61:in `fastfile_path'
  /Users/marcelofabri/.rbenv/versions/2.5.3/lib/ruby/gems/2.5.0/bundler/gems/fastlane-ad44e90daa08/fastlane/lib/fastlane/cli_tools_distributor.rb:181:in `available_lanes'
  /Users/marcelofabri/.rbenv/versions/2.5.3/lib/ruby/gems/2.5.0/bundler/gems/fastlane-ad44e90daa08/fastlane/lib/fastlane/cli_tools_distributor.rb:85:in `take_off'
  /Users/marcelofabri/.rbenv/versions/2.5.3/lib/ruby/gems/2.5.0/bundler/gems/fastlane-ad44e90daa08/bin/fastlane:23:in `<top (required)>'
  /Users/marcelofabri/.rbenv/versions/2.5.3/lib/ruby/gems/2.5.0/bin/fastlane:23:in `load'
  /Users/marcelofabri/.rbenv/versions/2.5.3/lib/ruby/gems/2.5.0/bin/fastlane:23:in `<top (required)>'

marcelofabri on 6 Dec 2018

I assume some of those should be printed with double quotes to fill the values 😅

marcelofabri on 6 Dec 2018

❤1

Fair assumption... Maybe I should learn a bit of this ruby stuff. Or not commit from an iPhone 🐒

foo2 is available on git. Try again if you can.

(What environment are you running this on? OS version? System ruby?)

janpio on 6 Dec 2018

macOS Mojave, ruby 2.5.3p105 (2018-10-18 revision 65156) [x86_64-darwin18] via rbenv.

I'll try to run again from the updated branch later today.

marcelofabri on 7 Dec 2018

Here's what I get:

[20:53:47]: Loading 'Metadata/secondary_first_sub_category.txt'...
[20:53:47]: Loading 'Metadata/secondary_second_sub_category.txt'...
fastlane_folder: 1
fastlane_folder: 2: './fastlane/'
fastlane_folder: 3: './fastlane/'
fastlane_folder: 4: './fastlane/'
fastlane_folder: 5: './fastlane/'
fastlane_folder: 6: './fastlane/'
fastlane_folder: 7: './fastlane/'
DEBUG: self.path = './fastlane/', Dir.getwd = '/Users/russ/Projects/HappyScale'
[20:53:48]: #<ArgumentError: different prefix: "/" and "fastlane">
[20:53:48]: /System/Library/Frameworks/Ruby.framework/Versions/2.3/usr/lib/ruby/2.3.0/pathname.rb:520:in `relative_path_from'
/Users/russ/.bundle/ruby/2.3.0/fastlane-8740885f1eb1/deliver/lib/deliver/html_generator.rb:38:in `render_relative_path'
(erb):240:in `block (3 levels) in render'
(erb):239:in `each'
(erb):239:in `each_with_index'
(erb):239:in `block (2 levels) in render'
(erb):233:in `each'
(erb):233:in `block in render'
(erb):157:in `each'
(erb):157:in `render'
/System/Library/Frameworks/Ruby.framework/Versions/2.3/usr/lib/ruby/2.3.0/erb.rb:864:in `eval'
/System/Library/Frameworks/Ruby.framework/Versions/2.3/usr/lib/ruby/2.3.0/erb.rb:864:in `result'
/Users/russ/.bundle/ruby/2.3.0/fastlane-8740885f1eb1/deliver/lib/deliver/html_generator.rb:56:in `render'
/Users/russ/.bundle/ruby/2.3.0/fastlane-8740885f1eb1/deliver/lib/deliver/html_generator.rb:13:in `run'
/Users/russ/.bundle/ruby/2.3.0/fastlane-8740885f1eb1/deliver/lib/deliver/runner.rb:180:in `validate_html'
/Users/russ/.bundle/ruby/2.3.0/fastlane-8740885f1eb1/deliver/lib/deliver/runner.rb:117:in `upload_metadata'
/Users/russ/.bundle/ruby/2.3.0/fastlane-8740885f1eb1/deliver/lib/deliver/runner.rb:38:in `run'
/Users/russ/.bundle/ruby/2.3.0/fastlane-8740885f1eb1/fastlane/lib/fastlane/actions/upload_to_app_store.rb:16:in `run'
/Users/russ/.bundle/ruby/2.3.0/fastlane-8740885f1eb1/fastlane/lib/fastlane/runner.rb:253:in `block (2 levels) in execute_action'
/Users/russ/.bundle/ruby/2.3.0/fastlane-8740885f1eb1/fastlane/lib/fastlane/actions/actions_helper.rb:50:in `execute_action'
/Users/russ/.bundle/ruby/2.3.0/fastlane-8740885f1eb1/fastlane/lib/fastlane/runner.rb:232:in `block in execute_action'
/Users/russ/.bundle/ruby/2.3.0/fastlane-8740885f1eb1/fastlane/lib/fastlane/runner.rb:227:in `chdir'
/Users/russ/.bundle/ruby/2.3.0/fastlane-8740885f1eb1/fastlane/lib/fastlane/runner.rb:227:in `execute_action'
/Users/russ/.bundle/ruby/2.3.0/fastlane-8740885f1eb1/fastlane/lib/fastlane/runner.rb:157:in `trigger_action_by_name'
/Users/russ/.bundle/ruby/2.3.0/fastlane-8740885f1eb1/fastlane/lib/fastlane/fast_file.rb:154:in `method_missing'
Fastfile:19:in `block in parsing_binding'
/Users/russ/.bundle/ruby/2.3.0/fastlane-8740885f1eb1/fastlane/lib/fastlane/lane.rb:33:in `call'
/Users/russ/.bundle/ruby/2.3.0/fastlane-8740885f1eb1/fastlane/lib/fastlane/runner.rb:49:in `block in execute'
/Users/russ/.bundle/ruby/2.3.0/fastlane-8740885f1eb1/fastlane/lib/fastlane/runner.rb:45:in `chdir'
/Users/russ/.bundle/ruby/2.3.0/fastlane-8740885f1eb1/fastlane/lib/fastlane/runner.rb:45:in `execute'
/Users/russ/.bundle/ruby/2.3.0/fastlane-8740885f1eb1/fastlane/lib/fastlane/lane_manager.rb:56:in `cruise_lane'
/Users/russ/.bundle/ruby/2.3.0/fastlane-8740885f1eb1/fastlane/lib/fastlane/command_line_handler.rb:36:in `handle'
/Users/russ/.bundle/ruby/2.3.0/fastlane-8740885f1eb1/fastlane/lib/fastlane/commands_generator.rb:108:in `block (2 levels) in run'
/Library/Ruby/Gems/2.3.0/gems/commander-fastlane-4.4.6/lib/commander/command.rb:178:in `call'
/Library/Ruby/Gems/2.3.0/gems/commander-fastlane-4.4.6/lib/commander/command.rb:153:in `run'
/Library/Ruby/Gems/2.3.0/gems/commander-fastlane-4.4.6/lib/commander/runner.rb:476:in `run_active_command'
/Users/russ/.bundle/ruby/2.3.0/fastlane-8740885f1eb1/fastlane_core/lib/fastlane_core/ui/fastlane_runner.rb:76:in `run!'
/Library/Ruby/Gems/2.3.0/gems/commander-fastlane-4.4.6/lib/commander/delegates.rb:15:in `run!'
/Users/russ/.bundle/ruby/2.3.0/fastlane-8740885f1eb1/fastlane/lib/fastlane/commands_generator.rb:333:in `run'
/Users/russ/.bundle/ruby/2.3.0/fastlane-8740885f1eb1/fastlane/lib/fastlane/commands_generator.rb:41:in `start'
/Users/russ/.bundle/ruby/2.3.0/fastlane-8740885f1eb1/fastlane/lib/fastlane/cli_tools_distributor.rb:122:in `take_off'
/Users/russ/.bundle/ruby/2.3.0/fastlane-8740885f1eb1/bin/fastlane:23:in `<top (required)>'
/Library/Ruby/Gems/2.3.0/bin/fastlane:22:in `load'
/Library/Ruby/Gems/2.3.0/bin/fastlane:22:in `<top (required)>'
/Library/Ruby/Gems/2.3.0/gems/bundler-1.16.1/lib/bundler/cli/exec.rb:75:in `load'
/Library/Ruby/Gems/2.3.0/gems/bundler-1.16.1/lib/bundler/cli/exec.rb:75:in `kernel_load'
/Library/Ruby/Gems/2.3.0/gems/bundler-1.16.1/lib/bundler/cli/exec.rb:28:in `run'
/Library/Ruby/Gems/2.3.0/gems/bundler-1.16.1/lib/bundler/cli.rb:424:in `exec'
/Library/Ruby/Gems/2.3.0/gems/bundler-1.16.1/lib/bundler/vendor/thor/lib/thor/command.rb:27:in `run'
/Library/Ruby/Gems/2.3.0/gems/bundler-1.16.1/lib/bundler/vendor/thor/lib/thor/invocation.rb:126:in `invoke_command'
/Library/Ruby/Gems/2.3.0/gems/bundler-1.16.1/lib/bundler/vendor/thor/lib/thor.rb:387:in `dispatch'
/Library/Ruby/Gems/2.3.0/gems/bundler-1.16.1/lib/bundler/cli.rb:27:in `dispatch'
/Library/Ruby/Gems/2.3.0/gems/bundler-1.16.1/lib/bundler/vendor/thor/lib/thor/base.rb:466:in `start'
/Library/Ruby/Gems/2.3.0/gems/bundler-1.16.1/lib/bundler/cli.rb:18:in `start'
/Library/Ruby/Gems/2.3.0/gems/bundler-1.16.1/exe/bundle:30:in `block in <top (required)>'
/Library/Ruby/Gems/2.3.0/gems/bundler-1.16.1/lib/bundler/friendly_errors.rb:122:in `with_friendly_errors'
/Library/Ruby/Gems/2.3.0/gems/bundler-1.16.1/exe/bundle:22:in `<top (required)>'
/usr/local/bin/bundle:22:in `load'
/usr/local/bin/bundle:22:in `<main>'
[20:53:48]: Could not render HTML preview. Do you still want to continue?

roostr on 7 Dec 2018

❤1

More logs!

bundle exec fastlane scan
take_off
[✔] 🚀 
take_off: after require
take_off: after lang
fastlane_folder: 1
fastlane_folder: 2: './fastlane/'
fastlane_folder: 3: './fastlane/'
fastlane_folder: 4: './fastlane/'
fastlane_folder: 5: './fastlane/'
fastlane_folder: 6: './fastlane/'
fastlane_folder: 7: './fastlane/'
DEBUG: self.path = './fastlane/', Dir.getwd = 'project-folder'
take_off: after dotenv
take_off: after updatechecker
take_off: after tool_name (= 'scan'), before if ...
available_lanes
fastfile_path
fastlane_folder: 1
fastlane_folder: 2: './fastlane/'
fastlane_folder: 3: './fastlane/'
fastlane_folder: 4: './fastlane/'
fastlane_folder: 5: './fastlane/'
fastlane_folder: 6: './fastlane/'
fastlane_folder: 7: './fastlane/'
DEBUG: self.path = './fastlane/', Dir.getwd = 'project-folder/fastlane'
fastfile_path: Fastfile.swift
fastlane_folder: 1
fastlane_folder: 2: ''
fastlane_folder: 3: ''
fastlane_folder: 4: ''
fastlane_folder: 5: ''
fastlane_folder: 6: ''
fastlane_folder: 7: ''
DEBUG: self.path = '', Dir.getwd = 'project-folder'
bundler: failed to load command: fastlane (/Users/marcelofabri/.rbenv/versions/2.5.3/lib/ruby/gems/2.5.0/bin/fastlane)
TypeError: no implicit conversion of nil into String
  /Users/marcelofabri/.rbenv/versions/2.5.3/lib/ruby/gems/2.5.0/bundler/gems/fastlane-8740885f1eb1/fastlane_core/lib/fastlane_core/fastlane_folder.rb:60:in `join'
  /Users/marcelofabri/.rbenv/versions/2.5.3/lib/ruby/gems/2.5.0/bundler/gems/fastlane-8740885f1eb1/fastlane_core/lib/fastlane_core/fastlane_folder.rb:60:in `fastfile_path'
  /Users/marcelofabri/.rbenv/versions/2.5.3/lib/ruby/gems/2.5.0/bundler/gems/fastlane-8740885f1eb1/fastlane/lib/fastlane/cli_tools_distributor.rb:182:in `available_lanes'
  /Users/marcelofabri/.rbenv/versions/2.5.3/lib/ruby/gems/2.5.0/bundler/gems/fastlane-8740885f1eb1/fastlane/lib/fastlane/cli_tools_distributor.rb:85:in `take_off'
  /Users/marcelofabri/.rbenv/versions/2.5.3/lib/ruby/gems/2.5.0/bundler/gems/fastlane-8740885f1eb1/bin/fastlane:23:in `<top (required)>'
  /Users/marcelofabri/.rbenv/versions/2.5.3/lib/ruby/gems/2.5.0/bin/fastlane:23:in `load'
  /Users/marcelofabri/.rbenv/versions/2.5.3/lib/ruby/gems/2.5.0/bin/fastlane:23:in `<top (required)>'

marcelofabri on 7 Dec 2018

Perfect @marcelofabri 💐

So usually the value is set between steps 1 and 2 - value ||= "./#{FOLDER_NAME}/" if File.directory?("./#{FOLDER_NAME}/"). That makes sense, as FOLDER_NAME is fastlane and you have a folder ./fastlane.

It works correctly for the first call of self.path inside fastlane_folder:
https://github.com/fastlane/fastlane/pull/13840/files#diff-b689c89143bd5d85e0a0fefd34aa2e85R56
But then the second call fails:
https://github.com/fastlane/fastlane/pull/13840/files#diff-b689c89143bd5d85e0a0fefd34aa2e85R60

The second call has a different working directory (project-folder instead of project-folder/fastlane), but as we can see in the first log output of the method this shouldn't be a problem!

Why is the working directory changing before and after the output of fastfile_path: Fastfile.swift anyway? self.path isn't doing it, the code in self.fastfile_path isn't doing that either. What can change a working directory without executing code?

Where are you executing bundle exec fastlane scan?

I am confused.

So I built another version foo3 that has even _more_ output, maybe this helps us to understand what is happening... Any suggestions what I should add to debug this further? I am really out of ideas.

janpio on 7 Dec 2018

I looked at your output as well @roostr - but am afraid this is a _different_ and possibly new issue! Could you please copy this in a new issue and add some information around it?
Back in https://github.com/fastlane/fastlane/issues/13512 the errors also were #<TypeError: no implicit conversion of nil into String> - which made them a match to possibly be the same core problem as this one. What you posted is different.

janpio on 7 Dec 2018

@janpio I'm also not sure how this could be happening. I'm running bundle exec fastlane scan.

here's some more logs:

$ bundle exec fastlane scan
take_off
[✔] 🚀 
take_off: after require
take_off: after lang
DEBUG: Dir.getwd = 'project-folder'
all files in current wd = '["Cartfile.resolved", "CHANGELOG.md", "Makefile", "Project.xcodeproj", "Cartfile", "InternalFrameworks", "Tests", "Carthage", "README.md", "Cartfile.private", "API", "Pods", "CONTRIBUTING.md", "Configs", "Scripts", "fastlane", "Dangerfile", "Project.xcworkspace", "Gemfile", "Gemfile.lock", "Project", "Brewfile", "Podfile", "build", "swiftgen.yml", "Podfile.lock", "Vendor"]'
fastlane_folder: 1
DEBUG: fastlane_folder_exists = 'true'
fastlane_folder: 2: './fastlane/'
fastlane_folder: 3: './fastlane/'
fastlane_folder: 4: './fastlane/'
fastlane_folder: 5: './fastlane/'
fastlane_folder: 6: './fastlane/'
fastlane_folder: 7: './fastlane/'
DEBUG: self.path = './fastlane/', Dir.getwd = 'project-folder'
take_off: after dotenv
take_off: after updatechecker
take_off: after tool_name (= 'scan'), before if ...
available_lanes
fastfile_path 0: Dir.getwd = 'project-folder'
DEBUG: Dir.getwd = 'project-folder'
all files in current wd = '["Cartfile.resolved", "CHANGELOG.md", "Makefile", "Project.xcodeproj", "Cartfile", "InternalFrameworks", "Tests", "Carthage", "README.md", "Cartfile.private", "API", "Pods", "CONTRIBUTING.md", "Configs", "Scripts", "fastlane", "Dangerfile", "Project.xcworkspace", "Gemfile", "Gemfile.lock", "Project", "Brewfile", "Podfile", "build", "swiftgen.yml", "Podfile.lock", "Vendor"]'
fastlane_folder: 1
DEBUG: fastlane_folder_exists = 'true'
fastlane_folder: 2: './fastlane/'
fastlane_folder: 3: './fastlane/'
fastlane_folder: 4: './fastlane/'
fastlane_folder: 5: './fastlane/'
fastlane_folder: 6: './fastlane/'
fastlane_folder: 7: './fastlane/'
DEBUG: self.path = './fastlane/', Dir.getwd = 'project-folder'
fastfile_path 1: Fastfile.swift: Dir.getwd = 'project-folder'
DEBUG: Dir.getwd = 'project-folder'
all files in current wd = '["Cartfile.resolved", "CHANGELOG.md", "Makefile", "Project.xcodeproj", "Cartfile", "InternalFrameworks", "Tests", "Carthage", "README.md", "Cartfile.private", "API", "Pods", "CONTRIBUTING.md", "Configs", "Scripts", "fastlane", "Dangerfile", "Project.xcworkspace", "Gemfile", "Gemfile.lock", "Project", "Brewfile", "Podfile", "build", "swiftgen.yml", "Podfile.lock", "Vendor"]'
fastlane_folder: 1
DEBUG: fastlane_folder_exists = 'false'
fastlane_folder: 2: ''
fastlane_folder: 3: ''
fastlane_folder: 4: ''
fastlane_folder: 5: ''
fastlane_folder: 6: ''
fastlane_folder: 7: ''
DEBUG: self.path = '', Dir.getwd = 'project-folder'
bundler: failed to load command: fastlane (/Users/marcelofabri/.rbenv/versions/2.5.3/lib/ruby/gems/2.5.0/bin/fastlane)
TypeError: no implicit conversion of nil into String
  /Users/marcelofabri/.rbenv/versions/2.5.3/lib/ruby/gems/2.5.0/bundler/gems/fastlane-de5dec310af3/fastlane_core/lib/fastlane_core/fastlane_folder.rb:66:in `join'
  /Users/marcelofabri/.rbenv/versions/2.5.3/lib/ruby/gems/2.5.0/bundler/gems/fastlane-de5dec310af3/fastlane_core/lib/fastlane_core/fastlane_folder.rb:66:in `fastfile_path'
  /Users/marcelofabri/.rbenv/versions/2.5.3/lib/ruby/gems/2.5.0/bundler/gems/fastlane-de5dec310af3/fastlane/lib/fastlane/cli_tools_distributor.rb:182:in `available_lanes'
  /Users/marcelofabri/.rbenv/versions/2.5.3/lib/ruby/gems/2.5.0/bundler/gems/fastlane-de5dec310af3/fastlane/lib/fastlane/cli_tools_distributor.rb:85:in `take_off'
  /Users/marcelofabri/.rbenv/versions/2.5.3/lib/ruby/gems/2.5.0/bundler/gems/fastlane-de5dec310af3/bin/fastlane:23:in `<top (required)>'
  /Users/marcelofabri/.rbenv/versions/2.5.3/lib/ruby/gems/2.5.0/bin/fastlane:23:in `load'
  /Users/marcelofabri/.rbenv/versions/2.5.3/lib/ruby/gems/2.5.0/bin/fastlane:23:in `<top (required)>'

marcelofabri on 7 Dec 2018

Ok, this is just absurd:

DEBUG: Dir.getwd = 'project-folder'
all files in current wd = '["Cartfile.resolved", "CHANGELOG.md", "Makefile", "Project.xcodeproj", "Cartfile", "InternalFrameworks", "Tests", "Carthage", "README.md", "Cartfile.private", "API", "Pods", "CONTRIBUTING.md", "Configs", "Scripts", "fastlane", "Dangerfile", "Project.xcworkspace", "Gemfile", "Gemfile.lock", "Project", "Brewfile", "Podfile", "build", "swiftgen.yml", "Podfile.lock", "Vendor"]'
fastlane_folder: 1
DEBUG: fastlane_folder_exists = 'false'

the code behind this is:

FOLDER_NAME = 'fastlane'
...
   puts("DEBUG: Dir.getwd = '#{Dir.getwd}'")
   puts("all files in current wd = '#{Dir.glob("*")}'")
   puts("fastlane_folder: 1")
   fastlane_folder_exists = File.directory?("./#{FOLDER_NAME}/")
   puts("DEBUG: fastlane_folder_exists = '#{fastlane_folder_exists}'")

We are in the correct folder, the fastlane folder exists, but checking for it via File.directory?("./fastlane/") returns false after it returned true the last 3 times.

It seems a bit like if the filesystem was completely gone here.

Wut?

janpio on 7 Dec 2018

Are you running your command in fastlane or in the project folder itself?

janpio on 7 Dec 2018

On the project folder.

Maybe this is nuts, but I wonder if removing the trailing slash would change something when calling File.directory?.

marcelofabri on 7 Dec 2018

Good idea: https://github.com/fastlane/fastlane/pull/13840/commits/3fa69a9fed11da27b3001341e3e32e0d880bfb68 I added a few additional checks.

You can try foo4 then.

TIL: https://stackoverflow.com/a/8590106/252627

janpio on 7 Dec 2018

$ bundle exec fastlane scan
take_off
[✔] 🚀 
take_off: after require
take_off: after lang
DEBUG: Dir.getwd = 'project-folder'
all files in current wd = '["Cartfile.resolved", "CHANGELOG.md", "Makefile", "Project.xcodeproj", "Cartfile", "InternalFrameworks", "Tests", "Carthage", "README.md", "Cartfile.private", "API", "Pods", "CONTRIBUTING.md", "Configs", "Scripts", "fastlane", "Dangerfile", "Project.xcworkspace", "Gemfile", "Gemfile.lock", "Project", "Brewfile", "Podfile", "build", "swiftgen.yml", "Podfile.lock", "Vendor"]'
fastlane_folder: 1
DEBUG: fastlane_folder_is_directory = 'true' (File.directory?(./FOLDER_NAME/))
DEBUG: fastlane_folder_exists_no_trailing_slash = 'true' (File.directory?(./FOLDER_NAME))
DEBUG: fastlane_folder_exists_no_slashes_at_all = 'true' (File.directory?(FOLDER_NAME))
DEBUG: fastlane_folder_is_a_file = 'false' (File.file?(./FOLDER_NAME/))
DEBUG: fastlane_folder_exists = 'false' (File.exist?(./FOLDER_NAME/))
DEBUG: fastfile_exists_in_fastlane_folder = 'false' (File.exist?(./FOLDER_NAME/Fastfile))
fastlane_folder: 2: ''
fastlane_folder: 3: ''
fastlane_folder: 4: ''
fastlane_folder: 5: ''
fastlane_folder: 6: ''
fastlane_folder: 7: ''
DEBUG: self.path = '', Dir.getwd = 'project-folder'
take_off: after dotenv
take_off: after updatechecker
take_off: after tool_name (= 'scan'), before if ...
available_lanes
fastfile_path 0: Dir.getwd = 'project-folder'
DEBUG: Dir.getwd = 'project-folder'
all files in current wd = '["Cartfile.resolved", "CHANGELOG.md", "Makefile", "Project.xcodeproj", "Cartfile", "InternalFrameworks", "Tests", "Carthage", "README.md", "Cartfile.private", "API", "Pods", "CONTRIBUTING.md", "Configs", "Scripts", "fastlane", "Dangerfile", "Project.xcworkspace", "Gemfile", "Gemfile.lock", "Project", "Brewfile", "Podfile", "build", "swiftgen.yml", "Podfile.lock", "Vendor"]'
fastlane_folder: 1
DEBUG: fastlane_folder_is_directory = 'true' (File.directory?(./FOLDER_NAME/))
DEBUG: fastlane_folder_exists_no_trailing_slash = 'true' (File.directory?(./FOLDER_NAME))
DEBUG: fastlane_folder_exists_no_slashes_at_all = 'true' (File.directory?(FOLDER_NAME))
DEBUG: fastlane_folder_is_a_file = 'false' (File.file?(./FOLDER_NAME/))
DEBUG: fastlane_folder_exists = 'false' (File.exist?(./FOLDER_NAME/))
DEBUG: fastfile_exists_in_fastlane_folder = 'false' (File.exist?(./FOLDER_NAME/Fastfile))
fastlane_folder: 2: ''
fastlane_folder: 3: ''
fastlane_folder: 4: ''
fastlane_folder: 5: ''
fastlane_folder: 6: './'
fastlane_folder: 7: './'
DEBUG: self.path = './', Dir.getwd = 'project-folder'
fastfile_path 1: Fastfile.swift: Dir.getwd = 'project-folder'
DEBUG: Dir.getwd = 'project-folder'
all files in current wd = '["Cartfile.resolved", "CHANGELOG.md", "Makefile", "Project.xcodeproj", "Cartfile", "InternalFrameworks", "Tests", "Carthage", "README.md", "Cartfile.private", "API", "Pods", "CONTRIBUTING.md", "Configs", "Scripts", "fastlane", "Dangerfile", "Project.xcworkspace", "Gemfile", "Gemfile.lock", "Project", "Brewfile", "Podfile", "build", "swiftgen.yml", "Podfile.lock", "Vendor"]'
fastlane_folder: 1
DEBUG: fastlane_folder_is_directory = 'true' (File.directory?(./FOLDER_NAME/))
DEBUG: fastlane_folder_exists_no_trailing_slash = 'true' (File.directory?(./FOLDER_NAME))
DEBUG: fastlane_folder_exists_no_slashes_at_all = 'true' (File.directory?(FOLDER_NAME))
DEBUG: fastlane_folder_is_a_file = 'false' (File.file?(./FOLDER_NAME/))
DEBUG: fastlane_folder_exists = 'false' (File.exist?(./FOLDER_NAME/))
DEBUG: fastfile_exists_in_fastlane_folder = 'false' (File.exist?(./FOLDER_NAME/Fastfile))
fastlane_folder: 2: ''
fastlane_folder: 3: ''
fastlane_folder: 4: ''
fastlane_folder: 5: ''
fastlane_folder: 6: ''
fastlane_folder: 7: ''
DEBUG: self.path = '', Dir.getwd = 'project-folder'
bundler: failed to load command: fastlane (/Users/marcelofabri/.rbenv/versions/2.5.3/lib/ruby/gems/2.5.0/bin/fastlane)
TypeError: no implicit conversion of nil into String
  /Users/marcelofabri/.rbenv/versions/2.5.3/lib/ruby/gems/2.5.0/bundler/gems/fastlane-52cd41b5f946/fastlane_core/lib/fastlane_core/fastlane_folder.rb:78:in `join'
  /Users/marcelofabri/.rbenv/versions/2.5.3/lib/ruby/gems/2.5.0/bundler/gems/fastlane-52cd41b5f946/fastlane_core/lib/fastlane_core/fastlane_folder.rb:78:in `fastfile_path'
  /Users/marcelofabri/.rbenv/versions/2.5.3/lib/ruby/gems/2.5.0/bundler/gems/fastlane-52cd41b5f946/fastlane/lib/fastlane/cli_tools_distributor.rb:182:in `available_lanes'
  /Users/marcelofabri/.rbenv/versions/2.5.3/lib/ruby/gems/2.5.0/bundler/gems/fastlane-52cd41b5f946/fastlane/lib/fastlane/cli_tools_distributor.rb:85:in `take_off'
  /Users/marcelofabri/.rbenv/versions/2.5.3/lib/ruby/gems/2.5.0/bundler/gems/fastlane-52cd41b5f946/bin/fastlane:23:in `<top (required)>'
  /Users/marcelofabri/.rbenv/versions/2.5.3/lib/ruby/gems/2.5.0/bin/fastlane:23:in `load'
  /Users/marcelofabri/.rbenv/versions/2.5.3/lib/ruby/gems/2.5.0/bin/fastlane:23:in `<top (required)>'

marcelofabri on 7 Dec 2018

Ok, foo4 was buggy - but that gave me an idea for foo5 - which can now be used to test.

Do you have the same results with other commands like bundle exec fastlane gym or anything like that? Maybe also just bundle exec fastlane?

janpio on 7 Dec 2018

@janpio have you made any changes that could fix the issue?

I've trying reproducing it for a while and I couldn't anymore - will keep trying.

marcelofabri on 7 Dec 2018

Not really, I added more checks for different paths and files and some output of files/folders and files/folders inside a folder. Mabye one of those somehow "reset" whatever blocked the actual code?

(But if this would be the case, it would have to be very indirect - I made sure that the actual check that was in there and returning false where it should be true was done _before_ all my debug stuff.)

janpio on 7 Dec 2018

I created a branch janpio-path_nil_old that is what we had as foo3 and where you could still reproduce the problem. Switching over to that should show if you can reproduce it. If yes, and another try with foo5 still shows it fixed, we will have to investigate what is going on here and what change was relevant.

janpio on 7 Dec 2018

OK, I was able to reproduce it with foo5:

bundle exec fastlane scan
take_off
[✔] 🚀 
take_off: after require
take_off: after lang
DEBUG: Dir.getwd = 'project-folder'
all files in current wd = '["Cartfile.resolved", "CHANGELOG.md", "Makefile", "Project.xcodeproj", "Cartfile", "InternalFrameworks", "Tests", "Carthage", "README.md", "Cartfile.private", "API", "Pods", "CONTRIBUTING.md", "Configs", "Scripts", "fastlane", "Dangerfile", "Project.xcworkspace", "Gemfile", "Gemfile.lock", "Project", "Brewfile", "Podfile", "build", "swiftgen.yml", "Podfile.lock", "Vendor"]'
all files in current wd/fastlane = '["fastlane/Gymfile", "fastlane/test_output", "fastlane/Pluginfile", "fastlane/Appfile", "fastlane/Fastfile", "fastlane/README.md", "fastlane/Scanfile", "fastlane/actions", "fastlane/build", "fastlane/Matchfile", "fastlane/report.xml", "fastlane/helpers"]'
fastlane_folder: 1
DEBUG: fastlane_folder_is_directory = 'true' (File.directory?(./FOLDER_NAME/))
DEBUG: fastlane_folder_exists_no_trailing_slash = 'true' (File.directory?(./FOLDER_NAME))
DEBUG: fastlane_folder_exists_no_slashes_at_all = 'true' (File.directory?(FOLDER_NAME))
DEBUG: fastlane_folder_is_a_file = 'false' (File.file?(./FOLDER_NAME/))
DEBUG: fastlane_folder_exists = 'false' (File.exist?(./FOLDER_NAME/))
DEBUG: fastfile_exists_in_fastlane_folder = 'false' (File.exist?(./FOLDER_NAME/Fastfile))
fastlane_folder: 2: './fastlane/'
fastlane_folder: 3: './fastlane/'
fastlane_folder: 4: './fastlane/'
fastlane_folder: 5: './fastlane/'
fastlane_folder: 6: './fastlane/'
fastlane_folder: 7: './fastlane/'
DEBUG: self.path = './fastlane/', Dir.getwd = 'project-folder'
take_off: after dotenv
take_off: after updatechecker
take_off: after tool_name (= 'scan'), before if ...
available_lanes
fastfile_path 0: Dir.getwd = 'project-folder'
DEBUG: Dir.getwd = 'project-folder'
all files in current wd = '["Cartfile.resolved", "CHANGELOG.md", "Makefile", "Project.xcodeproj", "Cartfile", "InternalFrameworks", "Tests", "Carthage", "README.md", "Cartfile.private", "API", "Pods", "CONTRIBUTING.md", "Configs", "Scripts", "fastlane", "Dangerfile", "Project.xcworkspace", "Gemfile", "Gemfile.lock", "Project", "Brewfile", "Podfile", "build", "swiftgen.yml", "Podfile.lock", "Vendor"]'
all files in current wd/fastlane = '["fastlane/Gymfile", "fastlane/test_output", "fastlane/Pluginfile", "fastlane/Appfile", "fastlane/Fastfile", "fastlane/README.md", "fastlane/Scanfile", "fastlane/actions", "fastlane/build", "fastlane/Matchfile", "fastlane/report.xml", "fastlane/helpers"]'
fastlane_folder: 1
DEBUG: fastlane_folder_is_directory = 'false' (File.directory?(./FOLDER_NAME/))
DEBUG: fastlane_folder_exists_no_trailing_slash = 'false' (File.directory?(./FOLDER_NAME))
DEBUG: fastlane_folder_exists_no_slashes_at_all = 'false' (File.directory?(FOLDER_NAME))
DEBUG: fastlane_folder_is_a_file = 'false' (File.file?(./FOLDER_NAME/))
DEBUG: fastlane_folder_exists = 'false' (File.exist?(./FOLDER_NAME/))
DEBUG: fastfile_exists_in_fastlane_folder = 'false' (File.exist?(./FOLDER_NAME/Fastfile))
fastlane_folder: 2: ''
fastlane_folder: 3: ''
fastlane_folder: 4: ''
fastlane_folder: 5: ''
fastlane_folder: 6: './'
fastlane_folder: 7: './'
DEBUG: self.path = './', Dir.getwd = 'project-folder/fastlane'
fastfile_path 1: Fastfile.swift: Dir.getwd = 'project-folder/fastlane'
DEBUG: Dir.getwd = 'project-folder/fastlane'
all files in current wd = '["Gymfile", "test_output", "Pluginfile", "Appfile", "Fastfile", "README.md", "Scanfile", "actions", "build", "Matchfile", "report.xml", "helpers"]'
all files in current wd/fastlane = '[]'
fastlane_folder: 1
DEBUG: fastlane_folder_is_directory = 'false' (File.directory?(./FOLDER_NAME/))
DEBUG: fastlane_folder_exists_no_trailing_slash = 'false' (File.directory?(./FOLDER_NAME))
DEBUG: fastlane_folder_exists_no_slashes_at_all = 'false' (File.directory?(FOLDER_NAME))
DEBUG: fastlane_folder_is_a_file = 'false' (File.file?(./FOLDER_NAME/))
DEBUG: fastlane_folder_exists = 'false' (File.exist?(./FOLDER_NAME/))
DEBUG: fastfile_exists_in_fastlane_folder = 'false' (File.exist?(./FOLDER_NAME/Fastfile))
fastlane_folder: 2: ''
fastlane_folder: 3: ''
fastlane_folder: 4: ''
fastlane_folder: 5: ''
fastlane_folder: 6: ''
fastlane_folder: 7: ''
DEBUG: self.path = '', Dir.getwd = 'project-folder'
bundler: failed to load command: fastlane (/Users/marcelofabri/.rbenv/versions/2.5.3/lib/ruby/gems/2.5.0/bin/fastlane)
TypeError: no implicit conversion of nil into String
  /Users/marcelofabri/.rbenv/versions/2.5.3/lib/ruby/gems/2.5.0/bundler/gems/fastlane-412d44145785/fastlane_core/lib/fastlane_core/fastlane_folder.rb:79:in `join'
  /Users/marcelofabri/.rbenv/versions/2.5.3/lib/ruby/gems/2.5.0/bundler/gems/fastlane-412d44145785/fastlane_core/lib/fastlane_core/fastlane_folder.rb:79:in `fastfile_path'
  /Users/marcelofabri/.rbenv/versions/2.5.3/lib/ruby/gems/2.5.0/bundler/gems/fastlane-412d44145785/fastlane/lib/fastlane/cli_tools_distributor.rb:182:in `available_lanes'
  /Users/marcelofabri/.rbenv/versions/2.5.3/lib/ruby/gems/2.5.0/bundler/gems/fastlane-412d44145785/fastlane/lib/fastlane/cli_tools_distributor.rb:85:in `take_off'
  /Users/marcelofabri/.rbenv/versions/2.5.3/lib/ruby/gems/2.5.0/bundler/gems/fastlane-412d44145785/bin/fastlane:23:in `<top (required)>'
  /Users/marcelofabri/.rbenv/versions/2.5.3/lib/ruby/gems/2.5.0/bin/fastlane:23:in `load'
  /Users/marcelofabri/.rbenv/versions/2.5.3/lib/ruby/gems/2.5.0/bin/fastlane:23:in `<top (required)>'

marcelofabri on 7 Dec 2018

I was also able to reproduce it using gym:

$ bundle exec fastlane gym
take_off
[✔] 🚀 
take_off: after require
take_off: after lang
DEBUG: Dir.getwd = 'project-folder'
all files in current wd = '["Cartfile.resolved", "CHANGELOG.md", "Makefile", "Project.xcodeproj", "Cartfile", "InternalFrameworks", "Tests", "Carthage", "README.md", "Cartfile.private", "API", "Pods", "CONTRIBUTING.md", "Configs", "Scripts", "fastlane", "Dangerfile", "Project.xcworkspace", "Gemfile", "Gemfile.lock", "Project", "Brewfile", "Podfile", "build", "swiftgen.yml", "Podfile.lock", "Vendor"]'
all files in current wd/fastlane = '["fastlane/Gymfile", "fastlane/test_output", "fastlane/Pluginfile", "fastlane/Appfile", "fastlane/Fastfile", "fastlane/README.md", "fastlane/Scanfile", "fastlane/actions", "fastlane/build", "fastlane/Matchfile", "fastlane/report.xml", "fastlane/helpers"]'
fastlane_folder: 1
DEBUG: fastlane_folder_is_directory = 'true' (File.directory?(./FOLDER_NAME/))
DEBUG: fastlane_folder_exists_no_trailing_slash = 'true' (File.directory?(./FOLDER_NAME))
DEBUG: fastlane_folder_exists_no_slashes_at_all = 'true' (File.directory?(FOLDER_NAME))
DEBUG: fastlane_folder_is_a_file = 'false' (File.file?(./FOLDER_NAME/))
DEBUG: fastlane_folder_exists = 'false' (File.exist?(./FOLDER_NAME/))
DEBUG: fastfile_exists_in_fastlane_folder = 'false' (File.exist?(./FOLDER_NAME/Fastfile))
fastlane_folder: 2: './fastlane/'
fastlane_folder: 3: './fastlane/'
fastlane_folder: 4: './fastlane/'
fastlane_folder: 5: './fastlane/'
fastlane_folder: 6: './fastlane/'
fastlane_folder: 7: './fastlane/'
DEBUG: self.path = './fastlane/', Dir.getwd = 'project-folder'
take_off: after dotenv
take_off: after updatechecker
take_off: after tool_name (= 'gym'), before if ...
available_lanes
fastfile_path 0: Dir.getwd = 'project-folder'
DEBUG: Dir.getwd = 'project-folder'
all files in current wd = '["Cartfile.resolved", "CHANGELOG.md", "Makefile", "Project.xcodeproj", "Cartfile", "InternalFrameworks", "Tests", "Carthage", "README.md", "Cartfile.private", "API", "Pods", "CONTRIBUTING.md", "Configs", "Scripts", "fastlane", "Dangerfile", "Project.xcworkspace", "Gemfile", "Gemfile.lock", "Project", "Brewfile", "Podfile", "build", "swiftgen.yml", "Podfile.lock", "Vendor"]'
all files in current wd/fastlane = '[]'
fastlane_folder: 1
DEBUG: fastlane_folder_is_directory = 'false' (File.directory?(./FOLDER_NAME/))
DEBUG: fastlane_folder_exists_no_trailing_slash = 'false' (File.directory?(./FOLDER_NAME))
DEBUG: fastlane_folder_exists_no_slashes_at_all = 'false' (File.directory?(FOLDER_NAME))
DEBUG: fastlane_folder_is_a_file = 'false' (File.file?(./FOLDER_NAME/))
DEBUG: fastlane_folder_exists = 'false' (File.exist?(./FOLDER_NAME/))
DEBUG: fastfile_exists_in_fastlane_folder = 'false' (File.exist?(./FOLDER_NAME/Fastfile))
fastlane_folder: 2: ''
fastlane_folder: 3: ''
fastlane_folder: 4: ''
fastlane_folder: 5: ''
fastlane_folder: 6: './'
fastlane_folder: 7: './'
DEBUG: self.path = './', Dir.getwd = 'project-folder/fastlane'
fastfile_path 1: Fastfile.swift: Dir.getwd = 'project-folder/fastlane'
DEBUG: Dir.getwd = 'project-folder/fastlane'
all files in current wd = '["Gymfile", "test_output", "Pluginfile", "Appfile", "Fastfile", "README.md", "Scanfile", "actions", "build", "Matchfile", "report.xml", "helpers"]'
all files in current wd/fastlane = '[]'
fastlane_folder: 1
DEBUG: fastlane_folder_is_directory = 'false' (File.directory?(./FOLDER_NAME/))
DEBUG: fastlane_folder_exists_no_trailing_slash = 'false' (File.directory?(./FOLDER_NAME))
DEBUG: fastlane_folder_exists_no_slashes_at_all = 'true' (File.directory?(FOLDER_NAME))
DEBUG: fastlane_folder_is_a_file = 'false' (File.file?(./FOLDER_NAME/))
DEBUG: fastlane_folder_exists = 'false' (File.exist?(./FOLDER_NAME/))
DEBUG: fastfile_exists_in_fastlane_folder = 'false' (File.exist?(./FOLDER_NAME/Fastfile))
fastlane_folder: 2: ''
fastlane_folder: 3: ''
fastlane_folder: 4: ''
fastlane_folder: 5: ''
fastlane_folder: 6: ''
fastlane_folder: 7: ''
DEBUG: self.path = '', Dir.getwd = 'project-folder'
bundler: failed to load command: fastlane (/Users/marcelofabri/.rbenv/versions/2.5.3/lib/ruby/gems/2.5.0/bin/fastlane)
TypeError: no implicit conversion of nil into String
  /Users/marcelofabri/.rbenv/versions/2.5.3/lib/ruby/gems/2.5.0/bundler/gems/fastlane-412d44145785/fastlane_core/lib/fastlane_core/fastlane_folder.rb:79:in `join'
  /Users/marcelofabri/.rbenv/versions/2.5.3/lib/ruby/gems/2.5.0/bundler/gems/fastlane-412d44145785/fastlane_core/lib/fastlane_core/fastlane_folder.rb:79:in `fastfile_path'
  /Users/marcelofabri/.rbenv/versions/2.5.3/lib/ruby/gems/2.5.0/bundler/gems/fastlane-412d44145785/fastlane/lib/fastlane/cli_tools_distributor.rb:182:in `available_lanes'
  /Users/marcelofabri/.rbenv/versions/2.5.3/lib/ruby/gems/2.5.0/bundler/gems/fastlane-412d44145785/fastlane/lib/fastlane/cli_tools_distributor.rb:85:in `take_off'
  /Users/marcelofabri/.rbenv/versions/2.5.3/lib/ruby/gems/2.5.0/bundler/gems/fastlane-412d44145785/bin/fastlane:23:in `<top (required)>'
  /Users/marcelofabri/.rbenv/versions/2.5.3/lib/ruby/gems/2.5.0/bin/fastlane:23:in `load'
  /Users/marcelofabri/.rbenv/versions/2.5.3/lib/ruby/gems/2.5.0/bin/fastlane:23:in `<top (required)>'

marcelofabri on 7 Dec 2018

Ok, these two outputs have a few interesting things:

In the second "pass" it used to get the correct value between fastlane_folder: 1 and fastlane_folder: 2 all the time. Now it somehow has to go down to fastlane_folder: 6: './' until it gets a value: value ||= "./" if File.basename(Dir.getwd) == FOLDER_NAME && File.exist?('Fastfile'). If this is to be trusted, it now thinks that the last part of the working directory is fastlane. A few lines above we output the wd as project-folder. That doesn't make sense.
In the third "pass" the wd used to be Dir.getwd = 'project-folder' as well. Now it is `Dir.getwd = 'project-folder/fastlane'. Maybe because of the wrong result of 1) somehow?
In the gym run there is something interesting: all files in current wd/fastlane = '["fastlane/Gymfile", "fastlane/test_output", "fastlane/Pluginfile", "fastlane/Appfile", "fastlane/Fastfile", "fastlane/README.md", "fastlane/Scanfile", "fastlane/actions", "fastlane/build", "fastlane/Matchfile", "fastlane/report.xml", "fastlane/helpers"]' is suddenly replaced with all files in current wd/fastlane = '[]' - although the line before showed that wd is as usual and contains a fastlane folder.

Somehow ruby suddenly has problems reading files.

Wasn't there some stuff with how many files could have been opened on Linux and Mac, possibly https://stackoverflow.com/questions/40597488/ruby-too-many-open-files-rb-sysopen? Although that should throw an exception instead of giving wrong information.

Anything special about your file system, background programs running or such things?

janpio on 7 Dec 2018

Unfortunately I'm out of ideas too. I can't think about anything special other than the fact that it's using APFS.

marcelofabri on 7 Dec 2018

foo7 is ready, almost no actual changes but prettier output - maybe that will help us think (and enable me to create a StackOverflow question with some output an external person might even unterstand).

janpio on 7 Dec 2018

I looked at your output as well @roostr - but am afraid this is a _different_ and possibly new issue! Could you please copy this in a new issue and add some information around it?
Back in #13512 the errors also were #<TypeError: no implicit conversion of nil into String> - which made them a match to possibly be the same core problem as this one. What you posted is different.

Thank you @janpio. I looked into it more, and it was happening because I was passing a full path for my screenshots_path parameter to upload_to_app_store. The failure happened because passing a relative path and absolute path into the ruby method was not allowed.

I fixed it like this:

        upload_to_app_store(
                app_identifier: "REDACTED",
                username: "REDACTED",
-               screenshots_path: "/Users/russ/Projects/HappyScale/Artwork/Screenshots/5-Captioned",
+               screenshots_path: "./Artwork/Screenshots/5-Captioned",
                metadata_path: "Metadata",

I'm not sure which version of fastlane this broke for me at, but it was within the last 3 months. I'm satisfied now that I have a working solution.

roostr on 7 Dec 2018

Thanks @roost - I created https://github.com/fastlane/fastlane/issues/13853 anyway ;)

janpio on 7 Dec 2018

Thank you, @janpio !

roostr on 7 Dec 2018

Pasting the full output here without redacting anything just to make sure I'm not messing something:

$ bundle exec fastlane scan
0 take_off
[✔] 🚀 
0 take_off: after require
0 take_off: after lang
3 path: Dir.getwd = '/Users/marcelofabri/instashopper-ios' - START 
3 path: Dir.getwd = '/Users/marcelofabri/instashopper-ios' - files in current wd = '["Cartfile.resolved", "CHANGELOG.md", "Makefile", "InstaShopper.xcodeproj", "Cartfile", "InternalFrameworks", "Tests", "Carthage", "README.md", "Cartfile.private", "ShopperAPI", "Pods", "CONTRIBUTING.md", "Configs", "Scripts", "fastlane", "Dangerfile", "InstaShopper.xcworkspace", "Gemfile", "Gemfile.lock", "InstaShopper", "Brewfile", "Podfile", "build", "swiftgen.yml", "Podfile.lock", "Vendor"]'
3 path: Dir.getwd = '/Users/marcelofabri/instashopper-ios' - files in current wd/fastlane = '["fastlane/Gymfile", "fastlane/test_output", "fastlane/Pluginfile", "fastlane/Appfile", "fastlane/Fastfile", "fastlane/README.md", "fastlane/Scanfile", "fastlane/actions", "fastlane/build", "fastlane/Matchfile", "fastlane/report.xml", "fastlane/helpers"]'
3 path: Dir.getwd = '/Users/marcelofabri/instashopper-ios' - fastlane_folder_is_directory =                    'true' (File.directory?(./FOLDER_NAME/))
3 path: Dir.getwd = '/Users/marcelofabri/instashopper-ios' - fastlane_folder_is_directory_no_trailing_slash =  'true' (File.directory?(./FOLDER_NAME))
3 path: Dir.getwd = '/Users/marcelofabri/instashopper-ios' - fastlane_folder_is_directory_no_slashes_at_all =  'true' (File.directory?(FOLDER_NAME))
3 path: Dir.getwd = '/Users/marcelofabri/instashopper-ios' - fastlane_folder_is_a_file =                       'false' (File.file?(./FOLDER_NAME/))
3 path: Dir.getwd = '/Users/marcelofabri/instashopper-ios' - fastlane_folder_exists =                          'false' (File.exist?(./FOLDER_NAME/)),
3 path: Dir.getwd = '/Users/marcelofabri/instashopper-ios' - fastfile_exists_in_fastlane_folder =              'false' (File.exist?(./FOLDER_NAME/Fastfile))
3 path: Dir.getwd = '/Users/marcelofabri/instashopper-ios' - fastfile_exists =                                 'false' (File.exist?(Fastfile))
3 path: Dir.getwd = '/Users/marcelofabri/instashopper-ios' - 1: ''
3 path: Dir.getwd = '/Users/marcelofabri/instashopper-ios' - 2: './fastlane/'
3 path: Dir.getwd = '/Users/marcelofabri/instashopper-ios' - 3: './fastlane/'
3 path: Dir.getwd = '/Users/marcelofabri/instashopper-ios' - 4: './fastlane/'
3 path: Dir.getwd = '/Users/marcelofabri/instashopper-ios' - 5: './fastlane/'
3 path: Dir.getwd = '/Users/marcelofabri/instashopper-ios' - 6: './fastlane/'
3 path: Dir.getwd = '/Users/marcelofabri/instashopper-ios' - 7: './fastlane/'
3 path: Dir.getwd = '/Users/marcelofabri/instashopper-ios' - END value = './fastlane/'

0 take_off: after dotenv
0 take_off: after updatechecker
0 take_off: after tool_name (= 'scan'), before if ...
1 available_lanes
2 fastfile_path 0: Dir.getwd = '/Users/marcelofabri/instashopper-ios'
3 path: Dir.getwd = '/Users/marcelofabri/instashopper-ios' - START 
3 path: Dir.getwd = '/Users/marcelofabri/instashopper-ios' - files in current wd = '["Cartfile.resolved", "CHANGELOG.md", "Makefile", "InstaShopper.xcodeproj", "Cartfile", "InternalFrameworks", "Tests", "Carthage", "README.md", "Cartfile.private", "ShopperAPI", "Pods", "CONTRIBUTING.md", "Configs", "Scripts", "fastlane", "Dangerfile", "InstaShopper.xcworkspace", "Gemfile", "Gemfile.lock", "InstaShopper", "Brewfile", "Podfile", "build", "swiftgen.yml", "Podfile.lock", "Vendor"]'
3 path: Dir.getwd = '/Users/marcelofabri/instashopper-ios' - files in current wd/fastlane = '["fastlane/Gymfile", "fastlane/test_output", "fastlane/Pluginfile", "fastlane/Appfile", "fastlane/Fastfile", "fastlane/README.md", "fastlane/Scanfile", "fastlane/actions", "fastlane/build", "fastlane/Matchfile", "fastlane/report.xml", "fastlane/helpers"]'
3 path: Dir.getwd = '/Users/marcelofabri/instashopper-ios/fastlane' - fastlane_folder_is_directory =                    'false' (File.directory?(./FOLDER_NAME/))
3 path: Dir.getwd = '/Users/marcelofabri/instashopper-ios/fastlane' - fastlane_folder_is_directory_no_trailing_slash =  'false' (File.directory?(./FOLDER_NAME))
3 path: Dir.getwd = '/Users/marcelofabri/instashopper-ios/fastlane' - fastlane_folder_is_directory_no_slashes_at_all =  'false' (File.directory?(FOLDER_NAME))
3 path: Dir.getwd = '/Users/marcelofabri/instashopper-ios/fastlane' - fastlane_folder_is_a_file =                       'false' (File.file?(./FOLDER_NAME/))
3 path: Dir.getwd = '/Users/marcelofabri/instashopper-ios/fastlane' - fastlane_folder_exists =                          'false' (File.exist?(./FOLDER_NAME/)),
3 path: Dir.getwd = '/Users/marcelofabri/instashopper-ios/fastlane' - fastfile_exists_in_fastlane_folder =              'false' (File.exist?(./FOLDER_NAME/Fastfile))
3 path: Dir.getwd = '/Users/marcelofabri/instashopper-ios/fastlane' - fastfile_exists =                                 'true' (File.exist?(Fastfile))
3 path: Dir.getwd = '/Users/marcelofabri/instashopper-ios/fastlane' - 1: ''
3 path: Dir.getwd = '/Users/marcelofabri/instashopper-ios/fastlane' - 2: ''
3 path: Dir.getwd = '/Users/marcelofabri/instashopper-ios/fastlane' - 3: ''
3 path: Dir.getwd = '/Users/marcelofabri/instashopper-ios/fastlane' - 4: ''
3 path: Dir.getwd = '/Users/marcelofabri/instashopper-ios/fastlane' - 5: ''
3 path: Dir.getwd = '/Users/marcelofabri/instashopper-ios/fastlane' - 6: './'
3 path: Dir.getwd = '/Users/marcelofabri/instashopper-ios/fastlane' - 7: './'
3 path: Dir.getwd = '/Users/marcelofabri/instashopper-ios/fastlane' - END value = './'

2 fastfile_path 1: Dir.getwd = '/Users/marcelofabri/instashopper-ios' - Fastfile.swift
3 path: Dir.getwd = '/Users/marcelofabri/instashopper-ios' - START 
3 path: Dir.getwd = '/Users/marcelofabri/instashopper-ios' - files in current wd = '["Cartfile.resolved", "CHANGELOG.md", "Makefile", "InstaShopper.xcodeproj", "Cartfile", "InternalFrameworks", "Tests", "Carthage", "README.md", "Cartfile.private", "ShopperAPI", "Pods", "CONTRIBUTING.md", "Configs", "Scripts", "fastlane", "Dangerfile", "InstaShopper.xcworkspace", "Gemfile", "Gemfile.lock", "InstaShopper", "Brewfile", "Podfile", "build", "swiftgen.yml", "Podfile.lock", "Vendor"]'
3 path: Dir.getwd = '/Users/marcelofabri/instashopper-ios/fastlane' - files in current wd/fastlane = '[]'
3 path: Dir.getwd = '/Users/marcelofabri/instashopper-ios/fastlane' - fastlane_folder_is_directory =                    'false' (File.directory?(./FOLDER_NAME/))
3 path: Dir.getwd = '/Users/marcelofabri/instashopper-ios/fastlane' - fastlane_folder_is_directory_no_trailing_slash =  'false' (File.directory?(./FOLDER_NAME))
3 path: Dir.getwd = '/Users/marcelofabri/instashopper-ios/fastlane' - fastlane_folder_is_directory_no_slashes_at_all =  'false' (File.directory?(FOLDER_NAME))
3 path: Dir.getwd = '/Users/marcelofabri/instashopper-ios/fastlane' - fastlane_folder_is_a_file =                       'false' (File.file?(./FOLDER_NAME/))
3 path: Dir.getwd = '/Users/marcelofabri/instashopper-ios/fastlane' - fastlane_folder_exists =                          'false' (File.exist?(./FOLDER_NAME/)),
3 path: Dir.getwd = '/Users/marcelofabri/instashopper-ios/fastlane' - fastfile_exists_in_fastlane_folder =              'false' (File.exist?(./FOLDER_NAME/Fastfile))
3 path: Dir.getwd = '/Users/marcelofabri/instashopper-ios' - fastfile_exists =                                 'false' (File.exist?(Fastfile))
3 path: Dir.getwd = '/Users/marcelofabri/instashopper-ios' - 1: ''
3 path: Dir.getwd = '/Users/marcelofabri/instashopper-ios' - 2: ''
3 path: Dir.getwd = '/Users/marcelofabri/instashopper-ios' - 3: ''
3 path: Dir.getwd = '/Users/marcelofabri/instashopper-ios' - 4: ''
3 path: Dir.getwd = '/Users/marcelofabri/instashopper-ios' - 5: ''
3 path: Dir.getwd = '/Users/marcelofabri/instashopper-ios' - 6: ''
3 path: Dir.getwd = '/Users/marcelofabri/instashopper-ios' - 7: ''
3 path: Dir.getwd = '/Users/marcelofabri/instashopper-ios' - END value = ''

bundler: failed to load command: fastlane (/Users/marcelofabri/.rbenv/versions/2.5.3/lib/ruby/gems/2.5.0/bin/fastlane)
TypeError: no implicit conversion of nil into String
  /Users/marcelofabri/.rbenv/versions/2.5.3/lib/ruby/gems/2.5.0/bundler/gems/fastlane-80d2dfbb37ce/fastlane_core/lib/fastlane_core/fastlane_folder.rb:88:in `join'
  /Users/marcelofabri/.rbenv/versions/2.5.3/lib/ruby/gems/2.5.0/bundler/gems/fastlane-80d2dfbb37ce/fastlane_core/lib/fastlane_core/fastlane_folder.rb:88:in `fastfile_path'
  /Users/marcelofabri/.rbenv/versions/2.5.3/lib/ruby/gems/2.5.0/bundler/gems/fastlane-80d2dfbb37ce/fastlane/lib/fastlane/cli_tools_distributor.rb:182:in `available_lanes'
  /Users/marcelofabri/.rbenv/versions/2.5.3/lib/ruby/gems/2.5.0/bundler/gems/fastlane-80d2dfbb37ce/fastlane/lib/fastlane/cli_tools_distributor.rb:85:in `take_off'
  /Users/marcelofabri/.rbenv/versions/2.5.3/lib/ruby/gems/2.5.0/bundler/gems/fastlane-80d2dfbb37ce/bin/fastlane:23:in `<top (required)>'
  /Users/marcelofabri/.rbenv/versions/2.5.3/lib/ruby/gems/2.5.0/bin/fastlane:23:in `load'
  /Users/marcelofabri/.rbenv/versions/2.5.3/lib/ruby/gems/2.5.0/bin/fastlane:23:in `<top (required)>'

marcelofabri on 7 Dec 2018

❤1

Thanks @marcelofabri - that is pretty good to read now I think.
Still doesn't make any sense though to me :/

The working directory of the script is mutating somehow. This sometimes causes the wrong path, and sometimes no path, to be returned.

Do you always get the exact same result for each script run now or is it different for you as well?

I have an idea: What is your Fastfile? Can you share it here or send it to [email protected] please? Any other files added that have normal ruby code in them (Appfile, Scanfile etc.)?

janpio on 7 Dec 2018

Do you always get the exact same result for each script run now or is it different for you as well?

You mean if it consistently fails? It seems random (I need to try a few times to get a crash).

We have a somewhat complex setup with custom actions and several lanes, so unfortunately I don't think I can share them 😢

marcelofabri on 7 Dec 2018

Ok, can you make sure there is no Dir.chdir (or similar) stuff going on _outside_ of a lane or method that could be executed when the file is loaded? I am trying to think of reasons for that directory change, especially to ./fastlane. This is not random and has to be triggered by something somehow.

janpio on 7 Dec 2018

Yeah, I've checked that: we have two Dir.chdir but inside actions (and using the closure version). We have 2 sh("cd ...") as well, but inside lanes.

marcelofabri on 7 Dec 2018

👍1

And another thought: Can you run the command in fastlane and see what happens? Can you also produce a crash here?

janpio on 7 Dec 2018

It seems that it doesn't crash in that case 🤔

I wonder if there's a chance of another thread changing the current directory 😬

marcelofabri on 8 Dec 2018

Are you running multiple fastlane instances at the same time? How are you running fastlane - normal Mac terminal app?

janpio on 8 Dec 2018

Just one instance, from Terminal.app (using zsh if that matters). But I've seen there're a few usages of Thread inside fastlane itself.

marcelofabri on 8 Dec 2018

There actually doesn't happen much in fastlane before your crash: bin/fastlane takes the call, cli_tools_distributor.rb runs take_off and that calls (first something to do with the env var loading which uses the path once, that's the first pass that is alway okay in your output - and then) available_lanes which goes into FastlaneCore::FastlaneFolder.fastfile_path which tries to load self.path 3 times, and the second one crashes for you. No process spawning, shell stuff, Dir.* or anything :/

janpio on 8 Dec 2018

Total luck shot, but can you try emptying your Fastfile and try to reproduce? This happens in a method that wants to show the available lanes - but from my reading of the code _before_ the file is actually touched (the method is used to get the path to know where it is), but let's just make sure...

janpio on 8 Dec 2018

Now a question on Stack Overflow: https://stackoverflow.com/questions/53678098/what-can-cause-the-working-directory-to-be-changed-if-dir-chdir-is-not-called

janpio on 8 Dec 2018

I've removed all contents of my Fastfile and was still able to reproduce it 😭

marcelofabri on 8 Dec 2018

Nah, this is pretty much what I hoped, if it had fixed the issue it would have meant I didn't understand the code running there and we had to start over :/

Which gives me an idea: Please set the FASTLANE_SKIP_UPDATE_CHECK env var and see if it is still reproducible. That is one part of the code I don't really understand and does indeed use Thread.

janpio on 8 Dec 2018

@janpio still trying to reproduce, but so far I couldn't when setting FASTLANE_SKIP_UPDATE_CHECK.

I'll keep trying, but wanted to give you a heads up that you might be on the right path 💯

marcelofabri on 8 Dec 2018

(That would actually be not very good, as I couldn't find any code in there that would explain the working directory to change)

janpio on 8 Dec 2018

Okay, it would help to know a bit more about your environments. Please fill this "form":

OS and OS version:
Ruby version:
Ruby install method:
File system:
What is running at the same time:

@Falken42 @cojoj @eriadam @jamezilla @ronocod @mathaeus @cesarferreira @dloic

Also, it would be awesome if more people could use my custom branch with additional output and post their result - if they can still reproduce the problem and what output they are getting.

janpio on 8 Dec 2018

@janpio: I still only very rarely see this happen (nowhere near as often as the others here have been able to repro it), but here's my environment as a datapoint to help narrow things down:

OS and OS version: macOS High Sierra 10.13.6 (17G3025)
Ruby version: ruby 2.5.1p57 (2018-03-29 revision 63029) [x86_64-darwin17]
Ruby install method: Homebrew (brew install ruby)
File system:
- OSX System Disk: Mac OS Extended (Journaled)
- fastlane execution disk: USB External Physical Volume / Mac OS Extended (Journaled)
What is running at the same time: only gitlab-runner (with concurrent = 2 so both macOS and iOS builds are performed simultaneously, but only the iOS build calls out to fastlane)

falken42 on 9 Dec 2018

❤1

@janpio I'm visiting from the SO post because it's a little easier to have the discussion here than in the comments boxes. I don't use fastlane or write ios/swift code I just think this is an interesting Ruby behavior so I'd like to try and help.

Have you tried throwing the pry/pry-byebug gems into your fastlane gemspec (you'll want to add them as dependencies, rather than development dependencies I think)?

You could use a rescue TypeError => e with a binding.pry inside that self.path method to inspect the environment when it errors. Note: Don't use a branch with that in your production code or a CI runner

I'm trying to recreate a local environment to reproduce the error. I pushed up a public repo here: https://github.com/jaydorsey/FastlaneDemo

Everything seems to work (e.g. I can run bundle exec fastlane ios custom_lane) but I'm not 100% sure I'm recreating what you guys are testing. Mainly because I couldn't make it break. If someone can take a look at it I'd appreciate it?

I did get all the debug output, but even when I ran this 100 times in a loop nothing broke. Ruby 2.5.3 with readline support, macos 10.14.1.

As I mentioned on the SO post, one immediate change that could be made is making the value ||= assignments straight up returns because ||= isn't thread safe. I think a return is a little more expressive as well. Would still like to see if the root cause can be identified so if anyone is able to make a PR on my repo with a reproducible (even intermittently) change that would be cool 👍

jaydorsey on 9 Dec 2018

👍1

Good call re Stackoverflow 👍

I think your demo project looks pretty good, @marcelofabri should be able to confirm if he can reproduce the problem with this project as well. I never could reproduce this at all on my machines :/

What would we look for in the environment when using binding.pry? (tbh I don't actually know ruby and have no experience with pry - so I might need some pointers here)

Regarding ||= I will repeat what I replied on StackOverflow for all the other people following this issue:

I could probably neutralize this problem by e.g. using return instead of ||=, caching the path instead of "recalculating" it by checking the file system every time the method is called. That would definitely decrease the number of affected users, but the underlying problem would still exist - somehow the working directory is changing in strange ways. If I optimize it to only a few calls that interact with the FS, instead of the hundreds we have right now, those few can still fail and crash the script :/

janpio on 9 Dec 2018

When the pry runs you'll basically be at an IRB-like console, in the middle of execution. You can look at things like ENV variables, Dir.getwd, etc. and run & evaluate each of the statements in the self.path method so you can make sure the results are evaluating like you expect them to.

If this is using threads what you might see is your spitting out more data & you have to kill the console (CTRL-C just like normal). That would be a pretty good indication that this part of the code is running in a thread somewhere. I'm still digging my way thru the source code to trace the behavior back.

jaydorsey on 10 Dec 2018

(bundle exec fastlane scan: bin/fastlane => take_off => availables_lanes is the crashing path for @marcelofabri - in there, 3rd time self.path is called it crashes for him)

janpio on 10 Dec 2018

@janpio

OS and OS version: macOS 10.13.6 (17G3025)
Ruby version: ruby 2.5.3p105 (2018-10-18 revision 65156) [x86_64-darwin17]
Ruby install method: Probably Homebrew
File system: APFS
What is running at the same time: difficult to say as we don't have control over the VM provided by Bitrise, but our setup is not doing things in parallel

dloic on 10 Dec 2018

👍1

While thinking about how to further debug this, I noticed that _I_ was one of the people reporting issues with this: https://github.com/fastlane/fastlane/issues/12789#issuecomment-438623197. I had totally forgotten about that.

First I looked for the 2 cases I had logged in that issue to see what environment they occured:

https://janpio.visualstudio.com/fastlane-e2e-tests/_build/results?buildId=256 - Here it is the Linux job, Ubuntu 1604
https://janpio.visualstudio.com/fastlane-e2e-tests/_build/results?buildId=117 - This is a Windows only build

So it doesn't seem to be bound to macOS :/

I also looked at the crashing commands: bundle exec fastlane screengrab init and bundle exec fastlane supply init --package_name=tools.fastlane.androidapp --json-key="D:\a\_temp\google_play_key.json". Both crashed in the available_lanes method, as it does for @marcelofabri

Then I set out to replicate what I was getting in that issue, where I had uses fastlane on Azure Devops Pipelines. This turned out to not be as easy as I hoped: Creating a minimal project that just run bundle exec fastlane scan multiple times to try to cause the crash did not work - the project did exactly what it should do and did not crash, no matter how often I restarted it.

I looked at my original issue https://github.com/fastlane/fastlane/issues/13680 again and noticed that I had been doing something special there: I was executing a fastlane lane, that was executing another lane via sh. Still, replicating this in a test project couldn't trigger the crash.

So I went back to one of the builds above (https://janpio.visualstudio.com/fastlane-e2e-tests/_build/results?buildId=117, the Windows one) and rerun the exact same code that had caused a crash back then. (It's great that CI can do rerun a specific commit from the past! 0aca5b2d2cd47ea4900f8df661816a8b600a3481) And boom, crash: https://janpio.visualstudio.com/fastlane-e2e-tests/_build/results?buildId=335

Strangely it did not crash with the same command bundle exec fastlane supply init --package_name=tools.fastlane.androidapp --json-key="D:\a\_temp\google_play_key.json" as before, but bundle exec fastlane screengrab init this time.

It is also not reproducible 100% (https://janpio.visualstudio.com/fastlane-e2e-tests/_build/results?buildId=336 e.g. only failed because my code is broken...), but that is expected after the reports from @marcelofabri. Lucky that it broke on the first try, otherwise I would have moved on.

I created a new branch based on the 0aca5b2d2cd47ea4900f8df661816a8b600a3481 commit, but couldn't reproduce the issue using that branch _at all_ - and I tried a lot of time.

So... yeah that didn't help at all 🤷‍♀️

janpio on 10 Dec 2018

I saw discussion around FASTLANE_SKIP_UPDATE_CHECK. In our setup, fastlane is also updated before the lane is executed.

Should we try disabling it also on our side?

dloic on 10 Dec 2018

The update is a parallel thread thing, so just wanted to exclude side effects from this - but as @marcelofabri could still replicate it with this env var being set, this doesn't seem to be the culprit. If you have the time, please still try it and report to us - maybe it works for you and thus gives us a hint what is going on.

janpio on 10 Dec 2018

I was getting the same error and it appears to be gone after following the tip here and downgrading to ruby 2.4.3 using rvm. Perhaps this helps narrow things down a bit.

I will update this post if the issue returns.

hvaughan3 on 10 Dec 2018

@janpio I wasn't able to replicate it with the env var being set (but I haven't tried since Friday)

marcelofabri on 10 Dec 2018

Oh sorry, this issue is getting complicated to keep in my head ;) So yes @dloic, then this is definitely worth testing.

janpio on 10 Dec 2018

@janpio I'm looking at that old build (335), and I also cloned that same repo locally and ran the bundle exec fastlane all command and got the same error (with a join and a TypeError)

Looking at the full debug output in that build and my build locally it doesn't like that fastlane_folder.rb is the cause, it's another shell script that's causing the error. I wonder if the root cause of the original issue might be an unhandled shell command/exception? Or an over-rescued exception in one of the shell runner commands?

Do you have any more examples (full output) we can look at? The other example in this thread looks like it's truncated to just show the puts statements.

bundler: failed to load command: fastlane (/Users/jay/.asdf/installs/ruby/2.5.3/lib/ruby/gems/2.5.0/bin/fastlane)
TypeError: no implicit conversion of nil into String
  /Users/jay/.asdf/installs/ruby/2.5.3/lib/ruby/gems/2.5.0/gems/fastlane-2.108.0/fastlane_core/lib/fastlane_core/fastlane_folder.rb:50:in `join'

jaydorsey on 11 Dec 2018

❤1

My setup on Azure Pipelines (the CI build 335 etc) and that repo is "special", in that I start a normal fastlane run and use the sh action to start another fastlane run inside it. So Pipelines executes the bundle exec fastlane all command, which itself then starts another bundle exec fastlane screengrab init (in the case of 335) which then crashes. That is why the Stacktrace there looks different and "convoluted".

The "real" stacktrace of the "inner" fastlane is output as soon as it crashes. That crash is then bubbled up to the "outer" fastlane, which doesn't handle it and adds its own stacktrace for that.

For the other people reporting here in this issue, they do no such funny stuff but just execute a command manually or on CI and encounter the crash with the output and stacktrace they posted.

It might be a coincidence that my matrjoschka style setup triggers a similar problem - or might just be another way to trigger the exact same problem.

That being said:
https://janpio.visualstudio.com/fastlane-e2e-tests/_build/results?buildId=117 and https://janpio.visualstudio.com/fastlane-e2e-tests/_build/results?buildId=256 (Linux) are two other full outputs of my setup.

But https://github.com/fastlane/fastlane/issues/12789#issue-335105535 https://github.com/fastlane/fastlane/issues/12789#issuecomment-430482522 https://github.com/fastlane/fastlane/issues/12789#issuecomment-443172562 https://github.com/fastlane/fastlane/issues/12789#issuecomment-445028889 https://github.com/fastlane/fastlane/issues/12789#issuecomment-445049874 are also full outputs (some with the pure release of fastlane, some with different stages of our puts debugging included) - they are just very short as fastlane crashes very early in the "startup" and isn't wrapper in another fastlane run.

janpio on 11 Dec 2018

👍1

@janpio Will do a test with FASTLANE_SKIP_UPDATE_CHECK and let you know.

dloic on 11 Dec 2018

❤1

Hi!

I did wait a bit to provide results but since we set FASTLANE_SKIP_UPDATE_CHECK=1, we've not seen the issue :/. Before we had it everyday and now nothing since three days.

dloic on 14 Dec 2018

👍1

Okay, that means both @dloic and @marcelofabri report that setting FASTLANE_SKIP_UPDATE_CHECK=1 fixes the issue. I will try to understand all code affected by that better and see if any of that makes sense. Thanks for the feedback!

janpio on 14 Dec 2018

So this is in the path that gets executed:
https://github.com/fastlane/fastlane/blob/f4849a575c07b7931f2832d2214c95751996e151/fastlane/lib/fastlane/cli_tools_distributor.rb#L63-L64

This "spawns" 2 threads running a method each:
https://github.com/fastlane/fastlane/blob/f4849a575c07b7931f2832d2214c95751996e151/fastlane_core/lib/fastlane_core/update_checker/update_checker.rb#L12-L33

The first method is here:
https://github.com/fastlane/fastlane/blob/f4849a575c07b7931f2832d2214c95751996e151/fastlane_core/lib/fastlane_core/update_checker/update_checker.rb#L125-L134

And the second here:
https://github.com/fastlane/fastlane/blob/f4849a575c07b7931f2832d2214c95751996e151/fastlane_core/lib/fastlane_core/update_checker/update_checker.rb#L117-L123

The second one looks totally harmless, it just requests the rubygems info, extracts the version and saves it for later use. The first one has its own env var to disable it: FASTLANE_SKIP_UPDATE_CHECK.

Can you try to _not_ set FASTLANE_SKIP_UPDATE_CHECK but FASTLANE_OPT_OUT_USAGE and see if you can reproduce the error? (Best also as a baseline try to reproduce with _none_ of the env vars set again to make sure you can still reproduce it at all) @dloic @marcelofabri

janpio on 14 Dec 2018

I looked into the code being executed in those threads a bit. There is indeed some code that does Dir.chdir(File.expand_path('..', path)) do in there. Could something like this be the culprit @jaydorsey for what we are seeing?

janpio on 14 Dec 2018

For documentation:

https://github.com/fastlane/fastlane/blob/f4849a575c07b7931f2832d2214c95751996e151/fastlane_core/lib/fastlane_core/update_checker/update_checker.rb#L125-L134
https://github.com/fastlane/fastlane/blob/3e643b4678936c0aacf8748a366062b9c9b40567/fastlane_core/lib/fastlane_core/analytics/app_identifier_guesser.rb#L13-L27
if you follow the methods called in here you finally end up at:
https://github.com/fastlane/fastlane/blob/3e643b4678936c0aacf8748a366062b9c9b40567/credentials_manager/lib/credentials_manager/appfile_config.rb#L25-L54
This is method that tries to get the Android or iOS package name from the Appfile.

janpio on 14 Dec 2018

Question to everyone here who had this problem: Can you confirm that you have an Appfile in your project?

Update: I could recreate it locally when creating an Appfile! Seems we are on track...

janpio on 14 Dec 2018

Yes we do have Appfile

dloic on 14 Dec 2018

👍1

Could it be related to a race condition with Dir.chdir(File.expand_path('..', path)) line 34 in appfile_config.rb?

dloic on 14 Dec 2018

Yep, that's exactly what I am thinking right now. Dir.chdir is not thread safe, so this could theoretically mess with the working directory in unexpected ways. I am currently trying to find a better repro now that I have that information, but can't :/

janpio on 14 Dec 2018

I also have an app file (getting the error every start)

chnbr on 14 Dec 2018

I don't have an Appfile but I still get the error as described in #13512.

✅ fastlane environment ✅

Stack

| Key | Value |
| --------------------------- | -------------------------------------------------------------------- |
| OS | 10.14.1 |
| Ruby | 2.5.1 |
| Bundler? | true |
| Git | git version 2.17.1 |
| Installation Source | ~/Development/iOS/willhabeniOS/vendor/bundle/ruby/2.5.0/bin/fastlane |
| Host | Mac OS X 10.14.1 (18B75) |
| Ruby Lib Dir | /usr/local/Cellar/ruby/2.5.1/lib |
| OpenSSL Version | OpenSSL 1.0.2o 27 Mar 2018 |
| Is contained | false |
| Is homebrew | false |
| Is installed via Fabric.app | false |
| Xcode Path | /Applications/Xcode.app/Contents/Developer/ |
| Xcode Version | 10.1 |

System Locale

| Variable | Value | |
| -------- | ----------- | - |
| LANG | en_US.UTF-8 | ✅ |
| LC_ALL | en_US.UTF-8 | ✅ |
| LANGUAGE | | |

fastlane files:

No Fastfile found

No Appfile found

fastlane gems

| Gem | Version | Update-Status |
| -------- | ------- | ------------- |
| fastlane | 2.111.0 | ✅ Up-To-Date |

Loaded fastlane plugins:

No plugins Loaded

Loaded gems

| Gem | Version |
| ------------------------- | ------------ |
| did_you_mean | 1.2.0 |
| bundler | 1.16.3 |
| CFPropertyList | 2.3.6 |
| concurrent-ruby | 1.1.3 |
| i18n | 1.2.0 |
| minitest | 5.11.3 |
| thread_safe | 0.3.6 |
| tzinfo | 1.2.5 |
| activesupport | 5.2.2 |
| public_suffix | 2.0.5 |
| addressable | 2.5.2 |
| atomos | 0.1.3 |
| babosa | 1.0.2 |
| claide | 1.0.2 |
| clamp | 0.6.5 |
| colored | 1.2 |
| colored2 | 3.1.2 |
| highline | 1.7.10 |
| commander-fastlane | 4.4.6 |
| declarative | 0.0.10 |
| declarative-option | 0.1.0 |
| digest-crc | 0.4.1 |
| unf_ext | 0.0.7.5 |
| unf | 0.1.4 |
| domain_name | 0.5.20180417 |
| dotenv | 2.5.0 |
| emoji_regex | 0.1.1 |
| excon | 0.62.0 |
| multipart-post | 2.0.0 |
| faraday | 0.15.4 |
| http-cookie | 1.0.3 |
| faraday-cookie_jar | 0.0.6 |
| faraday_middleware | 0.12.2 |
| fastimage | 2.1.5 |
| gh_inspector | 1.1.3 |
| jwt | 2.1.0 |
| memoist | 0.16.0 |
| multi_json | 1.13.1 |
| os | 1.0.0 |
| signet | 0.11.0 |
| googleauth | 0.6.7 |
| httpclient | 2.8.3 |
| mime-types-data | 3.2018.0812 |
| mime-types | 3.2.2 |
| uber | 0.1.0 |
| representable | 3.0.4 |
| retriable | 3.1.2 |
| google-api-client | 0.23.9 |
| google-cloud-env | 1.0.5 |
| google-cloud-core | 1.2.7 |
| google-cloud-storage | 1.15.0 |
| json | 2.1.0 |
| mini_magick | 4.5.1 |
| multi_xml | 0.6.0 |
| plist | 3.4.0 |
| rubyzip | 1.2.2 |
| security | 0.1.3 |
| naturally | 2.2.0 |
| simctl | 1.6.5 |
| slack-notifier | 2.3.2 |
| terminal-notifier | 1.8.0 |
| unicode-display_width | 1.4.0 |
| terminal-table | 1.8.0 |
| tty-screen | 0.6.5 |
| tty-cursor | 0.6.0 |
| tty-spinner | 0.9.0 |
| word_wrap | 1.0.0 |
| nanaimo | 0.2.6 |
| xcodeproj | 1.7.0 |
| rouge | 2.0.7 |
| xcpretty | 0.3.0 |
| xcpretty-travis-formatter | 1.0.0 |
| mini_portile2 | 2.3.0 |
| nokogiri | 1.8.5 |
| slather | 2.4.6 |

generated on: 2018-12-15

mathaeus on 15 Dec 2018

Thanks @chnbr.
Thanks @mathaeus, that fits as well: Your fastlane crashes a lot later, where there can already be a lot of other code being executed that spawns threads that can interfere by changing the working directory. If my current understanding turns out to be right regarding the "crashes directly after startup" situation, we will have to investigate #13512 under these aspects again. I am pretty sure there is something else going on.

janpio on 15 Dec 2018

Thanks @janpio for the quick answer. The consequence in the case of #13512 is also not _that_ tragic imo since one can continue with deliver, you just don't see the HTML preview. Which I can live with for the moment personally, but yeah, sooner or later it would be nice of course to see that fixed :)

Let us know if you need more info.

mathaeus on 15 Dec 2018

@mathaeus I am not sharing your considering #13512 "not that tragic". I currently have a new submission in 18 languages, completely fresh metadata and completely new screenshots. In such a case the html preview is "essential" to review if everything in every language is alright. Sure, you could do the review once it is uploaded in appstoreconnect, but this is very inconvenient, although it does work of course.

chnbr on 15 Dec 2018

Good news: I think I got it: #13918

janpio on 15 Dec 2018

👍2

@janpio: Great to hear, thanks for the fix!

I'd just like to confirm one thing: approximately what version of fastlane started using the analytics code with the offending call to Dir.chdir on a separate thread?

Going back through our fastlane version timeline, we basically saw this:

2.41.0: first fastlane version we began using (no error)
2.64.1: last fastlane version that didn't exhibit this issue
2.85.0: first fastlane version that did exhibit this issue

So if the analytics were added somewhere between 2.64.1 and 2.85.0, it's fairly safe to say that's indeed the problem.

falken42 on 15 Dec 2018

That code was actually in place since May 2017, which makes this ~2.30.0+. But as it is a race condition, it is basically a timing thing. Any change in the startup process could move the two pieces that clash closer together or further apart. I also think, that the computer/processor being used influences the probability of the race condition being triggered.

janpio on 15 Dec 2018

Yeah, that definitely sounds reasonable. Thanks for checking!

falken42 on 16 Dec 2018

@janpio Thanks for investigating this so deeply! 💯

marcelofabri on 17 Dec 2018

I can confirm we're still able to reproduce without setting FASTLANE_SKIP_UPDATE_CHECK. Will now try with FASTLANE_OPT_OUT_USAGE=1 and let you know.

dloic on 19 Dec 2018

Hi!

A quick update, I can confirm, setting FASTLANE_OPT_OUT_USAGE=1, we cannot reproduce the error ;)

dloic on 7 Jan 2019

@janpio With https://github.com/fastlane/fastlane/pull/13918 out, does it means the issue has been completely resolved or is there any other part of the code with problematic chdir?

dloic on 7 Jan 2019

No, that was all that I could identify.
Please don't tell me you still have the problem?

janpio on 9 Jan 2019

Ahah, I've not tried yet with the latest version as I was not able to update the CI on our side. I was just wondering if you fixed all paths you discovered. If so, we'll update soon and let you know, so we can hopefully close this issue :)

dloic on 10 Jan 2019

❤1

Please make sure to update to the latest fastlane version and check if that solves the issue. Let us know if that works for you by adding a comment :+1:

fastlane-bot on 9 Feb 2019

This issue will be auto-closed because there hasn't been any activity for a few months. Feel free to open a new one if you still experience this problem :+1:

fastlane-bot on 17 Feb 2019

Was this page helpful?

0 / 5 - 0 ratings