Aws-toolkit-jetbrains: Invalid SAM CLI executable configured 2

[enochdwelo]

When I try to set my SAM CLI executable I receive the following error: "Cannot Save Settings: Invalid SAM CLI executable configured"

Very similar to #577 but not quite.

Screenshots

Your Environment

• OS: MACOSX 10.14.6
• JetBrains' Product: PyCharm
• JetBrains' Product Version: 2019.2.1
• Toolkit Version: 1.5
• SAM CLI Version: version 0.19.0
• JVM/Python Version: Python 3.7.4

SAM CLI runs just fine when run directly from my console

[zhangzhx]

Thanks for reporting this issue. What is the result by running /usr/local/bin/sam --version in your terminal?

[enochdwelo]

$ /usr/local/bin/sam --version
SAM CLI, version 0.19.0
$

As I said, it runs just fine from my console

[zhangzhx]

Did you install aws-sam-cli, by pip or Homebrew? From the error message, it seems the dependencies are messed up.

[enochdwelo]

I installed it via Homebrew per the instructions here: https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-sam-cli-install-mac.html#serverless-sam-cli-install-mac-sam-cli

[shipmaster]

I ran into this exact issue using SAM CLI 0.22.0 installed using homebrew which is the recommended version in the doc. I am also using PyCharm 2019.2 on Mac OS 10.14.6

[kiiadi]

Does the SAM CLI run successfully from a terminal? Is it the same executable? eg. if you run a which sam is that the path as what's configured in the IDE?

[shipmaster]

It does run successfully from the terminal and its the same executable.
$ which sam
/usr/local/bin/sam
$ sam --version
SAM CLI, version 0.22.0

And the log from the IDE log points to the same path

java.lang.IllegalStateException: Traceback (most recent call last):
File "/usr/local/bin/sam", line 6, in
from pkg_resources import load_entry_point
File "/usr/local/lib/python3.7/site-packages/pkg_resources/__init__.py", line 3241, in
@_call_aside
File "/usr/local/lib/python3.7/site-packages/pkg_resources/__init__.py", line 3225, in _call_aside
f(args, *kwargs)
File "/usr/local/lib/python3.7/site-packages/pkg_resources/__init__.py", line 3254, in _initialize_master_working_set
working_set = WorkingSet._build_master()
File "/usr/local/lib/python3.7/site-packages/pkg_resources/__init__.py", line 583, in _build_master
ws.require(__requires__)
File "/usr/local/lib/python3.7/site-packages/pkg_resources/__init__.py", line 900, in require
needed = self.resolve(parse_requirements(requirements))
File "/usr/local/lib/python3.7/site-packages/pkg_resources/__init__.py", line 786, in resolve
raise DistributionNotFound(req, requirers)
pkg_resources.DistributionNotFound: The 'aws-sam-cli==0.22.0' distribution was not found and is required by the application
at software.aws.toolkits.jetbrains.services.lambda.sam.SamVersionCache.getFileInfo(SamVersionCache.kt:21)
at software.aws.toolkits.jetbrains.services.lambda.sam.SamVersionCache.getFileInfo(SamVersionCache.kt:11)
at software.aws.toolkits.jetbrains.utils.FileInfoCache$evaluate$4.run(FileInfoCache.kt:56)
at com.intellij.openapi.application.impl.ApplicationImpl$1.run(ApplicationImpl.java:294)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:834)

[flado]

Same issue on my MacOs with SAM installed from: https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-sam-cli-install-mac.html#serverless-sam-cli-install-mac-sam-cli

/usr/local/bin/sam
SAM CLI, version 0.31.0

IntelliJ IDEA 2019.2.4 (Ultimate Edition)
Build #IU-192.7142.36, built on October 29, 2019

[justinmk3]

• How are you starting the IDE (from a shell, or from a GUI icon)?
• Does your environment have $PYTHONPATH or $PYTHONHOME set? Try this from the IDE:
  
  
  
  1. open Terminal
  
  
  2. env | grep PYTHON
  
  
• Have you set the SAM CLI path in the AWS Toolkit preferences? If so, what was set there? Screenshot of dialog follows:

Analysis

All reports here mention /usr/local/lib/python3.7/site-packages/... But I would expect homebrew-installed sam to invoke the python it ships with, from the homebrew location similar to:

/usr/local/Cellar/aws-sam-cli/0.31.0/libexec/lib/python3.7/...

If I rename aws_sam_cli-0.31.0-py3.7.egg-info at that location, I can provoke a similar error:

$ cd /usr/local/Cellar/aws-sam-cli/0.31.0/libexec/lib/python3.7/site-packages
$ mv aws_sam_cli-0.31.0-py3.7.egg-info aws_sam_cli-0.31.0-py3.7.egg-info.bk
$ sam
Traceback (most recent call last):
  File "/usr/local/bin/sam", line 6, in <module>
    from pkg_resources import load_entry_point
  File "/usr/local/Cellar/aws-sam-cli/0.31.0/libexec/lib/python3.7/site-packages/pkg_resources/__init__.py", line 3251, in <module>
    @_call_aside
  File "/usr/local/Cellar/aws-sam-cli/0.31.0/libexec/lib/python3.7/site-packages/pkg_resources/__init__.py", line 3235, in _call_aside
    f(*args, **kwargs)
  File "/usr/local/Cellar/aws-sam-cli/0.31.0/libexec/lib/python3.7/site-packages/pkg_resources/__init__.py", line 3264, in _initialize_master_working_set
    working_set = WorkingSet._build_master()
  File "/usr/local/Cellar/aws-sam-cli/0.31.0/libexec/lib/python3.7/site-packages/pkg_resources/__init__.py", line 583, in _build_master
    ws.require(__requires__)
  File "/usr/local/Cellar/aws-sam-cli/0.31.0/libexec/lib/python3.7/site-packages/pkg_resources/__init__.py", line 900, in require
    needed = self.resolve(parse_requirements(requirements))
  File "/usr/local/Cellar/aws-sam-cli/0.31.0/libexec/lib/python3.7/site-packages/pkg_resources/__init__.py", line 786, in resolve
    raise DistributionNotFound(req, requirers)
pkg_resources.DistributionNotFound: The 'aws-sam-cli==0.31.0' distribution was not found and is required by the application

[flado]

it seems that /usr/local/bin/sam is a link to the _sam_ location :

0 lrwxr-xr-x 1 flado admin 36B 8 Nov 15:26 /usr/local/bin/sam -> ../Cellar/aws-sam-cli/0.31.0/bin/sam

By setting the full path as: /usr/local/Cellar/aws-sam-cli/0.31.0/bin/sam in the IDE / SAM CLI exe the issue is not present anymore. It might be that the plugin does not follow links?

[justinmk3]

By setting the full path as: /usr/local/Cellar/aws-sam-cli/0.31.0/bin/sam in the IDE / SAM CLI exe the issue is not present anymore

@flado good info, thanks! What happens if you set the path to /usr/local/bin/sam explicitly?

I am wondering if the auto-discovered sam decided by the IDE/toolkit is different than /usr/local/bin/sam.

[abrooksv]

it seems that /usr/local/bin/sam is a link to the _sam_ location :

0 lrwxr-xr-x 1 flado admin 36B 8 Nov 15:26 /usr/local/bin/sam -> ../Cellar/aws-sam-cli/0.31.0/bin/sam

By setting the full path as: /usr/local/Cellar/aws-sam-cli/0.31.0/bin/sam in the IDE / SAM CLI exe the issue is not present anymore. It might be that the plugin does not follow links?

Pretty sure the raw string eventually makes it down to Java's ProcessBuilder class.

[flado]

@justinmk3 The auto-discovery returns: /usr/local/bin/sam (if Auto-discovery was able to return: /usr/local/Cellar/aws-sam-cli/0.31.0/bin/sam, then would not be any issue)... also when i setup manually as: "/usr/local/bin/sam" I get :

[justinmk3]

Strange. I suppose it wouldn't hurt to resolve the symlink, but we might be hiding the root cause then...

[abrooksv]

If you go to Help->Debug Log Settings... and paste #com.intellij.execution.configurations.GeneralCommandLine, and try again, debug lines will show in the idea.log (Help->Show logs in finder...)

Search for GeneralCommandLine in the log

[flado]

@abrooksv the Stacktrace in IntelliJ:

2019-11-14 12:28:14,741 [13345457]   INFO - ces.lambda.sam.SamVersionCache - Evaluating /usr/local/bin/sam 
2019-11-14 12:28:14,742 [13345458]  DEBUG - figurations.GeneralCommandLine - Executing [/usr/local/bin/sam --info] 
2019-11-14 12:28:14,742 [13345458]  DEBUG - figurations.GeneralCommandLine - Building process with commands: [/usr/local/bin/sam, --info] 
2019-11-14 12:28:14,950 [13345666]   INFO - ces.lambda.sam.SamVersionCache - Failed to evaluate /usr/local/bin/sam 
java.lang.IllegalStateException: Traceback (most recent call last):
  File "/usr/local/bin/sam", line 6, in <module>
    from pkg_resources import load_entry_point
  File "/usr/local/lib/python3.7/site-packages/pkg_resources/__init__.py", line 3251, in <module>
    @_call_aside
  File "/usr/local/lib/python3.7/site-packages/pkg_resources/__init__.py", line 3235, in _call_aside
    f(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/pkg_resources/__init__.py", line 3264, in _initialize_master_working_set
    working_set = WorkingSet._build_master()
  File "/usr/local/lib/python3.7/site-packages/pkg_resources/__init__.py", line 583, in _build_master
    ws.require(__requires__)
  File "/usr/local/lib/python3.7/site-packages/pkg_resources/__init__.py", line 900, in require
    needed = self.resolve(parse_requirements(requirements))
  File "/usr/local/lib/python3.7/site-packages/pkg_resources/__init__.py", line 786, in resolve
    raise DistributionNotFound(req, requirers)
pkg_resources.DistributionNotFound: The 'aws-sam-cli==0.31.0' distribution was not found and is required by the application
    at software.aws.toolkits.jetbrains.services.lambda.sam.SamVersionCache.getFileInfo(SamVersionCache.kt:21)
    at software.aws.toolkits.jetbrains.services.lambda.sam.SamVersionCache.getFileInfo(SamVersionCache.kt:11)
    at software.aws.toolkits.jetbrains.utils.FileInfoCache.getValue(FileInfoCache.kt:24)
    at software.aws.toolkits.jetbrains.utils.FileInfoCache.getValue(FileInfoCache.kt:18)
    at software.aws.toolkits.jetbrains.utils.CachingAsyncEvaluator$evaluate$4.run(CachingAsyncEvaluator.kt:60)
    at com.intellij.openapi.application.impl.ApplicationImpl$1.run(ApplicationImpl.java:294)
    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at java.base/java.lang.Thread.run(Thread.java:834)

In cmd line:

$ /usr/local/bin/sam --info
{
  "version": "0.31.0"
}

[abrooksv]

Oh...it looks like the additional log lines were not added until 2019.3

[justinlarose]

I had aws-sam-cli already installed via Homebrew, but I installed the python package via pip install aws-sam-cliand this resolved the "DistributionNotFound" problem.

[justinmk3]

@justinlarose Thanks for noting that.

Oh...it looks like the additional log lines were not added until 2019.3

@flado are you still experiencing this issue? If so, can you update PyCharm to 2019.3.x (and also update AWS Toolkit), then get the debug log and post the result here?

[forgivegod]

Still happens in 2020.1.1

[justinmk3]

Recent version of the toolkit/Intellij should expose more useful logs for this now. If you're experiencing this issue, please try the instructions here and post the logs: https://github.com/aws/aws-toolkit-jetbrains/issues/1178#issuecomment-553669183

[forgivegod]

related issue for 2020.01.01 / 1.16-201 : https://github.com/aws/aws-toolkit-jetbrains/issues/1757

[Pauldic]

I had this issue on Mac following the documentation's guide but resolved it by setting $PYTHONPATH to my python path in ~/.bash_profile

SAM CLI, version 1.2.0
Pycharm 2020.2.1
Python 3.7.6