watchman-make not working on python3

[schrockn]

Hello! According to your docs pywatchman now supports python3. However I'm running into a python 2/3 compat issue.

$ watchman-make -p '**/*.py' -t unittest
Traceback (most recent call last):
  File "/Users/schrockn/code/venvs/clarify/bin/watchman-make", line 168, in <module>
    args = parser.parse_args()
  File "/usr/local/Cellar/python/3.6.4_3/Frameworks/Python.framework/Versions/3.6/lib/python3.6/argparse.py", line 1730, in parse_args
    args, argv = self.parse_known_args(args, namespace)
  File "/usr/local/Cellar/python/3.6.4_3/Frameworks/Python.framework/Versions/3.6/lib/python3.6/argparse.py", line 1762, in parse_known_args
    namespace, args = self._parse_known_args(args, namespace)
  File "/usr/local/Cellar/python/3.6.4_3/Frameworks/Python.framework/Versions/3.6/lib/python3.6/argparse.py", line 1968, in _parse_known_args
    start_index = consume_optional(start_index)
  File "/usr/local/Cellar/python/3.6.4_3/Frameworks/Python.framework/Versions/3.6/lib/python3.6/argparse.py", line 1908, in consume_optional
    take_action(action, args, option_string)
  File "/usr/local/Cellar/python/3.6.4_3/Frameworks/Python.framework/Versions/3.6/lib/python3.6/argparse.py", line 1836, in take_action
    action(self, namespace, argument_values, option_string)
  File "/Users/schrockn/code/venvs/clarify/bin/watchman-make", line 103, in __call__
    if isinstance(values, basestring):
NameError: name 'basestring' is not defined

Watchman version 4.9.0

$ watchman --version
4.9.0

Python version 3.6.4

$ python3 --version
Python 3.6.4

pywatchman 1.4.1 (from pip list)

pywatchman          1.4.1

Thanks again for open sourcing this awesome project.

[ltankey]

basestring is used on L103 of the watchman-make file. Needs to be updated to use str for Python3.6

[wez]

Ah, the problem there is that watchman-make isn't py3 compatible, but the underlying library is.

This is the diff I have queued up internally:

--- a/fbcode/watchman/python/bin/watchman-make  Tue Aug 14 13:13:38 2018 -0700
+++ b/fbcode/watchman/python/bin/watchman-make  Tue Aug 14 19:32:23 2018 -0700
@@ -9,6 +9,12 @@
 import pywatchman


+if pywatchman.compat.PYTHON3:
+    STRING_TYPES = (str, bytes)
+else:
+    STRING_TYPES = (str, unicode)  # noqa: F821
+
+
 def patterns_to_terms(pats):
     # convert a list of globs into the equivalent watchman expression term
     if pats is None or len(pats) == 0:
@@ -104,7 +110,7 @@
             targets = []
             setattr(namespace, self.dest, targets)

-        if isinstance(values, basestring):
+        if isinstance(values, STRING_TYPES):
             values = [values]

         if namespace.pattern is None or len(namespace.pattern) == 0:

[schrockn]

Thanks for looking into this! Also, Hi Wez!

[wez]

Hey Nick :) It's always fun to see folks using our opensource stuff in their lives after FB :)

[schrockn]

@wez when will this be available in a public form?

[ambis]

Would like to see a release of this.

Homebrew version 4.9.0 works ok, but if you have a fresh install you'll get the latest one which is broken (4.9.0_2). And homebrew doesn't allow installing older versions, only switching to an older already locally available version. So you are screwed if you don't have the older 4.9.0...

[elzii]

if you're being roadblocked by this, open line 103 of /usr/local/Cellar/watchman/4.9.0_2/libexec/bin/watchman-make and change

from
if isinstance(values, basestring):

to
if isinstance(values, str):

hold over until patch goes through

[maxlapides]

It looks like a fix was merged into this project over a year ago, but when I run brew install watchman today, it seems to still have the issue. Has this patch not shipped to Homebrew yet? Or am I doing something wrong?

EDIT: for the next lost soul, pip install pywatchman fixed this issue for me.

[lazdmx]

I suppose the issue must be reopened since problem is still with us (at least watchman-make and at least under macOS). Thumbs Ups and hearts obviously failed to make any progress

[jfredett]

Also seeing same issue as @lazdmx on Arch, so likely not isolated to OSX. I see on line 103 of the installed script the only reference to basestring is in the 'DefineTarget' class' __call method. Hope that helps the folks bughunting.

[rsms]

watchman on Homebrew (4.9.0_3) is still unpatched. In class DefineTarget there is a isinstance(values, basestring) which has clearly been fixed source but perhaps whoever maintains the Homebrew package needs to create a new release.

Github fails to show history or blame on https://github.com/Homebrew/homebrew-core/blob/master/Formula/watchman.rb so I’m not sure who to poke, but I’m guessing that the Homebrew maintainer is also active in this repo.

[ambis]

As @maxlapides mentioned pip install pywatchman works like a charm, if you happen to have python2 version of pip installed.

With pip3 install pywatchman you are going to hit the NameError: name 'basestring' is not defined error.

The latest MacOS does not ship with Python2, and homebrew has deleted the python@2 (sigh.. like all py2 code would've just magically disappeared) formula, there is no easy way to get py2 on a new MacOS install.

So back to square A.

[thoward27]

I had to modify /usr/local/Cellar/watchman/4.9.0_4/libexec/bin/watchman-make for this to work on MacOS. basestring -> str.

[drewsymo]

Noooo :(

Still failing on latest brew install

[jfrolich]

would be great to have this fixed.

[vickylance]

This issue is closed by accident it should be reopened. As its still present.