Swift: Add support for Python 3

[dan-zheng]

Currently, Swift is hard-coded to use Python 2.7.
Adding Python 3 support is non-trivial because the Python.swift standard library file calls Python C API functions specific to Python 2 (such as PyString_AsString).

There are multiple possible solutions (e.g. separate Python2/Python3 modules), but the best design isn't clear. Separate modules (e.g. import Python2 and import Python3) is unideal because that implies not having a unified Python interface, with separate PythonObject2 and PythonObject3 structs for instance. On the other hand, the Python version becomes explicit and clear in the code.

An interesting solution is to offer a single Python module and to make Python version configurable via runtime flag, e.g. swift --use-python3.

This is great because Python version would be dynamic: rather than building the compiler explicitly with support for Python 2/3, users can specify the version when invoking swift. This also resembles the way Python scripts are run: for a single main.py file, users can run either python2 main.py or python3 main.py.

However, the implementation is less straightforward and runtime calls to dlopen may be required.

[roman-kh]

The other important thing to consider is to support Python virtual environments which are widely used throughout development and production.

[johndpope]

@roman-kh - I threw some rough notes together on Xcode to configure to a specific python conda environment getting https://github.com/johndpope/SwiftReinforce

[eldar]

Hi there. I was wondering, given that Python 2 will cease to exist in 1 year and it is generally recommended to stop its support in current projects, why is it even a question? Why would a new project like this one even aim at a dual support?

[dan-zheng]

Update: I'm a fan of the the dlopen/dlsym approach for unified Python 2/3 support presented by @pvieito in this mailing list thread and implemented in PythonKit.

I think it's the right approach for configurable Python support. It works with any Python installation on any platform. Python version and library path can be specified via environment variables.

With @pvieito's permission, I support merging PythonKit into the Swift Python module. Some implementation details need to be tweaked (e.g. remove dependency on Foundation for file I/O and reading environment variables). The PythonLibrary.getSymbol<T>(name: String, signature: T.Type) -> T idea is nice and general.

What do others think?

---

EDIT @eldar: PythonKit's approach to Python support is general and can support multiple Python versions. With it, we can support Python 3 and choose whether to support Python 2. Since macOS still ships with Python 2 AFAIK, I think continuing support for both versions is prudent.

[rxwei]

See announcement: https://groups.google.com/a/tensorflow.org/forum/#!topic/swift/Rzpgfr0XUcw

Thank you again, @pvieito and @dan-zheng!

[KhushalDugar]

If Swift supports Python 3.x now please update the readme because it is misleading.

[dan-zheng]

If Swift supports Python 3.x now please update the readme because it is misleading.

What section requires updating exactly?

[anandijain]

https://github.com/apple/swift/tree/tensorflow#system-requirements
Under system requirements, there's the following:
Please make sure you use Python 2.x. Python 3.x is not supported currently.

[iamWing]

https://github.com/apple/swift/tree/tensorflow#system-requirements
Under system requirements, there's the following:
Please make sure you use Python 2.x. Python 3.x is not supported currently.

As is for today the document still states that Python 3.x is not supported currently. I'm not entirely sure if we can remove this line or not, but I'm using Python 3.6 on it and it seems working just fine.