Swift: Build 𝛁Swift without TensorFlow

[rahulbhalley]

Based on @BradLarson's response here it looks like it must be possible to build Swift with _differentiation_ feature (excluding TensorFlow module). Currently every toolchain released by Google's S4TF team includes TensorFlow module in it whereas I want to build and use 𝛁Swift without it (just differentiation).

Can somebody please help me with that?

cc @dan-zheng @dabrahams @rxwei

[dan-zheng]

Sure, try using Swift development snapshot toolchains from https://swift.org/download! These are built from apple/swift master branch.

You can use differentiable programming features by importing the _Differentiation module: Differentiable protocol, @differentiable and @derivative attributes, and differential operators.

This is a bit different from Swift for TensorFlow toolchains (apple/swift tensorflow branch), which don't currently require the import but will someday.

[rahulbhalley]

Hey @dan-zheng, I downloaded Swift 5.2.3 RELEASE 2020-04-28 (a) toolchain from swift.org and following happened. ☹️

Um, I am able to import _Differentiation. I can define differentiable functions with @differentiable attribute.

But there are none of the followings (therefore, I cannot use them):

1. differentiable APIs (gradient(at:in:), valueWithGradient(at:in:), and their variants)
2. define custom derivatives (differentiating(wrt:)), and so on.

Also defining custom differentiable type in S4TF toolchain is simple as:

struct Some: Differentiable {
  var x, y: Float
}

But using the apple/swift master toolchain from swift.org I cannot do this and am forced to give type for typealias TangentVector = type. Weird.

It looks like the apple/swift master toolchains are not yet ready to be entitled as 𝛁Swift. If so, I request you to please reopen this issue.

Or am I missing something that is halting me from doing all differentiation with this toolchain?

[dan-zheng]

Hi Rahul,

Sorry for not clarifying! Please download a development snapshot toolchain, those are built ~nightly from apple/swift master branch and have latest differentiable programming changes.

Release toolchains are built less frequently, so they won't have latest differentiable programming changes. Swift 5.2.3 was released in March 2020 before differentiable programming features were fully upstreamed to master.

[dan-zheng]

I personally use swiftenv for managing toolchains on macOS (both from swift.org/download and S4TF) and recommend it! https://github.com/tensorflow/swift/issues/360

[rahulbhalley]

Hmm, that development master toolchain still have TensorFlow module.

Wait. It works as you suggested @dan-zheng and does not include TensorFlow module!! Thank you so much. This is exactly what I was looking for! 🎉🥳

Sorry, I mistakenly selected S4TF toolchain when I made my above false comment and was quick to reply.

[rahulbhalley]

There is no @differentiating(wrt:) attribute to define my own derivative for a function.

Any timeline on when will this (and other differentiation features) will land into apple/swift master?

cc @dan-zheng

[dan-zheng]

There is no @differentiating(wrt:) attribute to define my own derivative for a function.

@differentiating(foo, wrt: ...) was renamed to @derivative(of: foo, wrt: ...) a few months ago.

Here's the related section from the Differentiable Programming Manifesto.
We plan to start Differentiable Swift usage documentation soon that should be more accessible.