Turing.jl: Unable to load Turing.jl

[chameroy]

Hi,
I'm unable to load Turing.jl. Here is the error I receive when attempting to load Turing.jl into a Jupyter notebook (using 0.62 on Windows 10):

LoadError: LoadError: LoadError: invalid subtyping in definition of HMC
while loading C:\Users#####.julia\v0.6\Turing\src\samplers\hmc.jl, in expression starting on line 521
while loading C:\Users#####.julia\v0.6\Turing\src\samplers/sampler.jl, in expression starting on line 29
while loading C:\Users#####.julia\v0.6\Turing\src\Turing.jl, in expression starting on line 72

Stacktrace:
[1] include_from_node1(::String) at .\loading.jl:576
[2] include(::String) at .\sysimg.jl:14
[3] include_from_node1(::String) at .\loading.jl:576
[4] include(::String) at .\sysimg.jl:14
[5] include_from_node1(::String) at .\loading.jl:576
[6] eval(::Module, ::Any) at .\boot.jl:235
[7] _require(::Symbol) at .\loading.jl:490
[8] require(::Symbol) at .\loading.jl:405
[9] include_string(::String, ::String) at .\loading.jl:522

[xukai92]

Hi, are you in the master branch of Turing.jl?

From while loading C:\Users#####.julia\v0.6\Turing\src\samplers\hmc.jl, in expression starting on line 521 it seems not?

[chameroy]

I should be (I think). Whatever is installed when I use Pkg.add().

[xukai92]

Can you try Pkg.checkout("Turing", "master")

[revendrat]

Am able to load through the above command, but getting an error while running the following example given in the introduction. Any suggestions on what to do?

using Turing, Distributions
using Gadfly

@model gdemo(x) = begin
  s ~ InverseGamma(2,3)
  m ~ Normal(0,sqrt(s))
  x[1] ~ Normal(m, sqrt(s))
  x[2] ~ Normal(m, sqrt(s))
  return s, m
end

c = sample(gdemo([1.5, 2]), PG(50,300));
describe(c)

Error message below:

error compiling resample!: error compiling fork: error compiling copy: could not load library "libtask"
dlopen(libtask.dylib, 1): image not found
step(::Function, ::Turing.Sampler{Turing.PG}, ::Turing.VarInfo) at pgibbs.jl:72
macro expansion at util.jl:293 [inlined]
#sample#99(::Bool, ::Void, ::Int64, ::Function, ::Function, ::Turing.PG) at pgibbs.jl:112
sample(::Function, ::Turing.PG) at pgibbs.jl:87
include_string(::String, ::String) at loading.jl:522
include_string(::String, ::String, ::Int64) at eval.jl:30
include_string(::Module, ::String, ::String, ::Int64, ::Vararg{Int64,N} where N) at eval.jl:34
(::Atom.##49#53{String,Int64,String})() at eval.jl:50
withpath(::Atom.##49#53{String,Int64,String}, ::Void) at utils.jl:30
withpath(::Function, ::String) at eval.jl:38
macro expansion at eval.jl:49 [inlined]
(::Atom.##48#52{Dict{String,Any}})() at task.jl:80

[xukai92]

Please do

Pkg.build("Turing")

[revendrat]

Thank you! When I gave this build command, got the following error. Does this I have to install xcode & homebrew?

Pkg.build("Turing")

INFO: Building Homebrew
Error: The following formula:
  git
cannot be installed as a binary package and must be built from source.
Install the Command Line Tools:
  xcode-select --install

Error: Git must be installed and in your PATH!
==============================[ ERROR: Homebrew ]===============================

LoadError: failed process: Process(`/Applications/JuliaPro-0.6.1.1.app/Contents/Resources/pkgs-0.6.1.1/v0.6/Homebrew/deps/usr/bin/brew update --force`, ProcessExited(1)) [1]
while loading /Applications/JuliaPro-0.6.1.1.app/Contents/Resources/pkgs-0.6.1.1/v0.6/Homebrew/deps/build.jl, in expression starting on line 2

================================================================================
INFO: Building Stan
INFO: Building SpecialFunctions
INFO: Building Rmath
INFO: Building Cairo
INFO: Building Turing
xcrun: error: invalid active developer path (/Library/Developer/CommandLineTools), missing xcrun at: /Library/Developer/CommandLineTools/usr/bin/xcrun
===============================[ ERROR: Turing ]================================

LoadError: failed process: Process(`make`, ProcessExited(1)) [1]
while loading /Applications/JuliaPro-0.6.1.1.app/Contents/Resources/pkgs-0.6.1.1/v0.6/Turing/deps/build.jl, in expression starting on line 1

================================================================================

================================[ BUILD ERRORS ]================================

WARNING: Turing and Homebrew had build errors.

 - packages with build errors remain installed in /Applications/JuliaPro-0.6.1.1.app/Contents/Resources/pkgs-0.6.1.1/v0.6
 - build the package(s) and all dependencies with `Pkg.build("Turing", "Homebrew")`
 - build a single package by running its `deps/build.jl` script

================================================================================

[xukai92]

It seems that you haven't installed the Command Line Tools from Xcode? I'm not sure about the error but please try install it.

[revendrat]

Installed XCode, and is working fine now. Thank you!

[xukai92]

Great! You're welcome

[chameroy]

I reinstalled using Pkg.checkout("Turing", "master"). I can now get it to load. However, more problems arise. First, because Gadfly is a requirement, some of my packages are downgraded: DataFrames, CSV, among some others, which breaks some of the code I have written using the most recent versions of those packages. Second, I tried running the simple Gaussian model example presented in the introduction in both a notebook and in command line. In both cases, when I try to sample,

c = sample(gdemo([1.5, 2]), PG(50, 300)), Julia crashes.

[chameroy]

Here is the command line output:

Please submit a bug report with steps to reproduce this fault, and any error messages that follow (in their entirety). Thanks.
Exception: EXCEPTION_ACCESS_VIOLATION at 0x6b5f5340 -- jl_eh_restore_state at /home/Administrator/buildbot/worker/package_win64/build/src/home/Administrator/buildbot/worker/package_win64/build/src\julia.h:1549 [inlined]
jl_pop_handler at /home/Administrator/buildbot/worker/package_win64/build/src/home/Administrator/buildbot/worker/package_win64/build/src\rtutils.c:236
while loading no file, in expression starting on line 0
jl_eh_restore_state at /home/Administrator/buildbot/worker/package_win64/build/src/home/Administrator/buildbot/worker/package_win64/build/src\julia.h:1543 [inlined]
jl_pop_handler at /home/Administrator/buildbot/worker/package_win64/build/src/home/Administrator/buildbot/worker/package_win64/build/src\rtutils.c:236
try_yieldto at .\event.jl:180
schedule_and_wait at .\event.jl:134
unknown function (ip: 00000000030E72FE)
jl_call_fptr_internal at /home/Administrator/buildbot/worker/package_win64/build/src/home/Administrator/buildbot/worker/package_win64/build/src\julia_internal.h:339 [inlined]
jl_call_method_internal at /home/Administrator/buildbot/worker/package_win64/build/src/home/Administrator/buildbot/worker/package_win64/build/src\julia_internal.h:358 [inlined]
jl_apply_generic at /home/Administrator/buildbot/worker/package_win64/build/src/home/Administrator/buildbot/worker/package_win64/build/src\gf.c:1926
produce at C:\Users\Chameroy.julia\v0.6\Turing\src\trace\taskcopy.jl:62
unknown function (ip: 00000000030E701A)
jl_call_fptr_internal at /home/Administrator/buildbot/worker/package_win64/build/src/home/Administrator/buildbot/worker/package_win64/build/src\julia_internal.h:339 [inlined]
jl_call_method_internal at /home/Administrator/buildbot/worker/package_win64/build/src/home/Administrator/buildbot/worker/package_win64/build/src\julia_internal.h:358 [inlined]
jl_apply_generic at /home/Administrator/buildbot/worker/package_win64/build/src/home/Administrator/buildbot/worker/package_win64/build/src\gf.c:1926
macro expansion at C:\Users\Chameroy.julia\v0.6\Turing\src\core\compiler.jl:63 [inlined]

gdemo_model#714 at .\REPL[4]:4

3 at C:\Users\Chameroy.julia\v0.6\Turing\src\trace\trace.jl:54

jl_call_fptr_internal at /home/Administrator/buildbot/worker/package_win64/build/src/home/Administrator/buildbot/worker/package_win64/build/src\julia_internal.h:339 [inlined]
jl_call_method_internal at /home/Administrator/buildbot/worker/package_win64/build/src/home/Administrator/buildbot/worker/package_win64/build/src\julia_internal.h:358 [inlined]
jl_apply_generic at /home/Administrator/buildbot/worker/package_win64/build/src/home/Administrator/buildbot/worker/package_win64/build/src\gf.c:1926
jl_apply at /home/Administrator/buildbot/worker/package_win64/build/src/home/Administrator/buildbot/worker/package_win64/build/src\julia.h:1424 [inlined]
start_task at /home/Administrator/buildbot/worker/package_win64/build/src/home/Administrator/buildbot/worker/package_win64/build/src\task.c:267
Allocations: 38529339 (Pool: 38518555; Big: 10784); GC: 92

[xukai92]

Did you run Pkg.build("Turing")

[chameroy]

No, after having done that, it appears to be working. Is it possible to use Turing with Plots.jl instead of Gadfly? Installing Gadfly downgrades some packages. If not, is it planned?

[xukai92]

The reason Turing.jl has some dependencies on specific packages is because Turing.jl depends on Mamba.jl which depends on some packages including Gadfly.jl.

You can certainly use Plots.jl with Turing.jl. I personally did this and in some of the notebook examples we have we also use Plots.jl. What specific problem do you meet?

[chameroy]

The problem is not with Turing.jl, but with Gadfly.jl. Currently, it appears that installing Gadfly.jl causes some packages to downgrade, notably DataFrames.jl and CSV.jl, which breaks code that I'm working on. But as you said, Turing.jl depends on Mamba.jl which in turn depends on Gadfly.jl, so my problem really has nothing to do with Turing.jl per say. Turing.jl appears to be working now.

Which Turing.jl examples are using Plots.jl?

Would you happen to know the status of Gadfly.jl? Is it still under development, or will it eventually be replaced by Plots.jl?

Thanks very much for the help.

[xukai92]

For examples this one uses PyPlot.jl directly https://github.com/yebai/Turing.jl/blob/master/example-models/notebooks/BayesHmm.ipynb and this one uses Plots.jl https://github.com/xukai92/TuringDemo/blob/master/Differential%20Equation/turing-de.jl.ipynb

I'm not sure the status of Gadfly.jl. Maybe @ChrisRackauckas knows.

[ChrisRackauckas]

Gadfly's development has slowed ever since the main developer left. It has two contributors who work a little bit on it, totaling 100 commits over the last year, so it seems like it's mostly in maintenance mode. I don't think it will be going away any time soon, but it's definitely not the main Julia plotting library anymore and I would only recommend it if you're really wedded to Grammar of Graphics APIs (even though, I hope that Plots.jl or eventually Makie.jl gets a GoG API so we can dump Gadfly).

[chameroy]

Thanks for the much appreciated insight Chris.

Is it possible use Plots.jl in place of Gadfly.jl in the examples in which Gadfly.jl is currently used? For example, I've tried using Plots.jl in introduction.ipynb but I get the following error:

No user recipe defined for Turing.Chain

Is there currently a workaround for this or will this be possible with future changes to Turing.jl?

[xukai92]

You can always extract samples from Turing.Chain and do anything with them by yourself. E.g.

@model gdemo(x) = begin
  s ~ InverseGamma(2,3)
  m ~ Normal(0,sqrt(s))
  x[1] ~ Normal(m, sqrt(s))
  x[2] ~ Normal(m, sqrt(s))
  return s, m
end

c = sample(gdemo([1.5, 2]), PG(50,300));

s_samples = c[:s]
m_samples = c[:m]

s_samples and m_samples are just normal Julia arrays. You can use any plotting libraries to plot the density or something.

[chameroy]

Thanks for the help Kai! I'm going to close this out because it's gotten off of the original topic.