Julia: Slowdown in function if previous call results in UndefVarError

[non-Jedi]

If you call the below function before importing Iterators.flatten, subsequent calls to prime_factors1 are slower than they would otherwise be with more allocations:

julia> function prime_factors1(n::Integer)
           factors = Int[]
           # Sadly, using an iterator on a hot loop like this is a bit too slow.
           for i in flatten((2, 3:2:n))
               while n % i == 0
                   push!(factors, i)
                   n = div(n, i)
               end
               n == 1 && break
           end
           factors
       end
prime_factors1 (generic function with 1 method)

julia> prime_factors1(98623589)
ERROR: UndefVarError: flatten not defined
Stacktrace:
 [1] prime_factors1(::Int64) at ./REPL[1]:4
 [2] top-level scope at REPL[2]:1

julia> using .Iterators: flatten

julia> @btime prime_factors1($98623589);
  34.837 ms (822935 allocations: 28.30 MiB)

In a fresh session without the call to prime_factors1 before importing Iterators.flatten:

julia> function prime_factors1(n::Integer)
           factors = Int[]
           # Sadly, using an iterator on a hot loop like this is a bit too slow.
           for i in flatten((2, 3:2:n))
               while n % i == 0
                   push!(factors, i)
                   n = div(n, i)
               end
               n == 1 && break
           end
           factors
       end
prime_factors1 (generic function with 1 method)

julia> using .Iterators: flatten

julia> using BenchmarkTools

julia> @btime prime_factors1($98623589);
  1.786 ms (4 allocations: 208 bytes)

@MasonProtter also encountered this behavior on 1.4.1

julia> versioninfo()
Julia Version 1.4.1
Commit 381693d3df* (2020-04-14 17:20 UTC)
Platform Info:
  OS: Linux (x86_64-unknown-linux-gnu)
  CPU: Intel(R) Core(TM) i5-5300U CPU @ 2.30GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-8.0.1 (ORCJIT, broadwell)

[MasonProtter]

Just to head off a possible line of inquiry, I checked to make sure this could be reproduced without Revise loaded and the presence or absense of Revise doesn't seem to matter.

[vchuravy]

Yes this is expected behavior. There is no system inplace to invalidate code that had the potential to be an undefined variable error. This falls under the global variables are slow category ;) You could probably teach Revise this.

[cmcaine]

Is there some reason not to invalidate here, beyond that no one has written that system yet?

[vtjnash]

We just don't have any system for invalidating code right now.

[cossio]

Here is a simpler example I ran into today.

julia> f(x) = erf(x);
julia> f(1)
ERROR: UndefVarError: erf not defined
Stacktrace:
 [1] f(::Int64) at ./REPL[1]:1
 [2] top-level scope at REPL[2]:1
julia> using BenchmarkTools, SpecialFunctions
julia> x = 1; @btime f($x);
  27.645 ns (1 allocation: 16 bytes)

Whereas if I import SpecialFunctions at the beginning, there is no allocation:

julia> using BenchmarkTools, SpecialFunctions
julia> f(x) = erf(x);
julia> x = 1; @btime f($x);
  22.195 ns (0 allocations: 0 bytes)

On Julia 1.4.1

[StefanKarpinski]

Yes, it's easy to do. Also not a problem in any correct program.