Julia: MethodError when add '@threads' to 'for' loop

[aqzas]

Hi guys, I meet a problem when I trying to paralleling my code using multi-thread

for (lgt,val) in gp
     absorb!(lgt, val, the_ERR)
end

to

Threads.@threads for (lgt,val) in gp
        absorb!(lgt, val, the_ERR)
    end

gp is a Dict{Int, Array{Myread,1}}()
Myread is my custom struct
function of absorb is complicate to explain, but generally it will filter out some item in the Array according to the lgt variable

Everything goes well without @threads, but after add @threadand a MethodError will occur and output data in val, like this

[jpsamaroo]

This question is better asked on Discourse: https://discourse.julialang.org/
@threads is tricky to get right, so I'm sure someone on Discourse can help work you through this issue.

EDIT: Looks like I misrepresented the source of the issue, sorry!

[KristofferC]

This doesn't look like a question, it looks like a reported crash in the optimizer or something.

[yuyichao]

It's not a compiler issue. Thread loop doesn't support arbitrary iterators since it needs to know the size.

[KristofferC]

Ok, I missed this was an unpacking loop. Could the macro detect this?

[yuyichao]

No that's impossible. (By which I mean the thing to detect is the method error.......... You can generate code to try to predict the error (check method applicability for example) but for the macro to detect it is impossible.)

[jpsamaroo]

@aqzas would you be able to craft a Minimum Working Example so that we can debug this on our local systems? It would also be great if you could post that error you got as text (instead of an image) so that we can see the whole thing.

[yuyichao]

You should get that error if you run any thread loop on dict.

[aqzas]

@yuyichao Thank for you reply, I vaguely feel that's the problem before

@jpsamaroo
Though the problem is solved as @yuyichao said, here is the example you can reproduce it.

gp = Dict(
1=>[1,2,3,4,5,6,7],
2=>[1,2,3,4,5,6,7],
3=>[1,2,4,5,3,5,6,36]
)

function absorb!(key, val)
    filter!(x->x>key, val)
    return
end

Threads.@threads for (key, val) in gp
    absorb!(key, val)
end

@show gp

Julia version 1.0.0
MacOS 10.13.3

[BCRARL]

Currently the documentation of @threads doesn't mention these limitations. A literal reading of the documentation suggests that any iterator is possible and in particular that an iterator over a finite collection will work. I had the same issue using eachrow in 1.1.0

[StefanKarpinski]

Best to quote @threads so that you don't ping the GitHub user who has that handle.

[JeffBezanson]

The error reporting should be better here now but we should add this limitation to the documentation for @threads.

[JeffBezanson]

Also duplicate of #18263.

[KristofferC]

Closing as dup.