Agents.jl: Type specificity of properties dicts

[fbanning]

Hiya,

consider the following

using Agents
mutable struct MyAgent <: AbstractAgent
    id::Int
end
properties = Dict(:par1 => 1, :par2 => 1.0, :par3 => "Test")
model = ABM(MyAgent; properties = properties)
model_step!(model) = begin
    a = model.par1 * model.par2
end

julia> @code_warntype model_step!(model)
Variables
  #self#::Core.Compiler.Const(model_step!, false)
  model::AgentBasedModel{Nothing,MyAgent,typeof(fastest),Dict{Symbol,Any},Random.MersenneTwister}
  a::Any

Body::Any
1 ─ %1 = Base.getproperty(model, :par1)::Any
│   %2 = Base.getproperty(model, :par2)::Any
│   %3 = (%1 * %2)::Any
│        (a = %3)
└──      return %3

Users would assume that par1::Int64 and par2::Float64 so that a::Float64. However, that's not the case and this reduces code performance afaik.

Expand for further illustrative examples

For a multi-type dictionary with reduced type variation Dict{Symbol,Real} a similar problem occurs:

properties = Dict(:par1 => 1, :par2 => 1.0, :par3 => 1)
model = ABM(MyAgent; properties = properties)

julia> @code_warntype model_step!(model)
Variables
  #self#::Core.Compiler.Const(model_step!, false)
  model::AgentBasedModel{Nothing,MyAgent,typeof(fastest),Dict{Symbol,Real},Random.MersenneTwister}
  a::Any

Body::Any
1 ─ %1 = Base.getproperty(model, :par1)::Real
│   %2 = Base.getproperty(model, :par2)::Real
│   %3 = (%1 * %2)::Any
│        (a = %3)
└──      return %3



md5-97f8100819081d1e6da3dbb6b75faeb6



```julia
julia> @code_warntype model_step!(model)
Variables
  #self#::Core.Compiler.Const(model_step!, false)
  model::AgentBasedModel{Nothing,MyAgent,typeof(fastest),Dict{Symbol,Int64},Random.MersenneTwister}
  a::Int64

Body::Int64
1 ─ %1 = Base.getproperty(model, :par1)::Int64
│   %2 = Base.getproperty(model, :par2)::Int64
│   %3 = (%1 * %2)::Int64
│        (a = %3)
└──      return %3

A few things to note here:

1. Dicts are inherently unperformant for models using properties of multiple types (e.g. Dict{Symbol,Any}). The compiler uses type Any for all dict values when they are used for calculations in the model and thus cannot properly dispatch because of missing type information.

2. Providing properties as a mutable struct instead circumvents this type inference problem.
   See the following example:

   mutable struct Properties
       par1::Int
       par2::Float64
       par3::String
   end
   properties = Properties(1,1.0,"Test")
   model = ABM(MyAgent; properties = properties)
   

   ```julia
   julia> @code_warntype model_step!(model)
   Variables
   #self#::Core.Compiler.Const(model_step!, false)
   model::AgentBasedModel{Nothing,MyAgent,typeof(fastest),Properties,Random.MersenneTwister}
   a::Float64

   Body::Float64
   1 ─ %1 = Base.getproperty(model, :par1)::Int64
   │ %2 = Base.getproperty(model, :par2)::Float64
   │ %3 = (%1 * %2)::Float64
   │ (a = %3)
   └── return %3
   `` As usual we can easily access the struct fields throughmodel.properties.fieldname~~(but not throughmodel.fieldname`)~~ to modify them during the model run.

3. Idle thought: Should we find that some functionality is missing, AbstractProperties could be defined as a supertype for specifying any necessary methods. This is not really needed as of now, because its fully functional as it is. But it would be nice to be able to write model.fieldname again, for example.

So for me three questions arise which I'd like to hear your thoughts on:

1. Is it sensible to use Dicts for defining model properties if we know that the compiler doesn't know about the elements' types in the case of mixed type properties?
2. Do you know of any showstoppers regarding the use of a properties struct as demonstrated above?
3. If the answer to 2. is no, should we maybe add a paragraph in the docs which explicitly explains the option to use structs? It's mentioned here but I think it deserves a bit more explanation (maybe even showing its usage in an example).

Best wishes :)

P.s.: While this is an abstract example, it is not arbitrarily chose. Our model's properties are diverse (resulting in a Dict{Symbol,Any}) and I've encountered the aforementioned typing problems in our application. Personally I am going to switch to the struct approach and will most likely avoid using dicts for properties in production code with a certain degree of complexity from now on.

[Libbum]

Everything you've stated here is indeed the case & is something that perhaps needs to be highlighted in the up and coming performance section of the docs.

The Dict as default, struct as optional comes in play mostly for usability. If a model property needs to be added mid-run, then the struct option is less easy to manage. Not sure if this is something we do in any of our examples, but it's something I've needed privately in a few models of my own.

Some of the things we've been speaking of in terms of performance recently (#396, #421) come at a pretty heavy price in terms of usability. Considering our current benchmarks against other frameworks, we don't at present need to sacrifice everything.

That being said, your suggestion about AbstractProperties is a decent one, and that would certainly help bringing the properties struct option up to par with the Dict choice.

[fbanning]

Thank you for your comment, much appreciated.

Could you elaborate a bit on your need for dynamically adding model properties during the model runs? Generally I would assume that model properties are not generated ad-hoc but instead the need for them should be anticipated by the model creator. Thanks in advance if you can share any insights on this!

One more thing I just wondered about: How does the paramscans function work when properties are a struct instead of a dict? It's probably not yet suited for this use case, or is it? It's certainly not a big problem to continue using the dict approach here but indeed exactly parameter scans/sensitivity analyses show critical reaction to performance related issues and using a more performant approach (e.g. structs for properties) can make a huge difference in the end.

[Datseris]

Could you elaborate a bit on your need for dynamically adding model properties during the model runs?

This is possible:

function model_step!(model)
  if nagents(model) == 10 # do action once 10 agents have been generated
    model.properties[:new_key] = 4
  end
end

[fbanning]

This is possible:

function model_step!(model)
  if nagents(model) == 10 # do action once 10 agents have been generated
    model.properties[:new_key] = 4
  end
end

I know but I was wondering whether @Libbum had a case where the condition is unclear in advance, making it impossible to previously create the field in a struct.

[Datseris]

A simple scenario would be dynamically deciding the _name_ of the new property. E.g. some events output some integer number, and you create the key new_key_integer.

EDIT: Unfortunately this won't work with our current setup of data collection, but oh well.

[AayushSabharwal]

Small point:
If the struct doesn't need to be mutable (if they just contain some tuning parameters) NamedTuple seems to solve this too. without having to declare an extra struct. Using properties = (par1 = 1, par2 = 1.0, par3 = "Test"), I get the same output of @code_warntype as using a struct

[Libbum]

Could you elaborate a bit on your need for dynamically adding model properties during the model runs?

You're right about this too. I guess I just use it for convenience. Set here, check here. I could just make this a bool in a struct now that I think about it.

George has a better example in the above comment though.

There is no functional difference when doing model.key

Yeah, that's true. @fbanning:

properties = Dict(:par1 => 1, :par2 => 1.0, :par3 => "Test")
model = ABM(MyAgent; properties = properties)
julia> model.par1
1
#...
mutable struct Properties
               par1::Int
               par2::Float64
               par3::String
       end
properties = Properties(1,1.0,"Test")
model = ABM(MyAgent; properties = properties)
julia> model.par1
1

So a custom AbstractProperty is not needed.

[fbanning]

If the struct doesn't need to be mutable (if they just contain some tuning parameters) NamedTuple seems to solve this too

Good catch and an easy approach, thanks for pointing it out. Should definitely go in the docs as well.

There is no functional difference when doing model.key

Yeah, that's true. @fbanning:

Yep, sorry, I was mistaken there. I once encountered an error when trying to access a field via model.key and just assumed there is a method missing. Indeed it's working as expected. 👍🏻

[Libbum]

Cool, then I think all there is to this is to better document it, we don't need to change any code. Let's follow up in #454 if there's anything else.

If you can find that edge case that failed with model.key, that's something separate to investigate—open a new issue for that if it crops up.

[fbanning]

Great! If anyone wants to start the PR for the docs enhancement, feel free to use any of my abovementioned examples/ideas/explanations/whatever from this thread. :)