Agents.jl: Split `step!` into three functions

[Datseris]

As far as I can tell, step! has three functionaities, all different with each other.

1. Evolve the model.
2. Evolve the model _and_ collect data _per step_
3. Evolve the model and _aggregate data_ across time (steps).

I think this is too much conceptual load put into one function. If we simply _rename_ these functionalities into three functions, I think it will make this package easier to use.

Thoughts?

[Libbum]

Totally agreed. These three aspects of the function muddy the waters and limit the users ability to efficiently collect data. It also limits our ability to both collect and plot 'live', which is something we should be working towards seeing.

I'm still in the process of getting together a set of use cases that with conflict the current method and solutions. I'll consider how they would change if we had four separate methods (evolution, data collection, data aggregation, plotting) and report back soon if there are any edge cases I identify.

[Datseris]

data collection and data aggregation is the only case that can't be separated. You need the data first to aggregate them, and I am not sure whether splitting this into two pipelines (first collect, _then_ aggregate) is useful in the end.

The way I see it the aggregating function should also do the data collection. Thankfully, plotting is already detached from anything else.

I've thought about names for the processes as well: step!, collect!, aggregate!

[Libbum]

step! to me is singular, perhaps evolve! could be used.

collect! and aggregate! may not have to be explicitly separated. Similar to what's implemented now, the step! function does separate these two scenarios, although they can't be done at the same time.

step!(model, agent_step!, model_step!)
data = collect!(model, properties)
agg_data = collect!(model, agg_properties)
step!(model, agent_step!, model_step!)

may be enough?

An implementation problem we need to consider here relates to the collectors understanding what has happened since the last call to it.

step!(model, agent_step!, model_step!)
results = collect!(model, properties) #New dataframe
step!(model, agent_step!, model_step!, 30)
collect!(results, model, properties) #data appended to existing dataframe

Here, properties are values on explicit steps. This would be equivalent to

results = step!(model, agent_step!, model_step!, 31, properties; when = 1:30:31)

The separation means that the data collection method has no way of tracking the actual step number (the issue I solved with model.properties[:tick] in #151).

This would need be solved by adding a step (or tick) to the model rather than having the evolution method handle this.

[kavir1698]

Hi guys,
Sorry that I have not been able to contribute in the past few days. But I am following the discussions. About this PR, I am clear what will be the final shape of the API. After splitting step!, does the user have to write a loop to collect data? Specifically, when we want to collect aggregate data from each step, we do not want to first collect all possible data first, and then aggregate them. This is inefficient.

[Datseris]

This:

step!(model, agent_step!, model_step!)
results = collect(model, properties) #New dataframe
step!(model, agent_step!, model_step!, 30)
collect!(results, model, properties) #data appended to existing dataframe

is the way to go, but one also needs to think about aggregation. Possibly something like

aggregate(results, ...) _> aggregated dataframe

The difficulty I see is that collect! is agnostic to the step number, but obviously in such a data collection the step needs to be known.

---

@kavir1698 you need to tell us how aggregation is done now. Aren't all _relevant_ data first collected for every step and _then_ aggregation is done? This is what we propose here as well. You collect the data you want with collect! (only the fields necessary), and then you aggregate if you want.

What we discuss here is a strict improvement of the code base. No previous functionality is lost. The code is simply made more modular, easier to understand and easier to expose to the front user.

The user will not have to write a loop at the end. Of course we will also provide a high level version that does the loop and collects. But is so much better to make the inner level of the loop independent, clear, and part of the public API. Step for 1 step, collect the data you want for 1 step. That's it.

[kavir1698]

This:

step!(model, agent_step!, model_step!)
results = collect(model, properties) #New dataframe
step!(model, agent_step!, model_step!, 30)
collect!(results, model, properties) #data appended to existing dataframe

is the way to go

That is a lot of more writing for the same functionality. Moreover, what happens here is 30 steps are run and then data are collected from the last step.

Aren't all relevant data first collected for every step and then aggregation is done?

No, that could consume a lot of memory. At each step, data is aggregated and added to a data frame.

What we discuss here is a strict improvement of the code base. No previous functionality is lost. The code is simply made more modular, easier to understand and easier to expose to the front user.

Increasing modularity would be great, if it doesn't cost compactness too much.

[Libbum]

Moreover, what happens here is 30 steps are run and then data are collected from the last step.

Yep, that was the point of this example. Harking back to the issues in #151 and #188.

data is aggregated and added to a data frame.

Put in other words, does this mean 'we compute an aggregate value over our collection (either model.agents or model.properties) at each collection step, then append the result to a dataframe'?

[kavir1698]

Yep, that was the point of this example.

I meant that this example does not collect data during the first 29 steps.

'we compute an aggregate value over our collection (either model.agents or model.properties) at each collection step, then append the result to a dataframe'?

Yes!

[Libbum]

Cool. I think we're all on the same page with this, but we need to identify the best way to implement both a high and low level separation.

All of the current examples work great with a high level api, but the issues I pointed to last post both need access to a low level solution.

-----Original Message-----
From: "Ali R. Vahdati" [e-mail]
To: "JuliaDynamics/Agents.jl" [e-mail]
Cc: Tim DuBois [e-mail], Comment [e-mail]
Sent: Mon, 30 Mar 2020 21:25
Subject: Re: [JuliaDynamics/Agents.jl] Split step! into three functions (#187)

Yep, that was the point of this example.

I meant that this example does not collect data during the first 29 steps.

'we compute an aggregate value over our collection (either model.agents or model.properties) at each collection step, then append the result to a dataframe'?

Yes!

--
You are receiving this because you commented.
Reply to this email directly or view it on GitHub:
https://github.com/JuliaDynamics/Agents.jl/issues/187#issuecomment-606197024

[kavir1698]

Well, those issues can be solved by optionally returning _two_ data frames: one for the model object and one agents. I have been thinking about them and I think forcing everything into one data frame is not optimal.

[Datseris]

There are some things here that don't make much sense to me @kavir1698 , so I'll enumerate the issues I currently have with the data collection and with your statements.

1

Aren't all relevant data first collected for every step and then aggregation is done?

No, that could consume a lot of memory. At each step, data is aggregated and added to a data frame.

Either I confused you with my wording, or you confused me. If you have Dict(:wealth => [mean, median])., the only way to compute the median is to first have all data. I fail to see your argument "this consumes more memory". What I meant is that at every step you want to collect data, you first collect all the data and then you aggregate them. Every approach has to do this.

1.5

step!(model, agent_step!, model_step!)
results = collect(model, properties) #New dataframe
step!(model, agent_step!, model_step!, 30)
collect!(results, model, properties) #data appended to existing dataframe

That is a lot of more writing for the same functionality. Moreover, what happens here is 30 steps are run and then data are collected from the last step.

Literally every line in this code snippet does a different thing. How can you say it is "the same functionality"? You can still offer a high level wrapper that loops over step! and collect! , replicating the current functionality of step! with properties.

Data aggregation is a separate yet once again a trivial loop over step! and then calling aggregate!(existing_dataframe, model) is enough, to e.g. append the median of the agent wealth after the step!. This is what happens at the moment as well. I only propose to make it cleaner, more streamlined, and supported as part of the public API.

2

Increasing modularity would be great, if it doesn't cost compactness too much.

The current state of the data collection is not only the least modular, but also the least compact. There is a reason #167 is still open, and that reason is that the source code of the data collection is so complicated and with so many repetitions that I don't want to mess with it, afraid I will break everything.

A lot of the file data_collect.jl is duplicated code. For me this simply proves that the code we have can be simplified significantly.

If we care about code base sustainability, this has to be improved.

3

Put in other words, does this mean 'we compute an aggregate value over our collection (either model.agents or model.properties) at each collection step, then append the result to a dataframe'?

Yes

Obviously this would happen, what else? I am confused that this is discussed, because I can't imagine an alternative scenario. If you can, please show it to me, so I am not so confused anymore...

4

Harking back to the issues in #151 and #188.

Well, those issues can be solved by optionally returning two data frames: one for the model object and one agents. I have been thinking about them and I think forcing everything into one data frame is not optimal.

Here is where the new api I propose really shines. Whether returning two dataframes instead of one is better, I can't say yet, I need to see some working implementation. I also side with the two dataframe case.

With the proposed more modular approach we discuss here, this becomes trivial. Here is a sketch:

agent_df = init_agent_df(...)
model_df = init_model_df(...)
for i in when()
step!(...)
if i in when_agents()
collect!(agent_df, model, agent_properties, i)
end
if i in when_model()
collect!(model_df, model, model_propertes, i)
end
end

In fact, you don't even have to really support this, this is so crystal clear that the user can do it themselves, with a proper collect! function and a proper function initialize_dataframe(model, agent_properties)... etc.

---

In conclusion, I still don't see why we discuss this at such lengths. What was proposed in the original post is straight forward, doesn't remove anything from the existing functionality and it loses no performance, yet the benefits are:

• 3 functions that the user can learn. understand, and use in their own loops to e.g. make a specialized version of data collection
• MORE functionality
• LESS source code

[Libbum]

Here is a sketch

This is good and on the way to what we need. Some additional comments though:

We should also allow the possibility of a when_to_plot() if this sketch is wrapped up in a run command or something.

I don't think that model/agent is the only DF separation. Aggregate/collection is also perhaps required.
If I want to collect data from all individual agents every 10 steps, but an aggregate result over all agents each step, how would this method handle such a case?

[Datseris]

agent_df = init_agent_df(...)
model_df = init_model_df(...)
aggregated_df = init_aggregated_df(...)
for i in when()
step!(...)
if i in when_agents()
collect!(agent_df, model, agent_properties, i)
end
if i in when_model()
collect!(model_df, model, model_propertes, i)
end
if i in when_aggregate()
aggregate!(aggregated_df, model, aggragate_dict, i)
end
if i in when_plot()
plot()
end
end

As you can see, it perfectly fits all cases, and is a powerful API that can be used by a user really with minimal effort.

[Datseris]

We should also allow the possibility of a when_to_plot() if this sketch is wrapped up in a run command or something.

This is probably not great, because the run function would become extremely complicated (i.e. what step! is right now). We should make it simple and composable instead, and just provide two high level loop functions for the current 2 cases of data collection (standard collection and aggregation)

[Libbum]

This is 100% in line with my requirements and expectations. Just trying to align @kavir1698 too. Ali: any issues with this?

[kavir1698]

This is 100% in line with my requirements and expectations. Just trying to align @kavir1698 too. Ali: any issues with this?

Initially, I thought we were going to collect data from _all_ the steps and then aggregate them.
I understand your proposed approach now. For non-standard data collection, the user has to write all the loops and if statements. This is the lack of compactness I was worried about. But since we are going to have high level loop functions for standard cases, we are not losing anything.

[Datseris]

Exactly.

Now it remains to be seen when the monumental task of refactoring this API will be done :D

[Libbum]

Great. I follow your concern about compactness, but to me it seems that other than our simple example cases in the docs, users will seldom want this simplicity. Could be wrong of course, but having two layers of API accessible works either way.

Next step: who wants to lead this? It's currently a blocker for me, so I could start something...

[kavir1698]

I have a couple of hours now. I'll make a PR.