Autokeras: time_limit and metric arguments are no longer available?

[iXanthos]

Greetings to everyone,

As a part of an evaluation I wanted to try autokeras on multiple different datasets. My data are tabular, so I use StructuredDataClassifier and StructuredDataRegressor. The problem is that I cannot set a custom evaluation metric, nor set the time_limit of the model.fit function. In the examples I found online, everyone is setting their desired time_limit, but when I try to run a fit function with this parameter, it outputs the error: Unrecognized keyword arguments: {'time_limit': 360}. Is this functionality removed from the newest version of autokeras? Also, about the metric, I want to use AUC, as the performance metric, but neither the metric variable is recognized.

autokeras 1.0
python 3.6.9
OS Ubuntu 18.04

[haifeng-jin]

@iXanthos For time_limit, we changed it. We now use max_trials to control the search. It is an argument in the initializer of the classes.
metrics is also in the initializer. It has the same usage as the Keras metrics in the compile function.

[iXanthos]

@haifeng-jin so is there a way to set a strict time limit? If not, this is a step backwards in my opinion, as it makes autokeras unavailable for quantitative performance evaluation under strict time restrictions (99% of all the published works evaluate a tool or library under some time constraints).
Also, when you talk about the initializer, you mean the StructuredDataClassifier call, correct?

Thank you in advance.

[haifeng-jin]

@iXanthos Yes, that is correct. The StructuredDataClassifier call.

We will do some performance evaluation later for the new version.
It is hard to do a strict time limit, so, for now, we will not focus on this feature.

[dahlo]

@haifeng-jin do you have any suggestion on how to limit the time any other way, or a way to save the state of the program every n iterations/time units?

I run my code on a HPC center where i have to book my jobs with a time limit, and jobs are terminated when the timer runs out. It would be a shame if i can't save my model before that happens.

[haifeng-jin]

Currently, the is no way to limit the time. However, all the intermediate results and models are saved on disk. You may try to see what's in the folder. Those kids and models are generated by KerasTuner. Thanks.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.