Ignite: "=" in the checkpoint name may be incompatible with OSs

[Nic-Ma]

❓ Questions/Help/Support

Issue raised by Eric from MONAI team:
File names with operators in them may be valid names in Linux but that's not the case in other OSs, plus shells like bash really don't like them.

[sdesrozis]

@Nic-Ma Thank you for this question.

I don't know any problematic OS with = operator in filenames but even with Linux, you have to be careful and sometimes use quotes.

The filename policy is here
https://github.com/pytorch/ignite/blob/cc78b3bbd764da9ae7dee0df21676b2b8a50fba8/ignite/handlers/checkpoint.py#L93-L105

Maybe we could add a format based on regex that user could set (default is existing policy) ?

[Nic-Ma]

Hi @sdesrozis ,

Thanks for your quick response.
A regex or user-defined separator sounds good, the default still can be "=".
Thanks.

[Joel-hanson]

Shall I take this up @sdesrozis

[sdesrozis]

@Joel-hanson thank you to contribute on this issue 😊

The idea is to replace the operator = in filename of checkpoints. To do it, we can introduce a separator attribute (default is =) as suggested by @Nic-Ma. A more sophisticated option is to provide a string pattern like {filename_prefix}_{name}_{score_name}={score}.{ext} and process it by replacing labels.

What do you think?

[vfdev-5]

How about setting this pattern as a class member of Checkpoint:

class Checkpoint:

     filename_pattern="{filename_prefix}_{name}_{score_name}={score}.{ext}"

     ...

If user would like to change it, there is a possibliity.

[Joel-hanson]

@sdesrozis replacing labels seems like a good option and the suggestion by @vfdev-5 is worth considering. I will note down what I have in mind to be more clear before moving forward.

I will set the filename_pattern="{filename_prefix}_{name}_{score_name}={score}.{ext}" as a class member for the Checkpoint. The filename will have the default pattern if there is no change in the class member. If a user changes his filename_pattern, then by using str.format_map I will replace each label with its values.

When we implement likewise whenever a user provides the pattern to be {filename_prefix}_{name}_{score_name}, without having the either {score} or {ext} shall I append the default pattern to it like {filename_prefix}_{name}_{score_name}{score}.{ext} so that the file doesn't get replaced or not get saved without extension. Is there anything that I am missing out on? correct me if I am wrong.

[sdesrozis]

@Joel-hanson What you have in mind seems good 😉 str + format with fixed keywords (score, ext, etc.) looks good. It’s similar to what is done in logging format. It should be a good inspiration.

And you are right, we have multiple patterns depending of the usage context. IMO if user give a pattern, we should use it. Otherwise, legacy policy remains used. Let’s see if it fits class member approach ! What do you think ?

[vfdev-5]

When we implement likewise whenever a user provides the pattern to be {filename_prefix}_{name}_{score_name}, without having the either {score} or {ext} shall I append the default pattern to it like {filename_prefix}_{name}_{score_name}{score}.{ext} so that the file doesn't get replaced or not get saved without extension.

@Joel-hanson let's consider that if user changes the pattern he/she knows what he/she is doing.
So, no internal additional appends. For example, if pattern is

p = "{name}_{score}"
kw = {"name": "abc", "filename_prefix": "pref", "score_name":"acc", "score": 0.234, "ext": "pt"}
p.format(**kw)
> 'abc_0.234'

[Joel-hanson]

Thanks, @sdesrozis @vfdev-5 for the quick response. So I shall start implementing based on what @vfdev-5 has conveyed.

[vfdev-5]

@Joel-hanson any updates on this ? Can we help you somehow to start implementation and send a PR ?

[Joel-hanson]

Sorry @vfdev-5 was busy with some pending works will start tomorrow.

[Joel-hanson]

Sorry for the delay @vfdev-5 @sdesrozis
I have created a PR #1127 for the following issue. If the PR satisfies the requirement and the code quality, I shall add the required tests and also update the docs.