Fmt: basic_format_args with named args not working?

[abrownsword]

I have a map that contains my named arguments and I want to pass these to vformat. Based on previous issues, I have written the code like this (against 5.3.0):

std::vector<fmt::basic_format_arg<fmt::format_context>> args;   

for (auto& p : str2strMap)
    args.push_back(fmt::internal::make_arg<fmt::format_context>(
                   fmt::arg(p.first, p.second)));

return fmt::vformat(format, fmt::format_args(args.data(), args.size()));

While it compiles and links, when I execute it, the named arguments are never found and vformat throws an exception.

Should this work?

[vitaut]

AFAICS this works (update: this doesn't work =)):

#include <fmt/format.h>
#include <vector>

int main() {
  std::vector<fmt::basic_format_arg<fmt::format_context>> args;
  args.push_back(fmt::internal::make_arg<fmt::format_context>(fmt::arg("n", 42)));
  fmt::vprint("{n}", fmt::format_args(args.data(), args.size()));
}

What exception do you get?

[abrownsword]

vformat failed with argument not found

Your example works for me too. After some experimentation I've discovered that _only args pushed into the vector in a loop_ fail. And one of them works. This harkens back to something mentioned in one of the related issues where references are being retained to temporaries on the stack. I didn't understand what they were talking about in that discussion and assumed that as long as _my_ data's lifetime was ensured then things would work. Apparently not. It seems like make_arg is keeping a reference to a temporary that fmt::arg creates?

I just found a (pretty hideous) workaround... create a separate vector or array of named_args. Use fmt::arg() to initialize its elements, and then build the vector of basic_format_arg<> from those elements. The resulting vector then works with fmt::format_args(). Note: you need to either reserve the named_args size in advance, or add all of the named_args before starting to make basic_format_arg<> elements otherwise a vector resize might break whatever hidden references you've created thus far (and a segfault is apparently the likely result).

[vitaut]

It seems like make_arg is keeping a reference to a temporary that fmt::arg creates?

Right. Thinking more of it, my example is broken too. To make this work you should store named_arg objects in some container and pass stable references to make_arg.

[abrownsword]

Could this at least be clearly documented? Plus an example of how to do this sort of thing ‘right’ would be very useful to users of the library the first time they want to try this sort of a usage pattern.

[vitaut]

The APIs your are using are internal (thus the internal namespace) and not recommended for general use. Eventually the library might provide an API for constructing dynamic argument lists but we are not there yet.

[vitaut]

That said, PRs to improve the docs are welcome (except for internal APIs that are not documented intentionally.)

[abrownsword]

True, but I highly encourage you to get there as quickly as you can. This is must have functionality, IMO, and a significant win over competing alternatives.

[vitaut]

Could you describe your use case a bit and why you need to construct the argument list dynamically?

[abrownsword]

I have a few use cases, but they all boil down to wanting to externalize the format of whatever needs to be produced from the C++ code. So, for example, a config file might have a format string in it and that format string wants to extract zero or more named values from some data dictionary in the program.

(obviously you don't want to do this in security sensitive situations, or at least be very cautious/defensive about it)

[vitaut]

Added an issue: #1170

[abrownsword]

I just uncovered a subtle bug in the above code to build an explicit argument list. Internally it appears that the format list is traversed until a none_type is encountered. So it is necessary to push a default constructed fmt::basic_format_arg (i.e. one of type none_type) onto the args vector.

It would be awfully nice to have this formalized as a public API. It is very useful functionality to have. Ideally it would also:
1) throw an exception with the tag of an unspecified tag, should that happen.
2) allow more than one named arg list to be provided (searching them in order provided).
3) allow the arg list to be kept around, and perhaps have its values updated efficiently.

[vitaut]

Internally it appears that the format list is traversed until a none_type is encountered.

This happens only for packed arguments, not when you construct basic_format_args dynamically from an array of format_arg objects.

So it is necessary to push a default constructed fmt::basic_format_arg (i.e. one of type none_type) onto the args vector.

This is not needed.

[abrownsword]

Are you certain? The code was crashing at element 242 (when I only had 7), and adding a none_type arg at the end fixed the problem. If that wasn't the problem, I need to understand what the real problem is and fix it.

[vitaut]

For normal arguments, yes. However, it looks like named arguments code still relies on the empty terminating argument instead of using the size directly: https://github.com/fmtlib/fmt/blob/b615eca9641f12edb6713058437fd8e83c03fe09/include/fmt/format.h#L1265. This needs to be fixed.

[vitaut]

Fixed in https://github.com/fmtlib/fmt/commit/5360ab0b59cef4a93d6e4372721835882e82a6b7. Thanks for bringing this to my attention.