cookiecutter as lib: Support custom config_file names (e.g recipe.json instead of cookiecutter.json)

[danwos]

Hi all,
before creating a pull request I would like to get some feedback on the following feature.

I have implemented cookiecutter into my lib groundwork as solution for realizing project generation via templates.
My goal is to somehow white-label cookiecutter, so that my users are able to use own names for their repository configuration files and the related context variables.

Example: Instead of having a cookiecutter.json and accessing variables via cookiecutter.project_name, a user can define a my_template.json and access vars via my_template.project_name.

Good news: The technical work is done and I have a working prototype.
However, I needed to pull a lot of cookiecutter-functions into my repository to add this dynamic behavior.
Reason for this: "cookiecutter.json" and "cookiecutter" as context key are hard coded at many places and right now there is no way to overwrite these values.

The needed changes are not too complex, mostly only a function parameter with a default value must be added. E.g. the function find_template(repo_dir) was changed to

def find_template(repo_dir, context_key='cookiecutter'):
    ...
    for item in repo_dir_contents:
        if context_key in item and '{{' in item and '}}' in item:
        #  was: if "cookiecutter" in item and '{{' in item and '}}' in item:
   ...

By using "cookiecutter" or "cookiecutter.json" as defaults for these new function parameters, all changes should be backward compatible.

A developer could use this feature by calling cookiecutter() from main.py with an additional parameter:

from cookiecutter.main import cookiecutter

cookiecutter('cookiecutter-pypackage/', config_json="recipe.json)

That's all. _recipe.json_ is used instead of _cookiecutter.json_ and {{recipe.project_name}} can be used instead of {{cookiecutter.project_name}}

I have temporally collected all changed cookiecutter functions in this file: https://github.com/useblocks/groundwork/blob/recipes/groundwork/patterns/gw_recipe_pattern.py#L193

This feature would allow developers to use cookiecutter more like a lib and let it work in their application background. It allows to somehow "hide" the name cookiecutter, as it can be replaced by own names (template, recipe, plugin, ...)

I would like to add this feature to cookiecutter and I could also prepare the pull request.
But before doing this, I want to be sure that there are no constraints of implementing such a function.
So, do you see any drawbacks for this feature?

Daniel

[pydanny]

This feature has always been rejected on account:

• it adds unnecessary complexity to Cookiecutter.
• Makes identifying Cookiecutter templates much harder

Traditionally to get white labeling means you pay more for a product. We already get so many UNPAID help and feature requests, but at least we get some recognition for our work. Allowing this would remove that recognition in return for additional complexity.

Finally, considering that Groundwork is about reusable components, I think keeping Cookiecutter simpler, hence more robust, is the better way to go for Groundwork.

[danwos]

Hi pydanny,

I understand your points, if cookiecutter shall be a single application without the goal to provide an API for developers. In this case, it's important to use standardized names, so that all recipes can be used the same way. :+1:
But for me this feature request does not touch this. Or even better, this feature will not be usable, because for all cookiecutter cli commands cookiecutter() gets invoked with default values ("cookiecutter.json") and this can not be changed via cli. So I see no drawbacks on this front.

However, as library it should give the freedom to developers to decide by their own how to name configuration files.
I do not know much libraries, which claim specific names for needed files.
And if they do, the name is more function based, like configuration.py or app.conf.

Also removing the hard coded file names and context keys may reduce complexity and make it more maintainable, because the need to repeat this string again and again is reduced (DRY).
Maybe it is also possible to define the configuration file name "cookiecutter.json" only once in the code. For my understanding this would be the cleanest and most maintainable way...

The missing recognition is a general problem of libraries.
Which website visitor can tell, if a website is using django or flask? Only the developers do.
And as library this applies also to cookiecutter and wouldn't change, if the developer can define own names for configuration files.
I suppose that the recognition also becomes more, as the new flexibility may cover additional use cases and therefor convince more developers to use cookiecutter.

My personal goal is not hide cookiecutter from my users or developers. It will be named several times in the documentation and beside a small code example how to use it, all other information must be read on the official cookiecutter website.
But I want to hide complexity for my users on the first levels. And throwing a library name on them is perplexing. How to tell them in 10 secs that they must use "cookiecutter.json" for configuration without introducing and explaining cookiecutter?

Don't get me wrong, I ask the community for feedback because I have already imagine this kind of reservations and can fully understand them. But I see more benefits having this feature :)

[pydanny]

However, as library it should give the freedom to developers to decide by their own how to name configuration files.
I do not know much libraries, which claim specific names for needed files.
And if they do, the name is more function based, like configuration.py or app.conf.

• .travis.yml
• .gitignore
• ..pylintrc
• docker-compose.yml
• gulpfile.js
• pytest.ini
• .editorconfig
• etc

The nice thing about having a specifically named management file like cookiecutter.json or any of those listed above is there is much, much less chance of collision.

In any case, you don't have to use cookiecutter.json. You can define your own format or means of getting use input, then write a script to use the cookiecutter/main.py's cookiecutter() function via like so:

YOUR_CUSTOM_DATA = {}
cookiecutter(template_name, extra_context=YOUR_CUSTOM_DATA, no_input=True)

For what it's worth, a number of projects and organizations, large and small, leverage Cookiecutter this way.

[bittner]

@pydanny :+1: for pointing out how to use cookiecutter as a library! :smiley:

[danwos]

Ok, most of the given examples are applications, which are used via command line and not by importing them into python code. So for me, most of them are not designed to work like a library.

A nice example of some code, which can be used as application and as lib is buildout.
buildout normally works with a config-file called buildout.cfg. But you are free overwrite this name and use something different.
But for the moment I'm okay with using cookiecutter inside config names.

However, I tried to use the given workaround from @pydanny. But it does not work as expected.
Reason (as far as I understood): extra_context does not allow extra context, it only allows to override existing context parameters, which must be part of an existing cookiecutter.json / .cookiecutterrc file.

You can see it in the function apply_overwrites_to_context.

def apply_overwrites_to_context(context, overwrite_context):
    """Modify the given context in place based on the overwrite_context."""
    for variable, overwrite in overwrite_context.items():
        if variable not in context:
            # Do not include variables which are not used in the template
            continue
   ....

So this only allows some extra_context, if the variable is already part of the given context.
Therefore it must already be part of a cookiecutter.json file.

This gets invoked by:

    if extra_context:
        apply_overwrites_to_context(obj, extra_context)

As you can see, even the name extra_context becomes overwrite_context.

But maybe this is the supposed behavior and I understood something wrong.

So, is there any other way to mix given user input (cookiecutter.json) with generated, extra context, which must not be part of a cookiecutter.json file (and therefore the user never gets asked about it)?

In my use case, the user shall give input about some project names. But there are some additional parameters, which must be 100% generated and shall not be changeable by the user (during project generation). Examples for such generated parameters: Contact email addresses, database connection strings, timestamps...

Besides this discussion, cookiecutter works like a charm. So pretty much thanks for this great tool!

[hackebrot]

Hi @danwos 👋

Maybe this is what you're looking for, but pass in every variable that

must be 100% generated

to cookiecutter via extra context. 😄

[danwos]

Hey @hackebrot,
yes, this should work. And thanks for the hint! :+1:

So I add really every variable to my cookiecutter.json file.
The variables, which I need to calculate will get an underscore and are added to extra_context.
So cookiecutter does not ask the user for input.

This pretty workaround should be part of the _Calling Cookiecutter Functions From Python_ section.
I think it may solve the needs of some python developers.

[hackebrot]

This pretty workaround should be part of the Calling Cookiecutter Functions From Python section.

Cookiecutter is free and open source software, developed and managed by unpaid volunteers 😉 If you feel that parts of the documentation lack important information, please consider submitting a PR and we'll happily review it.