Templating: dotnet new -i not reporting errors and exclusions not working as expected

[dguerin]

Doing some work on this PR at the moment. https://github.com/eShopWorld/template-webapi-fabric/pull/23

On my final test to install the template it just bombs out with no errors:

nuget pack C:\[path]\template-webapi-fabric\src\Eshopworld.WebAPIFabric.Template.nuspec

create new dir, navigate to it and run

dotnet new -i C:\[path]\template-webapi-fabric\src\Eshopworld.WebAPIFabric.Template.1.0.8.nupkg

The previous version does work as expected ie: 1.0.7. This is what is also in the master branch and creating this template on my machine runs just fine.

After you check what has been installed with 1.0.8 you are left with just the standard set of templates as outlined here:

Templates                                         Short Name         Language          Tags
----------------------------------------------------------------------------------------------------------------------------
Console Application                               console            [C#], F#, VB      Common/Console
Class library                                     classlib           [C#], F#, VB      Common/Library
Unit Test Project                                 mstest             [C#], F#, VB      Test/MSTest
NUnit 3 Test Project                              nunit              [C#], F#, VB      Test/NUnit
NUnit 3 Test Item                                 nunit-test         [C#], F#, VB      Test/NUnit
xUnit Test Project                                xunit              [C#], F#, VB      Test/xUnit
Razor Page                                        page               [C#]              Web/ASP.NET
MVC ViewImports                                   viewimports        [C#]              Web/ASP.NET
MVC ViewStart                                     viewstart          [C#]              Web/ASP.NET
ASP.NET Core Empty                                web                [C#], F#          Web/Empty
ASP.NET Core Web App (Model-View-Controller)      mvc                [C#], F#          Web/MVC
ASP.NET Core Web App                              webapp             [C#]              Web/MVC/Razor Pages
ASP.NET Core with Angular                         angular            [C#]              Web/MVC/SPA
ASP.NET Core with React.js                        react              [C#]              Web/MVC/SPA
ASP.NET Core with React.js and Redux              reactredux         [C#]              Web/MVC/SPA
Razor Class Library                               razorclasslib      [C#]              Web/Razor/Library/Razor Class Library
ASP.NET Core Web API                              webapi             [C#], F#          Web/WebAPI
global.json file                                  globaljson                           Config
NuGet Config                                      nugetconfig                          Config
Web Config                                        webconfig                            Config
Solution File                                     sln                                  Solution

Examples:
    dotnet new mvc --auth Individual
    dotnet new xunit
    dotnet new --help

Additionally, the exclusions within the https://github.com/eShopWorld/template-webapi-fabric/blob/bc04a29e681554861ccf35d287fc1c1702bbe883/src/.template.config/template.json do not seem to work as expected. I've tried a few varieties of options here and played them within the modifiers array as suggested here https://github.com/dotnet/templating/issues/850 but to no avail.

Any help on both issues would be appreciated.

Thanks
Dave

[dguerin]

Got further on the above.

Exclusions don't work for some reason, really not sure why. I've tried a number of different options with the placement of the template.config relative to the content directory and to the source but it does not seem to obey the config. I've also tried with the modifier json object but still to no avail.

I have however got the nupkg creating and installed as expected. It required deleting all the bin, obj and package folders before we nuget pack it and after that install with dotnet new -i. So some success.

[bartelink]

I case it helps, I used dotnet pack to make the nupkg with exclusions done this way

[dguerin]

@bartelink will certainly give it a try.

[mlorbetske]

Sorry for the delay in responding. With regard to, the bin and obj directories causing the install to fail - if a nupkg passed to dotnet new -i contains DLLs, instead of searching the package for templates, it searches for extensions to dotnet new itself, so having DLLs present in the package will cause this behavior.

I think we should at least give some output during -i to indicate that's what's happening (at least in the case that we're attempting a component installation) - @seancpeters thoughts?

With regard to the exclusions, could you elaborate on what you're trying to do? From the contents of template.json, the source directive is setting the template root to a directory that's a peer to the directory containing template.json (the path in source is relative to one directory up from template.json - in a folder called content), and then attempting to exclude any folder within that's contents that match the names .vs or packages.

[dguerin]

@mlorbetske thanks for the reply.

Re the exclusions:
I was trying to remove the bins, obj directories and anything else that shouldn't be in the nupkg (eg: .vs and packages). Particularity I wanted to exclude the dlls which were breaking the build of the nupkg.

Regarding what the source directive is set to now, I've just left it as what it was in the first place and it does not work currently. Our our CI/CD flow, I'm building the nupkg before I unit test it so it as a "clean" solution, ie: with nothing above I want to exclude.

I've tried different combinations of "things" here to try and get the exclusions to work:

• setting the source to a difference directory.
• setting the exclusions within the modifiers directive.
• removing the source directive and moving the template.json to the child directory.

Any help would be great. A working sample would be great so I can then just fill it out with the reset of the exclusions I need.

[bartelink]

@dguerin kinda reiterating, but the repo I linked to has full scripts that produce correct packages containing two templates (the crux of it all is pretty much the wildcard pattern in the include/exclude list though)

[dguerin]

@bartelink I took a look at that repo when you posted but it's done at the proj level, not with the template.json file. So there is either a bug there or a misunderstanding on our part.

FYI @mlorbetske

As I said above I have a solution already where I'm building the nupkg in a build pipeline before I run any restores/build/tests so I have good clean nupkgs currently.

[bartelink]

The project level controls what goes into the nupkg, which is pretty important, no? i.e. that's where it _should_ be filtered IMO

The repo I linked also has some simple filters based on flags which then exclude some of the stuff in the package at instantiation time - the paths are relative to the location in the package in that context.

[mlorbetske]

Normally template.json isn't related to what goes in to the nuget package - it's a file that indicates the presence of a template and describes how it should be created from the available content. The project or nuspec, depending on your build setup is what determines what actually goes in the package.

[TheAngryByrd]

+1 I'm experiencing not being able to install from a local file path location on macOS

/usr/local/share/dotnet/dotnet  new  -i dist/MiniScaffold.0.14.0-beta001.nupkg
/usr/local/share/dotnet/dotnet  new  mini-scaffold -lang F# -n MyCoolLib --githubUsername CoolPersonNo2

Invalid input switch:
  --githubUsername
  CoolPersonNo2
Unable to determine the desired template from the input template name: mini-scaffold.
The following templates partially match the input. Be more specific with the template name and/or language.

dotnet --info
.NET Core SDK (reflecting any global.json):
 Version:   2.2.104
 Commit:    73f036d4ac

Runtime Environment:
 OS Name:     Mac OS X
 OS Version:  10.14
 OS Platform: Darwin
 RID:         osx.10.14-x64
 Base Path:   /usr/local/share/dotnet/sdk/2.2.104/

Host (useful for support):
  Version: 2.2.2
  Commit:  a4fd7b2c84

[bartelink]

do you need to backtick the # or similar ? brew install powershell && pwsh ;)

[TheAngryByrd]

no that doesn't matter, it's not installing the template from nuget package on disk:

Unable to determine the desired template from the input template name: mini-scaffold.

[mlorbetske]

@TheAngryByrd what you're experiencing may be a different problem (the original was about the bin/obj folders being included in the nupkg due to a misunderstanding about the role of template.json in the packaging process). Does the package you're trying to install contain DLLs? Another common cause of these installation failures is the inclusion of a template.json file within a node_modules folder that has been accidentally included in the nupkg

[TheAngryByrd]

So i just reinstalled the latest sdk and the problem has miraculously gone away ¯_(ツ)_/¯

[dguerin]

So i just reinstalled the latest sdk and the problem has miraculously gone away ¯_(ツ)_/¯

Cheers for the headups @TheAngryByrd, I'll give that a shot.

Sorry I haven't been as active on this as I wanted to be. Working on other stuff at the moment but I'll get back to this after.

[TheAngryByrd]

So back to this not working for me. Seems similar to https://github.com/dotnet/templating/issues/1773

[vlada-shubina]

The issue will be fixed with https://github.com/dotnet/templating/issues/2745, when implementing that epic install-uninstall was majorly reworked.

The error messages and the result of installation will be now shown to user. If installation was not successful, the error and the reason will be shown to user.

Draft example:

> dotnet  new  -i Take.Blip.Client.Templates::0.50.10 --nuget-source https://api.nuget.org/v3/index.json
The following template sources will be installed:
  Take.Blip.Client.Templates, version: 0.50.10

Warning: Take.Blip.Client.Templates::0.50.10 is not found in NuGet feeds https://api.nuget.org/v3/index.json.
'Take.Blip.Client.Templates::0.50.10' could not be installed, the package does not exist.

If installation was successful, the list of installed templates will be shown to user:

dotnet new3 -i Microsoft.Azure.WebJobs.ItemTemplates
The following sources will be installed:
  Microsoft.Azure.WebJobs.ItemTemplates

The template source Microsoft.Azure.WebJobs.ItemTemplates::3.1.1648 was successfully installed.
The following templates were installed:
Template Name                  Short Name  Language  Tags
-----------------------------  ----------  --------  ----------------------------------------------
DurableFunctionsOrchestration  durable     [C#]      Azure Function/Durable Functions Orchestration
SendGrid                       sendgrid    [C#]      Azure Function/Ouput/SendGrid
BlobTrigger                    blob        [C#],F#   Azure Function/Trigger/Blob
CosmosDBTrigger                cosmos      [C#],F#   Azure Function/Trigger/Cosmos DB
EventGridTrigger               eventgrid   [C#]      Azure Function/Trigger/EventGrid
EventHubTrigger                eventhub    [C#],F#   Azure Function/Trigger/EventHub
HttpTrigger                    http        [C#],F#   Azure Function/Trigger/Http
SignalRTrigger                 signalr     [C#]      Azure Function/Trigger/Http/SignalR
IotHubTrigger                  iothub      [C#]      Azure Function/Trigger/IotHub
RabbitMQTrigger                rqueue      [C#]      Azure Function/Trigger/RabbitMQ Queue
ServiceBusQueueTrigger         squeue      [C#]      Azure Function/Trigger/Service Bus/Queue
ServiceBusTopicTrigger         stopic      [C#]      Azure Function/Trigger/Service Bus/Topic
QueueTrigger                   queue       [C#]      Azure Function/Trigger/Storage Queue
TimerTrigger                   timer       [C#],F#   Azure Function/Trigger/Timer

[vlada-shubina]

Above was implemented and merged to main branch, only list of installed templates is shown to users now, closing.

[vlada-shubina]

Above was implemented and merged to main branch, only list of installed templates is shown to users now, closing.