Ptvs: Integrating with distutils/setuptools

[fxthomas]

Hello all,

Is there a clean and recommended way to integrate distutils/setuptools in a Visual Studio Python project?

I'm specifically looking at this from the context of a Python library developed under Visual Studio and distributed according to the Python Packaging Guidelines.

Ideally, any setup.py command could be run from Visual Studio, including OTOH :

• bdist_wheel to generate a Wheel package
• bdist_wininst to generate a Windows installer
• test for integrating unit/integration test frameworks (e.g. PyTest)

This is a hack, but adding this custom AfterBuild target in the .pyproj file generates a Windows installer when clicking on _Rebuild project_ :

<Target Name="AfterBuild">
  <RunPythonCommand Target="setup.py" TargetType="script" Arguments="bdist_wininst" />
</Target>

[huguesv]

This surprisingly doesn't come up often.

It could potentially be done by adding commands to the Python menu that appears when you right-click a Python project node in solution explorer.

As an example, the pylint command is defined in PythonRunPyLintCommand target of Microsoft.PythonTools.targets, I think that one is pretty similar to what you need. We have more examples of commands that are available in web projects, those are defined in Microsoft.PythonTools.Web.targets.

I suspect you might already be familiar with those, given that you've already discovered RunPythonCommand.

You could define them in your own .targets file that you import in all your projects.

[fxthomas]

Thanks for answering, and for the custom menu item suggestion, I was able to piece together how RunPythonCommand worked but did not notice I could integrate it that way!

Here are a few more questions I have that are related to distributing and developing this Python package, hoping you don't mind the huge wall of text :

• The Python library is part of a larger solution. When my custom AfterBuild target works, it doesn't react to code changes : I always need to click "Rebuild All" to build a wheel/installer package, otherwise it just says "up-to-date". Is there a way I can tell MSBuild to check that?
• Can I use the output of the Python project in another project of the same solution? I'm thinking of an installer project in this case.
• When developing the library, I need to either manually edit the Python search path inside my scripts (environments don't respect the project settings), or manually install my package in "development" mode with pip install --editable <path to project>. Otherwise, even Python environments started from VS cannot import my library out of the box. Any way to make that better?
• This is not that important in my case (yet), but there doesn't seem to be a way to generate multiple build outputs for multiple Python versions or architectures. Did I miss something?

[int19h]

With respect to rebuild - does your target define Inputs and Outputs? MSBuild is supposed to use those for up-to-date checks.

[fxthomas]

@int19h Ah, indeed. I'm not very familiar with MSBuild, I tried something like this without much success (blindly copy-pasting the @(Compile) from other projects) :

<Target Name="AfterBuild" Inputs="@(Compile)" Outputs="$(ProjectDir)dist\package-1.0.0-win64.exe">
  <RunPythonCommand Target="setup.py" TargetType="script" Arguments="bdist_wininst" />
</Target>

Any tips?

[int19h]

The snippet above is missing a quote in Outputs, and should result in the project not being loadable due to being invalid XML. If it's a copy/pasted snippet, then perhaps you were accidentally editing the wrong file?

If that's not the case, try setting MSBuild verbosity to diagnostic (Tools -> Options -> Projects and Solutions -> Build and Run) - build output should then be detailed enough to see when it was considering your target, and why exactly it wasn't run.

@(Compile) should be fine - it basically makes it depend on all project items that are defined as <Compile Include="...">, which should include all .py files by default.

[fxthomas]

Thanks. I'll try to increase verbosity and see what happens! The missing quote was just me cutting the project name out of the pasted text :)

[fxthomas]

Thanks for all the help and suggestions! Here is a more complete example that works : https://gist.github.com/fxthomas/5c601e3e0c1a091bcf56aed0f2960cfa. It's not perfect, but it's much closer to what I initially had in mind.

It works really well with Python 2.7 ; the last thing that's missing is that I'm getting this weird issue with Python 3.6. Do you have an idea?

------ Build started: Project: xxx, Configuration: Debug Any CPU ------
    xxxxx -> 
C:\...\...\xxxxxx.pyproj(133,5): error MSB4018: The "RunPythonCommand" task failed unexpectedly.
System.ComponentModel.Win32Exception (0x80004005): The parameter is incorrect
   at System.Diagnostics.Process.StartWithCreateProcess(ProcessStartInfo startInfo)
   at System.Diagnostics.Process.Start()
   at System.Diagnostics.Process.Start(ProcessStartInfo startInfo)
   at Microsoft.PythonTools.BuildTasks.RunPythonCommand.Execute()
   at Microsoft.Build.BackEnd.TaskExecutionHost.Microsoft.Build.BackEnd.ITaskExecutionHost.Execute()
   at Microsoft.Build.BackEnd.TaskBuilder.<ExecuteInstantiatedTask>d__26.MoveNext()
Done building project "xxxxx.pyproj" -- FAILED.

Build FAILED.
========== Build: 0 succeeded, 1 failed, 0 up-to-date, 0 skipped ==========

[zooba]

Are you working in Visual Studio 2015? There are changes required for how .pyproj files import the default targets in Visual Studio 2017 that I don't see in your gist.

If so, Python 3.6 isn't supported in Visual Studio 2015 and we are no longer updating its Python support for anything that isn't significantly world-breaking. That's a weird and unfortunate error message, but ultimately there's not much we can do about it now.

Though if you've configured a custom environment for it and haven't set the path environment variable to PYTHONPATH, that may cause the issue (as it'll end up with an environment variable with no name) - setting that property in your configured environment should fix it.

Also, great work on those commands! They look real nice.

[fxthomas]

Sorry for getting back late, and thanks again for the reply. Unfortunately switching to VS 2017 is not an option at the moment (part of a bigger solution that needs migrating), but the lack of 3.6 support in VS 2015 would explain why this doesn't work. At least I'm aware of it!

[kraigb]

@fxthomas I'm in the process of finishing up a docs article on creating custom commands. Would it be OK with you if I copied the commands you have in https://gist.github.com/fxthomas/5c601e3e0c1a091bcf56aed0f2960cfa as examples? (With attribution, of course.)

[fxthomas]

@kraigb No problem, go ahead!

Note that the custom menu items are great, but I ended up disabling the build integration : the Build vs. Rebuild issue I alluded to in earlier comments was in fact not fully fixed, and I did not have time to find out why.

[kraigb]

@fxthomas Thanks. The article is on https://docs.microsoft.com/en-us/visualstudio/python/defining-custom-python-project-commands now; I'll be adding your examples shortly.

[fxthomas]

There's a ton of useful info that I didn't cover in my gist, thanks for the detailed docs! :)