Virtual-environments: Stronger documentation required

[Sispheor]

Hello guys,

I would like to create my own self hosted runner for Github Action. Following the official doc I felt on this repo.

I would suggest to improve the entry point doc.

A new user has to dig in files to discover by himself that it is based on Packer, the current deployment is only compliant so far with Azure Cloud and powershell is required.

I can see that you have a script that help to deploy resources in Azure and another that wrap Packer but no clear instruction on how to use them. A table with variables name, type, comment and default would help a lot.

For example, a "storage_account" variable is declared but this resource is not created by the first script. Also, no explanation on the usage of this storage account, if it's mandatory, etc...

Thanks for your work.

[maxim-lobanov]

@Sispheor , thank you for your feedback. It is definitely something that we would like to improve.

@alepauly would you like us to take care about it?

[Sispheor]

I can help too.

Could you add a Gitter chat room for this repo?

[alepauly]

thanks @Sispheor, we definitely welcome your help in this area!

Could you add a Gitter chat room for this repo?

I would rather we continue to use this repo as the main place for communicating, along with https://github.community/ when necessary. Too many places to communicate in can dilute where to find info. Thanks for the suggestion though.

[rbellamy]

@Sispheor I've been working on something similar - and almost have a build that will target both Docker and AWS. Then there's https://github.com/nektos/act which uses a Docker image he created by hand-crafting something from this repo. That can be found at https://github.com/nektos/act-environments.

[rbellamy]

One open question I have that fits into this discussion is: what user is used to run the provisioning scripts on Linux? It's clear from the scripts that the user is NOT root, yet I find no indication of what user that might be, or how packer is configured to use that user.

[rbellamy]

I mispoke. _SOME_ of the provisioning scripts use sudo and others don't - making it unclear how a non-root user is supposed to run java-tools.sh for instance, while numerous other scripts DO use sudo.

At the time of this writing, there are 87 provisioning scripts:
```shell script
virtual-environments/images/linux master % ls -l */.sh | wc -l
87

Of which, 21 use `sudo`:
```shell script
virtual-environments/images/linux master % grep -Rl sudo * | uniq | wc -l  
21

[maxim-lobanov]

A few more points that should be mentioned in documentation (thanks to @haokanga):

1. OS Requirement. The script must run in Windows OS because some dependencies only exist on Windows. Symptom: Could not load file or assembly 'System.Windows.Forms, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089'. The system cannot find the file specified. when running on MacOS PowerShell.
2. The ps1 files require dependencies to be installed to the Azure PowerShell such as Install-Module AzureRM -AllowClobber. Symptom: Login-AzureRmAccount : The term 'Login-AzureRmAccount' is not recognized as the name of a cmdlet, function, script file, or operable program.

[haokanga]

I will post the discussion with @maxim-lobanov as in #1175 here about what content is necessary for a minimum viable documentation for the community.

There are 2 major missing parts for the documentation that slow down the community to rebuild the images.
The first is the environment requirements for the ps1 and Packer scripts, the second is a step-by-step instruction on how to manually execute the ps1 and packer scripts.

1. Environment Requirement for the ps1 and Packer scripts (e.g., OS, software, etc.)

I saw the Azure Pipelines that are used to build the images are using a custom Azure Pipeline Agent pool pool: ci-agent-pool https://github.com/actions/virtual-environments/blob/master/images.CI/azure-pipelines/image-generation.yml#L9, which are blackboxes for the users and there can be such hidden dependencies (e.g., AzureRM) as pitfalls.

And as @maxim-lobanov mentioned, the GitHub Actions team configured the image generation CI pool a long time ago so it definitely worth it to revisit the pool environment in order to decide the precise OS Requirement and Software/Dependencies requirement and share the information to the users.

It will be great if the GitHub Actions team can enable the community to run the ps1 and Packer scripts without any major setup overhead. Instead of maintaining yet another README.md to list the environment requirement and letting users to install and configure them, one efficient solution to addressing the OS and dependencies is to provide IaC scripts, so that the community can use the IaC scripts to provision VMs with all the dependencies needed by the ps1 and Packer scripts.

With the IaC scripts, the users will only need to:

• Run the IaC scripts to provision the VM, e.g., terraform apply (or az group deployment create, etc.)
• Connect to the VM
• git clone https://github.com/actions/virtual-environments.git
• Follow a step-by-step instructions documented by GitHub Actions team to manually execute the ps1 or packer scripts

The above IaC scripts will be a good and sufficient start.

One step further in future is provide the community with an solution that automates both the VM creation and the image build process, e.g., by using ARM templates + Azure Custom Script Extension, or Terraform + the remote-exec provisioner.

1. A step-by-step instruction on how to manually execute the ps1 and Packer scripts

The current codebase is fully written in favor of Azure Pipeline automation, and the users have to figure out all the moving parts and effectively "reverse-engineer automation back to manual commands".

So the community needs a step-by-step instructions provided by GitHub Actions team on how to manually execute the ps1 or packer scripts.

[vmapetr]

Hi @Sispheor !
We have merged extended documentation into main branch. Please, feel free to create another issue, if you think that some documentation topics can be added or improved.