Appendix: Azure Build Agent
An important part of the CI/CD process for the wwt-windows-client repository is to build the WWT installer MSI program. At the moment (July 2020), the Azure Pipelines Microsoft-hosted “agents” do not come equipped with the tooling needed to build such an installer. So, annoyingly, we need to create and run our own Windows VM to run these builds. This Appendix documents how such an agent is set up.
- Microsoft VM scale set agent docs
- Microsoft self-hosted agent docs
- Microsoft Visual Studio in VM docs
- Stack Overflow #1
Initialize Template Machine🔗
It doesn’t seem that there's any good way to set up the VM image automatically. So, the first step is to create a cloud VM and set it all up by hand.
- First-time setup steps, if needed:
- Install Azure command line tool
az loginif needed to log in to the WWT Azure subscription.
az account list -o tableto list logged-in subscriptions
az account set -s $SUBSCRIPTION_IDto set the default subscription
az group create --location westus --name devops-supportto create a containing Resource Group.
- Install Azure command line tool
- Create the VM:
The password must be at least twelve characters, one upper/lower/number/special. The machine name cannot be more than 15 characters. For the setup phase, it’s nice to use a beefier machine image.
az vm create \ --resource-group devops-support \ --name hostedagentbox \ --image Win2019Datacenter \ --size Standard_D2_v3 \ --admin-username wwt \ --admin-password $PASSWORD
- Note the public IP address that is printed out by the creation step.
It should now be possible to RDP into the machine. On Linux, something like:
xfreerdp /u:wwt /v:$IP_ADDR:3389 /size:1280x960
or, if you're on a HiDPI display, perhaps with:
... /size:2560x1600 /scale:180
In one instance it looked like the program hung, but I just needed to hit Enter (?). Now the graphical setup can begin:
- If prompted upon login, turn off network discoverability.
- In the Server Manager, go to "Local Server", then click on "IE Enhanced Security Configuration". Turn the settings to off. Otherwise, IE literally won't allow you to download files without jumping through a lot of hoops. (Ref).
- It seems to now be necessary to install Microsoft Edge in order to download VS2019. Go to the Microsoft Edge download page and install it. (Note: copy-paste of links into the RDP screen should work!) Select "Don't use" when IE prompts about adopting recommended security policies.
- Now use Edge to navigate to the Visual Studio 2019 Community downloads page. Login to a Microsoft account will be necessary; password copy-paste out of the host machine FTW.
- Download the
- Rename that file to just
vs_community.exe(just to make copy-paste easier in the next step).
- Run it in an elevated Powershell (which should be the default kind, since
we’re logged in as the admin user):
This command cribbed from the Visual Studio on a VM docs. I am told that it’s important to not start up the graphical interface to avoid triggering Visual Studio from wanting a license. If you typed it all right, the installer will run and take a while as it downloads several gigs of files. If you made a mistake, it will start up and exit pretty quickly without displaying a clear error message.
.\vs_community.exe --allWorkloads --includeRecommended --passive ` --add Microsoft.Net.Component.4.7.SDK ` --add Microsoft.Net.Component.4.7.TargetingPack ` --add Microsoft.Net.Component.4.6.2.SDK ` --add Microsoft.Net.Component.4.6.2.TargetingPack ` --add Microsoft.Net.ComponentGroup.4.7.DeveloperTools
- Reboot the machine after the install. (Not 100% sure but I strongly suspect this is necessary.)
- Download the Microsoft Visual Studio Installer Projects Visual
Studio extension. If you download with IE instead of Edge, it lands with a
.zipextension rather than
- Install the extension from the command line:
&"C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\Common7\IDE\VSIXInstaller.exe" InstallerProjects.vsix
Convert the VM to work in a scale set🔗
Now we start following the Microsoft VM scale set agent docs.
- “Generalize” the online template VM:
This will shut down the VM. The docs suggest that the process might take a very long time, but in at least some cases it’s pretty quick.
&"C:\Windows\System32\sysprep\sysprep.exe" /generalize /oobe /shutdown
- Deallocate the VM:
az vm deallocate --resource-group devops-support --name hostedagentbox
- Tell Azure to generalize it:
az vm generalize --resource-group devops-support --name hostedagentbox
- Create a VM image out of it:
az image create \ --resource-group devops-support \ --source hostedagentbox \ --name hostedagent-$YYYYMM
- Turn it into a VM scale set. Note that this command seems to work even in
steady-state, when the scale set already exists:
Note: I'd like to use the
az vmss create \ --resource-group devops-support \ --name agent-vmss \ --instance-count 2 \ --disable-overprovision \ --upgrade-policy-mode manual \ --load-balancer "" \ --vm-sku Standard_D2s_v3 \ --admin-username wwt \ --admin-password $PASSWORD \ --image hostedagent-$YYYYMM
Standard_D2_v3VM SKU, which doesn't call for fancy "premium disk", but when I tried that I got an error about it being required anyway.
- Finally, delete the VM:
az vm delete -g devops-support -n hostedagentbox
If you want to do a test build in the VM, you’ll need to apply the fix for the HRESULT = 8000000A error:
cd "C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\Common7\IDE\CommonExtensions\Microsoft\VSI\DisableOutOfProcBuild" .\DisableOutOfProcBuild.exe
We need to apply the same fix on-the-fly in the VMs since the fix is user-specific and the Azure provisioning uses a different user.
If you want the VM to to act as an Azure Pipelines build agent on its own, follow the Microsoft self-hosted agent docs. However, we’re setting up the image to run in a VM scale set, in which case the agent will automatically be provisioned upon VM boot. So for the main workflow you should not install the agent.
If setting up from scratch, after creating the scale set you need to follow the rest of the instructions to wire it up to Azure Pipelines. The agent pool is currently called "Custom Windows".