Skip to content

Changes

Page history
Update On Demand VMs authored by Piotr Szubiakowski's avatar Piotr Szubiakowski
Hide whitespace changes
Inline Side-by-side
ELT-Virtual-Machines/On-Demand-VMs.md
View page @ 28d266e3
**For the old version of the document compatible with DevEnv 3, please take a look at [Deprecated DevEnv 3 Documentation](https://gitlab.eso.org/ecs/eltsw-docs/-/wikis/ELT-Virtual-Machines/On-Demand-VMs?version_id=7da7939889ab505ee0973be3a20c380a56c05885)**
[[_TOC_]]
# Overview
Vagrant boxes containing the ELT DevEnv are provided to create on-demand virtual machines (VMs) for development and testing purposes. Currently the boxes are using VirtualBox as the virtualisation middleware, which allows one to create virtual machine instances locally on a laptop or workstation.
These boxes allow a developer to create, modify and destroy virtual machine instances in a simple manner using the Vagrant command line tool.
Since the VM is owned exclusively by the developer, they have full admin rights (i.e. passwordless sudo) and can modify the VM's and OS's configuration as required.
If something goes wrong with the modifications, Vagrant allows to quickly delete the broken VM and create a clean new instance.
## Vagrant help and documentation
Extensive documentation for installing and using the Vagrant tool can be found under https://www.vagrantup.com/docs.
ESO IT typically handles installation of Vagrant on ESO machines if this is not already available.
In addition, useful help information is provided on the command line by using the `--help` option as follows:
```
vagrant --help
```
For additional help about specific Vagrant commands (e.g. the _up_ command) add the `--help` option after the command name as follows:
```
vagrant up --help
```
# VirtualBox VMs
## Prerequisites
The following software must be installed and working properly on the laptop or workstation where the VirtualBox based Vagrant boxes will be instantiated and started:
* VirtualBox (https://www.virtualbox.org/wiki/Downloads)
* Vagrant (https://www.vagrantup.com/downloads)
Please follow the installation instructions for the respective 3rd party products listed above, if these are not already available. For ESO laptops and workstations, ESO IT can be contacted to install these if they are not already available.
**WARNING:** Do not install the VirtualBox Extension Pack on ESO machines.
This requires an additional license that ESO currently does not have.
Only the core VirtualBox product is under the GPL version 2 license and freely installable on ESO machines.
## Fetching the VM image
Before instantiating a new VirtuaBox VM, one needs to download the box and register it with Vagrant.
This is the only step that requires network access.
Once the Vagrant box is downloaded, any number of VM instances can be created on the local host without requiring network connection.
The command to download a Vagrant box for VirtualBox onto the local host is as follows:
```
vagrant box add eltdev/eltdev-4 --provider=virtualbox
```
Note that this step can take a long time, since a complete OVA image of the virtual machine with preinstalled OS and ELT software is downloaded to the local host.
However, once downloaded this step does not need to be repeated.
If a specific version of the ELT DevEnv is needed, e.g. 4.1.0, then the `--box-version` option can be used as follows:
```
vagrant box add eltdev/eltdev-4 --provider=virtualbox --box-version=4.1.0
```
To see a list of available box versions simply view https://app.vagrantup.com/eltdev in a web browser.
## Creating a new VM instance
Once a box is downloaded and registered with Vagrant, a new instance can be created.
This is done by first creating a new working directory and moving to that directory as follows:
```
mkdir testbox
cd testbox
```
Note that the created VM in VirtualBox will derive its name from the directory name.
A file named _Vagrantfile_ must be created within the working directory.
This file can be created and edited manually (see the Vagrant documentation for the supported syntax),
or alternatively a default version can be created with the following command:
```
vagrant init eltdev/eltdev-4
```
Note that the generated _Vagrantfile_ can be modified after the above command is run to customise the VM configuration before creating it.
If multiple versions of the _eltdev_ box have been downloaded and registered with Vagrant it may be necessary to select a specific version with the following example command:
```
vagrant init eltdev --box-version 4.1.0
```
The next step is to run the following command that creates the new VM instance and boots the OS:
```
vagrant up
```
Note that this step may take a significant amount of time the first time this command is run,
since it will import the OVA VM image into VirtualBox.
All subsequent invocations of `vagrant up` for this box version should complete in a mater of 30-60 seconds,
since the linked-clone mechanism is used to reuse the disk image of the first imported VM in a copy-on-write manner.
If Vagrant was not installed from the downloads provided by https://www.vagrantup.com,
but is provided by the OS distribution itself,
and the `vagrant up` command fails,
it may be necessary to explicitly specify that VirtualBox should be used as follows:
```
vagrant up --provider=virtualbox
```
## Logging into the VM
When the VM instance has fully booted it is possible to SSH into the VM with the following command:
```
vagrant ssh
```
X11 forwarding can be enabled for the SSH connection by using the following command instead:
```
vagrant up -- -X
```
Any additional SSH options can be explicitly added to the command line after the `--`.
To view the GUI console of the VM, i.e. what would be seen on a physically attached screen, one should start the VirtualBox GUI application itself and navigate to the corresponding VM instance.
A button called _Show_ should appear at the top, alternatively this button can be found in the context menu by right clicking on the relevant VM.
By clicking on this button the VM's console window should appear.
## Copying files to and from the VM
For convenience the working directory (_testbox_ in the above examples) is mounted as a VirtualBox shared folder under _/vagrant_ in the guest OS in the VM.
Therefore copying files into the VM can be done by moving files into the working directory on the host machine, and then accessing these files from within the VM under _/vagrant_.
Any file created under the _/vagrant_ path within the guest OS in the VM will be automatically reflected in the working directory on the host machine.
Therefore by copying a file within the VM to _/vagrant_ one can then easily access the file on the host.
Note that as long as a network is available,
one can also exchange files by uploading or downloading files to an FTP or other cloud based storage solution.
## Deleting the VM and cleanup
Once the virtual machine instance is no longer needed it can be destroyed with the following command:
```
vagrant destroy
```
This will ask the user if they are sure and if the answer is yes,
then Vagrant will attempt to gracefully shutdown the VM and delete the associated disk files.
To avoid the interactivity one can use the `--force` option to destroy the VM immediately and without requiring to provide confirmation. For example:
```
vagrant destroy --force
```
The Vagrantfile in the working directory can be deleted only after the VM has been destroyed.
Note that Vagrant creates a _.vagrant_ hidden directory in the working directory where Vagrantfile is placed.
This directory should also not be deleted before the VM is destroyed.
## Unregistering a box
If no instances exist and no new instances of a VirtualBox based Vagrant box shall be created for a particular box version, the box can be deregistered from Vagrant and VirtualBox to recover disk space.
Since the linked-clones mechanism is used with Vagrant and VirtualBox,
one first needs to delete the base VM instance in VirtualBox that Vagrant creates.
To identify this instance look at the contents of the _master_id_ file for the corresponding Vagrant box under the _$VAGRANT_HOME_ directory.
This usually defaults to _~/.vagrant.d/_.
As an example, for a default installation and configuration of Vagrant, the location of the _master_id_ file for the _eltdev_ box version 4.1.0 is found under:
```
~/.vagrant.d/boxes/eltdev-VAGRANTSLASH-eltdev-4/4.1.0/virtualbox/master_id
```
This file will contain a UUID that can be passed to the VirtualBox `VBoxManage` command as follows:
```
VM_UUID=$(cat ~/.vagrant.d/boxes/eltdev-VAGRANTSLASH-eltdev-4/4.1.0/virtualbox/master_id)
VBoxManage unregistervm $VM_UUID --delete
```
Another useful command to list all VMs registered with VirtualBox is the following:
```
VBoxManage list vms
```
It is also possible to delete the base VM instance using the VirtuaBox GUI application.
Once the base VM is unregistered with VirtualBox, the box can then be unregistered with Vagrant.
This is done with the following example command to unregister a specific version of the box:
```
vagrant box remove eltdev --provider=virtualbox --box-version=4.1.0
```
Note that the box version must correspond to the one that was unregistered earlier from VirtualBox.
The following may be a useful command to list all currently registered boxes:
```
vagrant box list
[[_TOC_]]
# Overview
Vagrant boxes containing the ELT DevEnv are provided to create on-demand virtual machines (VMs) for development and testing purposes. Currently the boxes are using VirtualBox as the virtualisation middleware, which allows one to create virtual machine instances locally on a laptop or workstation.
These boxes allow a developer to create, modify and destroy virtual machine instances in a simple manner using the Vagrant command line tool.
Since the VM is owned exclusively by the developer, they have full admin rights (i.e. passwordless sudo) and can modify the VM's and OS's configuration as required.
If something goes wrong with the modifications, Vagrant allows to quickly delete the broken VM and create a clean new instance.
## Vagrant help and documentation
Extensive documentation for installing and using the Vagrant tool can be found under https://www.vagrantup.com/docs.
ESO IT typically handles installation of Vagrant on ESO machines if this is not already available.
In addition, useful help information is provided on the command line by using the `--help` option as follows:
```
vagrant --help
```
For additional help about specific Vagrant commands (e.g. the _up_ command) add the `--help` option after the command name as follows:
```
vagrant up --help
```
# VirtualBox VMs
## Prerequisites
The following software must be installed and working properly on the laptop or workstation where the VirtualBox based Vagrant boxes will be instantiated and started:
* VirtualBox (https://www.virtualbox.org/wiki/Downloads)
* Vagrant (https://www.vagrantup.com/downloads)
Please follow the installation instructions for the respective 3rd party products listed above, if these are not already available. For ESO laptops and workstations, ESO IT can be contacted to install these if they are not already available.
**WARNING:** Do not install the VirtualBox Extension Pack on ESO machines.
This requires an additional license that ESO currently does not have.
Only the core VirtualBox product is under the GPL version 2 license and freely installable on ESO machines.
## Fetching the VM image
Before instantiating a new VirtuaBox VM, one needs to download the box and register it with Vagrant.
This is the only step that requires network access.
Once the Vagrant box is downloaded, any number of VM instances can be created on the local host without requiring network connection.
The command to download a Vagrant box for VirtualBox onto the local host is as follows:
```
vagrant box add eltdev/eltdev-4 --provider=virtualbox
```
Note that this step can take a long time, since a complete OVA image of the virtual machine with preinstalled OS and ELT software is downloaded to the local host.
However, once downloaded this step does not need to be repeated.
If a specific version of the ELT DevEnv is needed, e.g. 4.1.0, then the `--box-version` option can be used as follows:
```
vagrant box add eltdev/eltdev-4 --provider=virtualbox --box-version=4.1.0
```
To see a list of available box versions simply view https://app.vagrantup.com/eltdev in a web browser.
## Creating a new VM instance
Once a box is downloaded and registered with Vagrant, a new instance can be created.
This is done by first creating a new working directory and moving to that directory as follows:
```
mkdir testbox
cd testbox
```
Note that the created VM in VirtualBox will derive its name from the directory name.
A file named _Vagrantfile_ must be created within the working directory.
This file can be created and edited manually (see the Vagrant documentation for the supported syntax),
or alternatively a default version can be created with the following command:
```
vagrant init eltdev/eltdev-4
```
Note that the generated _Vagrantfile_ can be modified after the above command is run to customise the VM configuration before creating it.
If multiple versions of the _eltdev_ box have been downloaded and registered with Vagrant it may be necessary to select a specific version with the following example command:
```
vagrant init eltdev --box-version 4.1.0
```
The next step is to run the following command that creates the new VM instance and boots the OS:
```
vagrant up
```
Note that this step may take a significant amount of time the first time this command is run,
since it will import the OVA VM image into VirtualBox.
All subsequent invocations of `vagrant up` for this box version should complete in a mater of 30-60 seconds,
since the linked-clone mechanism is used to reuse the disk image of the first imported VM in a copy-on-write manner.
If Vagrant was not installed from the downloads provided by https://www.vagrantup.com,
but is provided by the OS distribution itself,
and the `vagrant up` command fails,
it may be necessary to explicitly specify that VirtualBox should be used as follows:
```
vagrant up --provider=virtualbox
```
## Logging into the VM
When the VM instance has fully booted it is possible to SSH into the VM with the following command:
```
vagrant ssh
```
X11 forwarding can be enabled for the SSH connection by using the following command instead:
```
vagrant up -- -X
```
Any additional SSH options can be explicitly added to the command line after the `--`.
To view the GUI console of the VM, i.e. what would be seen on a physically attached screen, one should start the VirtualBox GUI application itself and navigate to the corresponding VM instance.
A button called _Show_ should appear at the top, alternatively this button can be found in the context menu by right clicking on the relevant VM.
By clicking on this button the VM's console window should appear.
## Copying files to and from the VM
For convenience the working directory (_testbox_ in the above examples) is mounted as a VirtualBox shared folder under _/vagrant_ in the guest OS in the VM.
Therefore copying files into the VM can be done by moving files into the working directory on the host machine, and then accessing these files from within the VM under _/vagrant_.
Any file created under the _/vagrant_ path within the guest OS in the VM will be automatically reflected in the working directory on the host machine.
Therefore by copying a file within the VM to _/vagrant_ one can then easily access the file on the host.
Note that as long as a network is available,
one can also exchange files by uploading or downloading files to an FTP or other cloud based storage solution.
## Deleting the VM and cleanup
Once the virtual machine instance is no longer needed it can be destroyed with the following command:
```
vagrant destroy
```
This will ask the user if they are sure and if the answer is yes,
then Vagrant will attempt to gracefully shutdown the VM and delete the associated disk files.
To avoid the interactivity one can use the `--force` option to destroy the VM immediately and without requiring to provide confirmation. For example:
```
vagrant destroy --force
```
The Vagrantfile in the working directory can be deleted only after the VM has been destroyed.
Note that Vagrant creates a _.vagrant_ hidden directory in the working directory where Vagrantfile is placed.
This directory should also not be deleted before the VM is destroyed.
## Unregistering a box
If no instances exist and no new instances of a VirtualBox based Vagrant box shall be created for a particular box version, the box can be deregistered from Vagrant and VirtualBox to recover disk space.
Since the linked-clones mechanism is used with Vagrant and VirtualBox,
one first needs to delete the base VM instance in VirtualBox that Vagrant creates.
To identify this instance look at the contents of the _master_id_ file for the corresponding Vagrant box under the _$VAGRANT_HOME_ directory.
This usually defaults to _~/.vagrant.d/_.
As an example, for a default installation and configuration of Vagrant, the location of the _master_id_ file for the _eltdev_ box version 4.1.0 is found under:
```
~/.vagrant.d/boxes/eltdev-VAGRANTSLASH-eltdev-4/4.1.0/virtualbox/master_id
```
This file will contain a UUID that can be passed to the VirtualBox `VBoxManage` command as follows:
```
VM_UUID=$(cat ~/.vagrant.d/boxes/eltdev-VAGRANTSLASH-eltdev-4/4.1.0/virtualbox/master_id)
VBoxManage unregistervm $VM_UUID --delete
```
Another useful command to list all VMs registered with VirtualBox is the following:
```
VBoxManage list vms
```
It is also possible to delete the base VM instance using the VirtuaBox GUI application.
Once the base VM is unregistered with VirtualBox, the box can then be unregistered with Vagrant.
This is done with the following example command to unregister a specific version of the box:
```
vagrant box remove eltdev --provider=virtualbox --box-version=4.1.0
```
Note that the box version must correspond to the one that was unregistered earlier from VirtualBox.
The following may be a useful command to list all currently registered boxes:
```
vagrant box list
```
\ No newline at end of file