README.md

<!--
// © University of Southampton IT Innovation Centre, 2017
//
// Copyright in this software belongs to University of Southampton
// IT Innovation Centre of Gamma House, Enterprise Road,
// Chilworth Science Park, Southampton, SO16 7NS, UK.
//
// This software may not be used, sold, licensed, transferred, copied
// or reproduced in whole or in part in any manner or form or in or
// on any media by any person other than in accordance with the terms
// of the Licence Agreement supplied with the software, or otherwise
// without the prior written consent of the copyright owners.
//
// This software is distributed WITHOUT ANY WARRANTY, without even the
// implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR
// PURPOSE, except where stated in the Licence Agreement supplied with
// the software.
//
//      Created By :            Michael Boniface
//      Created Date :          18-12-2017
//      Created for Project :   FLAME
-->

# FLAME Cross Layer Management and Control
#### Version: 1.0.0

### About this document

#### Authors

|Authors|Organisation|
|-|-|
|[Michael Boniface](mailto:mjb@it-innovation.soton.ac.uk)|[University of Southampton, IT Innovation Centre](http://www.it-innovation.soton.ac.uk)|
|[Simon Crowle](mailto:sgc@it-innovation.soton.ac.uk)|[University of Southampton, IT Innovation Centre](http://www.it-innovation.soton.ac.uk)|

### Contents

#### Documentation

Implementation documentation and discussion can be found in the docs directory.

#### Testing

Testing is implemented using pytest using the following convention:

* The testing environment is Vagrant/Virtualbox for a base VM with LXD is installed for specific containers.
* Tests are written in python using pytest
* Related tests are stored in a python module `src/test/clmctest/<testmodule>` to create a suite of tests. All tests are stored in files test_*.py, there can be many tests per file, and many files per module
* Tests are executed against a set of LXC containers described in `src/test/clmctest/rspec.json`.
* Tests are executed from the test-runner container on the VM using install python modules
* CI testing is on givry using nested LXD containers

Create a single VM with LXC installed and configured with lxcbr0 configured for the network 172.40.231.0/24

```shell
vagrant up
```

SSH into the VM:

```shell
vagrant ssh
```

The containers are controlled using a script called `/vagrant/scripts/test/fixtures.sh`

```shell
Usage: fixture.sh create|start|stop|destroy [-f config_file] [-r repo_root] [-c container_name|all]
```

The containers created are defined an rspec.json file, there's an example here: `/vagrant/src/test/clmctest/rspec.json`. The `fixtures.sh` script defaults to look for a `rspec.json` in the current directory, you can specify a specific `rspec.json` file using the `-f` option.

To create|start|stop|destroy specific services use the `-c` option e.g.

```shell
/vagrant/scripts/test/fixture.sh create -f /vagrant/src/test/clmctest/rspec.json -c clmc-service
```

The installation of several of the services depend on accessing the Nexus binary repository (for the custom Telegraf agent). To do this, a username and password for the repository must be specified in a `reporc` file in the user's home directory, e.g.

```shell
REPO_USER=itinnov.flame.integration
REPO_PASS=xxxxxxx
```

Create all the services needed for integration tests:

```shell
sudo su
/vagrant/scripts/test/fixture.sh create -f /vagrant/src/test/clmctest/rspec.json -c all
```

As part of the clmc-service installation, the service's unit tests have been run. The fixture script will fail if any individual service installation fails to install (or fails its tests).

Attach to the test-runner to run the tests

```shell
lxc-attach -n test-runner
```

Build and install the CLMC test Python module:

```shell
cd /vagrant/src/test
python setup.py sdist --dist-dir=../../build
pip3 install /vagrant/build/clmctest-<version>.tar.gz
```

The following module is unit tests:

```shell
pytest -s --pyargs clmctest.scripts
```

The following modules are integration tests:

```shell
pytest -s --pyargs clmctest.inputs
pytest -s --pyargs clmctest.monitoring
```

#### CI Testing


A lxd container is setup on givry called `flame-clmc-ci`. The container is priviledged and allows for nested containers. The container was created using the following commands

```
lxc launch ubuntu:18.04 flame-clmc-ci -c security.privileged=true -c security.nesting=true
```

the container is then started and LXD initialised

```
lxc exec flame-clmc-ci -- bash
lxd init --auto --storage-backend dir 
lxc network create lxcbr0 ipv6.address=none ipv4.address=172.40.231.1/24 ipv4.nat=true

```

Now configure ip table persistence, this is needed for the fixtures script to work but not for CI

```
apt update
echo iptables-persistent iptables-persistent/autosave_v4 boolean true | sudo debconf-set-selections
echo iptables-persistent iptables-persistent/autosave_v6 boolean true | sudo debconf-set-selections
apt -y install iptables-persistent
```

The install git runner file was just copied to the container from a tmp director

```
lxc file push /home/mjb/dev/flame-clmc/scripts/test/install-git-runner.sh flame-clmc-ci/root/
lxc exec flame-clmc-ci -- chmod 755 ./install-git-runner.sh
lxc exec flame-clmc-ci -- ./install-git-runner.sh
```

Note that the branch must be "protected" for the secret repo rc variables to be passed to the CI script