Skip to content
Snippets Groups Projects
Commit 2f77a2f5 authored by Michael Boniface's avatar Michael Boniface
Browse files

Revert "Merge branch 'pm_dev' into 'integration'" 

This reverts merge request !85
parent 4f266893
No related branches found
No related tags found
No related merge requests found
Show whitespace changes
Inline Side-by-side
README.md
+ 9
− 15
View file @ 2f77a2f5
...@@ -50,8 +50,6 @@ Testing is implemented using pytest using the following convention: ...@@ -50,8 +50,6 @@ Testing is implemented using pytest using the following convention:
* Tests are executed from the test-runner container on the VM using install python modules * Tests are executed from the test-runner container on the VM using install python modules
* CI testing is on givry using nested LXD containers * CI testing is on givry using nested LXD containers
##### Create VM
Create a single VM with LXC installed and configured with lxcbr0 configured for the network 172.40.231.0/24 Create a single VM with LXC installed and configured with lxcbr0 configured for the network 172.40.231.0/24
```shell ```shell
...@@ -78,22 +76,18 @@ To create|start|stop|destroy specific services use the `-c` option e.g. ...@@ -78,22 +76,18 @@ To create|start|stop|destroy specific services use the `-c` option e.g.
/vagrant/scripts/test/fixture.sh create -f /vagrant/src/test/clmctest/rspec.json -c clmc-service /vagrant/scripts/test/fixture.sh create -f /vagrant/src/test/clmctest/rspec.json -c clmc-service
``` ```
##### Create `reporc` file
The installation of several of the services depend on accessing the Nexus binary repository (for the custom Telegraf agent). To do this, a username and password for the repository must be specified in a `reporc` file in the user's home directory, e.g. The installation of several of the services depend on accessing the Nexus binary repository (for the custom Telegraf agent). To do this, a username and password for the repository must be specified in a `reporc` file in the user's home directory, e.g.
```shell ```shell
REPO_USER=itinnov.flame.integration REPO_USER=itinnov.flame.integration
REPO_PASS=xxxxxxx REPO_PASS=xxxxxxx
``` ```
##### Build all services
Create all the services needed for integration tests: Create all the services needed for integration tests:
```shell ```shell
sudo /vagrant/scripts/test/fixture.sh create -f /vagrant/src/test/clmctest/rspec.json -c all sudo su
/vagrant/scripts/test/fixture.sh create -f /vagrant/src/test/clmctest/rspec.json -c all
``` ```
As part of the clmc-service installation, the service's unit tests have been run. The fixture script will fail if any individual service installation fails to install (or fails its tests). As part of the clmc-service installation, the service's unit tests have been run. The fixture script will fail if any individual service installation fails to install (or fails its tests).
...@@ -101,15 +95,15 @@ As part of the clmc-service installation, the service's unit tests have been run ...@@ -101,15 +95,15 @@ As part of the clmc-service installation, the service's unit tests have been run
Attach to the test-runner to run the tests Attach to the test-runner to run the tests
```shell ```shell
sudo lxc exec test-runner -- bash lxc-attach -n test-runner
``` ```
Build and install the CLMC test Python module: Build and install the CLMC test Python module:
```shell ```shell
cd /opt/clmc/src/test cd /vagrant/src/test
python3 setup.py sdist --dist-dir=../../build python setup.py sdist --dist-dir=../../build
pip3 install /opt/clmc/build/clmctest-<version>.tar.gz pip3 install /vagrant/build/clmctest-<version>.tar.gz
``` ```
The following module is unit tests: The following module is unit tests:
......
Vagrantfile
+ 6
− 25
View file @ 2f77a2f5
...@@ -21,8 +21,6 @@ ...@@ -21,8 +21,6 @@
$lxc_script = <<-SCRIPT $lxc_script = <<-SCRIPT
export DEBIAN_FRONTEND=noninteractive
apt-get update apt-get update
apt-get install python3 python3-setuptools jq -y apt-get install python3 python3-setuptools jq -y
...@@ -31,7 +29,6 @@ echo iptables-persistent iptables-persistent/autosave_v4 boolean true | sudo deb ...@@ -31,7 +29,6 @@ echo iptables-persistent iptables-persistent/autosave_v4 boolean true | sudo deb
echo iptables-persistent iptables-persistent/autosave_v6 boolean true | sudo debconf-set-selections echo iptables-persistent iptables-persistent/autosave_v6 boolean true | sudo debconf-set-selections
apt-get -y install iptables-persistent apt-get -y install iptables-persistent
lxd init --auto --storage-backend dir lxd init --auto --storage-backend dir
lxc network create lxcbr0 ipv6.address=none ipv4.address=172.40.231.1/24 ipv4.nat=true lxc network create lxcbr0 ipv6.address=none ipv4.address=172.40.231.1/24 ipv4.nat=true
...@@ -48,7 +45,6 @@ timedatectl set-ntp no ...@@ -48,7 +45,6 @@ timedatectl set-ntp no
#done #done
apt-get install ntp -y apt-get install ntp -y
echo "timesync set to ntpd" echo "timesync set to ntpd"
# set timezone to London # set timezone to London
...@@ -58,29 +54,14 @@ SCRIPT ...@@ -58,29 +54,14 @@ SCRIPT
Vagrant.configure("2") do |config| Vagrant.configure("2") do |config|
config.vm.box = "ubuntu/bionic64" config.vm.box = "ubuntu/bionic64"
config.vm.box_version = '20190627.1.0' config.disksize.size = '50GB'
# use public network by allowing the IP to be assigned via DHCP
#config.vm.network "public_network", use_dhcp_assigned_default_route: true
# forward ports to host
#config.vm.network "forwarded_port", guest: 80, host: 80
# increase default disk size, it requires to install the plugin first
# e.g. vagrant plugin install vagrant-disksize
config.disksize.size = '15GB'
# forward X11
config.ssh.forward_agent = true
config.ssh.forward_x11 = true
# provision the VM
config.vm.provider "virtualbox" do |vb| config.vm.provider "virtualbox" do |vb|
vb.cpus = 2 vb.cpus = 4
#vb.memory = "8192" vb.memory = "8192"
vb.memory = "12288"
end end
#config.vm.network "forwarded_port", guest: 8888, host: 8888
# Install lxc # Install lxc
config.vm.provision :shell, inline: $lxc_script config.vm.provision :shell, inline: $lxc_script
end end
scripts/test/fixture.sh
+ 21
− 48
View file @ 2f77a2f5
...@@ -28,12 +28,6 @@ repo_root="/vagrant" ...@@ -28,12 +28,6 @@ repo_root="/vagrant"
target_root="/opt/clmc" target_root="/opt/clmc"
config_file="rspec.json" config_file="rspec.json"
# make sure UID is root
if [ "$EUID" -ne 0 ]; then
echo "please run as root (requires to change iptables)"
exit
fi
usage() { usage() {
echo "Usage: $0 create|start|stop|destroy [-f config_file] [-r repo_root] [-c container_name|all]" 1>&2 echo "Usage: $0 create|start|stop|destroy [-f config_file] [-r repo_root] [-c container_name|all]" 1>&2
echo " -f defaults to '${config_file}'" 1>&2 echo " -f defaults to '${config_file}'" 1>&2
...@@ -42,14 +36,6 @@ usage() { ...@@ -42,14 +36,6 @@ usage() {
exit 1 exit 1
} }
failed() {
msg=$1
sn=$2
echo "failed to: $msg for service $service_name"
delete $sn
exit 1
}
create() { create() {
service_name=$1 service_name=$1
config_file=$2 config_file=$2
...@@ -61,28 +47,25 @@ create() { ...@@ -61,28 +47,25 @@ create() {
echo $SERVICE echo $SERVICE
ip=$(echo $SERVICE | jq -r '.ip_address') ip=$(echo $SERVICE | jq -r '.ip_address')
lxc init ubuntu:16.04 ${service_name} || failed "crate lxc image" ${service_name} lxc init ubuntu:16.04 ${service_name}
echo "Creating network: ${service_name}" echo "Creating network: ${service_name}"
lxc network attach lxcbr0 ${service_name} eth0 || failed "attach lxcbr0" ${service_name} lxc network attach lxcbr0 ${service_name} eth0
lxc config device set ${service_name} eth0 ipv4.address ${ip} || failed "config eth0" ${service_name} lxc config device set ${service_name} eth0 ipv4.address ${ip}
lxc config set ${service_name} security.privileged true || failed "config security" ${service_name} lxc config set ${service_name} security.privileged true
# copy flame clmc files into the root container # copy flame clmc files into the root container
echo "Copying files to rootfs" echo "Copying files to rootfs"
container_dir="/var/lib/lxd/containers/"${service_name}"/rootfs" container_dir="/var/lib/lxd/containers/"${service_name}"/rootfs"
container_repo_root=${container_dir}${target_root} container_repo_root=${container_dir}${target_root}
mkdir -p ${container_repo_root} || failed "create target directory" ${service_name} mkdir -p ${container_repo_root}
cp -f ${repo_root}/reporc ${container_repo_root} || failed "copy reporc" ${service_name} cp -f ${repo_root}/reporc ${container_repo_root}
cp -rf ${repo_root}/scripts ${container_repo_root} || failed "copy scripts" ${service_name} cp -rf ${repo_root}/scripts ${container_repo_root}
cp -rf ${repo_root}/src ${container_repo_root} || failed "copy src" ${service_name} cp -rf ${repo_root}/src ${container_repo_root}
chown -R 100000:100000 ${container_repo_root} || failed "change permissions" ${service_name} chown -R 100000:100000 ${container_repo_root}
# start the container # start the container
echo "Starting: ${service_name}" echo "Starting: ${service_name}"
lxc start ${service_name} lxc start ${service_name}
# add delay to avoid while-loop lxc command failing initially
sleep 1
while :; do while :; do
echo "Waiting for container to start: ${service_name}" echo "Waiting for container to start: ${service_name}"
lxc file pull ${service_name}/etc/resolv.conf - | grep -q nameserver && break lxc file pull ${service_name}/etc/resolv.conf - | grep -q nameserver && break
...@@ -100,7 +83,8 @@ create() { ...@@ -100,7 +83,8 @@ create() {
echo "clmc-service installation failed with exit code ${exit_code}" echo "clmc-service installation failed with exit code ${exit_code}"
exit 1 exit 1
fi fi
elif [ ${service_name} == "test-runner" ]; then elif [ ${service_name} == "test-runner" ]
then
cmd="${target_root}/src/test/clmctest/services/pytest/install.sh" cmd="${target_root}/src/test/clmctest/services/pytest/install.sh"
lxc exec ${service_name} -- ${cmd} lxc exec ${service_name} -- ${cmd}
exit_code=$? exit_code=$?
...@@ -131,9 +115,8 @@ create() { ...@@ -131,9 +115,8 @@ create() {
lxc exec ${service_name} --env REPO_ROOT="${target_root}" -- ${cmd} lxc exec ${service_name} --env REPO_ROOT="${target_root}" -- ${cmd}
# check that telegraf installed (it may not have if the reporc file was not present or Nexus server was down) # check that telegraf installed (it may not have if the reporc file was not present or Nexus server was down)
if lxc-attach -n ${service_name} -- ls /etc/telegraf |& grep 'ls: cannot access'; then
if lxc exec ${service_name} -- ls /etc/telegraf |& grep 'ls: cannot access'; then echo "Telegraf agent failed to install (check reporc?)"
echo "Telegraf agent failed to install for ${service_name} (check reporc?)"
exit 1 exit 1
fi fi
...@@ -141,14 +124,13 @@ create() { ...@@ -141,14 +124,13 @@ create() {
lxc exec ${service_name} -- service telegraf stop lxc exec ${service_name} -- service telegraf stop
# copy telegraf configuration templates # copy telegraf configuration templates
echo "copy telegraf configuration templates" cp -f ${repo_root}/scripts/clmc-agent/telegraf.conf ${container_dir}/etc/telegraf/
lxc file push -r scripts/clmc-agent/telegraf.conf ${service_name}/etc/telegraf/ cp -f ${repo_root}/scripts/clmc-agent/telegraf_output.conf ${container_dir}/etc/telegraf/telegraf.d/
lxc file push -r scripts/clmc-agent/telegraf_output.conf ${service_name}/etc/telegraf/telegraf.d/
# copy the 'host' config into all service containers # copy the 'host' config into all service containers
lxc file push -r src/test/clmctest/services/host/telegraf*.conf ${service_name}/etc/telegraf/telegraf.d/ cp -f ${repo_root}/src/test/clmctest/services/host/telegraf*.conf ${container_dir}/etc/telegraf/telegraf.d/
# copy the service-specific config # copy the service-specific config
lxc file push -r src/test/clmctest/services/${sf_package_id}/telegraf*.conf ${service_name}/etc/telegraf/telegraf.d/ cp -f ${repo_root}/src/test/clmctest/services/${sf_package_id}/telegraf*.conf ${container_dir}/etc/telegraf/telegraf.d/
lxc exec ${service_name} -- chown -R 100000:100000 /etc/telegraf/ chown -R 100000:100000 ${container_dir}/etc/telegraf/
# replace telegraf template with container parameters # replace telegraf template with container parameters
cmd="${target_root}/scripts/clmc-agent/configure.sh ${location} ${sfc_id} ${sfc_instance_id} ${sf_package_id} ${sf_id} ${sf_endpoint_id} ${influxdb_url}" cmd="${target_root}/scripts/clmc-agent/configure.sh ${location} ${sfc_id} ${sfc_instance_id} ${sf_package_id} ${sf_id} ${sf_endpoint_id} ${influxdb_url}"
...@@ -185,19 +167,10 @@ stop() { ...@@ -185,19 +167,10 @@ stop() {
service_name=$1 service_name=$1
if lxc info ${service_name}; then if lxc info ${service_name}; then
echo "Stopping container: ${service_name}" echo "Stopping container: ${service_name}"
lxc stop ${service_name} lxc stop -n ${service_name}
fi fi
} }
delete() {
service_name=$1
if lxc list | grep -q ${service_name}; then
echo "Deleting container: ${service_name}"
lxc delete ${service_name}
fi
}
destroy() { destroy() {
service_name=$1 service_name=$1
config_file=$2 config_file=$2
......
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment