@@ -56,14 +56,14 @@ Testing is implemented using pytest.
The installation script is here:
`test/services/pytest/install.sh`
`sudo clmctest/services/pytest/install.sh`
using the following convention:
* Tests are written in python using pytest
* Related tests are stored in a python module `test/<testmodule>` to create a suite of tests. All tests are stored in files test_*.py, there can be many tests per file, and many files per module
* Related tests are stored in a python module `clmctest/<testmodule>` to create a suite of tests. All tests are stored in files test_*.py, there can be many tests per file, and many files per module
* Each test module has a rspec.yml that provides the baseline "fixture" for the tests in the module
* Tests are executed against fixtures. Fixtures are modular "setups" created for a test, that are inserted into the python code using dependancy injection. This offers more flexibility than the *unit style testing. The baseline deployment is created using `vagrant up` with an appropriate rspec, and the pytest fixture reads the rspec.yml and makes the configuration available to the test.
* Tests are executed against fixtures. Fixtures are modular "setups" created for a test, that are inserted into the python code using dependancy injection. This offers more flexibility than the unit style testing. The baseline deployment is created using `vagrant up` with an appropriate rspec, and the pytest fixture reads the rspec.yml and makes the configuration available to the test.
* Tests are executed from a guest VM (not the host) in the repo root using the command `pytest test/<testmodule>`
* Pytest will scan the directory for all tests including in files test_*.py and run them
...
...
@@ -75,7 +75,7 @@ To set up a simulation of the adaptive streaming use case scenario first install
and then execute the following command
`vagrant --fixture=streaming-sim -- up`
`vagrant --fixture=monitoring -- up`
This will provision the following VMs clmc-service, ipendpoint1, ipendpoint2
...
...
@@ -89,7 +89,7 @@ The **clmc-service** vm includes influx, Kapacitor and Chronograf. The following
@@ -99,20 +99,19 @@ The next step is to generate the test data, which could be done in two ways.
First option is to run a python script to generate the test data sets
`python3 test/streaming-sim/StreamingSim.py`
`python3 clmctest/monitoring/StreamingSim.py`
This script could also be used to clear the generated data by using the '-c' option
`python3 test/streaming-sim/StreamingSim.py -c`
`python3 clmctest/monitoring/StreamingSim.py -c`
The second option is to directly run the testing module, which will detect if the data was generated, and if not, will automatically
generate the data before executing the tests. Keep in mind that if the test data is being generated using this way, a 10 seconds timeout
is given after the generation is finished so that the data could properly be inserted into the database. If the data was already generated
using the first option, only the tests would be executed.
#### Running the monitoring tests
The command for running the testing module is
The second option is to directly run the testing module, which will detect if the data was generated, and if not, will automatically generate the data before executing the tests. Keep in mind that if the test data is being generated using this way, a 10 seconds timeout is given after the generation is finished so that the data could properly be inserted into the database. If the data was already generated using the first option, only the tests would be executed.
The `-s` option in the command is used to output prints used in the test code and is, therefore, optional.
...
...
@@ -121,3 +120,49 @@ If pytest is not installed, an easy solution is to use the Python Package Index
`sudo apt-get install python3-pip`
`pip3 install pytest`
#### Configuration status modelling and monitoring
FLAME _endpoints_ (VMs created and managed by the SFEMC) and media service _media components_ (processes that realise the execution of the media service) both undergo changes in configuration state during the lifetime of a media service's deployment. Observations of these state changes are recorded in the CLMC under named measurement sets, for example 'endpoint_config' and '\<media component name\>_config' for endpoint and media component labels respectively. In each case, all recordable states of the endpoint/media component are enumerated as columns within the measurement set (see respective state models below for details).
Observation of these states will be performed by a third party - for example, a Telegraf plugin will continuously __report__ on the state of an NGINX service to the CLMC using a _fixed_ interval (say 10 seconds). During this _reporting_ period, the actual state of the NGINX service will be sampled (polled) by the plugin several times (say 10 each second). During any reporting period, the NGINX service _may_ transition from one state to another:
| State observation # | State |
| --- | --- |
| 1 | stopped |
| 2 | stopped |
| 3 | stopped |
| 4 | stopped |
| 5 | starting |
| 6 | starting |
| 7 | starting |
| 8 | starting |
| 9 | starting |
| 10 | starting |
_Above: example observations within a single reporting period of a media component configuration state_
Therefore each report will include for each state:
* The total time in the state for the reporting period
* The avarage time in the state for the reporting period
##### Endpoint configuration state model
##### Media component configuration state model
A media component configuration state model consists of the following states:
* stopped
* starting [transitional]
* running
* stopping [transitional]
An example measurement row for a media component configuration states is below:
In this example, the _reporting period_ is 10 seconds and with an observation rate of 1/second; the observed states were 'stopped' (4 observations) and 'starting' (6 observations).
