Skip to content
Snippets Groups Projects
Commit 627d4aaf authored by Nikolay Stanchev's avatar Nikolay Stanchev
Browse files

[ Issue #57 ] - adjusted some cinfgurations of the streaming setup and fixed the README file

parent 881934bc
No related branches found
No related tags found
No related merge requests found
Hide whitespace changes
Inline Side-by-side
test/streaming/manual.md
+ 27
− 31
View file @ 627d4aaf
......@@ -25,34 +25,34 @@
# CLMC Adaptive Streaming Test
This test streams mpeg-dash video using the two apache servers monitored by Telegraf configured with a default apache plugin and a net_response plugin. The data is stored in the `clmc-service` using database `CLMCMetrics` and measurements `apache` and `net_response`
This test streams mpeg-dash video using the two nginx servers monitored by Telegraf configured with a default apache plugin and a net_response plugin. The data is stored in the `clmc-service` using database `CLMCMetrics` and measurements `nginx` and `net_response`
The following command brings up the services
`vagrant --infra=streaming up`
`vagrant --fixture=streaming up`
* clmc-service: configured with influx, kapacitor, chornograf
* apache1@DC1, apache2@DC2: configured with apache and a test video located at http://192.168.50.11:80/test_video/stream.mpd on the internal vbox network and at http://localhost:8081/test_video/stream.mpd if accessing from the host machine
* nginx1@DC1, nginx2@DC2: configured with nginx and a test video located at http://192.168.50.11:80/test_video/stream.mpd on the internal vbox network and at http://localhost:8081/test_video/stream.mpd if accessing from the host machine
### Run the test set-up
`vagrant --infra=streaming ssh clmc-service -- "sudo /vagrant/test/streaming/setup.sh /vagrant/test/streaming"`
`vagrant --fixture=streaming ssh clmc-service -- "sudo /vagrant/test/streaming/setupCLMC.sh /vagrant/test/streaming"`
`vagrant --fixture=streaming ssh nginx1 -- "sudo /vagrant/test/streaming/setupNGINX.sh"`
### Run the automated test
To run the load test using the following command (here, the last parameter '15' refers to the number of VLC player clients to be launched):
`vagrant --infra=streaming ssh loadtest-streaming -- "/vagrant/test/streaming/run.sh /home/ubuntu/test/streaming http://192.168.50.11/test_video/stream.mpd 15`
`vagrant --fixture=streaming ssh loadtest-streaming -- "sudo /vagrant/test/streaming/run.sh /home/ubuntu/test/streaming http://192.168.50.11/test_video/stream.mpd 15`
This test currently just generates the load and does not have any assertions. It breaks at 1000.
### Manual test
And then point your browser to:
And then point your browser to the Chronograf dashboard:
`http://localhost:8888`
### Manual test
## Manual set-up of Chronograf's CLMC data source
If you __do not__ want to run the automatic set-up, basic entry to the Chronograf dashboard is as follows:
......@@ -75,7 +75,7 @@ Start the VLC Player
The test video is the FLAME project video and it can be viewed at the following location.
`Enter the network URL: http://localhost:8081/test-video/stream.mpd for apache1 server`
`Enter the network URL: http://localhost:8081/test_video/stream.mpd for nginx1 server`
The video should play.
......@@ -83,9 +83,9 @@ The video should play.
Open Chronograph by entering the following URL into a browser on the host http://localhost:8888. Your CLMC data source, Kapacitor and demonstration dashboard should be ready for you to explore.
Press the Data Explorer in the menu and select the apache measurement and create a query such as
Press the Data Explorer in the menu and select the nginx measurement and create a query such as
`SELECT mean("BytesPerSec") AS "mean_BytesPerSec" FROM "CLMCMetrics"."autogen"."apache" WHERE time > now() - 5m GROUP BY time(10s)`
`SELECT mean("requests") AS "mean_requests" FROM "CLMCMetrics"."autogen"."nginx" WHERE time > now() - 1h GROUP BY time(10s)`
## KPI triggers
......@@ -94,40 +94,38 @@ In this demonstrator an example KPI rule has been set up in Kapacitor which fire
```
dbrp "CLMCMetrics"."autogen"
// Apache 1 rule
// Nginx 1 rule
// -------------
var a1Data = batch
|query(''' SELECT mean("ReqPerSec") AS "mean_RPS" FROM "CLMCMetrics"."autogen"."apache" WHERE "ipendpoint"='adaptive_streaming_I1_apache1' ''')
var n1Data = batch
|query(''' SELECT mean("active") AS "mean_active" FROM "CLMCMetrics"."autogen"."nginx" WHERE "ipendpoint"='adaptive_streaming_I1_nginx1' ''')
.period(5s)
.every(5s)
var a1Alert = a1Data
varn n1Alert = n1Data
|alert()
.id('{{ .Name }}/adaptive_streaming_I1_apache1')
.message('{{ .ID }} is {{ .Level }} Mean Requests Per Second: {{ index .Fields "mean_RPS" }}')
.warn(lambda: "mean_RPS" > 0.2)
.crit(lambda: "mean_RPS" > 0.5)
.id('{{ .Name }}/adaptive_streaming_I1_nginx1')
.message('{{ .ID }} is {{ .Level }} Mean active connections: {{ index .Fields "mean_active" }}')
.warn(lambda: "mean_active" > 10)
.slack()
.log( '/tmp/RPSLoad.log' )
// Apache 2 rule
// Nginx 2 rule
// -------------
var a2Data = batch
|query(''' SELECT mean("ReqPerSec") AS "mean_RPS" FROM "CLMCMetrics"."autogen"."apache" WHERE "ipendpoint"='adaptive_streaming_I1_apache2' ''')
var n2Data = batch
|query(''' SELECT mean("active") AS "mean_active" FROM "CLMCMetrics"."autogen"."nginx" WHERE "ipendpoint"='adaptive_streaming_I1_nginx2' ''')
.period(5s)
.every(5s)
var a2Alert = a2Data
var n2Alert = n2Data
|alert()
.id('{{ .Name }}/adaptive_streaming_I1_apache2')
.message('{{ .ID }} is {{ .Level }} Mean Requests Per Second: {{ index .Fields "mean_RPS" }}')
.warn(lambda: "mean_RPS" > 0.2)
.crit(lambda: "mean_RPS" > 0.5)
.id('{{ .Name }}/adaptive_streaming_I1_nginx2')
.message('{{ .ID }} is {{ .Level }} Mean active connections: {{ index .Fields "mean_active" }}')
.warn(lambda: "mean_active" > 10)
.slack()
.log( '/tmp/RPSLoad.log' )
```
Alerts are sent both an internal logging within the CLMC service file system and also to a FLAME demo Slack service:
Alerts are sent to both an internal logging within the CLMC service file system and also to a FLAME demo Slack service:
https://flamedemo-itinnov.slack.com
......@@ -136,5 +134,3 @@ Alerts can be found under the '#clmc' channel.
### Kapacitor rules in Chronograf's GUI
Additional rules can be added to this demonstrator either via the Chronograf GUI (see [here](https://docs.influxdata.com/chronograf/v1.4/introduction/getting-started/#4-connect-chronograf-to-kapacitor) for more information) or by using the Kapacitor HTTP API and TICKscript (for an introduction, [look here](https://docs.influxdata.com/kapacitor/v1.4/tick/)).
test/streaming/run.sh
+ 1
− 1
View file @ 627d4aaf
......@@ -48,7 +48,7 @@ COUNTER=0
MAX_CLIENTS=$3
while [ $COUNTER -lt $MAX_CLIENTS ]; do
# run cvlc headless, redirect stderr into stdout, pipe that into the report.sh script
cvlc -Vdummy --no-audio $STREAM_URI 2>&1 | /home/ubuntu/flame-clmc/test/streaming/report.sh ${COUNTER} &
cvlc -Vdummy --no-audio $STREAM_URI 2>&1 | /vagrant/test/streaming/report.sh ${COUNTER} &
sleep 1
let COUNTER=COUNTER+1
done
......
test/streaming/setup.sh → test/streaming/setupCLMC.sh 100755 → 100644
+ 0
− 18
View file @ 627d4aaf
......@@ -48,21 +48,3 @@ curl -i -X POST -H "Content-Type: application/json" http://localhost:9092/kapaci
# Set up dashboard
curl -i -X POST -H "Content-Type: application/json" http://localhost:8888/chronograf/v1/dashboards -d @$TEST_DIR/dashboard.json
# Copy test data
# Apache
# DEST_DIR="/var/www/html/"
# NGINX
DEST_DIR="/usr/share/nginx/html"
TEST_VIDEO="20180212104221flame-project-full.mp4"
TEST_VIDEO_ARCHIVE=$TEST_VIDEO".gz"
DEST_FILE=$DEST_DIR"/"$TEST_VIDEO_ARCHIVE
# Copy files for MPEG-DASH testing
curl "ftp://ftp.it-innovation.soton.ac.uk/testdata/video/"$TEST_VIDEO_ARCHIVE --user flame-rw:DR8ngj3ogSjd8gl -o $DEST_FILE
tar -xvf $DEST_FILE -C $DEST_DIR
rm -rf $DEST_FILE
mv $DEST_DIR"/"$TEST_VIDEO $DEST_DIR"/"test_video
\ No newline at end of file
test/streaming/setupNGINX.sh 0 → 100644
+ 41
− 0
View file @ 627d4aaf
#!/bin/bash
#/////////////////////////////////////////////////////////////////////////
#//
#// (c) University of Southampton IT Innovation Centre, 2018
#//
#// Copyright in this software belongs to University of Southampton
#// IT Innovation Centre of Gamma House, Enterprise Road,
#// Chilworth Science Park, Southampton, SO16 7NS, UK.
#//
#// This software may not be used, sold, licensed, transferred, copied
#// or reproduced in whole or in part in any manner or form or in or
#// on any media by any person other than in accordance with the terms
#// of the Licence Agreement supplied with the software, or otherwise
#// without the prior written consent of the copyright owners.
#//
#// This software is distributed WITHOUT ANY WARRANTY, without even the
#// implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR
#// PURPOSE, except where stated in the Licence Agreement supplied with
#// the software.
#//
#// Created By : Simon Crowle
#// Created Date : 14/02/2018
#// Created for Project : FLAME
#//
#/////////////////////////////////////////////////////////////////////////
# NGINX
DEST_DIR="/usr/share/nginx/html"
TEST_VIDEO="20180212104221flame-project-full.mp4"
TEST_VIDEO_ARCHIVE=$TEST_VIDEO".gz"
DEST_FILE=$DEST_DIR"/"$TEST_VIDEO_ARCHIVE
echo "ftp://ftp.it-innovation.soton.ac.uk/testdata/video/"$TEST_VIDEO_ARCHIVE
# Copy files for MPEG-DASH testing
curl "ftp://ftp.it-innovation.soton.ac.uk/testdata/video/"$TEST_VIDEO_ARCHIVE --user flame-rw:DR8ngj3ogSjd8gl -o $DEST_FILE
tar -xvf $DEST_FILE -C $DEST_DIR
rm -rf $DEST_FILE
mv $DEST_DIR"/"$TEST_VIDEO $DEST_DIR"/"test_video
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment