Skip to content
Snippets Groups Projects
Commit 627d4aaf authored by Nikolay Stanchev's avatar Nikolay Stanchev
Browse files

[ Issue #57 ] - adjusted some cinfgurations of the streaming setup and fixed the README file

parent 881934bc
Branches
No related tags found
No related merge requests found
Show whitespace changes
Inline Side-by-side
test/streaming/manual.md
+ 27
− 31
View file @ 627d4aaf
...@@ -25,34 +25,34 @@ ...@@ -25,34 +25,34 @@
# CLMC Adaptive Streaming Test # CLMC Adaptive Streaming Test
This test streams mpeg-dash video using the two apache servers monitored by Telegraf configured with a default apache plugin and a net_response plugin. The data is stored in the `clmc-service` using database `CLMCMetrics` and measurements `apache` and `net_response` This test streams mpeg-dash video using the two nginx servers monitored by Telegraf configured with a default apache plugin and a net_response plugin. The data is stored in the `clmc-service` using database `CLMCMetrics` and measurements `nginx` and `net_response`
The following command brings up the services The following command brings up the services
`vagrant --infra=streaming up` `vagrant --fixture=streaming up`
* clmc-service: configured with influx, kapacitor, chornograf * clmc-service: configured with influx, kapacitor, chornograf
* apache1@DC1, apache2@DC2: configured with apache and a test video located at http://192.168.50.11:80/test_video/stream.mpd on the internal vbox network and at http://localhost:8081/test_video/stream.mpd if accessing from the host machine * nginx1@DC1, nginx2@DC2: configured with nginx and a test video located at http://192.168.50.11:80/test_video/stream.mpd on the internal vbox network and at http://localhost:8081/test_video/stream.mpd if accessing from the host machine
### Run the test set-up ### Run the test set-up
`vagrant --infra=streaming ssh clmc-service -- "sudo /vagrant/test/streaming/setup.sh /vagrant/test/streaming"` `vagrant --fixture=streaming ssh clmc-service -- "sudo /vagrant/test/streaming/setupCLMC.sh /vagrant/test/streaming"`
`vagrant --fixture=streaming ssh nginx1 -- "sudo /vagrant/test/streaming/setupNGINX.sh"`
### Run the automated test ### Run the automated test
To run the load test using the following command (here, the last parameter '15' refers to the number of VLC player clients to be launched): To run the load test using the following command (here, the last parameter '15' refers to the number of VLC player clients to be launched):
`vagrant --infra=streaming ssh loadtest-streaming -- "/vagrant/test/streaming/run.sh /home/ubuntu/test/streaming http://192.168.50.11/test_video/stream.mpd 15` `vagrant --fixture=streaming ssh loadtest-streaming -- "sudo /vagrant/test/streaming/run.sh /home/ubuntu/test/streaming http://192.168.50.11/test_video/stream.mpd 15`
This test currently just generates the load and does not have any assertions. It breaks at 1000. This test currently just generates the load and does not have any assertions. It breaks at 1000.
And then point your browser to the Chronograf dashboard:
### Manual test
And then point your browser to:
`http://localhost:8888` `http://localhost:8888`
### Manual test
## Manual set-up of Chronograf's CLMC data source ## Manual set-up of Chronograf's CLMC data source
If you __do not__ want to run the automatic set-up, basic entry to the Chronograf dashboard is as follows: If you __do not__ want to run the automatic set-up, basic entry to the Chronograf dashboard is as follows:
...@@ -75,7 +75,7 @@ Start the VLC Player ...@@ -75,7 +75,7 @@ Start the VLC Player
The test video is the FLAME project video and it can be viewed at the following location. The test video is the FLAME project video and it can be viewed at the following location.
`Enter the network URL: http://localhost:8081/test-video/stream.mpd for apache1 server` `Enter the network URL: http://localhost:8081/test_video/stream.mpd for nginx1 server`
The video should play. The video should play.
...@@ -83,9 +83,9 @@ The video should play. ...@@ -83,9 +83,9 @@ The video should play.
Open Chronograph by entering the following URL into a browser on the host http://localhost:8888. Your CLMC data source, Kapacitor and demonstration dashboard should be ready for you to explore. Open Chronograph by entering the following URL into a browser on the host http://localhost:8888. Your CLMC data source, Kapacitor and demonstration dashboard should be ready for you to explore.
Press the Data Explorer in the menu and select the apache measurement and create a query such as Press the Data Explorer in the menu and select the nginx measurement and create a query such as
`SELECT mean("BytesPerSec") AS "mean_BytesPerSec" FROM "CLMCMetrics"."autogen"."apache" WHERE time > now() - 5m GROUP BY time(10s)` `SELECT mean("requests") AS "mean_requests" FROM "CLMCMetrics"."autogen"."nginx" WHERE time > now() - 1h GROUP BY time(10s)`
## KPI triggers ## KPI triggers
...@@ -94,40 +94,38 @@ In this demonstrator an example KPI rule has been set up in Kapacitor which fire ...@@ -94,40 +94,38 @@ In this demonstrator an example KPI rule has been set up in Kapacitor which fire
``` ```
dbrp "CLMCMetrics"."autogen" dbrp "CLMCMetrics"."autogen"
// Apache 1 rule // Nginx 1 rule
// ------------- // -------------
var a1Data = batch var n1Data = batch
|query(''' SELECT mean("ReqPerSec") AS "mean_RPS" FROM "CLMCMetrics"."autogen"."apache" WHERE "ipendpoint"='adaptive_streaming_I1_apache1' ''') |query(''' SELECT mean("active") AS "mean_active" FROM "CLMCMetrics"."autogen"."nginx" WHERE "ipendpoint"='adaptive_streaming_I1_nginx1' ''')
.period(5s) .period(5s)
.every(5s) .every(5s)
var a1Alert = a1Data varn n1Alert = n1Data
|alert() |alert()
.id('{{ .Name }}/adaptive_streaming_I1_apache1') .id('{{ .Name }}/adaptive_streaming_I1_nginx1')
.message('{{ .ID }} is {{ .Level }} Mean Requests Per Second: {{ index .Fields "mean_RPS" }}') .message('{{ .ID }} is {{ .Level }} Mean active connections: {{ index .Fields "mean_active" }}')
.warn(lambda: "mean_RPS" > 0.2) .warn(lambda: "mean_active" > 10)
.crit(lambda: "mean_RPS" > 0.5)
.slack() .slack()
.log( '/tmp/RPSLoad.log' ) .log( '/tmp/RPSLoad.log' )
// Apache 2 rule // Nginx 2 rule
// ------------- // -------------
var a2Data = batch var n2Data = batch
|query(''' SELECT mean("ReqPerSec") AS "mean_RPS" FROM "CLMCMetrics"."autogen"."apache" WHERE "ipendpoint"='adaptive_streaming_I1_apache2' ''') |query(''' SELECT mean("active") AS "mean_active" FROM "CLMCMetrics"."autogen"."nginx" WHERE "ipendpoint"='adaptive_streaming_I1_nginx2' ''')
.period(5s) .period(5s)
.every(5s) .every(5s)
var a2Alert = a2Data var n2Alert = n2Data
|alert() |alert()
.id('{{ .Name }}/adaptive_streaming_I1_apache2') .id('{{ .Name }}/adaptive_streaming_I1_nginx2')
.message('{{ .ID }} is {{ .Level }} Mean Requests Per Second: {{ index .Fields "mean_RPS" }}') .message('{{ .ID }} is {{ .Level }} Mean active connections: {{ index .Fields "mean_active" }}')
.warn(lambda: "mean_RPS" > 0.2) .warn(lambda: "mean_active" > 10)
.crit(lambda: "mean_RPS" > 0.5)
.slack() .slack()
.log( '/tmp/RPSLoad.log' ) .log( '/tmp/RPSLoad.log' )
``` ```
Alerts are sent both an internal logging within the CLMC service file system and also to a FLAME demo Slack service: Alerts are sent to both an internal logging within the CLMC service file system and also to a FLAME demo Slack service:
https://flamedemo-itinnov.slack.com https://flamedemo-itinnov.slack.com
...@@ -136,5 +134,3 @@ Alerts can be found under the '#clmc' channel. ...@@ -136,5 +134,3 @@ Alerts can be found under the '#clmc' channel.
### Kapacitor rules in Chronograf's GUI ### Kapacitor rules in Chronograf's GUI
Additional rules can be added to this demonstrator either via the Chronograf GUI (see [here](https://docs.influxdata.com/chronograf/v1.4/introduction/getting-started/#4-connect-chronograf-to-kapacitor) for more information) or by using the Kapacitor HTTP API and TICKscript (for an introduction, [look here](https://docs.influxdata.com/kapacitor/v1.4/tick/)). Additional rules can be added to this demonstrator either via the Chronograf GUI (see [here](https://docs.influxdata.com/chronograf/v1.4/introduction/getting-started/#4-connect-chronograf-to-kapacitor) for more information) or by using the Kapacitor HTTP API and TICKscript (for an introduction, [look here](https://docs.influxdata.com/kapacitor/v1.4/tick/)).
test/streaming/run.sh
+ 1
− 1
View file @ 627d4aaf
...@@ -48,7 +48,7 @@ COUNTER=0 ...@@ -48,7 +48,7 @@ COUNTER=0
MAX_CLIENTS=$3 MAX_CLIENTS=$3
while [ $COUNTER -lt $MAX_CLIENTS ]; do while [ $COUNTER -lt $MAX_CLIENTS ]; do
# run cvlc headless, redirect stderr into stdout, pipe that into the report.sh script # run cvlc headless, redirect stderr into stdout, pipe that into the report.sh script
cvlc -Vdummy --no-audio $STREAM_URI 2>&1 | /home/ubuntu/flame-clmc/test/streaming/report.sh ${COUNTER} & cvlc -Vdummy --no-audio $STREAM_URI 2>&1 | /vagrant/test/streaming/report.sh ${COUNTER} &
sleep 1 sleep 1
let COUNTER=COUNTER+1 let COUNTER=COUNTER+1
done done
......
test/streaming/setup.sh → test/streaming/setupCLMC.sh 100755 → 100644
+ 0
− 18
View file @ 627d4aaf
...@@ -48,21 +48,3 @@ curl -i -X POST -H "Content-Type: application/json" http://localhost:9092/kapaci ...@@ -48,21 +48,3 @@ curl -i -X POST -H "Content-Type: application/json" http://localhost:9092/kapaci
# Set up dashboard # Set up dashboard
curl -i -X POST -H "Content-Type: application/json" http://localhost:8888/chronograf/v1/dashboards -d @$TEST_DIR/dashboard.json curl -i -X POST -H "Content-Type: application/json" http://localhost:8888/chronograf/v1/dashboards -d @$TEST_DIR/dashboard.json
# Copy test data
# Apache
# DEST_DIR="/var/www/html/"
# NGINX
DEST_DIR="/usr/share/nginx/html"
TEST_VIDEO="20180212104221flame-project-full.mp4"
TEST_VIDEO_ARCHIVE=$TEST_VIDEO".gz"
DEST_FILE=$DEST_DIR"/"$TEST_VIDEO_ARCHIVE
# Copy files for MPEG-DASH testing
curl "ftp://ftp.it-innovation.soton.ac.uk/testdata/video/"$TEST_VIDEO_ARCHIVE --user flame-rw:DR8ngj3ogSjd8gl -o $DEST_FILE
tar -xvf $DEST_FILE -C $DEST_DIR
rm -rf $DEST_FILE
mv $DEST_DIR"/"$TEST_VIDEO $DEST_DIR"/"test_video
\ No newline at end of file
test/streaming/setupNGINX.sh 0 → 100644
+ 41
− 0
View file @ 627d4aaf
#!/bin/bash
#/////////////////////////////////////////////////////////////////////////
#//
#// (c) University of Southampton IT Innovation Centre, 2018
#//
#// Copyright in this software belongs to University of Southampton
#// IT Innovation Centre of Gamma House, Enterprise Road,
#// Chilworth Science Park, Southampton, SO16 7NS, UK.
#//
#// This software may not be used, sold, licensed, transferred, copied
#// or reproduced in whole or in part in any manner or form or in or
#// on any media by any person other than in accordance with the terms
#// of the Licence Agreement supplied with the software, or otherwise
#// without the prior written consent of the copyright owners.
#//
#// This software is distributed WITHOUT ANY WARRANTY, without even the
#// implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR
#// PURPOSE, except where stated in the Licence Agreement supplied with
#// the software.
#//
#// Created By : Simon Crowle
#// Created Date : 14/02/2018
#// Created for Project : FLAME
#//
#/////////////////////////////////////////////////////////////////////////
# NGINX
DEST_DIR="/usr/share/nginx/html"
TEST_VIDEO="20180212104221flame-project-full.mp4"
TEST_VIDEO_ARCHIVE=$TEST_VIDEO".gz"
DEST_FILE=$DEST_DIR"/"$TEST_VIDEO_ARCHIVE
echo "ftp://ftp.it-innovation.soton.ac.uk/testdata/video/"$TEST_VIDEO_ARCHIVE
# Copy files for MPEG-DASH testing
curl "ftp://ftp.it-innovation.soton.ac.uk/testdata/video/"$TEST_VIDEO_ARCHIVE --user flame-rw:DR8ngj3ogSjd8gl -o $DEST_FILE
tar -xvf $DEST_FILE -C $DEST_DIR
rm -rf $DEST_FILE
mv $DEST_DIR"/"$TEST_VIDEO $DEST_DIR"/"test_video
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment