Skip to content
Snippets Groups Projects
Commit 0d572515 authored by MJB's avatar MJB
Browse files

#60 restructured docs to be consistent with current implemenation, removed old... 

#60 restructured docs to be consistent with current implemenation, removed old discussions to separate doc files, renamed main document clmc-information-model.md
parent ccddb77b
No related branches found
No related tags found
No related merge requests found
Expand all Hide whitespace changes
Inline Side-by-side
Showing
with 869 additions and 496 deletions
docs/CLMC monitoring specification for a basic scenario.md deleted 100644 → 0
+ 0
− 117
View file @ ccddb77b
<!--
// © University of Southampton IT Innovation Centre, 2018
//
// Copyright in this software belongs to University of Southampton
// IT Innovation Centre of Gamma House, Enterprise Road,
// Chilworth Science Park, Southampton, SO16 7NS, UK.
//
// This software may not be used, sold, licensed, transferred, copied
// or reproduced in whole or in part in any manner or form or in or
// on any media by any person other than in accordance with the terms
// of the Licence Agreement supplied with the software, or otherwise
// without the prior written consent of the copyright owners.
//
// This software is distributed WITHOUT ANY WARRANTY, without even the
// implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR
// PURPOSE, except where stated in the Licence Agreement supplied with
// the software.
//
// Created By : Simon Crowle
// Created Date : 10-01-2018
// Created for Project : FLAME
-->
# Adaptive Streaming Use Case Scenario
## Infrastructure Slice
### *compute_node_config*
| compute_node_config | slice | location | comp_node | cpu | memory | storage | timestamp |
| --- | --- | --- | --- | --- | --- |--- | --- |
| compute_node_config | SLICE1 | locA | dc1 | 4 | 8 | 16 | 1515583926868000000 |
| compute_node_config | SLICE1 | locB | dc2 | 8 | 16 | 64 | 1515583926868000000 |
| compute_node_config | SLICE1 | locC | dc3 | 48 | 128 | 4000 | 1515583926868000000 |
### *network_config*
| network_config | slice | network | bandwidth | timestamp |
| --- | --- | --- | --- | --- | --- |--- |
| network_config | SLICE1 | data1 | 100 | 1515583926868000000 |
__How do we describe network configuration ?__
__What is a format of an infrastructure slices ?__
__What is the relevant information ?__
### *network_interface_config*
| network_interface_config | slice | comp_node | port | network | rx_constraint | tx_constraint | timestamp |
| --- | --- | --- | --- | --- | --- |--- |--- |
| network_config | SLICE1 | dc1 | enps03 | data1 | 1000 | 1000 | 1515583926868000000 |
| network_config | SLICE1 | dc2 | enps03 | data1 | 1000 | 1000 | 1515583926868000000 |
| network_config | SLICE1 | dc3 | enps03 | data1 | 1000 | 1000 | 1515583926868000000 |
## NAP
### ipendpoint_route
| ipendpoint_route | location | ipendpoint_id | cont_nav | avg_http_requests_fqdn_rate | avg_network_fqdn_latency | time |
| --- | --- | --- | --- | --- | --- | --- |
| ipendpoint_route | \<common tags> | DC1 | ipendpoint1 | http://netflix.com/scream | 386, | 50 | 1515583926868000000 |
## Media Service
There are various aggregated metrics we can calculate but in the use case scenario we postpone that till later.
### sfc_instance_config
`sfc_i_config,<common_tags>,state <fields> timestamp`
### sf_i_config
`sf_i_config,<common_tags>,state <fields> timestamp`
## IPEndpoint
All IPEndpoint measurements have the following global tags injected by a configured Telegraf agent
* location
* compute_node
* sfc
* sfc_i
* sf
* sfc_i
* ipendpoint
Also NOTE: the metrics provided in the measurements below are effectively a 'snapshot' of usage over a relatively small period of time. The length of this snapshot may vary, depending on the underlying implementation of the instrumentation, so we might have to assume this snapshot is essentially an average of a period of 1 second. Measuring 'usage' is dependent on the units, for example as a proportion of a resource or as a proportion of time.
### ipendpoint_config
| ipendpoint_config | location | sfc | sfc_i | sf | sf_i | ipendpoint | state | cpu| memory | storage |timestamp |
| --- | --- | --- | --- | --- | --- |--- | --- | --- | --- | --- | --- |
| ipendpoint_config | dc1 | MediaServiceTemplate | MediaServiceA | AdaptiveStreamingComp | AdaptiveStreamingComp1 | ipendpoint1 | placed | 2 | 4 | 16 | 1515583926868000000 |
| ipendpoint_config | dc2 | MediaServiceTemplate | MediaServiceA | AdaptiveStreamingComp | AdaptiveStreamingComp1 | ipendpoint2 | placed | 8 | 16 | 64 | 1515583926868000000 |
| ipendpoint_config | dc3 | MediaServiceTemplate | MediaServiceA | AdaptiveStreamingComp | AdaptiveStreamingComp1 | ipendpoint3 | placed | 48 | 128 | 4000 | 1515583926868000000 |
| ipendpoint_config | dc1 | MediaServiceTemplate | MediaServiceA | AdaptiveStreamingComp | AdaptiveStreamingComp1 | ipendpoint1 | booted | 2 | 4 | 16 | 1515583926868000000 |
| ipendpoint_config | dc2 | MediaServiceTemplate | MediaServiceA | AdaptiveStreamingComp | AdaptiveStreamingComp1 | ipendpoint2 | booted | 8 | 16 | 64 | 1515583926868000000 |
| ipendpoint_config | dc3 | MediaServiceTemplate | MediaServiceA | AdaptiveStreamingComp | AdaptiveStreamingComp1 | ipendpoint3 | booted | 48 | 128 | 4000 | 1515583926868000000 |
### cpu_usage
| cpu_usage | \<common tags> | cpu | avg_cpu_time_user | avg_cpu_time_idle | timestamp |
| --- | --- | --- | --- | --- |--- |
| cpu | \<common tags> | 1 | 40 | 5 | 1515583926868000000 |
### net_port_io
| net_port_io | \<common tags> | avg_packet_drop_rate | avg_packet_error_rate | rx_bytes_port_m | rx_packets_m | tx_bytes_port_m | tx_packets_port_m | timestamp |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| net_port_io | \<common tags> | 0.3 | 0.1 | 13567 | 768 | 8102 | 356 | 1515583926868000000 |
### mpegdash_service
| mpegdash_service_mon | \<common tags> | cont_nav | cont_rep | user_profile |avg_req_rate | avg_resp_time | peak_resp_time | avg_error_rate | avg_throughput | avg_quality_delivered | avg_startup_delay | avg_dropped_segments | timestamp |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |--- |
| mpegdash_service_mon | \<common tags> | http://netflix.com/scream | h264 | profileA | 10 | 40 | 230 | 0.2 | 200 | | 5 | 1200 | 2 | 1515583926868000000 |
docs/TestScenarios.md deleted 100644 → 0
+ 0
− 77
View file @ ccddb77b
<!--
// © University of Southampton IT Innovation Centre, 2018
//
// Copyright in this software belongs to University of Southampton
// IT Innovation Centre of Gamma House, Enterprise Road,
// Chilworth Science Park, Southampton, SO16 7NS, UK.
//
// This software may not be used, sold, licensed, transferred, copied
// or reproduced in whole or in part in any manner or form or in or
// on any media by any person other than in accordance with the terms
// of the Licence Agreement supplied with the software, or otherwise
// without the prior written consent of the copyright owners.
//
// This software is distributed WITHOUT ANY WARRANTY, without even the
// implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR
// PURPOSE, except where stated in the Licence Agreement supplied with
// the software.
//
// Created By : Rowan Powell
// Created Date : 05-01-2018
// Created for Project : FLAME
-->
# Test Scenarios
|author|
|------|
|Rowan Powell|
### Useful InfluxDB commands
| Action | Command example |
| ------ | --------------- |
| get top 3 entries from a database testDB | ```influx -database='testDB' -execute='SELECT * FROM response LIMIT 3'``` |
| show all metrics for a database | ```influx -execute 'SHOW MEASUREMENTS ON testDB'``` |
| show all databases | ```inflix -execute 'SHOW DATABASES'``` |
### Using Chronograf
open ```http://localhost:8888/sources/1/chronograf/data-explorer```
user: telegraf
password: metricsmetricsmetrics
### Scenario 1 - Linear user load increase
Simulating data from during normal usage (Users already present) and linearly increasing users
* Starting at 20 users and scaling up to 40 over the time period
* X% users using HD v Y% using SD
* A% using resource scream, B% using LegoStarWars
| Data | Database |
| ---- | -------- |
| Client requests | request |
| Server responses | response |
| server VM network performance | network |
| mpeg_dash reports | mpegdash_service |
| server configuration events | host_resources |
| VM state events | vm_res_alloc |
This table is written in shorthand
* ~: proportional to
* #: number of
| Measurement | Field | Relationships |
| ----------- | ----- | ------------- |
| response | cpuUsage | ~#clients |
| SF | avg_response_time | ~#requests and ~quality |
| SF | peak_repsonse_time | ~#requests and ~quality |
### Scenario 2 - Two Dash Servers
Simon has created a spec for this here: https://gitlab.it-innovation.soton.ac.uk/mjb/flame-clmc/blob/integration/docs/CLMC%20monitoring%20specification%20for%20a%20basic%20scenario.md
\ No newline at end of file
docs/adaptive-streaming-usecase-scenario.md deleted 100644 → 0
+ 0
− 122
View file @ ccddb77b
<!--
// © University of Southampton IT Innovation Centre, 2018
//
// Copyright in this software belongs to University of Southampton
// IT Innovation Centre of Gamma House, Enterprise Road,
// Chilworth Science Park, Southampton, SO16 7NS, UK.
//
// This software may not be used, sold, licensed, transferred, copied
// or reproduced in whole or in part in any manner or form or in or
// on any media by any person other than in accordance with the terms
// of the Licence Agreement supplied with the software, or otherwise
// without the prior written consent of the copyright owners.
//
// This software is distributed WITHOUT ANY WARRANTY, without even the
// implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR
// PURPOSE, except where stated in the Licence Agreement supplied with
// the software.
//
// Created By : Michael Boniface
// Created Date : 15-01-2018
// Created for Project : FLAME
-->
# Adaptive Streaming Use Case Scenario
![Scenario Topoligy](/docs/image/scenario-topology.jpg)
![Scenario Deployment](/docs/image/scenario-deployment.jpg)
## Infrastructure Slice
### *compute_node_config*
| compute_node_config | slice | location | comp_node | cpu | memory | storage | timestamp |
| --- | --- | --- | --- | --- | --- |--- | --- |
| compute_node_config | SLICE1 | locA | dc1 | 4 | 8 | 16 | 1515583926868000000 |
| compute_node_config | SLICE1 | locB | dc2 | 8 | 16 | 64 | 1515583926868000000 |
| compute_node_config | SLICE1 | locC | dc3 | 48 | 128 | 4000 | 1515583926868000000 |
### *network_config*
| network_config | slice | network | bandwidth | timestamp |
| --- | --- | --- | --- | --- | --- |--- |
| network_config | SLICE1 | data1 | 100 | 1515583926868000000 |
__How do we describe network configuration ?__
__What is a format of an infrastructure slices ?__
__What is the relevant information ?__
### *network_interface_config*
| network_interface_config | slice | comp_node | port | network | rx_constraint | tx_constraint | timestamp |
| --- | --- | --- | --- | --- | --- |--- |--- |
| network_config | SLICE1 | dc1 | enps03 | data1 | 1000 | 1000 | 1515583926868000000 |
| network_config | SLICE1 | dc2 | enps03 | data1 | 1000 | 1000 | 1515583926868000000 |
| network_config | SLICE1 | dc3 | enps03 | data1 | 1000 | 1000 | 1515583926868000000 |
## NAP
### ipendpoint_route
| ipendpoint_route | location | ipendpoint_id | cont_nav | avg_http_requests_fqdn_rate | avg_network_fqdn_latency | time |
| --- | --- | --- | --- | --- | --- | --- |
| ipendpoint_route | \<common tags> | DC1 | ipendpoint1 | http://netflix.com/scream | 386, | 50 | 1515583926868000000 |
## Media Service
There are various aggregated metrics we can calculate but in the use case scenario we postpone that till later.
### sfc_instance_config
`sfc_i_config,<common_tags>,state <fields> timestamp`
### sf_i_config
`sf_i_config,<common_tags>,state <fields> timestamp`
## IPEndpoint
All IPEndpoint measurements have the following global tags injected by a configured Telegraf agent
* location
* compute_node
* sfc
* sfc_i
* sf
* sfc_i
* ipendpoint
Also NOTE: the metrics provided in the measurements below are effectively a 'snapshot' of usage over a relatively small period of time. The length of this snapshot may vary, depending on the underlying implementation of the instrumentation, so we might have to assume this snapshot is essentially an average of a period of 1 second. Measuring 'usage' is dependent on the units, for example as a proportion of a resource or as a proportion of time.
### ipendpoint_config
| ipendpoint_config | location | sfc | sfc_i | sf | sf_i | ipendpoint | state | cpu| memory | storage |timestamp |
| --- | --- | --- | --- | --- | --- |--- | --- | --- | --- | --- | --- |
| ipendpoint_config | dc1 | MediaServiceTemplate | MediaServiceA | AdaptiveStreamingComp | AdaptiveStreamingComp1 | ipendpoint1 | placed | 2 | 4 | 16 | 1515583926868000000 |
| ipendpoint_config | dc2 | MediaServiceTemplate | MediaServiceA | AdaptiveStreamingComp | AdaptiveStreamingComp1 | ipendpoint2 | placed | 8 | 16 | 64 | 1515583926868000000 |
| ipendpoint_config | dc3 | MediaServiceTemplate | MediaServiceA | AdaptiveStreamingComp | AdaptiveStreamingComp1 | ipendpoint3 | placed | 48 | 128 | 4000 | 1515583926868000000 |
| ipendpoint_config | dc1 | MediaServiceTemplate | MediaServiceA | AdaptiveStreamingComp | AdaptiveStreamingComp1 | ipendpoint1 | booted | 2 | 4 | 16 | 1515583926868000000 |
| ipendpoint_config | dc2 | MediaServiceTemplate | MediaServiceA | AdaptiveStreamingComp | AdaptiveStreamingComp1 | ipendpoint2 | booted | 8 | 16 | 64 | 1515583926868000000 |
| ipendpoint_config | dc3 | MediaServiceTemplate | MediaServiceA | AdaptiveStreamingComp | AdaptiveStreamingComp1 | ipendpoint3 | booted | 48 | 128 | 4000 | 1515583926868000000 |
### cpu_usage
| cpu_usage | \<common tags> | cpu | avg_cpu_time_user | avg_cpu_time_idle | timestamp |
| --- | --- | --- | --- | --- |--- |
| cpu | \<common tags> | 1 | 40 | 5 | 1515583926868000000 |
### net_port_io
| net_port_io | \<common tags> | avg_packet_drop_rate | avg_packet_error_rate | rx_bytes_port_m | rx_packets_m | tx_bytes_port_m | tx_packets_port_m | timestamp |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| net_port_io | \<common tags> | 0.3 | 0.1 | 13567 | 768 | 8102 | 356 | 1515583926868000000 |
### mpegdash_service
| mpegdash_service_mon | \<common tags> | cont_nav | cont_rep | user_profile |avg_req_rate | avg_resp_time | peak_resp_time | avg_error_rate | avg_throughput | avg_quality_delivered | avg_startup_delay | avg_dropped_segments | timestamp |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |--- |
| mpegdash_service_mon | \<common tags> | http://netflix.com/scream | h264 | profileA | 10 | 40 | 230 | 0.2 | 200 | | 5 | 1200 | 2 | 1515583926868000000 |
docs/clmc-information-model.md 0 → 100644
+ 552
− 0
View file @ 0d572515
This diff is collapsed.
docs/flips-integration.md 0 → 100644
+ 164
− 0
View file @ 0d572515
# **Service Management and Control Decisions**
© University of Southampton IT Innovation Centre, 2018
This document describe possible approaches for integrating FLIPS monitoring with the CLMC.
## **Authors**
|Authors|Organisation|
|-|-|
|[Michael Boniface](mailto:mjb@it-innovation.soton.ac.uk)|[University of Southampton, IT Innovation Centre](http://www.it-innovation.soton.ac.uk)|
|[Simon Crowle](mailto:sgc@it-innovation.soton.ac.uk)|[University of Southampton, IT Innovation Centre](http://www.it-innovation.soton.ac.uk)|
### Integration with FLIPS Monitoring
FLIPS offers a scalable pub/sub system for distributing monitoring data. The architecture is described in the POINT monitoring specification https://drive.google.com/file/d/0B0ig-Rw0sniLMDN2bmhkaGIydzA/view. Some observations can be made
* MOOSE and CLMC provide similar functions in the architecture, the CLMC will not have access to MOOSE but will need to subscribe to data points provided by FLIPS
* The APIs for Moly and Blackadder are not provided therefore it's not possible to critically understand the correct implementation approach for agents and monitoring data distribution
* Individual datapoints need to be aggregated into measurements according to a sample rate
* We may need to use the blackadder API for distribution of monitoring data, replacing messaging systems such as AMQP with all buffering and pub/sub deployed on the nodes themselves rather than a central service.
There are a few architectural choices. The first below uses moly as an integration point for monitoring processes via a Telegraf output plugin with data inserted into influx using a blackadder API input plugin on another Telegraf agent running on the CLMC. In this case managing the subscriptions to nodes and data points is difficult. In addition, some data points will be individual from FLIPS monitoring whilst others will be in line protocol format from Telegraf. For the FLIPS data points a new input plugin would be required to aggregate individual data points into time-series measurements.
![FLIPSAgentArchitecture](/docs/image/flips-monitoring-architecture.jpg)
The second (currently preferred) choice only sends line protocol format over the wire. Here we develop telegraf input and output plugins for blackadder benefiting from the scalable nature of the pub/sub system rather than introducing RabbitMQ as a central server. In this case the agent on each node would be configured with input plugins for service, host and network . We'd deploy a new Telegraf input plugin for FLIPS data points on the node's agent by subscribing to blackadder locally and then publish the aggregated measurement using the line protocol back over blackadder to the CLMC. FLIPS can still publish data to MOOSE as required.
![FLIPSAgentArchitecture](/docs/image/flips-monitoring-architecture2.jpg)
The pub/sub protocol still needs some work as we don't want the CLMC to have to subscribe to nodes as they start and stop. We want the nodes to register with a known CLMC and then start publishing data to the CLMC according to a monitoring configuration (e.g. sample rate, etc). So we want a "monitoring topic" that nodes publish to and that the CLMC can pull data from. This topic is on the CLMC itself and note the nodes. Reading the FLIPS specification it seems that this is not how the nodes current distribute data, although could be wrong
#### Network Measurements
**net_port_config**
network config is concerned with any network io allocation/constraints for network rx/tx. Possible fields (but these are not available from the FLIPS monitoring specification)
`net_port_config,<common_tags>,port_id,port_state RX_USAGE_CONSTRAINT, TX_USAGE_CONSTRAINT, RX_THROUGHPUT_CONSTRAINT, TX_THROUGHPUT_CONSTRAINT timestamp`
Specific tags
* port_state
* port_id
**net_port_io**
All net_port_io measurements are monitoring by FLIPS. Note that RX_PACKETS_M seems to have inconsistent naming convention unless we are mistaken
`net_port_io,<common_tags>,port_id PACKET_DROP_RATE_M, PACKET_ERROR_RATE_M, RX_PACKETS_M, TX_PACKETS_PORT_M, RX_BYTES_PORT_M, TX_BYTES_PORT_M timestamp`
Specific tags
* port_id
**Worked Usage Scenario - MPEG-DASH**
The scenario aims to verify two aspects
* CLMC monitoring specification & data acquisition
* Support for initial decision making processes for FLAME (re)orchestration
The scenario is being developed in further document here:
https://gitlab.it-innovation.soton.ac.uk/mjb/flame-clmc/blob/integration/docs/CLMC%20monitoring%20specification%20for%20a%20basic%20scenario.md
The FLAME platform acquires a slice of the infrastructure resources (compute, RAM & storage [C1, C2, C3] and networking). A media service provider offers an MPEG-DASH service to end-users (via their video clients connected to NAPs on the FLAME platform). The service provider deploys surrogates of the MPEG-DASH service on all compute nodes [C1-C3]. All services (including NAPs) are monitored by the CLMC.
Over time a growing number of video clients use a MPEG-DASH service to stream movies on demand. As clients connect and make requests, the platform makes decisions and takes actions in order to maintain quality of service for the increasing number of clients demanding an MPEG-DASH service.
What are the possible criteria (based on metrics and analytics provided by the CLMC) that could be used to help NAP makes these decisions?
In this scenario what are the possible actions a NAP could take?
![Scenario](/docs/image/scenario.jpg)
![Scenario Measurements](/docs/image/scenario-measurements.jpg)
Platform actions
* Increase the resources available to MPEG-DASH surrogates
* This may not be possible if resources unavailable
* Vertical scaling may not solve the problem (i.e., I/O bottleneck)
* Re-route client requests to other MPEG-DASH services
* C1 – Closer to clients, but limited capability
* C3 – Greater capability but further away from clients
… note: NAP service end-point re-routing will need to take into account network factors AND compute resource availability related service KPIs; i.e., end-to-end performance
Service actions
* Lower overall service quality to clients… reduce overall resource usage
* Avg quality met: the ratio of average delivered quality out of requested quality
* Avg start up time: the average time taken before a video stream starts playing less than a threshold
* Avg video stalls: the percentage of stalls (dropped video segments that require re-sending) less than a threshold
# **MISC Measurements and Further Questions**
The following data points require further analysis
* CPU_UTILISATION_M: likely to be replaced by other metrics provided directly by Telegraf plugins
* END_TO_END_LATENCY_M: not clear what this measurement means, so needs clarification
* BUFFER_SIZES_M: needs clarification
* RX_PACKETS_IP_M: is this just NAP or all Nodes
* TX_PACKETS_IP_M: is this just NAP or all Nodes
The following fields need further analysis as they seem to relate to core ICN, most likely fields/measurements related to platform components
* FILE_DESCRIPTORS_TYPE_M
* MATCHES_NAMESPACE_M
* PATH_CALCULATIONS_NAMESPACE_M
* PUBLISHERS_NAMESPACE_M
* SUBSCRIBERS_NAMESPACE_M
The following fields relate to CID which I don't understand but jitter is an important metric so we need to find out.
* PACKET_JITTER_CID_M
* RX_BYTES_CID_M
* TX_BYTES_CID_M
Some questions
* Can a single value of jitter (e.g. avg jitter) be calculated from the set of measurements in PACKET_JITTER_CID_M message? What is the time period for the list of jitter measurements?
* What does CID mean? consecutive identical digits
__Can we measure network usage for a specific VM from FLIPS monitoring?__
__Some metrics from FLIPS contain 'port' label, others not, is this intended?__
QUESTIONS
1. Is the content navigation tag and fully qualified domain name (SDN based)? [Most likely: yes] although this may only be part of the URL?
#### Link Measurements
Links are established between VM/container instances, need to discuss what measurements make sense. Also the context for links could be between media services, therefore a link measurement should be within the platform context and NOT the media service context. Need a couple of scenarios to work this one out.
**link_config**
Link Tags
* link_name
* link_id
* source_node_id
* destination_node_id
* link_type
* link_state
**link_perf**
link perf is measured at the nodes, related to end_to_end_latency. Needs further work.
# Other Issues
**Trust in measurements**
If the agent is deployed in a VM/container that a tenant has root access then a tenant could change the configuration to fake measurements associated with network and host in an attempt gain benefit. This is a security risk. Some ideas include
* Deploy additional agents on hosts rather than agents to measure network and VM performance. Could be hard to differentiate between the different SFs deployed on a host
* Generate a hash from the agent configuration file that's checked within the monitoring message. Probably too costly and not part of the telegraf protocol
* Use unix permissions (e.g. surrogates are deployed within root access to them)
\ No newline at end of file
docs/image/flame-context.jpg
+2.40 KiB (+1%)
View file @ 0d572515
File suppressed by a .gitattributes entry or the file's encoding is unsupported.
docs/monitoring.md
+ 5
− 180
View file @ 0d572515
This diff is collapsed.
docs/service-management-decisions.md 0 → 100644
+ 148
− 0
View file @ 0d572515
# **Service Management and Control Decisions**
© University of Southampton IT Innovation Centre, 2017
This document describe the configuration and monitoring specification for cross-layer management and control within the FLAME platform. All information measured by the CLMC aims to improve management and control decisions made by the platform and/or media service providers against defined performance criteria such as increasing Quality of Experience and cost reduction.
## **Authors**
|Authors|Organisation|
|-|-|
|[Michael Boniface](mailto:mjb@it-innovation.soton.ac.uk)|[University of Southampton, IT Innovation Centre](http://www.it-innovation.soton.ac.uk)|
|[Simon Crowle](mailto:sgc@it-innovation.soton.ac.uk)|[University of Southampton, IT Innovation Centre](http://www.it-innovation.soton.ac.uk)|
## **Service Management and Control Decisions**
Service management decisions relate to processes for Service Request Management, Fault Management and Configuration Management. There are many possible management and control decisions and it is the purpose of the CLMC to provide decision makers with empirical knowledge to design and implement better policies. The FLAME architecture describes how the CLMC uses KPIs to measure performance and highlights examples of control policies such as shortest path routing to a SF and horizontal scaling of SFs in response to changes in workload. A Platform Provider and Media Service Provider will have KPI targets that are different and also not independent of each other. For example, allocating all of the resources needed for an expected peak workload of a media service when it is submitted for orchestration would guarantee a performance level . However, the outcome would typically produce low utilisation and increased costs due to peak workload only being of a fraction of the overall service operation time. The solution is to provide greater flexibility by exploiting points of variabilty within the system in relation to constraints. Constraints are imposed by policy (e.g. a limit on resource allocation) and technology limitations (e.g. VM boot time, horizontal/vertical scaling, routing).
The management and control processes implemented by the FLAME platform define the decisions, variability and constraints. The detail for the implementation of orchestration, management and control is under discussion and the following is based on a best understanding of what was described in the FLAME architecture.
### **An Elementary Starting Point: The Static Configuration Scenario**
The 1st scenario to consider is an entirely static configuration. In this scenario a media service provider defines explicitly the infrastructure resources needed for a media service. The static configuration is what is proposed by the adopted of the current TOSCA specification for the Alpha release. Here an MSP declares the resources needed to deliver a desired performance level (implicitly known to the MSP). In an extreme case, the approach results in a static infrastructure configuration where the MSP defines the entire topology including servers, links and resource requirements. This would include server locations (central, metro and edge DCs) and when the servers are needed. The most basic case is deploy everything now for the lifetime of the media service. This full declaration would statically configure surrogates through the explicit declaration of servers and software deployed on those services.
In this case, the Platform Provider is responsible for allocating the requested resources to the media service provider for the lifetime of the media service. The performance of the service is entirely related to the knowledge of the MSP and the workload over time.
Even this simple example leads important decisions
**D1: “How much infrastructure resource does a PP need from an IP?”**
The infrastructure resource (slice) defines a topology of compute, storage and network resources allocated by an IP to a PP. The slice is used by the PP to resource media services. The PP will allocate proportions of the slice to media services within the lifecycle of such services. In most cases, a PP will need to define resource management policies that define rules for allocation of resources considering that multiple media services are contending for such resources.
The capacity of the slice and the distribution of the resources within the infrastructure is a planning decision made by the PP based on a prediction of media service demand. The allocation of a slice has cost implications as from an IP’s perspective resources are dedicated to a PP. Depending on the business model and cost structures, the slice allocation would typically become a fixed cost to the PP and revenue for the IP. The PP must now allocate the slice to MSPs in the context of KPIs designed to maximise revenue from the slice.
Issues related to this decision include:
* What are the temporal constraints on a slice? Is there a defined end time, recurring subscription or is a slice perpetual?
* How fine grained are temporal constraints considering the fact that an IP has resource scarcity at edge DCs in comparison to metro and central DCs?
* What are the states of a slice? What causes the state transition?
* Can a slice be modified and if so how can the slice change?
*D1 CLMC outcome: a set of measurements describing an infrastructure slice.*
**D2: “How much infrastructure resource does a MSP need from a PP?”**
Once the PP has a slice then media services can be orchestrated, managed and controlled within the slice. Here the PP must consider the MSP infrastructure resource requirements. In the Alpha release FLAME adopts the current TOSCA specification where the MSPs define declaratively server resources required for each SF. The PP has no understanding of how a media service will behave in response to the resource allocation as that knowledge is within the MSP. In TOSCA++ FLAME is exploring KPI-based media service specifications where resource management knowledge forms part of the platform’s responsibility.
Issues related to this decision include:
* What are the temporal constraints on resource requirements within a TOSCA specification?
* How fine grained are the temporal constraints considering that a media service includes a set of media components with temporal resourcing requirements? E.g. media component A needs resource on Monday and media component B resource on Tuesday.
* What are the spatial constraints associated with the resource requirements? Does an MSP specify the precise DC (or set of DCs) where the SF needs or can be deployed? In effect, if the MSP says where the SF needs to be deployed this encodes the surrogate policy directly within the media service definition.
* How much variability are there is routing rules? How much of this is implicit within the platform implementation (e.g. coincidental multicast features)
*D2 CLMC outcome: a set of measurements describing media service infrastructure requirements.*
### **Where Next: Fast Variable Configuration Scenarios**
Variable configuration identifies configuration state that can change in the lifetime of a media service. Variability in configuration state includes:
* Vertically up and down scaling SF resources (i.e. compute, storage, network IO)
* Horizontally up and down scaling SF resources (i.e. replication)
* Distributing SFs by location (i.e. placement of a VM on an edge DC)
* Routing traffic between SFs (i.e. load balancing algorithms)
* Adapting content (i.e. reducing the resolution of a video stream)
Each transition in state is a decision that has a time in the lifecycle (when is it implemeted), a duration (how long does it take to implement), actor (who is responsible) and an expected outcome.
General issues reated to variable configuration include:
* What are the points of variability within the platform?
* How is variability configured, either through default platform policy or TOSCA templates?
* Where are we contributing innovation in variability? e.g. network + compute + service factors considered together
We now discuss key decisions associated variable configuration
**D3: “When should resources be allocated to a media service”?**
When a PP receives a request to orchestrate a media service the PP must decide on when to allocate infrastructure resources. Allocation has a temporal dimension defining a start time and end time. An allocation in the future can be seen as a commitment. Allocation is important for accounting purposes but even more important in situation of resource scarcity. In most public clouds, resources from a MSP perspective are assumed to be infinite and there’s little need to consider temporal constraints associated with resource allocations. As long as the MSP has budget to pay for the resources, public cloud providers will scale those resources as requested.
In FLAME we have resource scarcity and contention in edge DCs and therefore MSPs and the PP must find workable ways to negotiate allocation of resources over time. Different resource management policies can be considered.
* Allocate on request: PP allocates when the orchestration request is made. The PP would determine if sufficient infrastructure capacity exists considering the current commitments to other media services and if capacity is available then the resources would be allocated. This is a reservation for an MSP and is likely to result in underutilisation of the resources and increased costs for an MSP but may be needed to guarantee performance.
* Allocate on placement: PP allocates when the SFs are placed. The impact depends on the placement strategy as if SFs are placed when the MS is requested it will have the same effect to allocate all on request. If placement is selective based on factors such as utilisation/demand then some cost reduction may be achieved at the risk the resources might not be available. Note that placement does incur resource allocation to the MSP (e.g. storage and network I/O for ingest) but this is traded off with the potential to boot and respond to demand quickly.
* Allocate on boot: PP allocates when then SFs are booted if they are available. Here the VMs are placed with a defined resource that’s allocated when the machine boots. The PP needs to decide if the machine can be booted according to the utilisation by other VMs deployed on the server.
* Best effort with contention ratio: PP does not make any attempt to allocate resources but does place based on a defined contention ratio. Here there’s a high risk that performance is degraded by others competing for the resources
Some resource management constraints relate to peak usage rate, for example, 50M/s peak and 100G a month usage.
Issues related to this decision include:
* What is the resource management policy for Alpha?
* Do different policies apply for different types of infrastructure resources?
* How long does it take to allocate different types of infrastructure resources?
*D3 CLMC outcome: a set of measurements describing an allocation of infrastructure to a media service or SF over a time period*
**D4: “Where can a SF be placed over a time period”?**
When a media service tempalte is submitted for orchestration the PP must determine where SFs can be placed. Placement of a SF results in a VM being deployed on a server ready to be booted. Placement uses storage resources associated with a VM and network resources for ingest of the VM/content but does not utilise resources such as cpu, memory and data i/o incurred when the VM is used.
In alpha where no KPIs are provided, placement is a spatial/temporal decision based on a function of the following measurements
* infrastructure slice
* media service allocations
* SF server requirements
*The outcome is a set of server options where an SF could be placed within a time period. This outcome is not related to the CLMC monitoring beyond CLMC measurements providing input to placement functions*
**D5: “Where is a SF best placed over a time period”?**
The question of where is it best to place an SF is challenging and depends on responsibility for delivering KPIs. A PP define a KPI to achieve an utilisation target of 80% for servers and place VMs on servers according to an utilisation measurement. A MSP may have a KPI to achieve a response time of 200mS for 80% of requests and place VMs according to a request rate and location measurement.
*The outcome is a decision on where to place a SF. There’s no change to system state at this point just a decision to take an action now or in the future. *
**D6: “When is a SF placed”?**
The placement strategy is driven by KPIs such as response time. Placement takes time to transfer the VMs and content to a service. Placed VMs boot faster but they consume storage resources as a consequence.
A default PP strategy may be needed for the alpha release. For example, a strategy could be to place and boot in a metro or central DC where there’s less scarcity, and then selectively place/boot VMs in edge DCs on demand. However it’s not clear how such behaviour can be expressed in the TOSCA specification and how this relates to allocations.
A default policy could be that the PP can place a SF on any compute node in the network where there’s sufficient resources with a guarantee that there will be at least one instance of a SF, it’s then the PPs decision to create surrogates rather than have an explicit definition as per the static configuration scenario above. This policy is sensible as it moves towards the situation where the PP manages services based on KPIs, however it does require the PP to manage allocations over time in response to demand.
*D6 CLMC outcome: VM configuration measurement updating state to “placed”*
**D7: “When is a SF be booted?”**
The booting strategy is driven by KPIs such as response time. VMs take time to boot. Booted VMs are available to serve requests routed to them immediately. When SF’s are booted the VM consumes resources in accordance within the context of the applicable resource management policy (e.g. guaranteed allocation or with contention ratio)
*D5 CLMC outcome: VM configuration measurement updating state to “booted”*
**D8: “Which surrogate are requests routed to?”**
An SFC may have multiple surrogate services booted serving requests. A decision needs to be made on where to route requests. In a typical load balancing situation requests are routed using algorithms such as round robin and source-based. Routing to the closest surrogate may not deliver improved performance especially if the surrogate is deployed on a resource constrained server and the NAP is experiencing a high level of demand. In many typical load balancing scenarios, the servers are homogenous, network delay is not considered and requests are processed from a central point of access. In our scenario the server resources are heterogeneous, network delay is critical and requests enter from multiple points of access as defined by NAPs.
At this point it’s worth highlighting that we are considering E2E performance and that each step in an end to end process contributions to an overall performance. If we take latency (as a key benefit of the platform), the E2E latency is the sum of delays in network and servers contributing to a content delivery process as shown in the diagram below:
![E2E latency](/docs/image/e2e-latency.jpg)
If we the average delay for parts of a process over a time period we have some indication of best routing policy.
The variability factors that influence E2E latency include:
* Spatial/temporal demand
* Server placement and server resource allocation/contention over time
* Network routing and network resource allocation/contention over time
Issues related to this decision include:
* How are NAP routing decisions coordinated as requests are not sourced from a central point?
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment