diff --git a/docs/AlertsSpecification.md b/docs/AlertsSpecification.md index e4e1cc78d2501ac79baec43bb11594b2eba0a826..e97729aadb7549346a5d51474d0bf2e8b7066695 100644 --- a/docs/AlertsSpecification.md +++ b/docs/AlertsSpecification.md @@ -43,8 +43,7 @@ https://github.com/openstack/tosca-parser/blob/master/toscaparser/tests/data/pol The TOSCA Alerts Specification Document consists of two main sections - **metadata** and **policies**. Each **policy** contains a number of triggers. A **trigger** is a fully qualified specification for an alert. Full definitions and clarification of the structure of the document -is given in the following sections. An example of a valid alert specification -document will look like: +is given in the following sections. An example of a valid alert specification document will look like: ```yaml tosca_definitions_version: tosca_simple_profile_for_nfv_1_0_0 @@ -62,7 +61,7 @@ topology_template: policies: - high_latency_policy: - type: eu.ict-flame.policies.StateChange + type: eu.ict-flame.policies.Alert triggers: high_latency: description: This event triggers when the mean network latency in a given location exceeds a given threshold (in ms). @@ -77,11 +76,11 @@ topology_template: comparison_operator: gt action: implementation: - - http://sfemc.flame.eu/notify + - flame_sfemc - http://companyA.alert-handler.flame.eu/high-latency - low_requests_policy: - type: eu.ict-flame.policies.StateChange + type: eu.ict-flame.policies.Alert triggers: low_requests: description: | @@ -100,11 +99,11 @@ topology_template: comparison_operator: lt action: implementation: - - http://sfemc.flame.eu/notify + - flame_sfemc - http://companyA.alert-handler.flame.eu/low-requests - requests_diff_policy: - type: eu.ict-flame.policies.StateChange + type: eu.ict-flame.policies.Alert triggers: increase_in_requests: description: | @@ -116,8 +115,6 @@ topology_template: threshold: 100 # requests have increased by at least 100 granularity: 120 resource_type: - flame_sfc: companyA-VR - flame_sfci: companyA-VR-premium flame_sfp: storage flame_sf: storage-users flame_server: watershed @@ -125,7 +122,7 @@ topology_template: comparison_operator: gte action: implementation: - - http://sfemc.flame.eu/notify + - flame_sfemc decrease_in_requests: description: | This event triggers when the number of requests has decreased relative to the number of requests received @@ -142,10 +139,10 @@ topology_template: comparison_operator: lte action: implementation: - - http://sfemc.flame.eu/notify + - flame_sfemc - missing_measurement_policy: - type: eu.ict-flame.policies.StateChange + type: eu.ict-flame.policies.Alert triggers: missing_storage_measurements: description: This event triggers when the number of storage measurements reported falls below the threshold value. @@ -160,7 +157,7 @@ topology_template: flame_sfp: storage action: implementation: - - http://sfemc.flame.eu/notify + - http://companyA.alert-handler.flame.eu/missing-measurements ``` @@ -178,14 +175,14 @@ metadata: ##### Policies The ***policies*** section defines a list of policy nodes, each representing a fully qualified configuration for an -alert within CLMC. The format is the following: +alert within CLMC. Each policy must be of type eu.ict-flame.policies.Alert. The format is the following: ```yaml topology_template: policies: - <policy_identifier>: - type: eu.ict-flame.policies.StateChange + type: eu.ict-flame.policies.Alert triggers: <event identifier>: description: <optional description for the given event trigger> @@ -202,8 +199,8 @@ topology_template: comparison_operator: <logical operator to use for comparison, e.g. 'gt', 'lt', 'gte', etc. action: implementation: - - <HTTP Alert Handler URL - receives POST messages from Kapacitor when alerts trigger> - - <HTTP Alert Handler URL - receives POST messages from Kapacitor when alerts trigger> + - <flame_sfemc or HTTP Alert Handler URL - receives POST messages from Kapacitor when alerts trigger> + - <flame_sfemc or HTTP Alert Handler URL - receives POST messages from Kapacitor when alerts trigger> ... ... ``` @@ -239,19 +236,24 @@ the format is still the same for consistency. Therefore, using `<measurement>.*` the event_type is set to **threshold**. * **resource_type** - provides context for the given event - key-value pairs for the global tags of the CLMC Information Model. -This includes any of the following: `"flame_sfc", "flame_sfci", "flame_sfp", "flame_sf", "flame_sfe", "flame_server", "flame_location"`. -Keep in mind that filtering for **sfc** and **sfci** is automatically generated by extracting the metadata values from the alerts specification. +This includes any of the following: `"flame_sfp", "flame_sf", "flame_sfe", "flame_server", "flame_location"`. +Keep in mind that **flame_sfc** and **flame_sfci** are also part of the CLMC Information Model. However, filtering on +these tags is automatically generated and added to all InfluxDB queries by using the metadata values from the +alerts specification. Therefore, including **flame_sfc** and **flame_sfci** in the **resource_type** is considered INVALID. For more information on the global tags, please check the [documentation](monitoring.md). * **comparison_operator** - the logical operator to use for comparison - lt (less than), gt (greater than), lte (less than or equal to), etc. -* **implementation** - a list of the URLs of alert handlers to which alert data is sent when the event condition is true. +* **implementation** - a list of the URL entries for alert handlers to which alert data is sent when the event condition is true. +If the alert is supposed to be sent to SFEMC, then instead of typing a URL, use **flame_sfemc** - the configurator will generate the correct +SFEMC URL. ##### Event types * **threshold** - A threshold event type is an alert in which Kapacitor queries InfluxDB on specific metric in a given period of time -by using a query function such as *mean*, *median*, *mode*, etc. This value is then compared against a given threshold. If the +by using a query function such as *mean*, *median*, *mode*, etc. If the granularity is less than or equal to 60 seconds, then every measurement +point is monitored (improving performance), thus, ignoring the aggregation function. This value is then compared against a given threshold. If the result of the comparison operation is true, an alert is triggered. For example: ```yaml @@ -268,7 +270,7 @@ result of the comparison operation is true, an alert is triggered. For example: comparison_operator: gt action: implementation: - - http://sfemc.flame.eu/notify + - flame_sfemc - http://companyA.alert-handler.flame.eu/high-latency ``` @@ -313,7 +315,7 @@ threshold. If the result of the comparison operation is true, an alert is trigge comparison_operator: lte action: implementation: - - http://sfemc.flame.eu/notify + - flame_sfemc ``` This trigger specification will create an alert task in Kapacitor, which compares every **requests** value reported in @@ -345,7 +347,7 @@ For example: flame_sfp: storage action: implementation: - - http://sfemc.flame.eu/notify + - flame_sfemc ``` This trigger specification will create an alert task in Kapacitor, which monitors the number of points reported in @@ -361,3 +363,53 @@ For example: * even if you put something else for field value, it will be ignored - only the **measurement** name is used * **aggregation_method** is not required in this event type, any values provided will be ignored * **comparison operator** is not required in this event type, any values provided will be ignored + + +##### Alert messages + +Every alert handler registered in a trigger *action --> implementation* section receives an alert message when the trigger +event condition is true. This alert message is sent using an *HTTP POST* request to the URL of the alert handler. The alert +message is generated by Kapacitor and, currently, clmc-service has limited control over it. An alert message follows this format: + +```json +{ + "message": "TRUE", + "id": "<trigger_id>", + "level": "CRITICAL", + "duration": "<integer for the duration of alert - nanoseconds>", + "previousLevel": "<the previous level of the alert>", + "details": "<a context string with info on what triggered the alert>", + "time": "<timestamp of the alert occurrence>", + "data": { + "series": [ + { + "name": "<measurement name of the alert metric>", + "tags": { + "<tag name>": "<tag value>" + }, + "columns": [ + "<column name>" + ], + "values": [ + [ + "<values of each column name>" + ] + ] + } + ] + } +} +``` + +* message - currently, this is always set to "TRUE" +* id - trigger ID as defined in the alert specification document +* level - the level of the alert; currently all alerts that trigger have their level set as CRITICAL +* previousLevel - the previous level of the alert +* details - a string in the format "db=<db_name>,sfc=<sfc_id>,sfci=<sfc_instance_id>,policy=<policy_id>" providing context of the alert +* duration - integer, duration of the alert in nanoseconds +* time - timestamp of the point that triggered the alert +* data - describes the point(s) that triggered the alert +* data.series.name - the name of the measurement +* data.series.tags - (OPTIONAL) key-value pairs for all measurement tags +* data.series.columns - list of column names +* data.series.values - list of list of values, each nested list represents a measurement point and the values for each column diff --git a/scripts/clmc-service/install-clmc-service.sh b/scripts/clmc-service/install-clmc-service.sh index c6ba51b259638e1ec105ec3418535bc79038aa27..126ea5f6779ce279d320aaa5b1b0d13baca92c95 100755 --- a/scripts/clmc-service/install-clmc-service.sh +++ b/scripts/clmc-service/install-clmc-service.sh @@ -102,6 +102,7 @@ mkdir -p /opt/flame/clmc start_script_file="/opt/flame/clmc/start.sh" echo "#!/bin/bash" > $start_script_file echo "export WORKON_HOME=${HOME}/.virtualenvs" >> $start_script_file +echo "export SFEMC_FQDN=${SFEMC_FQDN}" >> $start_script_file echo "source /usr/local/bin/virtualenvwrapper.sh" >> $start_script_file echo "workon CLMC" >> $start_script_file echo "pserve ${REPO_ROOT}/src/service/production.ini &" >> $start_script_file diff --git a/scripts/clmc-service/install.sh b/scripts/clmc-service/install.sh index 90039438a70ee2c502a3d335f9174f80be87cdf9..f1a7aa1c828632351940b2ff5362e1c4650335a0 100755 --- a/scripts/clmc-service/install.sh +++ b/scripts/clmc-service/install.sh @@ -27,6 +27,11 @@ # Force fail on command fail (off for now as virtualenvwrapper install fails) set -euo pipefail +if [[ -z "${SFEMC_FQDN}" ]]; then + echo "Failed: cannot find SFEMC_FQDN environment variable." + exit 1 +fi + # Ensure everything runs in directory of the parent script cd `dirname $0` diff --git a/scripts/test/fixture.sh b/scripts/test/fixture.sh index 725e21d8e58a2df1f129c3c7e0a606d595d0d4ac..8c568ff2dc1ae36294af8484850bf3d79da9decd 100755 --- a/scripts/test/fixture.sh +++ b/scripts/test/fixture.sh @@ -52,7 +52,7 @@ create() { if [ ${service_name} == "clmc-service" ]; then cmd="/vagrant/scripts/clmc-service/install.sh" echo "Provisioning command ${cmd}" - lxc exec ${service_name} --env REPO_ROOT="/vagrant" -- ${cmd} + lxc exec ${service_name} --env REPO_ROOT="/vagrant" --env SFEMC_FQDN="sfemc.localhost" -- ${cmd} exit_code=$? if [ $exit_code != 0 ]; then echo "clmc-service installation failed with exit code ${exit_code}" diff --git a/src/service/clmcservice/__init__.py b/src/service/clmcservice/__init__.py index b710578a689fb7066c777e3143f8883c49e05bf1..70c6e76cad9a94cfca999f2b6f54ab717627c2b8 100644 --- a/src/service/clmcservice/__init__.py +++ b/src/service/clmcservice/__init__.py @@ -26,6 +26,7 @@ # Python standard libs from json import load from os.path import dirname, abspath +import os # PIP installed libs from pyramid.config import Configurator @@ -47,6 +48,8 @@ def main(global_config, **settings): DBSession.configure(bind=engine) # bind the engine to a DB session Base.metadata.bind = engine # bind the engine to the Base class metadata + settings['sfemc_fqdn'] = os.environ['SFEMC_FQDN'] # read the SFEMC FQDN from the OS environment + settings['influx_port'] = int(settings['influx_port']) # the influx port setting must be converted to integer instead of a string settings['kapacitor_port'] = int(settings['kapacitor_port']) # the kapacitor port setting must be converted to integer instead of a string diff --git a/src/service/clmcservice/alertsapi/alerts_specification_schema.py b/src/service/clmcservice/alertsapi/alerts_specification_schema.py index e2365bf21704b2b926ee8049badd956fca6330c2..ed2a5e749335db2d90a3da104512b3e6ff344a4f 100644 --- a/src/service/clmcservice/alertsapi/alerts_specification_schema.py +++ b/src/service/clmcservice/alertsapi/alerts_specification_schema.py @@ -35,11 +35,14 @@ from schema import Schema, And, Or, Optional, SchemaError # * metadata section must be present (with key-value pairs for sfc and sfci) # * policies section must be present (under the topology_template node) # * each policy must be associated with a triggers node (containing at least 1 trigger) -# * each policy is of type eu.ict-flame.policies.StateChange or eu.ict-flame.policies.Alert +# * each policy is of type eu.ict-flame.policies.Alert # * each trigger must specify event_type, metric, condition, and at least one handler in action/implementation # * the condition section must specify threshold, granularity, aggregation_method, comparison_operator +SFEMC = "flame_sfemc" # describes the keyword used for the SFEMC alert handler + + # Influx QL functions defined in the documentation https://docs.influxdata.com/influxdb/v1.6/query_language/functions/ INFLUX_QL_FUNCTIONS = ( "count", "mean", "median", "mode", "sum", "first", "last", "max", "min" @@ -62,7 +65,9 @@ URL_REGEX = re.compile( re.IGNORECASE) # Global tags allowed to be used for filtering in the trigger condition -CLMC_INFORMATION_MODEL_GLOBAL_TAGS = {"flame_sfc", "flame_sfci", "flame_sfp", "flame_sf", "flame_sfe", "flame_server", "flame_location"} +CLMC_INFORMATION_MODEL_GLOBAL_TAGS = {"flame_sfp", "flame_sf", "flame_sfe", "flame_server", "flame_location"} +# NOTICE that "flame_sfc", "flame_sfci" are not allowed, eventhough they are part of the CLMC Information Model +# This is because those two tags are automatically added to the InfluxDB queries - the required values are retrieved from the alert spec. metadata ALERTS_SPECIFICATION_SCHEMA = Schema({ "tosca_definitions_version": And(str, lambda v: v == "tosca_simple_profile_for_nfv_1_0_0"), @@ -76,7 +81,7 @@ ALERTS_SPECIFICATION_SCHEMA = Schema({ "policies": [ { str: { - "type": Or("eu.ict-flame.policies.StateChange", "eu.ict-flame.policies.Alert"), + "type": "eu.ict-flame.policies.Alert", "triggers": And({ str: { Optional("description"): str, @@ -94,7 +99,7 @@ ALERTS_SPECIFICATION_SCHEMA = Schema({ "action": { "implementation": [ - And(str, lambda s: URL_REGEX.match(s) is not None) + Or(SFEMC, And(str, lambda s: URL_REGEX.match(s) is not None)) ] } } diff --git a/src/service/clmcservice/alertsapi/tests.py b/src/service/clmcservice/alertsapi/tests.py index c142707c06f7723b7597ea1d9a4898b95c12a4c7..535db2350d8d3e19f038d275170a512f91c213b5 100644 --- a/src/service/clmcservice/alertsapi/tests.py +++ b/src/service/clmcservice/alertsapi/tests.py @@ -25,7 +25,7 @@ # Python standard libs from os import listdir -from os.path import isfile, join, dirname +from os.path import isfile, join, dirname, splitext # PIP installed libs import pytest @@ -62,7 +62,7 @@ class TestAlertsConfigurationAPI(object): """ self.registry = testing.setUp() - self.registry.add_settings({"kapacitor_host": "localhost", "kapacitor_port": 9092}) + self.registry.add_settings({"kapacitor_host": "localhost", "kapacitor_port": 9092, "sfemc_fqdn": "sfemc.localhost"}) yield @@ -272,6 +272,7 @@ def extract_alert_spec_data(alert_spec): :return: a tuple containing sfc_id and sfc_instance_id along with a list and a dictionary of generated IDs (alert IDs (list), topic IDs linked to handler IDs (dict)) """ + version = splitext(alert_spec.name)[0].split("-")[-1] # take the ending number of the alert spec file yaml_alert_spec = load(alert_spec) adjust_tosca_definitions_import(yaml_alert_spec) tosca_tpl = ToscaTemplate(yaml_dict_tpl=yaml_alert_spec) @@ -296,6 +297,10 @@ def extract_alert_spec_data(alert_spec): alert_ids.append((alert_id, alert_type)) for handler_url in trigger.trigger_tpl["action"]["implementation"]: + + if handler_url == "flame_sfemc": + handler_url = "http://sfemc.localhost/{0}/{1}/{2}/{3}".format(sfc, sfc_instance, policy_id, "trigger_id_{0}".format(version)) + handler_id = "{0}\n{1}\n{2}\n{3}\n{4}".format(sfc, sfc_instance, policy_id, trigger_id, handler_url) handler_id = AlertsConfigurationAPI.get_hash(handler_id) topic_handlers[topic_id].append((handler_id, handler_url)) diff --git a/src/service/clmcservice/alertsapi/utilities.py b/src/service/clmcservice/alertsapi/utilities.py index eba22f2c3e631b8548ed4d3c79dcff591d302187..53a144beb4797736746651526c2eafcf667c0d41 100644 --- a/src/service/clmcservice/alertsapi/utilities.py +++ b/src/service/clmcservice/alertsapi/utilities.py @@ -31,6 +31,7 @@ from yaml import load # CLMC-service imports from clmcservice import ROOT_DIR +from clmcservice.alertsapi.alerts_specification_schema import SFEMC CLMC_ALERTS_TOSCA_DEFINITIONS_REL_PATH = ["static", "flame_clmc_alerts_definitions.yaml"] @@ -57,18 +58,18 @@ def adjust_tosca_definitions_import(alert_spec): pass # nothing to replace if the import is not specified (either imports are missed, or no reference to the clmc tosca definitions file) -def get_resource_spec_topic_ids(resource_spec_reference): +def get_resource_spec_policy_triggers(resource_spec_reference): """ Tries to extract all event identifiers from a TOSCA resource specification :param resource_spec_reference: the resource specification file reference from the POST HTTP request - :return: sfc ID, sfc instance ID and the list of topic IDs + :return: sfc ID, sfc instance ID and the map of policy trigger IDs linked to the resource spec trigger id """ resource_spec = load(resource_spec_reference.file) - topic_ids = [] + policy_trigger_ids = {} sfc, sfc_i = resource_spec["metadata"]["sfc"], resource_spec["metadata"]["sfci"] policies = resource_spec["topology_template"]["policies"] @@ -79,25 +80,26 @@ def get_resource_spec_topic_ids(resource_spec_reference): if policy_object["type"] == "eu.ict-flame.policies.StateChange": triggers = policy_object["triggers"] - for trigger in triggers.values(): + for trigger_id in triggers: + trigger = triggers[trigger_id] event = trigger["condition"]["constraint"] source, event_id = event.split("::") if source.lower() == "clmc": # only take those event IDs that have clmc set as their source - topic_ids.append("{0}\n{1}".format(policy_id, event_id)) + policy_trigger_ids["{0}\n{1}".format(policy_id, event_id)] = trigger_id - return sfc, sfc_i, topic_ids + return sfc, sfc_i, policy_trigger_ids -def get_alert_spec_topic_ids(alerts_spec_tpl): +def get_alert_spec_policy_triggers(alerts_spec_tpl): """ Tries to extract all event identifiers from a TOSCA alerts specification :param alerts_spec_tpl: the alerts specification TOSCA template object - :return: the list of topic IDs + :return: the list of policy triggers """ - topic_ids = [] + policy_trigger_ids = [] for policy in alerts_spec_tpl.policies: policy_id = policy.name @@ -105,10 +107,12 @@ def get_alert_spec_topic_ids(alerts_spec_tpl): for trigger in policy.triggers: trigger_id = trigger.name - topic_id = "{0}\n{1}".format(policy_id, trigger_id) - topic_ids.append(topic_id) + # only include those triggers that actually POST to SFEMC + if SFEMC in trigger.trigger_tpl["action"]["implementation"]: + policy_trigger_string = "{0}\n{1}".format(policy_id, trigger_id) + policy_trigger_ids.append(policy_trigger_string) - return topic_ids + return policy_trigger_ids def fill_http_post_handler_vars(handler_id, handler_url): @@ -152,7 +156,7 @@ class TICKScriptTemplateFiller: return fill_function(**kwargs) @staticmethod - def _fill_threshold_batch_template_vars(db=None, measurement=None, field=None, influx_function=None, critical_value=None, + def _fill_threshold_batch_template_vars(sfc=None, sfci=None, policy=None, db=None, measurement=None, field=None, influx_function=None, critical_value=None, comparison_operator=None, alert_period=None, topic_id=None, event_id=None, where_clause=None, **kwargs): """ Creates a dictionary object ready to be posted to kapacitor to create a "threshold" task from template. @@ -174,6 +178,18 @@ class TICKScriptTemplateFiller: comparison_lambda = '"real_value" {0} {1}'.format(comparison_operator, critical_value) # build up lambda string, e.g. "real_value" >= 10 template_vars = { + "sfc": { + "type": "string", + "value": sfc + }, + "sfci": { + "type": "string", + "value": sfci + }, + "policy": { + "type": "string", + "value": policy + }, "db": { "type": "string", "value": db @@ -217,7 +233,7 @@ class TICKScriptTemplateFiller: return template_vars @staticmethod - def _fill_threshold_stream_template_vars(db=None, measurement=None, field=None, critical_value=None, + def _fill_threshold_stream_template_vars(sfc=None, sfci=None, policy=None, db=None, measurement=None, field=None, critical_value=None, comparison_operator=None, topic_id=None, event_id=None, where_clause=None, **kwargs): """ Creates a dictionary object ready to be posted to kapacitor to create a "threshold" task from template. @@ -239,6 +255,18 @@ class TICKScriptTemplateFiller: comparison_lambda = '"{0}" {1} {2}'.format(field, comparison_operator, critical_value) # build up lambda string, e.g. "real_value" >= 10 template_vars = { + "sfc": { + "type": "string", + "value": sfc + }, + "sfci": { + "type": "string", + "value": sfci + }, + "policy": { + "type": "string", + "value": policy + }, "db": { "type": "string", "value": db @@ -271,7 +299,7 @@ class TICKScriptTemplateFiller: return template_vars @staticmethod - def _fill_relative_template_vars(db=None, measurement=None, field=None, influx_function=None, critical_value=None, comparison_operator=None, + def _fill_relative_template_vars(sfc=None, sfci=None, policy=None, db=None, measurement=None, field=None, influx_function=None, critical_value=None, comparison_operator=None, alert_period=None, topic_id=None, event_id=None, where_clause=None, **kwargs): """ Creates a dictionary object ready to be posted to kapacitor to create a "relative" task from template. @@ -293,6 +321,18 @@ class TICKScriptTemplateFiller: comparison_lambda = '"diff" {0} {1}'.format(comparison_operator, critical_value) template_vars = { + "sfc": { + "type": "string", + "value": sfc + }, + "sfci": { + "type": "string", + "value": sfci + }, + "policy": { + "type": "string", + "value": policy + }, "db": { "type": "string", "value": db @@ -336,7 +376,7 @@ class TICKScriptTemplateFiller: return template_vars @staticmethod - def _fill_deadman_template_vars(db=None, measurement=None, critical_value=None, alert_period=None, topic_id=None, event_id=None, where_clause=None, **kwargs): + def _fill_deadman_template_vars(sfc=None, sfci=None, policy=None, db=None, measurement=None, critical_value=None, alert_period=None, topic_id=None, event_id=None, where_clause=None, **kwargs): """ Creates a dictionary object ready to be posted to kapacitor to create a "deadman" task from template. @@ -352,6 +392,18 @@ class TICKScriptTemplateFiller: """ template_vars = { + "sfc": { + "type": "string", + "value": sfc + }, + "sfci": { + "type": "string", + "value": sfci + }, + "policy": { + "type": "string", + "value": policy + }, "db": { "type": "string", "value": db diff --git a/src/service/clmcservice/alertsapi/views.py b/src/service/clmcservice/alertsapi/views.py index 74ce6e928c2495243c64dec30906d4f942755fb7..72681ac34eb3e398971cbf3fddb41140c14b81ad 100644 --- a/src/service/clmcservice/alertsapi/views.py +++ b/src/service/clmcservice/alertsapi/views.py @@ -34,8 +34,8 @@ from toscaparser.tosca_template import ToscaTemplate from requests import post # CLMC-service imports -from clmcservice.alertsapi.utilities import adjust_tosca_definitions_import, TICKScriptTemplateFiller, fill_http_post_handler_vars, get_resource_spec_topic_ids, get_alert_spec_topic_ids -from clmcservice.alertsapi.alerts_specification_schema import COMPARISON_OPERATORS, validate_clmc_alerts_specification +from clmcservice.alertsapi.utilities import adjust_tosca_definitions_import, TICKScriptTemplateFiller, fill_http_post_handler_vars, get_resource_spec_policy_triggers, get_alert_spec_policy_triggers +from clmcservice.alertsapi.alerts_specification_schema import COMPARISON_OPERATORS, SFEMC, validate_clmc_alerts_specification # initialise logger log = logging.getLogger('service_logger') @@ -100,15 +100,20 @@ class AlertsConfigurationAPI(object): alert_spec_reference = self.request.POST.get('alert-spec') resource_spec_reference = self.request.POST.get('resource-spec') + + # check that the resource specification file was sent + if not hasattr(resource_spec_reference, "file") or not hasattr(resource_spec_reference, "filename"): + raise HTTPBadRequest("Request to this API endpoint must include a (YAML) file input referenced as 'resource-spec' representing the TOSCA Resource Specification.") + try: - resource_spec_sfc, resource_spec_sfc_i, resource_spec_topic_ids = get_resource_spec_topic_ids(resource_spec_reference) + resource_spec_sfc, resource_spec_sfc_i, resource_spec_policy_triggers = get_resource_spec_policy_triggers(resource_spec_reference) except Exception as e: log.error("Couldn't extract resource specification event IDs due to error: {0}".format(e)) raise HTTPBadRequest("Couldn't extract resource specification event IDs - invalid TOSCA resource specification.") - # check that the specification file was sent + # check that the alerts specification file was sent if not hasattr(alert_spec_reference, "file") or not hasattr(alert_spec_reference, "filename"): - raise HTTPBadRequest("Request to this API endpoint must include a (YAML) file input referenced as 'alert-spec' representing the TOSCA Alerts Specification.") + raise HTTPBadRequest("Request to this API endpoint must include a (YAML) file input referenced as 'alert-spec' representing the TOSCA Alert Specification.") # extract alert specification file and filename alerts_input_filename = alert_spec_reference.filename @@ -136,19 +141,20 @@ class AlertsConfigurationAPI(object): if not valid_alert_spec: raise HTTPBadRequest("Request alert specification file could not be validated as a CLMC TOSCA alerts specification document.") - alert_spec_topic_ids = get_alert_spec_topic_ids(tosca_tpl) + alert_spec_policy_triggers = get_alert_spec_policy_triggers(tosca_tpl) sfc, sfc_instance = tosca_tpl.tpl["metadata"]["sfc"], tosca_tpl.tpl["metadata"]["sfci"] # do validation between the two TOSCA documents - self._compare_alert_and_resource_spec(sfc, sfc_instance, alert_spec_topic_ids, resource_spec_sfc, resource_spec_sfc_i, resource_spec_topic_ids) + self._compare_alert_and_resource_spec(sfc, sfc_instance, alert_spec_policy_triggers, resource_spec_sfc, resource_spec_sfc_i, resource_spec_policy_triggers) + + db = sfc # database per service function chain, named after the service function chain ID - db = sfc # ASSUMPTION: database per service function chain, named after the service function chain ID # two lists to keep track of any errors while interacting with the Kapacitor HTTP API alert_tasks_errors = [] alert_handlers_errors = [] # iterate through every policy and extract all triggers of the given policy - self._config_kapacitor_alerts(tosca_tpl, sfc, sfc_instance, db, kapacitor_host, kapacitor_port, alert_tasks_errors, alert_handlers_errors) + self._config_kapacitor_alerts(tosca_tpl, sfc, sfc_instance, db, kapacitor_host, kapacitor_port, resource_spec_policy_triggers, alert_tasks_errors, alert_handlers_errors) return_msg = {"msg": "Alerts specification has been successfully validated and forwarded to Kapacitor", "service_function_chain_id": sfc, "service_function_chain_instance_id": sfc_instance} @@ -163,16 +169,16 @@ class AlertsConfigurationAPI(object): return return_msg - def _compare_alert_and_resource_spec(self, alert_spec_sfc, alert_spec_sfc_instance, alert_spec_topics, resource_spec_sfc, resource_spec_sfc_instance, resource_spec_topics): + def _compare_alert_and_resource_spec(self, alert_spec_sfc, alert_spec_sfc_instance, alert_spec_policy_triggers, resource_spec_sfc, resource_spec_sfc_instance, resource_spec_policy_triggers): """ Compares the extracted values from the resource spec against the values from the alerts spec - validation that they refer to the same things, :param alert_spec_sfc: sfc from alert spec :param alert_spec_sfc_instance: sfc instance from alert spec - :param alert_spec_topics: policy/trigger IDs from alert spec + :param alert_spec_policy_triggers: policy/trigger IDs from alert spec :param resource_spec_sfc: sfc from resource spec :param resource_spec_sfc_instance: sfc instance from resource spec - :param resource_spec_topics: policy/trigger IDs from resource spec + :param resource_spec_policy_triggers: policy/trigger IDs from resource spec :raises: HTTP Bad Request if the two specifications are inconsistent """ @@ -183,14 +189,20 @@ class AlertsConfigurationAPI(object): if alert_spec_sfc_instance != resource_spec_sfc_instance: raise HTTPBadRequest("Different service function chain instance ID used in the alert and resource specification documents: {0} != {1}".format(alert_spec_sfc_instance, resource_spec_sfc_instance)) - alert_spec_topics_set = set(alert_spec_topics) - missing_topic_ids = [topic_id for topic_id in resource_spec_topics if topic_id not in alert_spec_topics_set] + alert_spec_triggers_set = set(alert_spec_policy_triggers) + resource_spec_triggers_set = set(resource_spec_policy_triggers) - if len(missing_topic_ids) > 0: - missing_topic_ids = [topic_id.replace("\n", " : ") for topic_id in missing_topic_ids] - raise HTTPBadRequest("Couldn't match the following policy triggers from the resource specification with triggers defined in the alerts specification: {0}".format(missing_topic_ids)) + # check for triggers defined in the resource specification but missing in the alerts specification + missing_policy_triggers = [policy_trigger_id.replace("\n", " : ") for policy_trigger_id in resource_spec_policy_triggers if policy_trigger_id not in alert_spec_triggers_set] + if len(missing_policy_triggers) > 0: + raise HTTPBadRequest("Couldn't match the following policy triggers from the resource specification with triggers defined in the alerts specification: {0}".format(missing_policy_triggers)) - def _config_kapacitor_alerts(self, tosca_tpl, sfc, sfc_instance, db, kapacitor_host, kapacitor_port, alert_tasks_errors, alert_handlers_errors): + # check for triggers defined in the alerts specification and posting to flame_sfemc, but missing in the resource specification + missing_policy_triggers = [policy_trigger_id.replace("\n", " : ") for policy_trigger_id in alert_spec_policy_triggers if policy_trigger_id not in resource_spec_triggers_set] + if len(missing_policy_triggers) > 0: + raise HTTPBadRequest("Couldn't match the following policy triggers from the alerts specification with triggers defined in the resource specification: {0}".format(missing_policy_triggers)) + + def _config_kapacitor_alerts(self, tosca_tpl, sfc, sfc_instance, db, kapacitor_host, kapacitor_port, resource_spec_policy_triggers, alert_tasks_errors, alert_handlers_errors): """ Configures the alerts task and alert handlers within Kapacitor. @@ -200,6 +212,7 @@ class AlertsConfigurationAPI(object): :param db: Influx database ID :param kapacitor_host: default host is localhost (CLMC service running on the same machine as Kapacitor) :param kapacitor_port: default value to use is 9092 + :param resource_spec_policy_triggers: the extracted policy-trigger strings from the resource specification :param alert_tasks_errors: the list for tracking errors while interacting with Kapacitor tasks :param alert_handlers_errors: the list for tracking errors while interacting with Kapacitor alert handlers @@ -210,6 +223,7 @@ class AlertsConfigurationAPI(object): for trigger in policy.triggers: event_id = trigger.name policy_id = policy.name + resource_spec_trigger_id = resource_spec_policy_triggers.get("{0}\n{1}".format(policy_id, event_id)) event_type = trigger.trigger_tpl["event_type"] template_id = "{0}-template".format(event_type) @@ -225,7 +239,7 @@ class AlertsConfigurationAPI(object): where_clause = None if "resource_type" in trigger.trigger_tpl["condition"]: tags = condition["resource_type"] - # make sure alert tasks are executing with queries for the given sfc and sfc instance + # make sure alert tasks are executing with queries for the given sfc and sfc instance - automatically add those tags using the metadata values tags["flame_sfc"] = sfc tags["flame_sfci"] = sfc_instance @@ -251,9 +265,9 @@ class AlertsConfigurationAPI(object): # built up the template vars dictionary depending on the event type (threshold, relative, etc.) # all extracted properties from the trigger are passed, the TICKScriptTemplateFiller entry point then forwards those to the appropriate function for template filling - template_vars = TICKScriptTemplateFiller.fill_template_vars(event_type, db=db, measurement=measurement, field=field, influx_function=influx_function, - critical_value=critical_value, comparison_operator=comparison_operator, alert_period=alert_period, - topic_id=topic_id, event_id=event_id, where_clause=where_clause) + template_vars = TICKScriptTemplateFiller.fill_template_vars(event_type, sfc=sfc, sfci=sfc_instance, policy=policy_id, db=db, measurement=measurement, + field=field, influx_function=influx_function, critical_value=critical_value, comparison_operator=comparison_operator, + alert_period=alert_period, topic_id=topic_id, event_id=event_id, where_clause=where_clause) # create and activate alert task through the kapacitor HTTP API kapacitor_api_tasks_url = "http://{0}:{1}/kapacitor/v1/tasks".format(kapacitor_host, kapacitor_port) @@ -283,9 +297,11 @@ class AlertsConfigurationAPI(object): http_handlers = trigger.trigger_tpl["action"]["implementation"] # subscribe all http handlers to the created topic - self._config_kapacitor_alert_handlers(kapacitor_host, kapacitor_port, sfc, sfc_instance, policy_id, topic_id, event_id, http_handlers, alert_handlers_errors) + self._config_kapacitor_alert_handlers(kapacitor_host, kapacitor_port, sfc, sfc_instance, policy_id, resource_spec_trigger_id, topic_id, event_id, + http_handlers, alert_handlers_errors) - def _config_kapacitor_alert_handlers(self, kapacitor_host, kapacitor_port, sfc, sfc_i, policy_id, topic_id, event_id, http_handlers, alert_handlers_errors): + def _config_kapacitor_alert_handlers(self, kapacitor_host, kapacitor_port, sfc, sfc_i, policy_id, trigger_id, + topic_id, event_id, http_handlers, alert_handlers_errors): """ Handles the configuration of HTTP Post alert handlers. @@ -294,6 +310,7 @@ class AlertsConfigurationAPI(object): :param sfc: service function chain identifier :param sfc_i: service function chain instance identifier :param policy_id: policy ID those triggers relate to + :param trigger_id: the resource specification trigger ID :param topic_id: topic ID built of sfc, sfc instance and event_id :param event_id: name of trigger :param http_handlers: list of handlers to subscribe @@ -302,6 +319,12 @@ class AlertsConfigurationAPI(object): kapacitor_api_handlers_url = "http://{0}:{1}/kapacitor/v1/alerts/topics/{2}/handlers".format(kapacitor_host, kapacitor_port, topic_id) for http_handler_url in http_handlers: + + # check for flame_sfemc entry, if found replace with sfemc FQDN + if http_handler_url == SFEMC: + sfemc_fqdn = self.request.registry.settings['sfemc_fqdn'] + http_handler_url = "http://{0}/{1}/{2}/{3}/{4}".format(sfemc_fqdn, sfc, sfc_i, policy_id, trigger_id) + handler_id = "{0}\n{1}\n{2}\n{3}\n{4}".format(sfc, sfc_i, policy_id, event_id, http_handler_url) handler_id = self.get_hash(handler_id) kapacitor_http_request_body = fill_http_post_handler_vars(handler_id, http_handler_url) diff --git a/src/service/clmcservice/static/flame_clmc_alerts_definitions.yaml b/src/service/clmcservice/static/flame_clmc_alerts_definitions.yaml index 4553746b081258c653d5b90e5d5798ae611b004f..51349a7be63d0ba81cdd3e4a98061d0c8a156aa5 100644 --- a/src/service/clmcservice/static/flame_clmc_alerts_definitions.yaml +++ b/src/service/clmcservice/static/flame_clmc_alerts_definitions.yaml @@ -2,6 +2,6 @@ tosca_definitions_version: tosca_simple_profile_for_nfv_1_0_0 policy_types: - eu.ict-flame.policies.StateChange: + eu.ict-flame.policies.Alert: - derived_from: tosca.policies.Update \ No newline at end of file + derived_from: tosca.policies.Root \ No newline at end of file diff --git a/src/service/development.ini b/src/service/development.ini index faf52a578c2e639d0a1cf872d0bb4792c8a80cd8..cbc930b6b81038ed0ef4b074d0ff0e674458e9b9 100644 --- a/src/service/development.ini +++ b/src/service/development.ini @@ -14,8 +14,6 @@ pyramid.default_locale_name = en pyramid.includes = pyramid_debugtoolbar pyramid_exclog exclog.ignore = -# Configuration file path -configuration_file_path = /etc/flame/clmc/service.conf network_configuration_path = /vagrant/src/service/resources/GraphAPI/network_config.json diff --git a/src/service/production.ini b/src/service/production.ini index 1716af09d14e2121556b2133aec17a640516805e..5acb36aa9855c249cd45d988d8d3598e4075da61 100644 --- a/src/service/production.ini +++ b/src/service/production.ini @@ -14,8 +14,6 @@ pyramid.default_locale_name = en pyramid.includes = pyramid_exclog exclog.ignore = -# Configuration file path -configuration_file_path = /etc/flame/clmc/service.conf network_configuration_path = /vagrant/src/service/resources/GraphAPI/network_config.json diff --git a/src/service/resources/TICKscript/deadman-template.tick b/src/service/resources/TICKscript/deadman-template.tick index 5762d4ad5505972b989f1561a36ede5e3bc099b7..730a4610b0d8bdf97164db121ed67fa7a3e2c95e 100644 --- a/src/service/resources/TICKscript/deadman-template.tick +++ b/src/service/resources/TICKscript/deadman-template.tick @@ -1,3 +1,9 @@ +var sfc string + +var sfci string + +var policy string + var db string // database per service function chain, so db is named after sfc var rp = 'autogen' // default value for the retention policy @@ -25,7 +31,7 @@ stream .where(whereClause) | deadman(throughputThreshold, alertPeriod) .id(eventID) - .details('db=' + db + ',measurement=' + measurement) + .details('db=' + db + ',sfc=' + sfc + ',sfci=' + sfci + ',policy=' + policy) .message(messageValue) .topic(topicID) .noRecoveries() \ No newline at end of file diff --git a/src/service/resources/TICKscript/relative-template.tick b/src/service/resources/TICKscript/relative-template.tick index 30ac95c962a6bf547c93935810beb2d108a38b3f..669954bfd91b8ee6aad243a1fba01e9a54ab4bb9 100644 --- a/src/service/resources/TICKscript/relative-template.tick +++ b/src/service/resources/TICKscript/relative-template.tick @@ -1,3 +1,9 @@ +var sfc string + +var sfci string + +var policy string + var db string // database per service function chain, so db is named after sfc var rp = 'autogen' // default value for the retention policy @@ -42,7 +48,7 @@ past .as('diff') | alert() .id(eventID) - .details('db=' + db + ',measurement=' + measurement) + .details('db=' + db + ',sfc=' + sfc + ',sfci=' + sfci + ',policy=' + policy) .crit(comparisonLambda) .message(messageValue) .topic(topicID) diff --git a/src/service/resources/TICKscript/threshold-batch-template.tick b/src/service/resources/TICKscript/threshold-batch-template.tick index 379a49c4fffb030f8f1775c658c1b25cf8f64d2e..f5f7a64b91006de1420b2a65232bc2ba55d8e44c 100644 --- a/src/service/resources/TICKscript/threshold-batch-template.tick +++ b/src/service/resources/TICKscript/threshold-batch-template.tick @@ -1,3 +1,9 @@ +var sfc string + +var sfci string + +var policy string + var db string // database per service function chain, so db is named after sfc var rp = 'autogen' // default value for the retention policy @@ -26,7 +32,7 @@ batch .every(alertPeriod) |alert() .id(eventID) - .details('db=' + db + ',measurement=' + measurement) + .details('db=' + db + ',sfc=' + sfc + ',sfci=' + sfci + ',policy=' + policy) .crit(comparisonLambda) .message(messageValue) .topic(topicID) diff --git a/src/service/resources/TICKscript/threshold-stream-template.tick b/src/service/resources/TICKscript/threshold-stream-template.tick index 6ee92fd2c03b7d68efa871bff02dc53b29f1c1fd..9eb886589943d4da76a1ea8699f1efe7afb365a6 100644 --- a/src/service/resources/TICKscript/threshold-stream-template.tick +++ b/src/service/resources/TICKscript/threshold-stream-template.tick @@ -1,3 +1,9 @@ +var sfc string + +var sfci string + +var policy string + var db string // database per service function chain, so db is named after sfc var rp = 'autogen' // default value for the retention policy @@ -23,7 +29,7 @@ stream .where(whereClause) | alert() .id(eventID) - .details('db=' + db + ',measurement=' + measurement) + .details('db=' + db + ',sfc=' + sfc + ',sfci=' + sfci + ',policy=' + policy) .crit(comparisonLambda) .message(messageValue) .topic(topicID) diff --git a/src/service/resources/tosca/test-data/clmc-validator/invalid/alerts_test_config-1.yaml b/src/service/resources/tosca/test-data/clmc-validator/invalid/alerts_test_config-1.yaml index eb47f97eb1b17ed8b91831faf613f49035e05675..7812392af8dc3e50301843c84c38adf4e67cc9e4 100644 --- a/src/service/resources/tosca/test-data/clmc-validator/invalid/alerts_test_config-1.yaml +++ b/src/service/resources/tosca/test-data/clmc-validator/invalid/alerts_test_config-1.yaml @@ -13,7 +13,7 @@ topology_template: policies: - high_latency_policy: - type: eu.ict-flame.policies.StateChange + type: eu.ict-flame.policies.Alert triggers: high_latency: description: This event triggers when the mean network latency in a given location exceeds a given threshold (in ms). @@ -28,10 +28,10 @@ topology_template: comparison_operator: gt action: implementation: - - http://sfemc.flame.eu/notify + - flame_sfemc - http://companyA.alert-handler.flame.eu/high-latency - requests_diff_policy: - type: eu.ict-flame.policies.StateChange + type: eu.ict-flame.policies.Alert triggers: increase_in_requests: description: | @@ -51,7 +51,7 @@ topology_template: implementation: - http://sfemc.flame.eu/notify - low_requests_policy: - type: eu.ict-flame.policies.StateChange + type: eu.ict-flame.policies.Alert triggers: low_requests: description: | diff --git a/src/service/resources/tosca/test-data/clmc-validator/invalid/alerts_test_config-10.yaml b/src/service/resources/tosca/test-data/clmc-validator/invalid/alerts_test_config-10.yaml index 67793a750eee7654c944c7504acb7e1c1d33a9f5..c2a855080832457c3f8a6f83975696542ea1b08f 100644 --- a/src/service/resources/tosca/test-data/clmc-validator/invalid/alerts_test_config-10.yaml +++ b/src/service/resources/tosca/test-data/clmc-validator/invalid/alerts_test_config-10.yaml @@ -13,7 +13,7 @@ topology_template: policies: - high_latency_policy: - type: eu.ict-flame.policies.StateChange + type: eu.ict-flame.policies.Alert triggers: high_latency: description: This event triggers when the mean network latency in a given location exceeds a given threshold (in ms). @@ -28,10 +28,10 @@ topology_template: comparison_operator: gt action: implementation: - - http://sfemc.flame.eu/notify + - flame_sfemc - http://companyA.alert-handler.flame.eu/high-latency - requests_diff_policy: - type: eu.ict-flame.policies.StateChange + type: eu.ict-flame.policies.Alert triggers: decrease_in_requests: description: | @@ -49,9 +49,9 @@ topology_template: comparison_operator: lte action: implementation: - - http://sfemc.flame.eu/notify + - flame_sfemc - low_requests_policy: - type: eu.ict-flame.policies.StateChange + type: eu.ict-flame.policies.Alert triggers: low_requests: description: | @@ -70,5 +70,5 @@ topology_template: comparison_operator: lt action: implementation: - - http://sfemc.flame.eu/notify + - flame_sfemc - http://companyA.alert-handler.flame.eu/low-requests diff --git a/src/service/resources/tosca/test-data/clmc-validator/invalid/alerts_test_config-11.yaml b/src/service/resources/tosca/test-data/clmc-validator/invalid/alerts_test_config-11.yaml new file mode 100644 index 0000000000000000000000000000000000000000..d9110e5a69aa61f631d5b836774257ce299ee92d --- /dev/null +++ b/src/service/resources/tosca/test-data/clmc-validator/invalid/alerts_test_config-11.yaml @@ -0,0 +1,75 @@ +tosca_definitions_version: tosca_simple_profile_for_nfv_1_0_0 + +description: TOSCA Alerts Configuration document + +imports: +- flame_clmc_alerts_definitions.yaml + +metadata: + sfc: companyA-VR + sfci: companyA-VR-premium + +topology_template: + + policies: + - high_latency_policy: + type: eu.ict-flame.policies.Alert + triggers: + high_latency: + description: This event triggers when the mean network latency in a given location exceeds a given threshold (in ms). + event_type: threshold + metric: network.latency + condition: + threshold: 45 + granularity: 120 + aggregation_method: mean + resource_type: + flame_sfc: companyA-VR # not allowed - sfc tag is automatically added in the filters + flame_location: watershed + comparison_operator: gt + action: + implementation: + - flame_sfemc + - http://companyA.alert-handler.flame.eu/high-latency + - requests_diff_policy: + type: eu.ict-flame.policies.Alert + triggers: + decrease_in_requests: + description: | + This event triggers when the number of requests has decreased relative to the number of requests received + 120 seconds ago. + event_type: relative + metric: storage.requests + condition: + threshold: -100 # requests have decreased by at least 100 + granularity: 120 + resource_type: + flame_sfp: storage + flame_sf: storage-users + flame_location: watershed + comparison_operator: lte + action: + implementation: + - http://sfemc.flame.eu/notify + - low_requests_policy: + type: eu.ict-flame.policies.Alert + triggers: + low_requests: + description: | + This event triggers when the last reported number of requests for a given service function + falls behind a given threshold. + event_type: threshold + metric: storage.requests + condition: + threshold: 5 + granularity: 60 + aggregation_method: last + resource_type: + flame_sfp: storage + flame_sf: storage-users + flame_location: watershed + comparison_operator: lt + action: + implementation: + - http://sfemc.flame.eu/notify + - http://companyA.alert-handler.flame.eu/low-requests diff --git a/src/service/resources/tosca/test-data/clmc-validator/invalid/alerts_test_config-12.yaml b/src/service/resources/tosca/test-data/clmc-validator/invalid/alerts_test_config-12.yaml new file mode 100644 index 0000000000000000000000000000000000000000..818823f8139ca1b8493a26e3d7fa1bcdb8be890d --- /dev/null +++ b/src/service/resources/tosca/test-data/clmc-validator/invalid/alerts_test_config-12.yaml @@ -0,0 +1,75 @@ +tosca_definitions_version: tosca_simple_profile_for_nfv_1_0_0 + +description: TOSCA Alerts Configuration document + +imports: +- flame_clmc_alerts_definitions.yaml + +metadata: + sfc: companyA-VR + sfci: companyA-VR-premium + +topology_template: + + policies: + - high_latency_policy: + type: eu.ict-flame.policies.Alert + triggers: + high_latency: + description: This event triggers when the mean network latency in a given location exceeds a given threshold (in ms). + event_type: threshold + metric: network.latency + condition: + threshold: 45 + granularity: 120 + aggregation_method: mean + resource_type: + flame_sfci: companyA-VR-premium # not allowed - sfci tag is automatically added in the filters + flame_location: watershed + comparison_operator: gt + action: + implementation: + - flame_sfemc + - http://companyA.alert-handler.flame.eu/high-latency + - requests_diff_policy: + type: eu.ict-flame.policies.Alert + triggers: + decrease_in_requests: + description: | + This event triggers when the number of requests has decreased relative to the number of requests received + 120 seconds ago. + event_type: relative + metric: storage.requests + condition: + threshold: -100 # requests have decreased by at least 100 + granularity: 120 + resource_type: + flame_sfp: storage + flame_sf: storage-users + flame_location: watershed + comparison_operator: lte + action: + implementation: + - http://sfemc.flame.eu/notify + - low_requests_policy: + type: eu.ict-flame.policies.Alert + triggers: + low_requests: + description: | + This event triggers when the last reported number of requests for a given service function + falls behind a given threshold. + event_type: threshold + metric: storage.requests + condition: + threshold: 5 + granularity: 60 + aggregation_method: last + resource_type: + flame_sfp: storage + flame_sf: storage-users + flame_location: watershed + comparison_operator: lt + action: + implementation: + - http://sfemc.flame.eu/notify + - http://companyA.alert-handler.flame.eu/low-requests diff --git a/src/service/resources/tosca/test-data/clmc-validator/invalid/alerts_test_config-2.yaml b/src/service/resources/tosca/test-data/clmc-validator/invalid/alerts_test_config-2.yaml index 8a06dcf2c47a9c75d6d4dd5516ec9eba11e278b9..da58e71b7c08918707ac362dc4328d306fd74115 100644 --- a/src/service/resources/tosca/test-data/clmc-validator/invalid/alerts_test_config-2.yaml +++ b/src/service/resources/tosca/test-data/clmc-validator/invalid/alerts_test_config-2.yaml @@ -13,7 +13,7 @@ topology_template: policies: - missing_measurement_policy: - type: eu.ict-flame.policies.StateChange + type: eu.ict-flame.policies.Alert triggers: missing_storage_measurements: description: This event triggers when the number of storage measurements reported falls below the threshold value. @@ -26,9 +26,9 @@ topology_template: flame_sfp: storage action: implementation: - - http://sfemc.flame.eu/notify + - flame_sfemc - low_requests_policy: - type: eu.ict-flame.policies.StateChange + type: eu.ict-flame.policies.Alert triggers: low_requests: description: | diff --git a/src/service/resources/tosca/test-data/clmc-validator/invalid/alerts_test_config-3.yaml b/src/service/resources/tosca/test-data/clmc-validator/invalid/alerts_test_config-3.yaml index 4ed49c37360320e59bd4c881b627217a83ffdd85..8a89ae9f89af2abbf6f5b380066a1041d863f9f1 100644 --- a/src/service/resources/tosca/test-data/clmc-validator/invalid/alerts_test_config-3.yaml +++ b/src/service/resources/tosca/test-data/clmc-validator/invalid/alerts_test_config-3.yaml @@ -13,7 +13,7 @@ topology_template: policies: - low_requests_policy: - type: eu.ict-flame.policies.StateChange + type: eu.ict-flame.policies.Alert triggers: low_requests: description: | @@ -32,4 +32,5 @@ topology_template: comparison_operator: lt action: implementation: - - http://companyA.alert-handler.flame.eu/high-latency \ No newline at end of file + - http://companyA.alert-handler.flame.eu/high-latency + - flame_sfemc \ No newline at end of file diff --git a/src/service/resources/tosca/test-data/clmc-validator/invalid/alerts_test_config-4.yaml b/src/service/resources/tosca/test-data/clmc-validator/invalid/alerts_test_config-4.yaml index 67c403af9c9edfc01e0b11b38bf1a7837f969baf..6a2884270313c3d2a3b5e4ba520a237947f7c666 100644 --- a/src/service/resources/tosca/test-data/clmc-validator/invalid/alerts_test_config-4.yaml +++ b/src/service/resources/tosca/test-data/clmc-validator/invalid/alerts_test_config-4.yaml @@ -13,7 +13,7 @@ topology_template: policies: - low_requests_policy: - type: eu.ict-flame.policies.StateChange + type: eu.ict-flame.policies.Alert triggers: low_requests: description: | diff --git a/src/service/resources/tosca/test-data/clmc-validator/invalid/alerts_test_config-5.yaml b/src/service/resources/tosca/test-data/clmc-validator/invalid/alerts_test_config-5.yaml index 735761389c4e4234c451efca3669303ce0108bed..b558ac97aff5b0e44c31f811f9fd8161e1d4e43d 100644 --- a/src/service/resources/tosca/test-data/clmc-validator/invalid/alerts_test_config-5.yaml +++ b/src/service/resources/tosca/test-data/clmc-validator/invalid/alerts_test_config-5.yaml @@ -13,7 +13,7 @@ topology_template: policies: - low_requests_policy: - type: eu.ict-flame.policies.StateChange + type: eu.ict-flame.policies.Alert triggers: low_requests: description: | @@ -33,4 +33,4 @@ topology_template: action: implementation: - http://companyA.alert-handler.flame.eu/high-latency - - sfemc-webhook # should be a valid URL address \ No newline at end of file + - sfemc-webhook # should be a valid URL address or flame_sfemc for the SFEMC URL generation \ No newline at end of file diff --git a/src/service/resources/tosca/test-data/clmc-validator/invalid/alerts_test_config-6.yaml b/src/service/resources/tosca/test-data/clmc-validator/invalid/alerts_test_config-6.yaml index ac13f6fe6eca8ce4b9b80233eba7c65572952360..7a7688658d90ef1dfc036410b752366976e488de 100644 --- a/src/service/resources/tosca/test-data/clmc-validator/invalid/alerts_test_config-6.yaml +++ b/src/service/resources/tosca/test-data/clmc-validator/invalid/alerts_test_config-6.yaml @@ -13,7 +13,7 @@ topology_template: policies: - high_latency_policy: - type: eu.ict-flame.policies.StateChange + type: eu.ict-flame.policies.Alert triggers: high_latency: description: This event triggers when the mean network latency in a given location exceeds a given threshold (in ms). @@ -29,10 +29,10 @@ topology_template: comparison_operator: gt action: implementation: - - http://sfemc.flame.eu/notify + - flame_sfemc - http://companyA.alert-handler.flame.eu/high-latency - low_requests_policy: - type: eu.ict-flame.policies.StateChange + type: eu.ict-flame.policies.Alert triggers: low_requests: description: | diff --git a/src/service/resources/tosca/test-data/clmc-validator/invalid/alerts_test_config-7.yaml b/src/service/resources/tosca/test-data/clmc-validator/invalid/alerts_test_config-7.yaml index acb00ed3ac6ff2380c90d12424d6b386b3a45498..312b5e4e1b27f8cde5c346173fd2d29e2d3819aa 100644 --- a/src/service/resources/tosca/test-data/clmc-validator/invalid/alerts_test_config-7.yaml +++ b/src/service/resources/tosca/test-data/clmc-validator/invalid/alerts_test_config-7.yaml @@ -13,7 +13,7 @@ topology_template: policies: - high_latency_policy: - type: eu.ict-flame.policies.StateChange + type: eu.ict-flame.policies.Alert triggers: high_latency: description: This event triggers when the mean network latency in a given location exceeds a given threshold (in ms). @@ -28,10 +28,10 @@ topology_template: comparison_operator: gt action: implementation: - - http://sfemc.flame.eu/notify + - flame_sfemc - http://companyA.alert-handler.flame.eu/high-latency - low_requests_policy: - type: eu.ict-flame.policies.StateChange + type: eu.ict-flame.policies.Alert triggers: low_requests: description: | @@ -50,10 +50,10 @@ topology_template: comparison_operator: lt action: implementation: - - http://sfemc.flame.eu/notify + - flame_sfemc - http://companyA.alert-handler.flame.eu/low-requests - missing_measurement_policy: - type: eu.ict-flame.policies.StateChange + type: eu.ict-flame.policies.Alert triggers: missing_storage_measurements: description: This event triggers when the number of storage measurements reported falls below the threshold value. diff --git a/src/service/resources/tosca/test-data/clmc-validator/invalid/alerts_test_config-8.yaml b/src/service/resources/tosca/test-data/clmc-validator/invalid/alerts_test_config-8.yaml index a2c3009707631525caa656294f0cf2461ba2a0b9..de10ba0aeba3acd63743dfe669c8838aa6698e25 100644 --- a/src/service/resources/tosca/test-data/clmc-validator/invalid/alerts_test_config-8.yaml +++ b/src/service/resources/tosca/test-data/clmc-validator/invalid/alerts_test_config-8.yaml @@ -13,11 +13,11 @@ topology_template: policies: - high_latency_policy: - type: eu.ict-flame.policies.StateChange + type: eu.ict-flame.policies.Alert triggers: # should specify at least 1 trigger - low_requests_policy: - type: eu.ict-flame.policies.StateChange + type: eu.ict-flame.policies.Alert triggers: low_requests: description: | @@ -36,10 +36,10 @@ topology_template: comparison_operator: lt action: implementation: - - http://sfemc.flame.eu/notify + - flame_sfemc - http://companyA.alert-handler.flame.eu/low-requests - requests_diff_policy: - type: eu.ict-flame.policies.StateChange + type: eu.ict-flame.policies.Alert triggers: increase_in_requests: description: | @@ -74,4 +74,4 @@ topology_template: comparison_operator: lte action: implementation: - - http://sfemc.flame.eu/notify \ No newline at end of file + - flame_sfemc \ No newline at end of file diff --git a/src/service/resources/tosca/test-data/clmc-validator/invalid/alerts_test_config-9.yaml b/src/service/resources/tosca/test-data/clmc-validator/invalid/alerts_test_config-9.yaml index 102c476d55265c73c22b46d889fa2a44d0e85930..15e50edd9a60ae250dd597de2c59c5b0654ad825 100644 --- a/src/service/resources/tosca/test-data/clmc-validator/invalid/alerts_test_config-9.yaml +++ b/src/service/resources/tosca/test-data/clmc-validator/invalid/alerts_test_config-9.yaml @@ -11,7 +11,7 @@ topology_template: policies: - high_latency_policy: - type: eu.ict-flame.policies.StateChange + type: eu.ict-flame.policies.Alert triggers: high_latency: description: This event triggers when the mean network latency in a given location exceeds a given threshold (in ms). @@ -26,10 +26,10 @@ topology_template: comparison_operator: gt action: implementation: - - http://sfemc.flame.eu/notify + - flame_sfemc - http://companyA.alert-handler.flame.eu/high-latency - low_requests_policy: - type: eu.ict-flame.policies.StateChange + type: eu.ict-flame.policies.Alert triggers: low_requests: description: | @@ -48,5 +48,5 @@ topology_template: comparison_operator: lt action: implementation: - - http://sfemc.flame.eu/notify + - flame_sfemc - http://companyA.alert-handler.flame.eu/low-requests diff --git a/src/service/resources/tosca/test-data/clmc-validator/valid/alerts_test_config-1.yaml b/src/service/resources/tosca/test-data/clmc-validator/valid/alerts_test_config-1.yaml index 0114a3546eaadda10e54ab0e051f67eebe303b40..c3b3a227f9825f29f570a44a2ad84959b4cbd716 100644 --- a/src/service/resources/tosca/test-data/clmc-validator/valid/alerts_test_config-1.yaml +++ b/src/service/resources/tosca/test-data/clmc-validator/valid/alerts_test_config-1.yaml @@ -13,7 +13,7 @@ topology_template: policies: - high_latency_policy: - type: eu.ict-flame.policies.StateChange + type: eu.ict-flame.policies.Alert triggers: high_latency: description: This event triggers when the mean network latency in a given location exceeds a given threshold (in ms). @@ -28,11 +28,11 @@ topology_template: comparison_operator: eq action: implementation: - - http://sfemc.flame.eu/notify + - flame_sfemc - http://companyA.alert-handler.flame.eu/high-latency - low_requests_policy: - type: eu.ict-flame.policies.StateChange + type: eu.ict-flame.policies.Alert triggers: low_requests: description: | @@ -51,5 +51,23 @@ topology_template: comparison_operator: lt action: implementation: - - http://sfemc.flame.eu/notify - - http://companyA.alert-handler.flame.eu/low-requests \ No newline at end of file + - flame_sfemc + - http://companyA.alert-handler.flame.eu/low-requests + + - missing_measurement_policy: + type: eu.ict-flame.policies.Alert + triggers: + missing_storage_measurements: + description: This event triggers when the number of storage measurements reported falls below the threshold value. + event_type: deadman + # deadman trigger instances monitor the whole measurement (storage in this case), so simply put a star for field value + # to be compliant with the <measurement>.<field> format + metric: storage.* + condition: + threshold: 10 # if requests are less than or equal to 0 (in other words, no measurements are reported) + granularity: 60 # check for for missing data for the last 60 seconds + resource_type: + flame_sfp: storage + action: + implementation: + - http://companyA.alert-handler.flame.eu/missing_measurements \ No newline at end of file diff --git a/src/service/resources/tosca/test-data/clmc-validator/valid/alerts_test_config-2.yaml b/src/service/resources/tosca/test-data/clmc-validator/valid/alerts_test_config-2.yaml index 5169f5d4d205f3f9121c5c390a2596235e05de49..a3ea67cafc811d0e313e866e9ddfc62b8e792030 100644 --- a/src/service/resources/tosca/test-data/clmc-validator/valid/alerts_test_config-2.yaml +++ b/src/service/resources/tosca/test-data/clmc-validator/valid/alerts_test_config-2.yaml @@ -12,8 +12,24 @@ metadata: topology_template: policies: + - high_latency_policy: + type: eu.ict-flame.policies.Alert + triggers: + high_latency: + # optional description - hence, valid + event_type: threshold + metric: network.latency + condition: + threshold: 45 + granularity: 120 + resource_type: + flame_location: watershed + comparison_operator: eq + action: + implementation: + - http://companyA.alert-handler.flame.eu/high-latency - requests_diff_policy: - type: eu.ict-flame.policies.StateChange + type: eu.ict-flame.policies.Alert triggers: decrease_in_requests: description: | @@ -25,18 +41,16 @@ topology_template: threshold: -100 # requests have decreased by at least 100 granularity: 120 resource_type: - flame_sfc: companyA-VR # sfc tag is also allowed, even though it is already included in the metadata - flame_sfci: companyA-VR-premium # sfci tag is also allowed, even though it is already included in the metadata flame_sfp: storage flame_sf: storage-users flame_location: watershed comparison_operator: lte action: implementation: - - http://sfemc.flame.eu/notify + - flame_sfemc - missing_measurement_policy: - type: eu.ict-flame.policies.StateChange + type: eu.ict-flame.policies.Alert triggers: missing_storage_measurements: description: This event triggers when the number of storage measurements reported falls below the threshold value. @@ -51,4 +65,4 @@ topology_template: flame_sfp: storage action: implementation: - - http://sfemc.flame.eu/notify \ No newline at end of file + - flame_sfemc \ No newline at end of file diff --git a/src/service/resources/tosca/test-data/clmc-validator/valid/alerts_test_config-3.yaml b/src/service/resources/tosca/test-data/clmc-validator/valid/alerts_test_config-3.yaml index ef1c542aa86bbcd08809b850c1bdfb28de2b693b..266fa3152806f66f52101e3230d7605e33a11772 100644 --- a/src/service/resources/tosca/test-data/clmc-validator/valid/alerts_test_config-3.yaml +++ b/src/service/resources/tosca/test-data/clmc-validator/valid/alerts_test_config-3.yaml @@ -13,7 +13,7 @@ topology_template: policies: - high_latency_policy: - type: eu.ict-flame.policies.StateChange + type: eu.ict-flame.policies.Alert triggers: high_latency: # optional description - hence, valid @@ -28,10 +28,10 @@ topology_template: # comparison operator is optional, default value is >= or "gte" action: implementation: - - http://sfemc.flame.eu/notify + - flame_sfemc - http://companyA.alert-handler.flame.eu/high-latency - low_requests_policy: - type: eu.ict-flame.policies.StateChange + type: eu.ict-flame.policies.Alert triggers: low_requests: # optional description - hence, valid @@ -48,5 +48,22 @@ topology_template: comparison_operator: lt action: implementation: - - http://sfemc.flame.eu/notify + - flame_sfemc - http://companyA.alert-handler.flame.eu/low-requests + - missing_measurement_policy: + type: eu.ict-flame.policies.Alert + triggers: + missing_storage_measurements: + description: This event triggers when the number of storage measurements reported falls below the threshold value. + event_type: deadman + # deadman trigger instances monitor the whole measurement (storage in this case), so simply put a star for field value + # to be compliant with the <measurement>.<field> format + metric: storage.* + condition: + threshold: 0 # if requests are less than or equal to 0 (in other words, no measurements are reported) + granularity: 60 # check for for missing data for the last 60 seconds + resource_type: + flame_sfp: storage + action: + implementation: + - http://companyA.alert-handler.flame.eu/no-measurements diff --git a/src/service/resources/tosca/test-data/clmc-validator/valid/alerts_test_config-4.yaml b/src/service/resources/tosca/test-data/clmc-validator/valid/alerts_test_config-4.yaml index 13808f97ec173a5e1187d1a2551875b60e74208d..3993c088f0a2fbe6fc9f7e388563d9a3fd072478 100644 --- a/src/service/resources/tosca/test-data/clmc-validator/valid/alerts_test_config-4.yaml +++ b/src/service/resources/tosca/test-data/clmc-validator/valid/alerts_test_config-4.yaml @@ -13,7 +13,7 @@ topology_template: policies: - high_latency_policy: - type: eu.ict-flame.policies.StateChange + type: eu.ict-flame.policies.Alert triggers: high_latency: # optional description - hence, valid @@ -29,10 +29,10 @@ topology_template: comparison_operator: neq action: implementation: - - http://sfemc.flame.eu/notify + - flame_sfemc - http://companyA.alert-handler.flame.eu/high-latency - low_requests_policy: - type: eu.ict-flame.policies.StateChange + type: eu.ict-flame.policies.Alert triggers: low_requests: # optional description - hence, valid @@ -46,10 +46,10 @@ topology_template: comparison_operator: lte action: implementation: - - http://sfemc.flame.eu/notify + - flame_sfemc - http://companyA.alert-handler.flame.eu/low-requests - missing_measurement_policy: - type: eu.ict-flame.policies.StateChange + type: eu.ict-flame.policies.Alert triggers: missing_storage_measurements: description: This event triggers when the number of storage measurements reported falls below the threshold value. @@ -65,4 +65,5 @@ topology_template: comparison_operator: gte # although events of type deadman do not use a comparison operator, the validator will not complain if one is given, it will simply ignore it action: implementation: - - http://sfemc.flame.eu/notify + - flame_sfemc + - http://companyA.alert-handler.flame.eu/missing_measurements diff --git a/src/service/resources/tosca/test-data/clmc-validator/valid/alerts_test_config-5.yaml b/src/service/resources/tosca/test-data/clmc-validator/valid/alerts_test_config-5.yaml index f03c52382b2eccbc268617a975cf018bdc8d7c18..83195048c010fb0b4f9f7ce9e5a06002e69f36ca 100644 --- a/src/service/resources/tosca/test-data/clmc-validator/valid/alerts_test_config-5.yaml +++ b/src/service/resources/tosca/test-data/clmc-validator/valid/alerts_test_config-5.yaml @@ -13,7 +13,7 @@ topology_template: policies: - high_latency_policy: - type: eu.ict-flame.policies.StateChange + type: eu.ict-flame.policies.Alert triggers: high_latency: # optional description - hence, valid @@ -28,10 +28,10 @@ topology_template: comparison_operator: gt action: implementation: - - http://sfemc.flame.eu/notify + - flame_sfemc - http://companyA.alert-handler.flame.eu/high-latency - low_requests_policy: - type: eu.ict-flame.policies.StateChange + type: eu.ict-flame.policies.Alert triggers: low_requests: # optional description - hence, valid @@ -45,10 +45,10 @@ topology_template: comparison_operator: lt action: implementation: - - https://sfemc.flame.eu/notify + - flame_sfemc - http://localhost:9999/low-requests # localhost url is also allowed - requests_diff_policy: - type: eu.ict-flame.policies.StateChange + type: eu.ict-flame.policies.Alert triggers: increase_in_requests: description: | @@ -67,4 +67,4 @@ topology_template: comparison_operator: gte action: implementation: - - http://sfemc.flame.eu/notify + - flame_sfemc \ No newline at end of file diff --git a/src/service/resources/tosca/test-data/resource-spec/resources_valid_test_config-1.yaml b/src/service/resources/tosca/test-data/resource-spec/resources_valid_test_config-1.yaml index ceb65829204591b0d9eec14429014f3a325778c6..a17b840e608e6a4af4654ae464d9dc516452b7f4 100644 --- a/src/service/resources/tosca/test-data/resource-spec/resources_valid_test_config-1.yaml +++ b/src/service/resources/tosca/test-data/resource-spec/resources_valid_test_config-1.yaml @@ -75,7 +75,7 @@ topology_template: properties: parent: service_paid triggers: - check_trigger: + trigger_id_1: description: Check high latency on relationships condition: constraint: clmc::high_latency @@ -93,7 +93,7 @@ topology_template: properties: parent: service_paid triggers: - check_trigger: + trigger_id_1: description: Check high latency on relationships condition: constraint: clmc::low_requests diff --git a/src/service/resources/tosca/test-data/resource-spec/resources_valid_test_config-2.yaml b/src/service/resources/tosca/test-data/resource-spec/resources_valid_test_config-2.yaml index cda9496ad069c82eeebe6b187f24c887a5d77446..f2530f59e6e03ee793977ff63bb89266586de5ac 100644 --- a/src/service/resources/tosca/test-data/resource-spec/resources_valid_test_config-2.yaml +++ b/src/service/resources/tosca/test-data/resource-spec/resources_valid_test_config-2.yaml @@ -74,7 +74,7 @@ topology_template: properties: parent: service_paid triggers: - check_trigger: + trigger_id_2: description: Check high latency on relationships condition: constraint: clmc::decrease_in_requests @@ -90,7 +90,7 @@ topology_template: properties: parent: service_paid triggers: - check_trigger: + trigger_id_2: description: Check high latency on relationships condition: constraint: clmc::missing_storage_measurements diff --git a/src/service/resources/tosca/test-data/resource-spec/resources_valid_test_config-3.yaml b/src/service/resources/tosca/test-data/resource-spec/resources_valid_test_config-3.yaml index 6190b25551dfdc78cba6dcd8ab4353f3c8791a65..4ca88b33a0fd4186b85e2668e66accb8787d6d2b 100644 --- a/src/service/resources/tosca/test-data/resource-spec/resources_valid_test_config-3.yaml +++ b/src/service/resources/tosca/test-data/resource-spec/resources_valid_test_config-3.yaml @@ -75,7 +75,7 @@ topology_template: properties: parent: service_paid triggers: - check_trigger: + trigger_id_3: description: Check high latency on relationships condition: constraint: clmc::high_latency @@ -92,7 +92,7 @@ topology_template: properties: parent: service_paid triggers: - check_trigger: + trigger_id_3: description: Check high latency on relationships condition: constraint: clmc::low_requests diff --git a/src/service/resources/tosca/test-data/resource-spec/resources_valid_test_config-4.yaml b/src/service/resources/tosca/test-data/resource-spec/resources_valid_test_config-4.yaml index fe46de4d3bbb0e3d204a17de4d8da831d159f2b2..496e972b73e5fad39016c118e1824cacbac33399 100644 --- a/src/service/resources/tosca/test-data/resource-spec/resources_valid_test_config-4.yaml +++ b/src/service/resources/tosca/test-data/resource-spec/resources_valid_test_config-4.yaml @@ -75,7 +75,7 @@ topology_template: properties: parent: service_paid triggers: - check_trigger: + trigger_id_4: description: Check high latency on relationships condition: constraint: clmc::high_latency @@ -92,7 +92,7 @@ topology_template: properties: parent: service_paid triggers: - check_trigger: + trigger_id_4: description: Check high latency on relationships condition: constraint: clmc::low_requests @@ -109,7 +109,7 @@ topology_template: properties: parent: service_paid triggers: - check_trigger: + trigger_id_4: description: Check high latency on relationships condition: constraint: clmc::missing_storage_measurements diff --git a/src/service/resources/tosca/test-data/resource-spec/resources_valid_test_config-5.yaml b/src/service/resources/tosca/test-data/resource-spec/resources_valid_test_config-5.yaml index 82a200cc1fa9cc44e285459bbaa1303829c34c8f..0466063c748d5a5f8a470a2cf253cfb9c5c31460 100644 --- a/src/service/resources/tosca/test-data/resource-spec/resources_valid_test_config-5.yaml +++ b/src/service/resources/tosca/test-data/resource-spec/resources_valid_test_config-5.yaml @@ -75,7 +75,7 @@ topology_template: properties: parent: service_paid triggers: - check_trigger: + trigger_id_5: description: Check high latency on relationships condition: constraint: clmc::high_latency @@ -92,7 +92,7 @@ topology_template: properties: parent: service_paid triggers: - check_trigger: + trigger_id_5: description: Check high latency on relationships condition: constraint: clmc::low_requests @@ -109,7 +109,7 @@ topology_template: properties: parent: service_paid triggers: - check_trigger: + trigger_id_5: description: Check high latency on relationships condition: constraint: clmc::increase_in_requests diff --git a/src/service/resources/tosca/test-data/tosca-parser/invalid/alerts_test_config-2.yaml b/src/service/resources/tosca/test-data/tosca-parser/invalid/alerts_test_config-2.yaml index 7bd6e44b5b14bb7431630a2549530ee62774d22f..ad61f4b578a0343a1b8e3597ee258ec7d16e7027 100644 --- a/src/service/resources/tosca/test-data/tosca-parser/invalid/alerts_test_config-2.yaml +++ b/src/service/resources/tosca/test-data/tosca-parser/invalid/alerts_test_config-2.yaml @@ -12,7 +12,7 @@ topology_template: policies: - high_latency_policy: - type: eu.ict-flame.policies.StateChange + type: eu.ict-flame.policies.Alert triggers: high_latency: description: This event triggers when the mean network latency in a given location exceeds a given threshold (in ms). @@ -27,10 +27,11 @@ topology_template: comparison_operator: gt action: implementation: + - flame_sfemc - http://sfemc.flame.eu/notify - http://companyA.alert-handler.flame.eu/high-latency - requests_diff_policy: - type: eu.ict-flame.policies.StateChange + type: eu.ict-flame.policies.Alert triggers: increase_in_requests: description: | diff --git a/src/service/resources/tosca/test-data/tosca-parser/invalid/alerts_test_config-3.yaml b/src/service/resources/tosca/test-data/tosca-parser/invalid/alerts_test_config-3.yaml index 94891fe31c116e93a49f9fc5718e6842849a0a9c..326d6b59833f805a506a1125efae9cf9824ab428 100644 --- a/src/service/resources/tosca/test-data/tosca-parser/invalid/alerts_test_config-3.yaml +++ b/src/service/resources/tosca/test-data/tosca-parser/invalid/alerts_test_config-3.yaml @@ -12,7 +12,7 @@ metadata: topology_template: # missing policies section - high_latency_policy: - type: eu.ict-flame.policies.StateChange + type: eu.ict-flame.policies.Alert triggers: high_latency: description: This event triggers when the mean network latency in a given location exceeds a given threshold (in ms). @@ -31,7 +31,7 @@ topology_template: - http://sfemc.flame.eu/notify - http://companyA.alert-handler.flame.eu/high-latency - low_requests_policy: - type: eu.ict-flame.policies.StateChange + type: eu.ict-flame.policies.Alert triggers: low_requests: description: | diff --git a/src/service/resources/tosca/test-data/tosca-parser/invalid/alerts_test_config-4.yaml b/src/service/resources/tosca/test-data/tosca-parser/invalid/alerts_test_config-4.yaml index 76b8a0cd04ca08872f104bbb3e3aff9a505b0de6..f1056c6cfd66a62627a0a5c55c1c5a5858e0d537 100644 --- a/src/service/resources/tosca/test-data/tosca-parser/invalid/alerts_test_config-4.yaml +++ b/src/service/resources/tosca/test-data/tosca-parser/invalid/alerts_test_config-4.yaml @@ -13,7 +13,7 @@ metadata: alerts: policies: - high_latency_policy: - type: eu.ict-flame.policies.StateChange + type: eu.ict-flame.policies.Alert triggers: high_latency: description: This event triggers when the mean network latency in a given location exceeds a given threshold (in ms). @@ -28,10 +28,11 @@ alerts: comparison_operator: gt action: implementation: + - flame_sfemc - http://sfemc.flame.eu/notify - http://companyA.alert-handler.flame.eu/high-latency - low_requests_policy: - type: eu.ict-flame.policies.StateChange + type: eu.ict-flame.policies.Alert triggers: low_requests: description: | diff --git a/src/service/resources/tosca/test-data/tosca-parser/valid/alerts_test_config-1.yaml b/src/service/resources/tosca/test-data/tosca-parser/valid/alerts_test_config-1.yaml index 928c25ed767ba906158e52c1649d76f184ae64b2..5df5321036a8239fc3b746fa491d621e268d7a13 100644 --- a/src/service/resources/tosca/test-data/tosca-parser/valid/alerts_test_config-1.yaml +++ b/src/service/resources/tosca/test-data/tosca-parser/valid/alerts_test_config-1.yaml @@ -13,7 +13,7 @@ topology_template: policies: - high_latency_policy: - type: eu.ict-flame.policies.StateChange + type: eu.ict-flame.policies.Alert triggers: high_latency: description: This event triggers when the mean network latency in a given location exceeds a given threshold (in ms). @@ -32,7 +32,7 @@ topology_template: - http://sfemc.flame.eu/notify - http://companyA.alert-handler.flame.eu/high-latency - low_requests_policy: - type: eu.ict-flame.policies.StateChange + type: eu.ict-flame.policies.Alert triggers: low_requests: description: | @@ -54,7 +54,7 @@ topology_template: - http://sfemc.flame.eu/notify - http://companyA.alert-handler.flame.eu/low-requests - requests_diff_policy: - type: eu.ict-flame.policies.StateChange + type: eu.ict-flame.policies.Alert triggers: increase_in_requests: description: | @@ -91,7 +91,7 @@ topology_template: implementation: - http://sfemc.flame.eu/notify - missing_measurement_policy: - type: eu.ict-flame.policies.StateChange + type: eu.ict-flame.policies.Alert triggers: missing_storage_measurements: description: This event triggers when the number of storage measurements reported falls below the threshold value. diff --git a/src/service/resources/tosca/test-data/tosca-parser/valid/alerts_test_config-2.yaml b/src/service/resources/tosca/test-data/tosca-parser/valid/alerts_test_config-2.yaml index 5924c13817555788959d792044500d7e0dfe00bb..07f1cb69fe23aacec07a2fddcc31c3a657624a82 100644 --- a/src/service/resources/tosca/test-data/tosca-parser/valid/alerts_test_config-2.yaml +++ b/src/service/resources/tosca/test-data/tosca-parser/valid/alerts_test_config-2.yaml @@ -13,7 +13,7 @@ topology_template: policies: - high_latency_policy: - type: eu.ict-flame.policies.StateChange + type: eu.ict-flame.policies.Alert triggers: high_latency: description: This event triggers when the mean network latency in a given location exceeds a given threshold (in ms). @@ -28,8 +28,9 @@ topology_template: implementation: - http://sfemc.flame.eu/notify - http://companyA.alert-handler.flame.eu/high-latency + - flame_sfemc - low_requests_policy: - type: eu.ict-flame.policies.StateChange + type: eu.ict-flame.policies.Alert triggers: low_requests: description: | @@ -47,7 +48,7 @@ topology_template: - http://sfemc.flame.eu/notify - http://companyA.alert-handler.flame.eu/low-requests - missing_measurement_policy: - type: eu.ict-flame.policies.StateChange + type: eu.ict-flame.policies.Alert triggers: missing_storage_measurements: event_type: deadman diff --git a/src/service/resources/tosca/test-data/tosca-parser/valid/alerts_test_config-3.yaml b/src/service/resources/tosca/test-data/tosca-parser/valid/alerts_test_config-3.yaml index 118797e8b5c1889b4c81a0cbeadecfd5cd141aeb..13f05f89427fcfba330254fc87198fba8eaeedf7 100644 --- a/src/service/resources/tosca/test-data/tosca-parser/valid/alerts_test_config-3.yaml +++ b/src/service/resources/tosca/test-data/tosca-parser/valid/alerts_test_config-3.yaml @@ -13,7 +13,7 @@ topology_template: policies: - high_latency_policy: - type: eu.ict-flame.policies.StateChange + type: eu.ict-flame.policies.Alert triggers: high_latency: description: This event triggers when the mean network latency in a given location exceeds a given threshold (in ms). diff --git a/src/service/resources/tosca/test-data/tosca-parser/valid/alerts_test_config-4.yaml b/src/service/resources/tosca/test-data/tosca-parser/valid/alerts_test_config-4.yaml index 64b19cbe37225546b4e5c73adfb83ee47c9d981d..db5a6d2dd3c3d6851e0437d19432dcae2be79038 100644 --- a/src/service/resources/tosca/test-data/tosca-parser/valid/alerts_test_config-4.yaml +++ b/src/service/resources/tosca/test-data/tosca-parser/valid/alerts_test_config-4.yaml @@ -13,7 +13,7 @@ topology_template: policies: - low_requests_policy: - type: eu.ict-flame.policies.StateChange + type: eu.ict-flame.policies.Alert triggers: low_requests: description: | @@ -32,9 +32,10 @@ topology_template: comparison_operator: lt action: implementation: + - flame_sfemc - http://companyA.alert-handler.flame.eu/low-requests - requests_diff_policy: - type: eu.ict-flame.policies.StateChange + type: eu.ict-flame.policies.Alert triggers: increase_in_requests: event_type: relative diff --git a/src/test/clmctest/alerts/alerts_test_config.yaml b/src/test/clmctest/alerts/alerts_test_config.yaml index fe293a0308352acb4f694e208708db7c1d3d06fa..6c422e7e6f78f8db8f3a9b89942d2c7dfba03074 100644 --- a/src/test/clmctest/alerts/alerts_test_config.yaml +++ b/src/test/clmctest/alerts/alerts_test_config.yaml @@ -13,7 +13,7 @@ topology_template: policies: - scale_nginx_policy: - type: eu.ict-flame.policies.StateChange + type: eu.ict-flame.policies.Alert triggers: high_requests: description: | @@ -45,6 +45,7 @@ topology_template: comparison_operator: gte action: implementation: + - flame_sfemc - http://172.40.231.200:9999/ increase_in_active_requests: description: This event triggers when the cpu system usage is too high. @@ -54,8 +55,6 @@ topology_template: threshold: 1 granularity: 10 resource_type: - flame_sfc: MS_Template_1 # value is already given in metadata so this is optional - flame_sfci: MS_I1 # value is already given in metadata so this is optional flame_sfp: nginx flame_sf: adaptive_streaming_nginx_I1 flame_location: DC1 @@ -63,9 +62,10 @@ topology_template: comparison_operator: gte action: implementation: + - flame_sfemc - http://172.40.231.200:9999/ - deadman_policy: - type: eu.ict-flame.policies.StateChange + type: eu.ict-flame.policies.Alert triggers: no_measurements: description: | @@ -76,12 +76,11 @@ topology_template: threshold: 0 granularity: 5 resource_type: - flame_sfc: MS_Template_1 # value is already given in metadata so this is optional - flame_sfci: MS_I1 # value is already given in metadata so this is optional flame_sfp: nginx flame_sf: adaptive_streaming_nginx_I1 flame_location: DC1 flame_server: DC1 action: implementation: + - flame_sfemc - http://172.40.231.200:9999/ \ No newline at end of file diff --git a/src/test/clmctest/alerts/resources_test_config.yaml b/src/test/clmctest/alerts/resources_test_config.yaml index 061c227317c1e4f6818e34b173b1f42eb8eb2907..161ab8bebff06877fd0c8a9dc186589df0f35fae 100644 --- a/src/test/clmctest/alerts/resources_test_config.yaml +++ b/src/test/clmctest/alerts/resources_test_config.yaml @@ -74,15 +74,6 @@ topology_template: properties: parent: service_paid triggers: - tigger_a: - condition: - constraint: clmc::high_requests - action: - frontend: - - - fqdn: frontend.app.ict-flame.eu - lifecycle_actions: - Bristol: eu.ict-flame.sfe.state.lifecycle.connected tigger_b: condition: constraint: clmc::increase_in_running_processes diff --git a/src/test/clmctest/alerts/test_alerts.py b/src/test/clmctest/alerts/test_alerts.py index 59fefb48eb0f7f5e3a88f13c5ba4f06943924f51..8f5ca464693428791094994753fb737d200e5b93 100644 --- a/src/test/clmctest/alerts/test_alerts.py +++ b/src/test/clmctest/alerts/test_alerts.py @@ -22,17 +22,63 @@ ## Created for Project : FLAME """ -from time import sleep +from time import sleep, strptime from requests import post, get from os import listdir from os.path import join, dirname +from json import load +from schema import Schema, And, Or, Optional, SchemaError from clmctest.alerts.alert_handler_server import LOG_TEST_FOLDER_PATH -CLMC_SERVICE_PORT = 9080 NGINX_PORT = 80 +def is_valid_timestamp(str_timestamp): + try: + strptime(str_timestamp, "%Y-%m-%dT%H:%M:%SZ") + return True + except ValueError: + return False + + +def is_valid_details_string(details): + try: + details_dict = {key.strip(): value.strip() for key,value in [item.split("=") for item in details.split(",")]} + return len(details_dict) == 4 and "db" in details_dict and "sfc" in details_dict and "sfci" in details_dict and "policy" in details_dict + except Exception: + return False + + +JSON_BODY_SCHEMA = Schema({ + "message": "TRUE", + "id": str, + "level": "CRITICAL", + "duration": int, + "previousLevel": str, + "details": And(str, is_valid_details_string), + "time": And(str, is_valid_timestamp), + "data": { + "series": [ + { + "name": str, + Optional("tags"): { + str: str + }, + "columns": [ + str + ], + "values": [ + [ + Or(str, int) + ] + ] + } + ] + } +}) + + class TestAlerts(object): def test_alert_triggers(self, rspec_config, set_up_tear_down_fixture): @@ -47,7 +93,7 @@ class TestAlerts(object): :param rspec_config: fixture from conftest.py """ - global CLMC_SERVICE_PORT, NGINX_PORT + global NGINX_PORT, JSON_BODY_SCHEMA clmc_service_host, nginx_host = None, None for host in rspec_config: @@ -66,11 +112,13 @@ class TestAlerts(object): with open(alerts_spec, 'rb') as alerts: with open(resources_spec, 'rb') as resources: files = {'alert-spec': alerts, 'resource-spec': resources} - response = post("http://{0}:{1}/alerts".format(clmc_service_host, CLMC_SERVICE_PORT), files=files) - assert response.status_code == 200 - clmc_service_response = response.json() - assert "triggers_specification_errors" not in clmc_service_response, "Unexpected error was returned for triggers specification" - assert "triggers_action_errors" not in clmc_service_response, "Unexpected error was returned for handlers specification" + response = post("http://{0}/clmc-service/alerts".format(clmc_service_host), files=files) + + assert response.status_code == 200 + clmc_service_response = response.json() + assert "triggers_specification_errors" not in clmc_service_response, "Unexpected error was returned for triggers specification" + assert "triggers_action_errors" not in clmc_service_response, "Unexpected error was returned for handlers specification" + print("Alert spec sent successfully") print("Wait 10 seconds for Kapacitor stream/batch tasks to start working...") @@ -85,4 +133,19 @@ class TestAlerts(object): print("Wait 15 seconds for Kapacitor to trigger alerts...") sleep(15) - assert len(listdir(LOG_TEST_FOLDER_PATH)) == 4, "4 log files must have been created - one for each alert defined in the specification." + alert_logs = listdir(LOG_TEST_FOLDER_PATH) + assert len(alert_logs) == 4, "4 log files must have been created - one for each alert defined in the specification." + + for alert_log in alert_logs: + alert_log_path = join(LOG_TEST_FOLDER_PATH, alert_log) + + with open(alert_log_path) as fh: + alert_json = load(fh) + + try: + JSON_BODY_SCHEMA.validate(alert_json) + valid = True + except SchemaError: + valid = False + + assert valid, "Alert log content is invalid - {0}".format(alert_log_path) diff --git a/src/test/clmctest/services/pytest/install.sh b/src/test/clmctest/services/pytest/install.sh index a545894cd663fe5e162bdf0e93492faa1f70f876..f63ff21e3523a3a17e8cd7cee115b2dfc549ca49 100755 --- a/src/test/clmctest/services/pytest/install.sh +++ b/src/test/clmctest/services/pytest/install.sh @@ -27,5 +27,5 @@ apt-get update apt-get -y install python3 python3-pip python-influxdb update-alternatives --install /usr/bin/python python /usr/bin/python3 10 -pip3 install pytest pyyaml +pip3 install pytest pyyaml schema pip3 install --upgrade influxdb