Extension `chaosdynatrace`¶


Version	0.2.0
Repository	https://github.com/chaostoolkit-incubator/chaostoolkit-dynatrace

Dynatrace support for the Chaos Toolkit.

Install¶

To be used from your experiment, this package must be installed in the Python environment where chaostoolkit already lives.

$ pip install chaostoolkit-dynatrace

Usage¶

To use this package, you must have access to a Dynatrace instance via DynatraceApi and be allowed to connect to it.

the access credentials to the api must be specified in the configuration section

{

    "configuration": {
        "dynatrace_base_url": "https://{your-environment-id}.live.dynatrace.com"
    },
    "secrets": {
        "dynatrace": {
            "token": "..."
        }
    }
}

Here is an example of how to get the failure rate of a service in Dynatrace. for this example, the api for validate de failure rate is Metric-v1

{
    "type": "probe",
    "name": "get-failure-rate-services",
    "provider": {
        "type": "python",
        "module": "chaosdynatrace.timeseries.v1.probes",
        "func": "failure_rate",
        "secrets": ["dynatrace"],
        "arguments": {
            "entity": "SERVICE-665B05BC92550119",
            "relative_time": "30mins",
            "failed_percentage": 1
        }
    }
}

The probe returns true if the api request failure percentage is less than the failed_percentage value or return false.

The extension also exports a control to send events to Dynatrace. For instance:

{
    "controls": [
        {
            "name": "dynatrace",
            "provider": {
                "type": "python",
                "secrets": ["dynatrace"],
                "module": "chaosdynatrace.events.v2.control"
            }
        }
    ]
}

This will send start/stop logs of the experiment events.

You can correlate them to traces using the Open Telemetry extension.

{
    "configuration": {
        "dynatrace_base_url": "https://{your-environment-id}.live.dynatrace.com",
        "tracing_provider": "opentelemetry",
        "tracing_opentelemetry_exporter": "oltp-http",
        "tracing_opentelemetry_collector_endpoint": "https://{your-environment-id}.live.dynatrace.com/api/v2/otlp/v1/traces",
        "tracing_opentelemetry_collector_headers": {
            "Authorization": "Api-Token <TOKEN>"
        }
    },
    "controls": [
        {
            "name": "opentracing",
            "provider": {
                "type": "python",
                "module": "chaostracing.control"
            }
        }
    ]

The logs and traces will be automatically correlated.

Contribute¶

If you wish to contribute more functions to this package, you are more than welcome to do so. Please, fork this project, make your changes following the usual PEP 8 code style, sprinkling with tests and submit a PR for review.

Develop¶

If you wish to develop on this project, make sure to install the development dependencies. But first, create a virtual environment and then install those dependencies.

$ pip install -r requirements-dev.txt -r requirements.txt

Then, point your environment to this directory:

$ pip install -e .

Now, you can edit the files and they will be automatically be seen by your environment, even when running from the chaos command locally.

Test¶

To run the tests for the project execute the following:

$ pytest

Exported Controls¶

control¶

This module exports controls covering the following phases of the execution of an experiment:

Level	Before	After
Experiment Loading	False	False
Experiment	False	True
Steady-state Hypothesis	True	True
Method	True	True
Rollback	True	True
Activities	True	True

In addition, the controls may define the followings:

Level	Enabled
Validate Control	False
Configure Control	True
Cleanup Control	True

To use this control module, please add the following section to your experiment:

JSONYAML

{
  "controls": [
    {
      "name": "chaosdynatrace",
      "provider": {
        "type": "python",
        "module": "chaosdynatrace.events.v2.control"
      }
    }
  ]
}

controls:
- name: chaosdynatrace
  provider:
    module: chaosdynatrace.events.v2.control
    type: python

This block may also be enabled at any other level (steady-state hypothesis or activity) to focus only on that level.

When enabled at the experiment level, by default, all sub-levels are also applied unless you set the automatic properties to false.

Exported Activities¶

v2¶

`get_aggregate_logs`¶


Type	probe
Module	chaosdynatrace.logs.v2.probes
Name	get_aggregate_logs
Return	mapping

Query the aggregate logs v2 endpoint matching the criteria.

Returns a list such as:

https://www.dynatrace.com/support/help/dynatrace-api/environment-api/log-monitoring-v2/get-search-logs

Signature:

def get_aggregate_logs(
        from_time: str = 'now',
        to_time: str = None,
        query: str = None,
        time_buckets: int = 1,
        max_group_values: int = 10,
        group_by: Union[str, List[str]] = None,
        configuration: Dict[str, Dict[str, str]] = None,
        secrets: Dict[str, Dict[str, str]] = None) -> Dict[str, Any]:
    pass

Arguments:

Name	Type	Default	Required
from_time	string	“now”	No
to_time	string	null	No
query	string	null	No
time_buckets	integer	1	No
max_group_values	integer	10	No
group_by	object	null	No

Usage:

JSONYAML

{
  "name": "get-aggregate-logs",
  "type": "probe",
  "provider": {
    "type": "python",
    "module": "chaosdynatrace.logs.v2.probes",
    "func": "get_aggregate_logs"
  }
}

name: get-aggregate-logs
provider:
  func: get_aggregate_logs
  module: chaosdynatrace.logs.v2.probes
  type: python
type: probe

`get_search_logs`¶


Type	probe
Module	chaosdynatrace.logs.v2.probes
Name	get_search_logs
Return	list

Query the search logs v2 endpoint matching the criteria.

Returns a list such as:

https://www.dynatrace.com/support/help/dynatrace-api/environment-api/log-monitoring-v2/get-search-logs

Signature:

def get_search_logs(
        from_time: str = 'now',
        to_time: str = None,
        query: str = None,
        limit: int = 1000,
        sort: str = None,
        configuration: Dict[str, Dict[str, str]] = None,
        secrets: Dict[str, Dict[str, str]] = None) -> List[Dict[str, Any]]:
    pass

Arguments:

Name	Type	Default	Required
from_time	string	“now”	No
to_time	string	null	No
query	string	null	No
limit	integer	1000	No
sort	string	null	No

Usage:

JSONYAML

{
  "name": "get-search-logs",
  "type": "probe",
  "provider": {
    "type": "python",
    "module": "chaosdynatrace.logs.v2.probes",
    "func": "get_search_logs"
  }
}

name: get-search-logs
provider:
  func: get_search_logs
  module: chaosdynatrace.logs.v2.probes
  type: python
type: probe

`query_data_points`¶


Type	probe
Module	chaosdynatrace.metrics.v2.probes
Name	query_data_points
Return	list

Query the metrics v2 endpoint for any data point matching the various parameters.

Returns a list such as:

[
    {
   'metricId': 'builtin:tech.generic.network.bytesRx:splitBy():avg:auto:sort(value(avg,descending)):limit(10)',
   'data': [
  {
 'dimensions': [],
 'dimensionMap': {},
 'timestamps': [
1647545100000,
1647545400000,
1647545700000,
1647546000000,
1647546300000,
1647546600000,
1647546900000
 ],
 'values': [
None,
9272.911109076605,
3565.171102555593,
3441.491649373372,
48242.974005126955,
8055.613537597656,
None
 ]
  }
   ]
    }
]

https://www.dynatrace.com/support/help/dynatrace-api/environment-api/metric-v2/get-data-points

Signature:

def query_data_points(
        metrics_selector: Union[str, List[str]] = '*',
        entity_selector: Union[str, List[str]] = None,
        resolution: str = '5m',
        from_time: str = 'now',
        to_time: str = None,
        configuration: Dict[str, Dict[str, str]] = None,
        secrets: Dict[str, Dict[str, str]] = None) -> List[Dict[str, Any]]:
    pass

Arguments:

Name	Type	Default	Required
metrics_selector	object	”*”	No
entity_selector	object	null	No
resolution	string	“5m”	No
from_time	string	“now”	No
to_time	string	null	No

Usage:

JSONYAML

{
  "name": "query-data-points",
  "type": "probe",
  "provider": {
    "type": "python",
    "module": "chaosdynatrace.metrics.v2.probes",
    "func": "query_data_points"
  }
}

name: query-data-points
provider:
  func: query_data_points
  module: chaosdynatrace.metrics.v2.probes
  type: python
type: probe

v1¶

`failure_rate`¶


Type	probe
Module	chaosdynatrace.timeseries.v1.probes
Name	failure_rate
Return	boolean

Validates the failure rate of a specific service. Returns true if the failure rate is less than the expected failure rate For more information check the api documentation. https://www.dynatrace.com/support/help/dynatrace-api/environment-api/metric-v1/

Signature:

def failure_rate(entity: str,
                 relative_time: str,
                 failed_percentage: int,
                 configuration: Dict[str, Dict[str, str]],
                 secrets: Dict[str, Dict[str, str]] = None) -> bool:
    pass

Arguments:

Name	Type	Required
entity	string	Yes
relative_time	string	Yes
failed_percentage	integer	Yes

Usage:

JSONYAML

{
  "name": "failure-rate",
  "type": "probe",
  "provider": {
    "type": "python",
    "module": "chaosdynatrace.timeseries.v1.probes",
    "func": "failure_rate",
    "arguments": {
      "entity": "",
      "relative_time": "",
      "failed_percentage": 0
    }
  }
}

name: failure-rate
provider:
  arguments:
    entity: ''
    failed_percentage: 0
    relative_time: ''
  func: failure_rate
  module: chaosdynatrace.timeseries.v1.probes
  type: python
type: probe

Extension chaosdynatrace¶

Install¶

Usage¶

Contribute¶

Develop¶

Test¶

Exported Controls¶

control¶

Exported Activities¶

v2¶

get_aggregate_logs¶

get_search_logs¶

query_data_points¶

v1¶

failure_rate¶

Extension `chaosdynatrace`¶

`get_aggregate_logs`¶

`get_search_logs`¶

`query_data_points`¶

`failure_rate`¶