Skip to content

Extension chaoshumio

Version 0.6.0
Repository https://github.com/chaostoolkit-incubator/chaostoolkit-humio

Build, Test, and Lint Python versions

This project is an extension for the Chaos Toolkit to target Humio.

Install

This package requires Python 3.7+

To be used from your experiment, this package must be installed in the Python environment where chaostoolkit already lives.

$ pip install -U chaostoolkit-humio

Humio Token

To use this extension, you will need one piece of information from Humio, the API token for a user.

Usage

This extension can be used a control on the experiment or a notification plugin of the Chaos Toolkit CLI itself. Usually, only one of these two methods is used at any given time as they serve similar purpose but feel free to combine them. The control approach is deeper because it logs down to the activity whereas notifications are much higher level.

This extension can also be used as a probe to fetch information from Humio.

Query Log Events

To use this extension as a probe as part of your experiment, use it as follows:

{
  "configuration": {
    "humio_url": {
      "type": "env",
      "key": "HUMIO_URL",
      "default": "https://cloud.humio.com"
    },
    "humio_repository": {
      "type": "env",
      "key": "HUMIO_REPOSITORY",
      "default": "sandbox"
    }
  },
  "secrets": {
    "humio": {
      "token": {
        "type": "env",
        "key": "HUMIO_TOKEN"
      }
    }
  },
  "steady-state-hypothesis": {
    "title": "Running experiment",
    "probes": [
      {
        "name": "run-humio-search-query",
        "type": "probe",
        "provider": {
          "type": "python",
          "module": "chaoshumio.probes",
          "func": "search_query",
          "secrets": [
            "humio"
          ],
          "arguments": {
            "qs": "count(as=_count)",
            "start": "24hours",
            "end": "now"
          }
        },
        "tolerance": {
          "name": "humio-query-result-value-greater-than",
          "type": "probe",
          "provider": {
            "type": "python",
            "module": "chaoshumio.tolerances",
            "func": "field_value_above",
            "arguments": {
              "field": "_count",
              "lower": 1
            }
          }
        }
      }
    ]
  }
}

In this example, we are using the search_query probe and validate it with a specific tolerance that can inspect the returned payload from Humio and ensure each value matches the required expectations.

Notification

To use this extension to push notifications, edit your chaostoolkit settings by adding the following payload:

notifications:
  -
    type: plugin
    module: chaoshumio.notification
    humio_url: https://myhumio.company.com
    token: my-token

By default all events will be forwarded to that channel. You may filter only those events you care for:

notifications:
  -
    type: plugin
    module: chaoshumio.notification
    humio_url: https://myhumio.company.com
    token: my-token
    events:
      - run-failed
      - run-started

Only sends those two events.

Control

To use this extension as a control over the experiment and send logs during the execution of the experiment to https://cloud.humio.com, add the following payload to your experiment:

{
    "secrets": {
        "humio": {
            "ingest_token": {
                "type": "env",
                "key": "HUMIO_INGEST_TOKEN"
            }
        }
    },
    "controls": [
        {
            "name": "humio-logger",
            "provider": {
                "type": "python",
                "module": "chaoshumio.control"
            }
        }
    ]
}

You do not need to set the secrets property in the provider block. In a control, all secrets are passed directly to each control asking for it.

If you want to send logs to a different Humio URL endpoint, specify the humio_url configuration parameter. The following shows how this parameter:

{
    "secrets": {
        "humio": {
            "ingest_token": {
                "type": "env",
                "key": "HUMIO_INGEST_TOKEN"
            }
        }
    },
    "configuration": {
        "humio_url": "https://myhumio.company.com"
    },
    "controls": [
        {
            "name": "humio-logger",
            "provider": {
                "type": "python",
                "module": "chaoshumio.control"
            }
        }
    ]
}

This will ensure the results of the experiment, steady-state, method, rollbacks and each activity are sent to Humio. The experiment itself will also be send initially.

Contribute

If you wish to contribute more functions to this package, you are more than welcome to do so. Please, fork this project, make your changes following the usual PEP 8 code style, sprinkling with tests and submit a PR for review.

The Chaos Toolkit projects require all contributors must sign a Developer Certificate of Origin on each commit they would like to merge into the master branch of the repository. Please, make sure you can abide by the rules of the DCO before submitting a PR.

Develop

If you wish to develop on this project, make sure to install the development dependencies. But first, create a virtual environment and then install those dependencies.

$ pip install .[dev]

Then, point your environment to this directory:

$ pip install -e .

Now, you can edit the files and they will be automatically be seen by your environment, even when running from the chaos command locally.

Test

To run the tests for the project execute the following:

$ pytest

Lint

Ensure your code is properly linted:

$ make format
$ make lint

Exported Controls

control

This module exports controls covering the following phases of the execution of an experiment:

Level Before After
Experiment Loading False False
Experiment True True
Steady-state Hypothesis False True
Method False True
Rollback False True
Activities False True

In addition, the controls may define the followings:

Level Enabled
Validate Control False
Configure Control True
Cleanup Control False

To use this control module, please add the following section to your experiment:

{
  "controls": [
    {
      "name": "chaoshumio",
      "provider": {
        "type": "python",
        "module": "chaoshumio.control"
      }
    }
  ]
}
controls:
- name: chaoshumio
  provider:
    module: chaoshumio.control
    type: python

This block may also be enabled at any other level (steady-state hypothesis or activity) to focus only on that level.

When enabled at the experiment level, by default, all sub-levels are also applied unless you set the automatic properties to false.

Exported Activities

control

notification


notify

Type
Module chaoshumio.notification
Name notify
Return null

Send a log message to the Humio ingest endpoint.

The settings must contain:

  • "token": a slack API token
  • "humio_url": the Humio endpoint to send the event to

If token is missing, no notification is sent. If humio_url is not specified then the default, https://cloud.humio.com, will be used.

Signature:

def notify(settings: Dict[str, Any], event: Dict[str, Any]) -> None:
    pass

Arguments:

Name Type Default Required
settings mapping Yes
event mapping Yes

Usage:

{
  "name": "notify",
  "type": "",
  "provider": {
    "type": "python",
    "module": "chaoshumio.notification",
    "func": "notify",
    "arguments": {
      "settings": {},
      "event": {}
    }
  }
}
name: notify
provider:
  arguments:
    event: {}
    settings: {}
  func: notify
  module: chaoshumio.notification
  type: python
type: ''

probes


search_query

Type probe
Module chaoshumio.probes
Name search_query
Return Any

Perform a search query against the Humio API and returns its result as-is.

Set result_as_text to true to get the result as a raw string, otherwise the probe returns a JSON payload.

Make sure to set the Humio token as part of the experiment secrets and the repository name as part of its configuration section using the humio_repository key.

See https://docs.humio.com/api/using-the-search-api-with-humio/#query

Signature:

def search_query(qs: str,
                 start: Union[int, str] = '24hours',
                 end: Union[int, str] = 'now',
                 tz_offset: int = 0,
                 params: Union[Dict[str, str], str, NoneType] = None,
                 result_as_text: Optional[bool] = False,
                 configuration: Optional[Dict[str, Dict[str, str]]] = None,
                 secrets: Optional[Dict[str, Dict[str, str]]] = None) -> Any:
    pass

Arguments:

Name Type Default Required
qs string Yes
start object “24hours” No
end object “now” No
tz_offset integer 0 No
params object null No
result_as_text object false No

Usage:

{
  "name": "search-query",
  "type": "probe",
  "provider": {
    "type": "python",
    "module": "chaoshumio.probes",
    "func": "search_query",
    "arguments": {
      "qs": ""
    }
  }
}
name: search-query
provider:
  arguments:
    qs: ''
  func: search_query
  module: chaoshumio.probes
  type: python
type: probe

tolerances


field_value_above

Type tolerance
Module chaoshumio.tolerances
Name field_value_above
Return boolean

Validate value at the given field to be above the given lower limit.

Signature:

def field_value_above(value: Any = None,
                      field: Optional[str] = None,
                      lower: float = 0.0) -> bool:
    pass

Arguments:

Name Type Default Required
value object null No
field object null No
lower number 0.0 No

Tolerances declare the value argument which is automatically injected by Chaos Toolkit as the output of the probe they are evaluating.

Usage:

{
  "steady-state-hypothesis": {
    "title": "...",
    "probes": [
      {
        "type": "probe",
        "tolerance": {
          "name": "field-value-above",
          "type": "tolerance",
          "provider": {
            "type": "python",
            "module": "chaoshumio.tolerances",
            "func": "field_value_above"
          }
        },
        "...": "..."
      }
    ]
  }
}
steady-state-hypothesis:
  probes:
  - '...': '...'
    tolerance:
      name: field-value-above
      provider:
        func: field_value_above
        module: chaoshumio.tolerances
        type: python
      type: tolerance
    type: probe
  title: '...'

field_value_between

Type tolerance
Module chaoshumio.tolerances
Name field_value_between
Return boolean

Validate value at the given field to be between the lower/upper boundaries.

Signature:

def field_value_between(value: Any = None,
                        field: Optional[str] = None,
                        lower: float = 0.0,
                        upper: float = 0.0) -> bool:
    pass

Arguments:

Name Type Default Required
value object null No
field object null No
lower number 0.0 No
upper number 0.0 No

Tolerances declare the value argument which is automatically injected by Chaos Toolkit as the output of the probe they are evaluating.

Usage:

{
  "steady-state-hypothesis": {
    "title": "...",
    "probes": [
      {
        "type": "probe",
        "tolerance": {
          "name": "field-value-between",
          "type": "tolerance",
          "provider": {
            "type": "python",
            "module": "chaoshumio.tolerances",
            "func": "field_value_between"
          }
        },
        "...": "..."
      }
    ]
  }
}
steady-state-hypothesis:
  probes:
  - '...': '...'
    tolerance:
      name: field-value-between
      provider:
        func: field_value_between
        module: chaoshumio.tolerances
        type: python
      type: tolerance
    type: probe
  title: '...'

field_value_under

Type tolerance
Module chaoshumio.tolerances
Name field_value_under
Return boolean

Validate value at the given field to be under the given upper limit.

Signature:

def field_value_under(value: Any = None,
                      field: Optional[str] = None,
                      upper: float = 0.0) -> bool:
    pass

Arguments:

Name Type Default Required
value object null No
field object null No
upper number 0.0 No

Tolerances declare the value argument which is automatically injected by Chaos Toolkit as the output of the probe they are evaluating.

Usage:

{
  "steady-state-hypothesis": {
    "title": "...",
    "probes": [
      {
        "type": "probe",
        "tolerance": {
          "name": "field-value-under",
          "type": "tolerance",
          "provider": {
            "type": "python",
            "module": "chaoshumio.tolerances",
            "func": "field_value_under"
          }
        },
        "...": "..."
      }
    ]
  }
}
steady-state-hypothesis:
  probes:
  - '...': '...'
    tolerance:
      name: field-value-under
      provider:
        func: field_value_under
        module: chaoshumio.tolerances
        type: python
      type: tolerance
    type: probe
  title: '...'