Extension chaoshumio¶

This project is an extension for the Chaos Toolkit to target Humio.

Install¶

This package requires Python 3.7+

To be used from your experiment, this package must be installed in the Python environment where chaostoolkit already lives.

$ pip install -U chaostoolkit-humio

Humio Token¶

To use this extension, you will need one piece of information from Humio, the API token for a user.

Usage¶

This extension can be used a control on the experiment or a notification plugin of the Chaos Toolkit CLI itself. Usually, only one of these two methods is used at any given time as they serve similar purpose but feel free to combine them. The control approach is deeper because it logs down to the activity whereas notifications are much higher level.

This extension can also be used as a probe to fetch information from Humio.

Query Log Events¶

To use this extension as a probe as part of your experiment, use it as follows:

{
  "configuration": {
    "humio_url": {
      "type": "env",
      "key": "HUMIO_URL",
      "default": "https://cloud.humio.com"
    },
    "humio_repository": {
      "type": "env",
      "key": "HUMIO_REPOSITORY",
      "default": "sandbox"
    }
  },
  "secrets": {
    "humio": {
      "token": {
        "type": "env",
        "key": "HUMIO_TOKEN"
      }
    }
  },
  "steady-state-hypothesis": {
    "title": "Running experiment",
    "probes": [
      {
        "name": "run-humio-search-query",
        "type": "probe",
        "provider": {
          "type": "python",
          "module": "chaoshumio.probes",
          "func": "search_query",
          "secrets": [
            "humio"
          ],
          "arguments": {
            "qs": "count(as=_count)",
            "start": "24hours",
            "end": "now"
          }
        },
        "tolerance": {
          "name": "humio-query-result-value-greater-than",
          "type": "probe",
          "provider": {
            "type": "python",
            "module": "chaoshumio.tolerances",
            "func": "field_value_above",
            "arguments": {
              "field": "_count",
              "lower": 1
            }
          }
        }
      }
    ]
  }
}

In this example, we are using the search_query probe and validate it with a specific tolerance that can inspect the returned payload from Humio and ensure each value matches the required expectations.

Notification¶

To use this extension to push notifications, edit your chaostoolkit settings by adding the following payload:

notifications:
  -
    type: plugin
    module: chaoshumio.notification
    humio_url: https://myhumio.company.com
    token: my-token

By default all events will be forwarded to that channel. You may filter only those events you care for:

notifications:
  -
    type: plugin
    module: chaoshumio.notification
    humio_url: https://myhumio.company.com
    token: my-token
    events:
      - run-failed
      - run-started

Only sends those two events.

Control¶

To use this extension as a control over the experiment and send logs during the execution of the experiment to https://cloud.humio.com, add the following payload to your experiment:

{
    "secrets": {
        "humio": {
            "ingest_token": {
                "type": "env",
                "key": "HUMIO_INGEST_TOKEN"
            }
        }
    },
    "controls": [
        {
            "name": "humio-logger",
            "provider": {
                "type": "python",
                "module": "chaoshumio.control"
            }
        }
    ]
}

You do not need to set the secrets property in the provider block. In a control, all secrets are passed directly to each control asking for it.

If you want to send logs to a different Humio URL endpoint, specify the humio_url configuration parameter. The following shows how this parameter:

{
    "secrets": {
        "humio": {
            "ingest_token": {
                "type": "env",
                "key": "HUMIO_INGEST_TOKEN"
            }
        }
    },
    "configuration": {
        "humio_url": "https://myhumio.company.com"
    },
    "controls": [
        {
            "name": "humio-logger",
            "provider": {
                "type": "python",
                "module": "chaoshumio.control"
            }
        }
    ]
}

This will ensure the results of the experiment, steady-state, method, rollbacks and each activity are sent to Humio. The experiment itself will also be send initially.

Contribute¶

If you wish to contribute more functions to this package, you are more than welcome to do so. Please, fork this project, make your changes following the usual PEP 8 code style, sprinkling with tests and submit a PR for review.

The Chaos Toolkit projects require all contributors must sign a Developer Certificate of Origin on each commit they would like to merge into the master branch of the repository. Please, make sure you can abide by the rules of the DCO before submitting a PR.

Develop¶

If you wish to develop on this project, make sure to install the development dependencies. But first, create a virtual environment and then install those dependencies.

$ pip install .[dev]

Then, point your environment to this directory:

$ pip install -e .

Now, you can edit the files and they will be automatically be seen by your environment, even when running from the chaos command locally.

Test¶

To run the tests for the project execute the following:

$ pytest

Lint¶

Ensure your code is properly linted:

$ make format
$ make lint

Exported Controls¶

control¶

This module exports controls covering the following phases of the execution of an experiment:

In addition, the controls may define the followings:

To use this control module, please add the following section to your experiment:

{
  "controls": [
    {
      "name": "chaoshumio",
      "provider": {
        "type": "python",
        "module": "chaoshumio.control"
      }
    }
  ]
}

controls:
- name: chaoshumio
  provider:
    module: chaoshumio.control
    type: python

This block may also be enabled at any other level (steady-state hypothesis or activity) to focus only on that level.

When enabled at the experiment level, by default, all sub-levels are also applied unless you set the automatic properties to false.

Exported Activities¶

control¶

notification¶

notify¶

Send a log message to the Humio ingest endpoint.

The settings must contain:

• "token": a slack API token
• "humio_url": the Humio endpoint to send the event to

If token is missing, no notification is sent. If humio_url is not specified then the default, https://cloud.humio.com, will be used.

Signature:

def notify(settings: Dict[str, Any], event: Dict[str, Any]) -> None:
    pass

Arguments:

Usage:

{
  "name": "notify",
  "type": "",
  "provider": {
    "type": "python",
    "module": "chaoshumio.notification",
    "func": "notify",
    "arguments": {
      "settings": {},
      "event": {}
    }
  }
}

name: notify
provider:
  arguments:
    event: {}
    settings: {}
  func: notify
  module: chaoshumio.notification
  type: python
type: ''

probes¶

search_query¶

Perform a search query against the Humio API and returns its result as-is.

Set result_as_text to true to get the result as a raw string, otherwise the probe returns a JSON payload.

Make sure to set the Humio token as part of the experiment secrets and the repository name as part of its configuration section using the humio_repository key.

See https://docs.humio.com/api/using-the-search-api-with-humio/#query

Signature:

def search_query(qs: str,
                 start: Union[int, str] = '24hours',
                 end: Union[int, str] = 'now',
                 tz_offset: int = 0,
                 params: Union[Dict[str, str], str, NoneType] = None,
                 result_as_text: Optional[bool] = False,
                 configuration: Optional[Dict[str, Dict[str, str]]] = None,
                 secrets: Optional[Dict[str, Dict[str, str]]] = None) -> Any:
    pass

Arguments:

Usage:

{
  "name": "search-query",
  "type": "probe",
  "provider": {
    "type": "python",
    "module": "chaoshumio.probes",
    "func": "search_query",
    "arguments": {
      "qs": ""
    }
  }
}

name: search-query
provider:
  arguments:
    qs: ''
  func: search_query
  module: chaoshumio.probes
  type: python
type: probe

tolerances¶

field_value_above¶

Validate value at the given field to be above the given lower limit.

Signature:

def field_value_above(value: Any = None,
                      field: Optional[str] = None,
                      lower: float = 0.0) -> bool:
    pass

Arguments:

Usage:

{
  "steady-state-hypothesis": {
    "title": "...",
    "probes": [
      {
        "type": "probe",
        "tolerance": {
          "name": "field-value-above",
          "type": "tolerance",
          "provider": {
            "type": "python",
            "module": "chaoshumio.tolerances",
            "func": "field_value_above"
          }
        },
        "...": "..."
      }
    ]
  }
}

steady-state-hypothesis:
  probes:
  - '...': '...'
    tolerance:
      name: field-value-above
      provider:
        func: field_value_above
        module: chaoshumio.tolerances
        type: python
      type: tolerance
    type: probe
  title: '...'

field_value_between¶

Validate value at the given field to be between the lower/upper boundaries.

Signature:

def field_value_between(value: Any = None,
                        field: Optional[str] = None,
                        lower: float = 0.0,
                        upper: float = 0.0) -> bool:
    pass

Arguments:

Usage:

{
  "steady-state-hypothesis": {
    "title": "...",
    "probes": [
      {
        "type": "probe",
        "tolerance": {
          "name": "field-value-between",
          "type": "tolerance",
          "provider": {
            "type": "python",
            "module": "chaoshumio.tolerances",
            "func": "field_value_between"
          }
        },
        "...": "..."
      }
    ]
  }
}

steady-state-hypothesis:
  probes:
  - '...': '...'
    tolerance:
      name: field-value-between
      provider:
        func: field_value_between
        module: chaoshumio.tolerances
        type: python
      type: tolerance
    type: probe
  title: '...'

field_value_under¶

Validate value at the given field to be under the given upper limit.

Signature:

def field_value_under(value: Any = None,
                      field: Optional[str] = None,
                      upper: float = 0.0) -> bool:
    pass

Arguments:

Usage:

{
  "steady-state-hypothesis": {
    "title": "...",
    "probes": [
      {
        "type": "probe",
        "tolerance": {
          "name": "field-value-under",
          "type": "tolerance",
          "provider": {
            "type": "python",
            "module": "chaoshumio.tolerances",
            "func": "field_value_under"
          }
        },
        "...": "..."
      }
    ]
  }
}

steady-state-hypothesis:
  probes:
  - '...': '...'
    tolerance:
      name: field-value-under
      provider:
        func: field_value_under
        module: chaoshumio.tolerances
        type: python
      type: tolerance
    type: probe
  title: '...'