Extension chaosaws¶

AWS extension for the Chaos Toolkit

Community • ChangeLog

Welcome to the Amazon Web Services (AWS) extension for Chaos Toolkit. The package aggregates activities to target your AWS infrastructure and explore your resilience via Chaos Engineering experiments.

Install¶

This package requires Python 3.8+

To be used from your experiment, this package must be installed in the Python environment where [chaostoolkit][] already lives.

$ pip install -U chaostoolkit-aws

Usage¶

To use the probes and actions from this package, add the following to your experiment file:

{
    "name": "stop-an-ec2-instance",
    "provider": {
        "type": "python",
        "module": "chaosaws.ec2.actions",
        "func": "stop_instance",
        "arguments": {
            "instance_id": "i-123456"
        }
    }
},
{
    "name": "create-a-new-policy",
    "provider": {
        "type": "python",
        "module": "chaosaws.iam.actions",
        "func": "create_policy",
        "arguments": {
            "name": "mypolicy",
            "path": "user/Jane",
            "policy": {
                "Version": "2012-10-17",
                "Statement": [
                    {
                        "Effect": "Allow",
                        "Action": [
                            "s3:ListAllMyBuckets",
                            "s3:GetBucketLocation"
                        ],
                        "Resource": "arn:aws:s3:::*"
                    }
                ]
            }
        }
    }
}

Or select one at random from an AZ:

{
    "name": "stop-an-ec2-instance-in-az-at-random",
    "provider": {
        "type": "python",
        "module": "chaosaws.ec2.actions",
        "func": "stop_instance",
        "arguments": {
            "az": "us-west-1"
        }
    }
}

That’s it!

Please explore the code to see existing probes and actions.

Configuration¶

Credentials¶

This extension uses the boto3 library under the hood. This library expects that you have properly configured your environment to connect and authenticate with the AWS services.

Use default profile from ~/.aws/credentials or ~/.aws/config¶

This is the most basic case, assuming your default profile is properly configured in ~/.aws/credentials (or ~/.aws/config), then you do not need to pass any specific credentials to the experiment.

Use a non-default profile from ~/.aws/credentials or ~/.aws/config¶

Assuming you have configure a profile in your ~/.aws/credentials (or ~/.aws/config) file, you may declare it in your experiment as follows:

{
    "configuration": {
        "aws_profile_name": "dev"
    }
}

Your ~/.aws/credentials should look like this:

[dev]
aws_access_key_id = XYZ
aws_secret_access_key = UIOPIY

Or, your ~/.aws/config should look like this:

[profile dev]
output = json
aws_access_key_id = XYZ
aws_secret_access_key = UIOPIY

Assume an ARN role from a non-default profile¶

Assuming you have configure a profile in your ~/.aws/config file with a specific ARN role you want to assume during the run:

{
    "configuration": {
        "aws_profile_name": "dev"
    }
}

Your ~/.aws/config should look like this:

[default]
output = json

[profile dev]
role_arn = arn:aws:iam::XXXXXXX:role/role-name
source_profile = default

Assume an ARN role from within the experiment¶

You mays also assume a role by declaring the role ARN in the experiment directly. In that case, the profile has no impact if you also set it.

    "configuration": {
        "aws_assume_role_arn": "arn:aws:iam::XXXXXXX:role/role-name",
        "aws_assume_role_session_name": "my-chaos"
    }

The aws_assume_role_session_name key is optional and will be set to "ChaosToolkit" when not provided.

When this approach is used, the extension performs a assume role call against the AWS STS service to fetch credentials dynamically.

Pass credentials explicitely¶

You can pass the credentials as a secret to the experiment definition as follows:

{
    "secrets": {
        "aws": {
            "aws_access_key_id": "your key",
            "aws_secret_access_key": "access key",
            "aws_session_token": "token",
        }
    }
}

Then, use it as follows:

{
    "name": "stop-an-ec2-instance",
    "provider": {
        "type": "python",
        "module": "chaosaws.ec2.actions",
        "func": "stop_instance",
        "secrets": ["aws"],
        "arguments": {
            "instance_id": "i-123456"
        }
    }
}

Setting the region¶

In additon to the authentication credentials, you must configure the region against which you want to use.

You can either declare it at the top level of the experiment, add:

{
    "configuration": {
        "aws_region": "us-east-1"
    }
}

or

{
    "configuration": {
        "aws_region": {
            "type": "env",
            "key": "AWS_REGION"
        }
    }
}

But you can also simply set either AWS_REGION or AWS_DEFAULT_REGION in your terminal session without declaring anything in the experiment.

If none of these are set, your experiment will likely fail.

Contribute¶

If you wish to contribute more functions to this package, you are more than welcome to do so. Please, fork this project, write unit tests to cover the proposed changes, implement the changes, ensure they meet the formatting standards by running pdm run format and pdm run lint.

The Chaos Toolkit projects require all contributors must sign a Developer Certificate of Origin on each commit they would like to merge into the master branch of the repository. Please, make sure you can abide by the rules of the DCO before submitting a PR.

Develop¶

If you wish to develop on this project, make sure to install the development dependencies. First installk PDM.

$ pdm install --dev

Now, you can edit the files and they will be automatically be seen by your environment, even when running from the chaos command locally.

Tests¶

To run the tests for the project execute the following:

$ pdm run test

Formatting and Linting¶

We use ruff to lint and format the code.

Before raising a Pull Request, we recommend you run formatting against your code with:

$ pdm run format

This will automatically format any code that doesn’t adhere to the formatting standards.

As some things are not picked up by the formatting, we also recommend you run:

$ pdm run lint

To ensure that any unused import statements/strings that are too long, etc. are also picked up.

Add new AWS API Support¶

Once you have setup your environment, you can start adding new AWS API support by adding new actions, probes and entire sub-packages for those.

Services supported by boto¶

This package relies on boto3 to wrap the API calls into a fluent Python API. Some newer AWS services are not yet available in boto3, in that case, you should read the next section.

Let’s say you want to support a new action in the EC2 sub-package.

Start by creating a new function in ec2/actions.py:

from chaoslib.types import Configuration, Secrets

from chaosaws import aws_client
from chaosaws.types import AWSResponse

def reboot_instance(instance_id: str, dry_run: bool=False,
                    configuration: Configuration=None,
                    secrets: Secrets=None) -> AWSResponse:
    """
    Reboot a given EC2 instance.
    """
    client = aws_client('ec2', configuration, secrets)
    return client.reboot_instances(InstanceIds=[instance_id], DryRun=dry_run)

As you can see, the actual code is straightforward. You first create a EC2 client and simply call the appropriate method on that client with the expected arguments. We return the action as-is so that it can be logged by the chaostoolkit, or even be used as part of a steady-state hypothesis probe (if this was a probe, not action that is).

You could decide to make more than one AWS API call but, it is better to keep it simple so that composition is easier from the experiment. Nonetheless, you may also compose those directly into a single action as well for specific use-cases.

Please refer to the Chaos Toolkit documentation to learn more about the configuration and secrets objects.

Once you have implemented that action, you must create at least one unit test for it in the tests/ec2/test_ec2_actions.py test module. For example:

from chaosaws.ec2.actions import reboot_instancex

@patch('chaosaws.ec2.actions.aws_client', autospec=True)
def test_reboot_instance(aws_client):
    client = MagicMock()
    aws_client.return_value = client
    inst_id = "i-1234567890abcdef0"
    response = reboot_instance(inst_id)
    client.reboot_instances.assert_called_with(
        InstanceIds=[inst_id], DryRun=False)

By using the built-in Python module to mock objects, we can mock the EC2 client and assert that we do indeed call the appropriate method with the right arguments. You are encouraged to write more than a single test for various conditions.

Finally, should you choose to add support for a new AWS API resource altogether, you should create the according sub-package.

Services not supported by boto (new AWS features)¶

If the support you want to provide is for a new AWS service that boto does not support yet, this requires direct call to the API endpoint via the requests package. Say we have a new service, not yet supported by boto3

from chaoslib.types import Configuration, Secrets

from chaosaws import signed_api_call
from chaosaws.types import AWSResponse

def terminate_worker_node(worker_node_id: str,
                          configuration: Configuration=None,
                          secrets: Secrets=None) -> AWSResponse:
    """
    Terminate a worker node.
    """
    params = {
        "DryRun": True,
        "WorkerNodeId.1": worker_node_id
    }
    response = signed_api_call(
        'some-new-service-name', path='/2018-01-01/worker/terminate',
        method='POST', params=params,
        configuration=configuration, secrets=secrets)
    return response.json()

Here is an example on existing API call (as a more concrete snippet):

from chaoslib.types import Configuration, Secrets

from chaosaws import signed_api_call

def stop_instance(instance_id: str, configuration: Configuration=None,
                  secrets: Secrets=None) -> str:
    response = signed_api_call(
        'ec2',
        configuration=configuration,
        secrets=secrets,
        params={
            "Action": "StopInstances",
            "InstanceId.1": instance_id,
            "Version": "2013-06-15"
        }
    )

    # this API returns XML, not JSON
    return response.text

When using the signed_api_call, you are responsible for the right way of passing the parameters. Basically, look at the AWS documentation for each API call.

WARNING: It should be noted that, whenever boto3 implements an API, this package should be updated accordingly, as boto3 is much more versatile and solid.

Make your new sub-package discoverable¶

Finally, if you have created a new sub-package entirely, you need to make its capability discoverable by the chaos toolkit. Simply amend the discover function in the chaosaws/__init__.py. For example, assuming a new eks sub-package, with actions and probes:

    activities.extend(discover_actions("chaosaws.eks.actions"))
    activities.extend(discover_probes("chaosaws.eks.probes"))

Exported Controls¶

upload¶

This module exports controls covering the following phases of the execution of an experiment:

In addition, the controls may define the followings:

To use this control module, please add the following section to your experiment:

{
  "controls": [
    {
      "name": "chaosaws",
      "provider": {
        "type": "python",
        "module": "chaosaws.s3.controls.upload"
      }
    }
  ]
}

controls:
- name: chaosaws
  provider:
    module: chaosaws.s3.controls.upload
    type: python

This block may also be enabled at any other level (steady-state hypothesis or activity) to focus only on that level.

When enabled at the experiment level, by default, all sub-levels are also applied unless you set the automatic properties to false.

Exported Activities¶

asg¶

attach_volume¶

Attaches ebs volumes that have been previously detached by CTK

: One of: asg_names: list: one or more asg names tags: list: key/value pairs to identify asgs by

tags are expected as a list of dictionary objects: [ {‘Key’: ‘TagKey1’, ‘Value’: ‘TagValue1’}, {‘Key’: ‘TagKey2’, ‘Value’: ‘TagValue2’}, … ]

Signature:

def attach_volume(
        asg_names: List[str] = None,
        tags: List[Dict[str, str]] = None,
        configuration: Dict[str, Dict[str, str]] = None,
        secrets: Dict[str, Dict[str, str]] = None) -> List[Dict[str, Any]]:
    pass

Arguments:

Usage:

{
  "name": "attach-volume",
  "type": "action",
  "provider": {
    "type": "python",
    "module": "chaosaws.asg.actions",
    "func": "attach_volume"
  }
}

name: attach-volume
provider:
  func: attach_volume
  module: chaosaws.asg.actions
  type: python
type: action

change_subnets¶

Adds/removes subnets on autoscaling groups

: One of: asg_names: a list of one or more asg names tags: a list of key/value pair to identify asg(s) by

subnets: a list of subnet IDs to associate to the ASG

tags are expected as a list of dictionary objects: [ {‘Key’: ‘TagKey1’, ‘Value’: ‘TagValue1’}, {‘Key’: ‘TagKey2’, ‘Value’: ‘TagValue2’}, … ]

Signature:

def change_subnets(subnets: List[str],
                   asg_names: List[str] = None,
                   tags: List[dict] = None,
                   configuration: Dict[str, Dict[str, str]] = None,
                   secrets: Dict[str, Dict[str, str]] = None):
    pass

Arguments:

Usage:

{
  "name": "change-subnets",
  "type": "action",
  "provider": {
    "type": "python",
    "module": "chaosaws.asg.actions",
    "func": "change_subnets",
    "arguments": {
      "subnets": []
    }
  }
}

name: change-subnets
provider:
  arguments:
    subnets: []
  func: change_subnets
  module: chaosaws.asg.actions
  type: python
type: action

describe_auto_scaling_groups¶

Returns AWS descriptions for provided ASG(s)

Params: OneOf: - asg_names: a list of asg names to describe - tags: a list of key/value pairs to collect ASG(s)

tags are expected as a list of dictionary objects: [ {‘Key’: ‘TagKey1’, ‘Value’: ‘TagValue1’}, {‘Key’: ‘TagKey2’, ‘Value’: ‘TagValue2’}, … ]

Signature:

def describe_auto_scaling_groups(
        asg_names: List[str] = None,
        tags: List[Dict[str, Any]] = None,
        configuration: Dict[str, Dict[str, str]] = None,
        secrets: Dict[str, Dict[str, str]] = None) -> Dict[str, Any]:
    pass

Arguments:

Usage:

{
  "name": "describe-auto-scaling-groups",
  "type": "probe",
  "provider": {
    "type": "python",
    "module": "chaosaws.asg.probes",
    "func": "describe_auto_scaling_groups"
  }
}

name: describe-auto-scaling-groups
provider:
  func: describe_auto_scaling_groups
  module: chaosaws.asg.probes
  type: python
type: probe

desired_equals_healthy¶

If desired number matches the number of healthy instances for each of the auto-scaling groups

Returns: bool

Signature:

def desired_equals_healthy(asg_names: List[str],
                           configuration: Dict[str, Dict[str, str]] = None,
                           secrets: Dict[str, Dict[str, str]] = None) -> bool:
    pass

Arguments:

Usage:

{
  "name": "desired-equals-healthy",
  "type": "probe",
  "provider": {
    "type": "python",
    "module": "chaosaws.asg.probes",
    "func": "desired_equals_healthy",
    "arguments": {
      "asg_names": []
    }
  }
}

name: desired-equals-healthy
provider:
  arguments:
    asg_names: []
  func: desired_equals_healthy
  module: chaosaws.asg.probes
  type: python
type: probe

desired_equals_healthy_tags¶

If desired number matches the number of healthy instances

for each of the auto-scaling groups matching tags provided

tags are expected as: [{ ‘Key’: ‘KeyName’, ‘Value’: ‘KeyValue’ }, … ]

Returns: bool

Signature:

def desired_equals_healthy_tags(
        tags: List[Dict[str, str]],
        configuration: Dict[str, Dict[str, str]] = None,
        secrets: Dict[str, Dict[str, str]] = None) -> bool:
    pass

Arguments:

Usage:

{
  "name": "desired-equals-healthy-tags",
  "type": "probe",
  "provider": {
    "type": "python",
    "module": "chaosaws.asg.probes",
    "func": "desired_equals_healthy_tags",
    "arguments": {
      "tags": []
    }
  }
}

name: desired-equals-healthy-tags
provider:
  arguments:
    tags: []
  func: desired_equals_healthy_tags
  module: chaosaws.asg.probes
  type: python
type: probe

detach_random_instances¶

Detaches one or more random instances from an autoscaling group

: One of: asg_names: a list of one or more asg names tags: a list of key/value pair to identify asg(s) by

One of:

instance_count: integer value of number of instances to detach instance_percent: 1-100, percent of instances to detach

decrement_capacity: boolean value to determine if the desired capacity
of the autoscaling group should be decreased

tags are expected as a list of dictionary objects: [ {‘Key’: ‘TagKey1’, ‘Value’: ‘TagValue1’}, {‘Key’: ‘TagKey2’, ‘Value’: ‘TagValue2’}, … ]

Signature:

def detach_random_instances(
        asg_names: List[str] = None,
        tags: List[dict] = None,
        instance_count: int = None,
        instance_percent: int = None,
        decrement_capacity: bool = False,
        configuration: Dict[str, Dict[str, str]] = None,
        secrets: Dict[str, Dict[str, str]] = None) -> Dict[str, Any]:
    pass

Arguments:

Usage:

{
  "name": "detach-random-instances",
  "type": "action",
  "provider": {
    "type": "python",
    "module": "chaosaws.asg.actions",
    "func": "detach_random_instances"
  }
}

name: detach-random-instances
provider:
  func: detach_random_instances
  module: chaosaws.asg.actions
  type: python
type: action

detach_random_volume¶

Detaches a random (non root) ebs volume from ec2 instances associated to an ASG

: One of: asg_names: a list of one or more asg names tags: a list of key/value pair to identify asg(s) by

force: force detach volume (default: true)

tags are expected as a list of dictionary objects: [ {‘Key’: ‘TagKey1’, ‘Value’: ‘TagValue1’}, {‘Key’: ‘TagKey2’, ‘Value’: ‘TagValue2’}, … ]

Signature:

def detach_random_volume(
        asg_names: List[str] = None,
        tags: List[Dict[str, str]] = None,
        force: bool = True,
        configuration: Dict[str, Dict[str, str]] = None,
        secrets: Dict[str, Dict[str, str]] = None) -> List[Dict[str, Any]]:
    pass

Arguments:

Usage:

{
  "name": "detach-random-volume",
  "type": "action",
  "provider": {
    "type": "python",
    "module": "chaosaws.asg.actions",
    "func": "detach_random_volume"
  }
}

name: detach-random-volume
provider:
  func: detach_random_volume
  module: chaosaws.asg.actions
  type: python
type: action

has_subnets¶

Determines if the provided autoscaling groups are in the provided subnets

:returns boolean

Signature:

def has_subnets(subnets: List[str],
                asg_names: List[str] = None,
                tags: List[Dict[str, str]] = None,
                configuration: Dict[str, Dict[str, str]] = None,
                secrets: Dict[str, Dict[str, str]] = None) -> bool:
    pass

Arguments:

Usage:

{
  "name": "has-subnets",
  "type": "probe",
  "provider": {
    "type": "python",
    "module": "chaosaws.asg.probes",
    "func": "has_subnets",
    "arguments": {
      "subnets": []
    }
  }
}

name: has-subnets
provider:
  arguments:
    subnets: []
  func: has_subnets
  module: chaosaws.asg.probes
  type: python
type: probe

instance_count_by_health¶

Reports the number of instances currently in the ASG by their health status

Params: OneOf: - asg_names: a list of asg names to describe - tags: a list of key/value pairs to collect ASG(s)

- count_healthy: boolean: true for healthy instance count,

false for unhealthy instance count

tags are expected as a list of dictionary objects: [ {‘Key’: ‘TagKey1’, ‘Value’: ‘TagValue1’}, {‘Key’: ‘TagKey2’, ‘Value’: ‘TagValue2’}, … ]

Signature:

def instance_count_by_health(asg_names: List[str] = None,
                             tags: List[Dict[str, str]] = None,
                             count_healthy: bool = True,
                             configuration: Dict[str, Dict[str, str]] = None,
                             secrets: Dict[str, Dict[str, str]] = None) -> int:
    pass

Arguments:

Usage:

{
  "name": "instance-count-by-health",
  "type": "probe",
  "provider": {
    "type": "python",
    "module": "chaosaws.asg.probes",
    "func": "instance_count_by_health"
  }
}

name: instance-count-by-health
provider:
  func: instance_count_by_health
  module: chaosaws.asg.probes
  type: python
type: probe

is_scaling_in_progress¶

Check if there is any scaling activity in progress for ASG matching tags

Returns: Boolean

Signature:

def is_scaling_in_progress(tags: List[Dict[str, str]],
                           configuration: Dict[str, Dict[str, str]] = None,
                           secrets: Dict[str, Dict[str, str]] = None) -> bool:
    pass

Arguments:

Usage:

{
  "name": "is-scaling-in-progress",
  "type": "probe",
  "provider": {
    "type": "python",
    "module": "chaosaws.asg.probes",
    "func": "is_scaling_in_progress",
    "arguments": {
      "tags": []
    }
  }
}

name: is-scaling-in-progress
provider:
  arguments:
    tags: []
  func: is_scaling_in_progress
  module: chaosaws.asg.probes
  type: python
type: probe

process_is_suspended¶

Determines if one or more processes on an ASG are suspended.

:returns Boolean

Signature:

def process_is_suspended(asg_names: List[str] = None,
                         tags: List[Dict[str, str]] = None,
                         process_names: List[str] = None,
                         configuration: Dict[str, Dict[str, str]] = None,
                         secrets: Dict[str, Dict[str, str]] = None) -> bool:
    pass

Arguments:

Usage:

{
  "name": "process-is-suspended",
  "type": "probe",
  "provider": {
    "type": "python",
    "module": "chaosaws.asg.probes",
    "func": "process_is_suspended"
  }
}

name: process-is-suspended
provider:
  func: process_is_suspended
  module: chaosaws.asg.probes
  type: python
type: probe

resume_processes¶

Resumes 1 or more suspended processes on a list of auto scaling groups.

If no process is specified, all suspended auto scaling processes will be resumed.

For a list of valid processes that can be suspended, reference: https://docs.aws.amazon.com/autoscaling/ec2/userguide/as-suspend-resume-processes.html

: One of: - asg_names: a list of one or more asg names to target - tags: a list of key/value pairs to identify the asgs by

tags are expected as a list of dictionary objects: [ {‘Key’: ‘TagKey1’, ‘Value’: ‘TagValue1’}, {‘Key’: ‘TagKey2’, ‘Value’: ‘TagValue2’}, … ]

Signature:

def resume_processes(
        asg_names: List[str] = None,
        tags: List[Dict[str, str]] = None,
        process_names: List[str] = None,
        configuration: Dict[str, Dict[str, str]] = None,
        secrets: Dict[str, Dict[str, str]] = None) -> Dict[str, Any]:
    pass

Arguments:

Usage:

{
  "name": "resume-processes",
  "type": "action",
  "provider": {
    "type": "python",
    "module": "chaosaws.asg.actions",
    "func": "resume_processes"
  }
}

name: resume-processes
provider:
  func: resume_processes
  module: chaosaws.asg.actions
  type: python
type: action

stop_random_instances¶

Terminates one or more random healthy instances associated to an ALB

A healthy instance is considered one with a status of ‘InService’

: - force: force stop the instances (default: False)

One Of:

• asg_names: a list of one or more asg names to target

• tags: a list of key/value pairs to identify the asgs by

  One Of: - instance_count: the number of instances to terminate - instance_percent: the percentage of instances to terminate - az: the availability zone to terminate instances

tags are expected as a list of dictionary objects: [ {‘Key’: ‘TagKey1’, ‘Value’: ‘TagValue1’}, {‘Key’: ‘TagKey2’, ‘Value’: ‘TagValue2’}, … ]

Signature:

def stop_random_instances(
        asg_names: List[str] = None,
        tags: List[Dict[str, str]] = None,
        instance_count: int = None,
        instance_percent: int = None,
        az: str = None,
        force: bool = False,
        configuration: Dict[str, Dict[str, str]] = None,
        secrets: Dict[str, Dict[str, str]] = None) -> List[Dict[str, Any]]:
    pass

Arguments:

Usage:

{
  "name": "stop-random-instances",
  "type": "action",
  "provider": {
    "type": "python",
    "module": "chaosaws.asg.actions",
    "func": "stop_random_instances"
  }
}

name: stop-random-instances
provider:
  func: stop_random_instances
  module: chaosaws.asg.actions
  type: python
type: action

suspend_processes¶

Suspends 1 or more processes on a list of auto scaling groups.

If no process is specified, all running auto scaling processes will be suspended.

For a list of valid processes that can be suspended, reference: https://docs.aws.amazon.com/autoscaling/ec2/userguide/as-suspend-resume-processes.html

: One of: - asg_names: a list of one or more asg names to target - tags: a list of key/value pairs to identify the asgs by

tags are expected as a list of dictionary objects: [ {‘Key’: ‘TagKey1’, ‘Value’: ‘TagValue1’}, {‘Key’: ‘TagKey2’, ‘Value’: ‘TagValue2’}, … ]

Signature:

def suspend_processes(
        asg_names: List[str] = None,
        tags: List[Dict[str, str]] = None,
        process_names: List[str] = None,
        configuration: Dict[str, Dict[str, str]] = None,
        secrets: Dict[str, Dict[str, str]] = None) -> Dict[str, Any]:
    pass

Arguments:

Usage:

{
  "name": "suspend-processes",
  "type": "action",
  "provider": {
    "type": "python",
    "module": "chaosaws.asg.actions",
    "func": "suspend_processes"
  }
}

name: suspend-processes
provider:
  func: suspend_processes
  module: chaosaws.asg.actions
  type: python
type: action

terminate_random_instances¶

Terminates one or more random healthy instances associated to an ALB

A healthy instance is considered one with a status of ‘InService’

: One Of: - asg_names: a list of one or more asg names to target - tags: a list of key/value pairs to identify the asgs by

One Of: - instance_count: the number of instances to terminate - instance_percent: the percentage of instances to terminate - az: the availability zone to terminate instances

tags are expected as a list of dictionary objects: [ {‘Key’: ‘TagKey1’, ‘Value’: ‘TagValue1’}, {‘Key’: ‘TagKey2’, ‘Value’: ‘TagValue2’}, … ]

Signature:

def terminate_random_instances(
        asg_names: List[str] = None,
        tags: List[Dict[str, str]] = None,
        instance_count: int = None,
        instance_percent: int = None,
        az: str = None,
        configuration: Dict[str, Dict[str, str]] = None,
        secrets: Dict[str, Dict[str, str]] = None) -> List[Dict[str, Any]]:
    pass

Arguments:

Usage:

{
  "name": "terminate-random-instances",
  "type": "action",
  "provider": {
    "type": "python",
    "module": "chaosaws.asg.actions",
    "func": "terminate_random_instances"
  }
}

name: terminate-random-instances
provider:
  func: terminate_random_instances
  module: chaosaws.asg.actions
  type: python
type: action

wait_desired_equals_healthy¶

Wait until desired number matches the number of healthy instances for each of the auto-scaling groups

Returns: Integer (number of seconds it took to wait) or sys.maxsize in case of timeout

Signature:

def wait_desired_equals_healthy(
        asg_names: List[str],
        configuration: Dict[str, Dict[str, str]] = None,
        timeout: Union[int, float] = 300,
        secrets: Dict[str, Dict[str, str]] = None) -> int:
    pass

Arguments:

Usage:

{
  "name": "wait-desired-equals-healthy",
  "type": "probe",
  "provider": {
    "type": "python",
    "module": "chaosaws.asg.probes",
    "func": "wait_desired_equals_healthy",
    "arguments": {
      "asg_names": []
    }
  }
}

name: wait-desired-equals-healthy
provider:
  arguments:
    asg_names: []
  func: wait_desired_equals_healthy
  module: chaosaws.asg.probes
  type: python
type: probe

wait_desired_equals_healthy_tags¶

Wait until desired number matches the number of healthy instances for each of the auto-scaling groups matching tags provided

tags are expected as: [{ ‘Key’: ‘KeyName’, ‘Value’: ‘KeyValue’ }, … ]

Returns: Integer (number of seconds it took to wait) or sys.maxsize in case of timeout

Signature:

def wait_desired_equals_healthy_tags(
        tags: List[Dict[str, str]],
        timeout: Union[int, float] = 300,
        configuration: Dict[str, Dict[str, str]] = None,
        secrets: Dict[str, Dict[str, str]] = None) -> int:
    pass

Arguments:

Usage:

{
  "name": "wait-desired-equals-healthy-tags",
  "type": "probe",
  "provider": {
    "type": "python",
    "module": "chaosaws.asg.probes",
    "func": "wait_desired_equals_healthy_tags",
    "arguments": {
      "tags": []
    }
  }
}

name: wait-desired-equals-healthy-tags
provider:
  arguments:
    tags: []
  func: wait_desired_equals_healthy_tags
  module: chaosaws.asg.probes
  type: python
type: probe

wait_desired_not_equals_healthy_tags¶

Wait until desired number doesn’t match the number of healthy instances for each of the auto-scaling groups matching tags provided

tags are expected as: [{ ‘Key’: ‘KeyName’, ‘Value’: ‘KeyValue’ }, … ]

Returns: Integer (number of seconds it took to wait) or sys.maxsize in case of timeout

Signature:

def wait_desired_not_equals_healthy_tags(
        tags: List[Dict[str, str]],
        timeout: Union[int, float] = 300,
        configuration: Dict[str, Dict[str, str]] = None,
        secrets: Dict[str, Dict[str, str]] = None) -> int:
    pass

Arguments:

Usage:

{
  "name": "wait-desired-not-equals-healthy-tags",
  "type": "probe",
  "provider": {
    "type": "python",
    "module": "chaosaws.asg.probes",
    "func": "wait_desired_not_equals_healthy_tags",
    "arguments": {
      "tags": []
    }
  }
}

name: wait-desired-not-equals-healthy-tags
provider:
  arguments:
    tags: []
  func: wait_desired_not_equals_healthy_tags
  module: chaosaws.asg.probes
  type: python
type: probe

awslambda¶

add_error¶

Introduce error from a lambda by using the Chaos Lambda Extension https://github.com/aws-cli-tools/chaos-lambda-extension

The error is the body passed as a json encoded string. The probability represents the likelihood of the fault to be injected on a particular exchange.

You can set the layer depending on your lambda architecture and region: see https://github.com/aws-cli-tools/chaos-lambda-extension/blob/main/LAYERS.md

Signature:

def add_error(
        function_name: str,
        body:
    str = '{"statusCode": 500, "body": "Greeting from Chaos Toolkit"}',
        probability: float = 0.9,
        layer:
    str = 'arn:aws:lambda:us-east-1:871265522301:layer:chaos-lambda-extension-x86_64-unknown-linux-gnu-release:12',
        configuration: Dict[str, Dict[str, str]] = None,
        secrets: Dict[str, Dict[str, str]] = None) -> Dict[str, Any]:
    pass

Arguments:

Usage:

{
  "name": "add-error",
  "type": "action",
  "provider": {
    "type": "python",
    "module": "chaosaws.awslambda.actions",
    "func": "add_error",
    "arguments": {
      "function_name": ""
    }
  }
}

name: add-error
provider:
  arguments:
    function_name: ''
  func: add_error
  module: chaosaws.awslambda.actions
  type: python
type: action

add_latency¶

Introduce latency from a lambda by using the Chaos Lambda Extension https://github.com/aws-cli-tools/chaos-lambda-extension

The latencyis in seconds between 1 and 900. The probability represents the likelihood of the fault to be injected on a particular exchange.

You can set the layer depending on your lambda architecture and region: see https://github.com/aws-cli-tools/chaos-lambda-extension/blob/main/LAYERS.md

Signature:

def add_latency(
        function_name: str,
        latency: int = 1,
        probability: float = 0.9,
        layer:
    str = 'arn:aws:lambda:us-east-1:871265522301:layer:chaos-lambda-extension-x86_64-unknown-linux-gnu-release:12',
        configuration: Dict[str, Dict[str, str]] = None,
        secrets: Dict[str, Dict[str, str]] = None) -> Dict[str, Any]:
    pass

Arguments:

Usage:

{
  "name": "add-latency",
  "type": "action",
  "provider": {
    "type": "python",
    "module": "chaosaws.awslambda.actions",
    "func": "add_latency",
    "arguments": {
      "function_name": ""
    }
  }
}

name: add-latency
provider:
  arguments:
    function_name: ''
  func: add_latency
  module: chaosaws.awslambda.actions
  type: python
type: action

add_layer¶

Add a new layer

:param function_name: Function name :param layer: The layer name :param configuration: AWS configuration data :param secrets: AWS secrets :return: AWSResponse

Signature:

def add_layer(function_name: str,
              layer: str,
              configuration: Dict[str, Dict[str, str]] = None,
              secrets: Dict[str, Dict[str, str]] = None) -> Dict[str, Any]:
    pass

Arguments:

Usage:

{
  "name": "add-layer",
  "type": "action",
  "provider": {
    "type": "python",
    "module": "chaosaws.awslambda.actions",
    "func": "add_layer",
    "arguments": {
      "function_name": "",
      "layer": ""
    }
  }
}

name: add-layer
provider:
  arguments:
    function_name: ''
    layer: ''
  func: add_layer
  module: chaosaws.awslambda.actions
  type: python
type: action

delete_event_source_mapping¶

Delete an event source mapping

:param event_uuid: The identifier of the event source mapping :param configuration: AWS configuration data :param secrets: AWS secrets :return: AWSResponse

Signature:

def delete_event_source_mapping(
        event_uuid: str,
        configuration: Dict[str, Dict[str, str]] = None,
        secrets: Dict[str, Dict[str, str]] = None) -> Dict[str, Any]:
    pass

Arguments:

Usage:

{
  "name": "delete-event-source-mapping",
  "type": "action",
  "provider": {
    "type": "python",
    "module": "chaosaws.awslambda.actions",
    "func": "delete_event_source_mapping",
    "arguments": {
      "event_uuid": ""
    }
  }
}

name: delete-event-source-mapping
provider:
  arguments:
    event_uuid: ''
  func: delete_event_source_mapping
  module: chaosaws.awslambda.actions
  type: python
type: action

delete_function_concurrency¶

Removes concurrency limit applied to the specified Lambda

Signature:

def delete_function_concurrency(
        function_name: str,
        configuration: Dict[str, Dict[str, str]] = None,
        secrets: Dict[str, Dict[str, str]] = None) -> Dict[str, Any]:
    pass

Arguments:

Usage:

{
  "name": "delete-function-concurrency",
  "type": "action",
  "provider": {
    "type": "python",
    "module": "chaosaws.awslambda.actions",
    "func": "delete_function_concurrency",
    "arguments": {
      "function_name": ""
    }
  }
}

name: delete-function-concurrency
provider:
  arguments:
    function_name: ''
  func: delete_function_concurrency
  module: chaosaws.awslambda.actions
  type: python
type: action

get_function_concurrency¶

Get configuration information of lambda by its function name

Signature:

def get_function_concurrency(
        function_name: str,
        configuration: Dict[str, Dict[str, str]] = None,
        secrets: Dict[str, Dict[str, str]] = None) -> bool:
    pass

Arguments:

Usage:

{
  "name": "get-function-concurrency",
  "type": "probe",
  "provider": {
    "type": "python",
    "module": "chaosaws.awslambda.probes",
    "func": "get_function_concurrency",
    "arguments": {
      "function_name": ""
    }
  }
}

name: get-function-concurrency
provider:
  arguments:
    function_name: ''
  func: get_function_concurrency
  module: chaosaws.awslambda.probes
  type: python
type: probe

get_function_memory_size¶

Get the configured memory size of a lambda function.

The returned memory size is specified in megabytes.

Signature:

def get_function_memory_size(function_name: str,
                             qualifier: str = None,
                             configuration: Dict[str, Dict[str, str]] = None,
                             secrets: Dict[str, Dict[str, str]] = None) -> int:
    pass

Arguments:

Usage:

{
  "name": "get-function-memory-size",
  "type": "probe",
  "provider": {
    "type": "python",
    "module": "chaosaws.awslambda.probes",
    "func": "get_function_memory_size",
    "arguments": {
      "function_name": ""
    }
  }
}

name: get-function-memory-size
provider:
  arguments:
    function_name: ''
  func: get_function_memory_size
  module: chaosaws.awslambda.probes
  type: python
type: probe

get_function_timeout¶

Get the configured timeout of a lambda function.

The returned timeout is specified in number of seconds.

Signature:

def get_function_timeout(function_name: str,
                         qualifier: str = None,
                         configuration: Dict[str, Dict[str, str]] = None,
                         secrets: Dict[str, Dict[str, str]] = None) -> int:
    pass

Arguments:

Usage:

{
  "name": "get-function-timeout",
  "type": "probe",
  "provider": {
    "type": "python",
    "module": "chaosaws.awslambda.probes",
    "func": "get_function_timeout",
    "arguments": {
      "function_name": ""
    }
  }
}

name: get-function-timeout
provider:
  arguments:
    function_name: ''
  func: get_function_timeout
  module: chaosaws.awslambda.probes
  type: python
type: probe

invoke_function¶

Invokes Lambda.

More information about request arguments are available in the documentation https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/lambda.html#Lambda.Client.invoke

Signature:

def invoke_function(
        function_name: str,
        function_arguments: Dict[str, Any] = None,
        invocation_type: str = 'RequestResponse',
        client_context: Dict[str, Any] = None,
        qualifier: str = None,
        configuration: Dict[str, Dict[str, str]] = None,
        secrets: Dict[str, Dict[str, str]] = None) -> Dict[str, Any]:
    pass

Arguments:

Usage:

{
  "name": "invoke-function",
  "type": "action",
  "provider": {
    "type": "python",
    "module": "chaosaws.awslambda.actions",
    "func": "invoke_function",
    "arguments": {
      "function_name": ""
    }
  }
}

name: invoke-function
provider:
  arguments:
    function_name: ''
  func: invoke_function
  module: chaosaws.awslambda.actions
  type: python
type: action

list_event_source_mapping¶

List event source mappings for the provided lambda function or ARN of the event source

:param source_arn: The ARN of the event source :param function_name: The name of the lambda function :param configuration: AWS configuration data :param secrets: AWS secrets :return: AWSResponse

Signature:

def list_event_source_mapping(
        source_arn: str = None,
        function_name: str = None,
        configuration: Dict[str, Dict[str, str]] = None,
        secrets: Dict[str, Dict[str, str]] = None) -> Dict[str, Any]:
    pass

Arguments:

Usage:

{
  "name": "list-event-source-mapping",
  "type": "probe",
  "provider": {
    "type": "python",
    "module": "chaosaws.awslambda.probes",
    "func": "list_event_source_mapping"
  }
}

name: list-event-source-mapping
provider:
  func: list_event_source_mapping
  module: chaosaws.awslambda.probes
  type: python
type: probe

put_function_concurrency¶

Throttles Lambda by setting reserved concurrency amount.

Signature:

def put_function_concurrency(
        function_name: str,
        concurrent_executions: int,
        configuration: Dict[str, Dict[str, str]] = None,
        secrets: Dict[str, Dict[str, str]] = None) -> Dict[str, Any]:
    pass

Arguments:

Usage:

{
  "name": "put-function-concurrency",
  "type": "action",
  "provider": {
    "type": "python",
    "module": "chaosaws.awslambda.actions",
    "func": "put_function_concurrency",
    "arguments": {
      "function_name": "",
      "concurrent_executions": 0
    }
  }
}

name: put-function-concurrency
provider:
  arguments:
    concurrent_executions: 0
    function_name: ''
  func: put_function_concurrency
  module: chaosaws.awslambda.actions
  type: python
type: action

put_function_memory_size¶

Sets the function memory size.

Input memory_size argument is specified in megabytes.

Signature:

def put_function_memory_size(
        function_name: str,
        memory_size: int,
        configuration: Dict[str, Dict[str, str]] = None,
        secrets: Dict[str, Dict[str, str]] = None) -> Dict[str, Any]:
    pass

Arguments:

Usage:

{
  "name": "put-function-memory-size",
  "type": "action",
  "provider": {
    "type": "python",
    "module": "chaosaws.awslambda.actions",
    "func": "put_function_memory_size",
    "arguments": {
      "function_name": "",
      "memory_size": 0
    }
  }
}

name: put-function-memory-size
provider:
  arguments:
    function_name: ''
    memory_size: 0
  func: put_function_memory_size
  module: chaosaws.awslambda.actions
  type: python
type: action

put_function_timeout¶

Sets the function timeout.

Input timeout argument is specified in seconds.

Signature:

def put_function_timeout(
        function_name: str,
        timeout: int,
        configuration: Dict[str, Dict[str, str]] = None,
        secrets: Dict[str, Dict[str, str]] = None) -> Dict[str, Any]:
    pass

Arguments:

Usage:

{
  "name": "put-function-timeout",
  "type": "action",
  "provider": {
    "type": "python",
    "module": "chaosaws.awslambda.actions",
    "func": "put_function_timeout",
    "arguments": {
      "function_name": "",
      "timeout": 0
    }
  }
}

name: put-function-timeout
provider:
  arguments:
    function_name: ''
    timeout: 0
  func: put_function_timeout
  module: chaosaws.awslambda.actions
  type: python
type: action

remove_error¶

Remove error and resources introduced with the add_error action.

Signature:

def remove_error(
        function_name: str,
        layer:
    str = 'arn:aws:lambda:us-east-1:871265522301:layer:chaos-lambda-extension-x86_64-unknown-linux-gnu-release:12',
        configuration: Dict[str, Dict[str, str]] = None,
        secrets: Dict[str, Dict[str, str]] = None) -> Dict[str, Any]:
    pass

Arguments:

Usage:

{
  "name": "remove-error",
  "type": "action",
  "provider": {
    "type": "python",
    "module": "chaosaws.awslambda.actions",
    "func": "remove_error",
    "arguments": {
      "function_name": ""
    }
  }
}