AWS Fargate

Deployed as a sidecar container using a task definition

In AWS Fargate, the ThreatStryker agents are deployed as a sidecar container using a task definition.

The ThreatStryker management console is installed separately outside the fargate and the installation procedure is the same as before.

note

Currently supported base operating systems of containers are Amazon Linux, Ubuntu, Debian, CentOS and RHEL

note

Please note the agent image "quay.io/deepfenceio/deepfence_agent:2.3.1-fargate" is different from other deployment methods.

Installing on AWS Fargate

1. Set up AWS ECS by following the steps outlined here: Set up to use AWS ECS

2. Refer Prerequisites for the actions performed in this step. Add the Deepfence Quay secrets provided to AWS secrets manager by following the steps outlined here: Introducing private registry authentication support for AWS Fargate and here: Private registry authentication for tasks

   You'll need to perform the following steps:

   1. Create an AWS ECS task execution IAM role.
   2. Create a secret so that fargate can pull the Deepfence agents from Quay using the username/password provided in the license email. Note the created secret name or ARN (you will need to enter them in task definition creation step later). You may need to create other secrets to access your own artifacts.
   3. Create policies (either managed or inlined policy) allowing access to your stored secrets and attach the created policies to the task IAM role. You also need to attach the AmazonECSTaskExecutionRolePolicy to the IAM role to run AWS ECS tasks.

3. Click on the task definition on the side panel to create a new task definition. Select "AWS Fargate" as launch type

   Use the following steps outlined below in "Fargate Task definition And Deployment" instructions to deploy the fargate agent.

   You can configure the task definition either through JSON or using the AWS UI.

4. Deploy your application on your cluster.

Create New Task Definition in Fargate

Create Task Definition

Click Create new Task Definition and select "AWS Fargate" as launch type.

New Fargate Task

Set Task Parameters

Edit the Task Definition Name, Task Role and Task Execution Role etc. as required. For the Task Role and Task Execution Role, you have to use the role created in IAM role creation step earlier. Specify Task memory and Task CPU according to your Requirements.

Update task definition and create agent container

Add the Deepfence Agent Sidecar Container

Click on the Add Container button to create a standard container for the ThreatStryker agent. Set image as quay.io/deepfenceio/deepfence_agent:2.3.1-fargate

Check the private repository authentication and add the secret name or ARN from IAM role creation step to access Deepfence Quay. In the environment section, DO NOT mark it as essential.

You need to note down the name of the agent container (deepfence-agent in our example), which you will have to specify in Volumes From section in application container task definition section later.

Finally, click the Add button to create the deepfence agent container:

Create the Sidecar Agent Container inside the Task Definition

Add the Main Container to your Application

Click on the Add Container button to create a new container for your application by following the additional steps outlined below. If you have more than one application container, you will have to repeat these steps for each container.

Configure Environment Variables for Fargate Application Container

The following environment variables are required for the ThreatStryker agent:

• DEEPFENCE_KEY: API key available in the management console UI(can be stored as a secret and later referred in environment using valuesFrom)
• MGMT_CONSOLE_URL: IP address of Management Console
• DF_SERVERLESS: Set to true for serverless instances
• MGMT_CONSOLE_URL_SCHEMA: Set to http or https depending on the schema used for management console
• MGMT_CONSOLE_PORT: Set to 80 or 443 depending on the port used for management console

Configuring Environment Variables for Fargate Application Container

If you are using json to configure your task definitions, you can use the following part in the appropriate container section of task definition json after copying the appropriate IP address and API Key.

"environment": [
    {
        "name": "DEEPFENCE_KEY",
        "value": "<deepfence-key>"
    },
    {
        "name": "MGMT_CONSOLE_URL",
        "value": "<MGMT_CONSOLE_URL>"
    },
    {
        "name": "DF_SERVERLESS",
        "value": "true"
    },
    {
        "name": "DF_DISABLE_FILE_MON",
        "value": ""
    },
    {
        "name": "DF_DISABLE_PROC_MON",
        "value": ""
    },
    {
        "name": "DF_DISABLE_TRAFFIC_ANALYSIS",
        "value": ""
    },
    {
        "name": "DF_DISABLE_LOCAL_TRAFFIC_FILTER",
        "value": ""
    },
    {
        "name": "DF_DISABLE_REVERSE_DNS",
        "value": "Y"
    },
    {
        "name": "DF_LOG_LEVEL",
        "value": "info"
    },
    {
        "name": "USER_DEFINED_TAGS",
        "value": ""
    },
    {
        "name": "DF_INSTALL_DIR",
        "value": "/deepfence"
    },
    {
        "name": "MGMT_CONSOLE_URL_SCHEMA",
        "value": "https"
    },
    {
        "name": "MGMT_CONSOLE_PORT",
        "value": "443"
    }
]

Configure Storage

Scroll down to Storage Section and click Add Volume from. In the Container dropdown select your application container and in Source container dropdown select the agent container to allow read/write from deepfence agent volume. Leave the Read only button unchecked as shown below.

Configure VolumesFrom Setting

If you are using json to configure your task definitions, you can copy the following settings to the appropriate container section of the json after changing the Container name:

"volumesFrom": [
   {
       "sourceContainer": "deepfence-agent",
       "readOnly": false
   }
],

Finally, click the Create button to create the task definition for the deployment.

Configure Correct Startup

Now that deepfence agent is available in the fargate instance, you need to invoke agent and application entrypoints to start the application with Deepfence enabled. This can be done in two ways:

Edit the Entry Point for the container

There are two ways to achieve this:

Change the Entrypoint: For this, you need to provide the ThreatStryker entrypoint and the Application entrypoint and arguments, as a comma delimited list in the Entry point field:

Method (1a): Invoking agent by changing the Entrypoint

If you are using json to configure your task definitions, then you can specify the entrypoint and/or command as follows using appropriate quoting:

"entryPoint": [
   "/deepfence/usr/local/bin/deepfence-entry-point-scratch.sh",
   "customer-entry-point.sh",
   "param1",
   "param2"
]

Change the Entrypoint and Command: Alternatively, you can provide the ThreatStryker entrypoint in the Entry point field and the Application entrypoint and arguments in the Command field as shown below:

Method (1b): Invoking agent by changing the Entrypoint and Command field

If you are using json to configure your task definitions, then you can specify the entrypoint and/or command as follows using appropriate quoting:

"entryPoint": [
   "/deepfence/usr/local/bin/deepfence-entry-point-scratch.sh"
],
"command": [
   "customer-entry-point.sh",
   "param1",
   "param2"
]

Prerequisites

Make sure you have the following information:

• Quay login, later referred as <QUAY_LOGIN>
• Quay password, later referred as <QUAY_PASSWORD>
• Management console URL/IP, later referred as <MGMT_CONSOLE_URL>
• Deepfence API key, later referred as <DEEPFENCE_KEY> (This key can be found from the management console, in the settings > User > API Key)

1. Add secret for quay.io registry login
   • Go to the secret manager dashboard from the AWS Console
   • Select "Store a new secret"
   • Select "Other type of secret"
   • Select "Plaintext" and paste the following:

     {
       "username": "<QUAY_LOGIN>",
       "password": "<QUAY_PASSWORD>"
     }

Create the secret and store the ARN. We will refer to it as <ARN_QUAY_CREDS>

2. Add secret for Deepfence API key
   • Go to the secret manager dashboard from the AWS Console
   • Select "Store a new secret"
   • Select "Other type of secret"
   • Select "Plaintext" and paste the following:

     {
       "deepfence_api_key": "<DEEPFENCE_KEY>"
     }

Create the secret and store the ARN. We will refer to it as <API_KEY_SECRET_ARN>

caution

Be careful with the double quotes, sometimes the AWS UI transforms them into a special character that is not recognized as valid JSON.

3. Create a new role (e.g.: deepfence-agent-role)
   • Go to the IAM dashboard from AWS Console
   • Go to Access management > roles
   • Select "Create Role",
   • Select "Custom trust policy"
   • Paste the following:

     {
       "Version": "2012-10-17",
       "Statement": [
         {
           "Effect": "Allow",
           "Principal": {
             "Service": "ecs-tasks.amazonaws.com"
           },
           "Action": "sts:AssumeRole"
         }
       ]
     }

Then continue:

• Search in the "Permissions policies" for "Task" > Select the following policy: AmazonECSTaskExecutionRolePolicy

• Again search in the "Permissions policies" for "Task" > Select the following policy: CloudWatchLogsFullAccess

• Click "Next", name the role deepfence-agent-role, then "Create role"

• Store the Role ARN. We will refer to it as <AGENT_TASK_ROLE_ARN>

• Search for your newly created role

• Click on it (deepfence-agent-role in our example)

• Select "Add permissions" > "Create inline policy" and add:

  {
      "Version": "2012-10-17",
      "Statement": [
          {
              "Effect": "Allow",
              "Action": [
                  "secretsmanager:GetSecretValue"
              ],
              "Resource": [
                  "<ARN_QUAY_CREDS>",
                  "<API_KEY_SECRET_ARN>"
              ]
          }
      ]
  }

• If you are using a custom KMS key for your secrets and not using the default key, you will also need to add the KMS key permissions to your inline policy:

  {
      "Version": "2012-10-17",
      "Statement": [
          {
              "Effect": "Allow",
              "Action": [
                  "kms:Decrypt",
                  "secretsmanager:GetSecretValue"
              ],
              "Resource": [
                  "<ARN_QUAY_CREDS>",
                  "<API_KEY_SECRET_ARN>",
                  "<custom_kms_key_arn>"
              ]
          }
      ]
  }

Then create the new policy.

Sample fargate task definition json with deepfence-agent sidecar

{
  "requiresCompatibilities": ["FARGATE"],
  "inferenceAccelerators": [],
  "containerDefinitions": [
    {
      "name": "python",
      "image": "python:latest",
      "cpu": 0,
      "portMappings": [
        {
          "name": "python-8000-tcp",
          "containerPort": 8000,
          "hostPort": 8000,
          "protocol": "tcp",
          "appProtocol": "http"
        }
      ],
      "essential": true,
      "entryPoint": [
        "/deepfence/usr/local/bin/deepfence-entry-point-scratch.sh"
      ],
      "command": ["python3", "-m", "http.server"],
      "environment": [
        {
          "name": "MGMT_CONSOLE_URL",
          "value": "<MGMT_CONSOLE_URL>"
        },
        {
          "name": "DF_SERVERLESS",
          "value": "true"
        },
        {
          "name": "DF_DISABLE_FILE_MON",
          "value": ""
        },
        {
          "name": "DF_DISABLE_PROC_MON",
          "value": ""
        },
        {
          "name": "DF_DISABLE_TRAFFIC_ANALYSIS",
          "value": ""
        },
        {
          "name": "DF_DISABLE_LOCAL_TRAFFIC_FILTER",
          "value": ""
        },
        {
          "name": "DF_DISABLE_REVERSE_DNS",
          "value": "Y"
        },
        {
          "name": "DF_LOG_LEVEL",
          "value": "info"
        },
        {
          "name": "USER_DEFINED_TAGS",
          "value": ""
        },
        {
          "name": "DF_INSTALL_DIR",
          "value": "/deepfence"
        },
        {
          "name": "MGMT_CONSOLE_URL_SCHEMA",
          "value": "https"
        },
        {
          "name": "MGMT_CONSOLE_PORT",
          "value": "443"
        }
      ],
      "mountPoints": [],
      "volumesFrom": [
        {
          "sourceContainer": "deepfence-agent",
          "readOnly": false
        }
      ],
      "secrets": [
        {
          "name": "DEEPFENCE_KEY",
          "valueFrom": "<API_KEY_SECRET_ARN>:deepfence_api_key::"
        }
      ],
      "dependsOn": [
        {
          "containerName": "deepfence-agent",
          "condition": "COMPLETE"
        }
      ],
      "logConfiguration": {
        "logDriver": "awslogs",
        "options": {
          "awslogs-create-group": "true",
          "awslogs-group": "/ecs/test-doc-python",
          "awslogs-region": "us-west-2",
          "awslogs-stream-prefix": "ecs",
          "mode": "non-blocking",
          "max-buffer-size": "25m"
        }
      }
    },
    {
      "name": "deepfence-agent",
      "image": "quay.io/deepfenceio/deepfence_agent:2.3.1-fargate",
      "repositoryCredentials": {
        "credentialsParameter": "<ARN_QUAY_CREDS>"
      },
      "cpu": 0,
      "portMappings": [],
      "essential": false,
      "environment": [],
      "mountPoints": [],
      "volumesFrom": [],
      "logConfiguration": {
        "logDriver": "awslogs",
        "options": {
          "awslogs-create-group": "true",
          "awslogs-group": "/ecs/test-doc-python",
          "awslogs-region": "us-west-2",
          "awslogs-stream-prefix": "ecs",
          "mode": "non-blocking",
          "max-buffer-size": "25m"
        }
      }
    }
  ],
  "volumes": [],
  "networkMode": "awsvpc",
  "memory": "4096",
  "cpu": "2048",
  "family": "test-doc-python",
  "executionRoleArn": "<AGENT_TASK_ROLE_ARN>",
  "taskRoleArn": "<AGENT_TASK_ROLE_ARN>",
  "runtimePlatform": {
    "cpuArchitecture": "X86_64",
    "operatingSystemFamily": "LINUX"
  },
  "tags": [],
  "placementConstraints": []
}