# Amazon Web Services (AWS)

:::warning
**Support Notice**

AWS Cloudtrail logs are ingested via S3 buckets. While it is possible to ingest logs after setting up only an S3 bucket, this is highly inefficient and can lead to performance issues. The reason is that the S3 API does not list "last modified", forcing us to download the entire list of files stored in a bucket. Instead, S3 will "publish" an "object created" event to an SQS queue. CyberSift filebeat is then setup to read these events from that queue and download the event instead of "polling" the S3 Bucket ARN and listing all files.  
[[AWS Documentation](https://docs.aws.amazon.com/AmazonS3/latest/userguide/ways-to-add-notification-config-to-bucket.html)] 

:::

# Requirements

| Step | Instruction | Notes |
|------|-------------|-------|
| 1    | Log in to the AWS Management Console | Navigate to the [AWS Management Console](https://aws.amazon.com/console/) and sign in to your AWS account. |
| 2    | **Open the SQS Service** | **Open the SQS Service**: Click on “**Services**” in the top left corner and then select “**SQS**” under the “**Application Integration**” section. Alternatively, you can type “SQS” in the search bar and select it. |
| 3    | **Create a New Queue** | In the Amazon SQS dashboard, click the “Create Queue” button. |

## Configure the SQS Queue

You’ll need to provide some information to configure the queue

| Field | Instruction |
|-------|-------------|
| **Queue Name** | Enter a name for your queue. The name must be unique within your AWS account and follow the naming rules (letters, numbers, hyphens, and underscores allowed) |
| **Standard Queue** | Make sure the “Standard Queue” option is selected. This is the default queue type. |
| **Other Settings** | You can configure additional settings like visibility timeout, message retention period, and other advanced settings as needed. These settings determine how messages are handled in the queue.<br> ![](https://outline.cybersift.io/api/files.get?sig=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJrZXkiOiJ1cGxvYWRzL2MwNWNhZDYzLTA0ZmEtNGFmZC04NjMwLTI3MjNjZDYwZWZkNS84NTNkMDU1YS0xMmYzLTQzOGEtYTgxMi0wNmFiMGU4OGI5YWMvaW1hZ2UucG5nIiwidHlwZSI6ImF0dGFjaG1lbnQiLCJpYXQiOjE3OTUxNDcyMDAsImV4cCI6MTc5NTIzMzYwMH0.Jta5QdsodoOQhicZsxWExImQPx4EAGiHkzHFpWGyo68 " =700x274") |
| **Create the Queue** | Once you’ve configured the queue settings, click the *“Create Queue”* button at the bottom of the page. |

## Configure S3 bucket

> Amazon S3 provides a feature called “S3 events” that allows you to trigger actions or notifications based on events that occur within your S3 buckets. The SQS Queue must be in the same region as the AWS S3 bucket. These events can include object creation, deletion, or modification, and they can be used to automate various tasks.

For the purposes of the CyberSift SIEM, only “object creation” events are of interest to us


1. **Create Bucket**: Create a S3 bucket if it already does not exist.
2. **Set Permissions**: Replace the access policy that's attached to the queue. In it, provide your Amazon SQS ARN, source bucket name:

   ```json
   {
       "Version": "2012-10-17",
       "Id": "example-ID",  
       "Statement": [
           {
               "Sid": "example-statement-ID",
               "Effect": "Allow",
               "Principal": {
                   "Service": "s3.amazonaws.com"
               },
               "Action": [
                   "SQS:SendMessage"
               ],
               "Resource": "SQS-queue-ARN", // <---- change this to your newly created SQS ARN
               "Condition": {
                   "ArnLike": {
                       "aws:SourceArn": "arn:aws:s3:*:*:awsexamplebucket1" // <-- change this to your S3 ARN
                   }
               }
           },
           {
               "Sid": "allow-receive-message",
               "Effect": "Allow",
               "Principal": {
                   "AWS": "arn:aws:iam::account-id:user/example_user" // <---- change this to your newly created SQS ARN
               },
               "Action": "SQS:ReceiveMessage",
               "Resource": "SQS-queue-ARN" // <---- change this to your newly created SQS ARN
           }
       ]
   }
   ```
3. **Define Event Notifications:** Navigate to the bucket properties and configure event notifications. Select “**Object Creation**”.
4. **Select the Destination**: Choose the destination for these events. In our case, we want to send events to an SQS queue.

   ![](https://outline.cybersift.io/api/files.get?sig=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJrZXkiOiJ1cGxvYWRzL2MwNWNhZDYzLTA0ZmEtNGFmZC04NjMwLTI3MjNjZDYwZWZkNS9mMmM4ZWUwNi1iZjU4LTQ5NmItODVjNy1kNzg3MThkYmE2ZmEvaW1hZ2UucG5nIiwidHlwZSI6ImF0dGFjaG1lbnQiLCJpYXQiOjE3OTUxNDcyMDAsImV4cCI6MTc5NTIzMzYwMH0._9yqPyL_0nbD4AIwRadnMZekrEP6hSCLsA8kMg5Bk68 " =700x327")

   
5. **Test the Configuration:** To verify your setup, you can upload an object to the S3 bucket and check if it triggers a message in the SQS queue.


---

# NiFi Provisioning 


:::warning
**Cybersift Support Required**

At this point in time, Cybersift will do the NiFi provisioning for you.  
  
Please provide the below details to support@cybersift.io


* The SQS ARN in URL format:

   `https://sqs.{REGION_ENDPOINT}.amazonaws.com/{ACCOUNT_NUMBER}/{QUEUE_NAME}`
* Credentials for the AWS user:

  **==Please provide both the below details using a service such as==** [**==https://onetimesecret.com/en/==**](https://onetimesecret.com/en/)
  * Access Key ID
  * Secret Access Key

:::


---


:::info
tags:

* Amazon Web Services (AWS) Logs Setup
* NiFi AWS Setup
* S3 Bucket
* Cloudtrail 

:::

---

**Documents**

- [Logs and Architecture](https://outline.cybersift.io/s/sentio-siem/doc/logs-and-architecture-acvUiDgvI5)
- [Sentio](https://outline.cybersift.io/s/sentio-siem/doc/sentio-KuYAbF5grr)
- [ISO27001:2022](https://outline.cybersift.io/s/sentio-siem/doc/iso270012022-7S5IzvF6a0)
- [FAQ](https://outline.cybersift.io/s/sentio-siem/doc/faq-vVyRNyNb60)