Skip to main content

Setting Up Amazon S3

Setting Up Amazon S3

Daniel
  • Updated

There are so many sources of data that don't provide direct, software API connections. You are probably using at least one. If you've got data but are in need of a way to get it to NinjaCat we can help. If you can get your data into an Amazon Web Services (AWS) S3 bucket we can target your S3 bucket and make that data available for Reports and Dashboards. This article is a basic overview of the steps to set up and use an AWS S3 Bucket.

 

Section Links

Access Requirements

Creating an IAM Policy and Granting Access

Connecting an AWS S3 Bucket Network

Creating an AWS S3 Mapping

Using AWS S3 Data in the Template Builder

 

Access Requirements

In order to use the NinjaCat AWS S3 integration, you’ll have to have access to Amazon AWS and permissions that allow you to:

  • Create an S3 bucket or identify an existing bucket containing files with supported file types and encodings
  • Note the region where the bucket is located (i.e. us-east-1)
  • For private or encrypted buckets, an AWS account with the ability to grant NinjaCat permission to read from the bucket.

 

Creating an IAM Policy and Granting Access

1. Log in to Amazon AWS

2. Click on "Services" near the top left1.png

3. Click on "All Services"2.png

4. Scroll down to and then click on "IAM"3.png

5. Click on "Policies"4.png

6. Click on "Create Policy"5.png

7. Click on the JSON tab6.png

8. Copy the following policy and paste it in the JSON tab, replacing {your-bucket-name} with the name of your S3 bucket:

{

    "Version": "2012-10-17",

    "Statement": [

        {

            "Effect": "Allow",

            "Action": [

                "s3:Get*",

                "s3:List*"

            ],

            "Resource": "arn:aws:s3:::{your-bucket-name}/*"

        },

        {

            "Effect": "Allow",

            "Action": [

                "s3:Get*",

                "s3:List*"

            ],

            "Resource": "arn:aws:s3:::{your-bucket-name}"

        }

    ]

}

9. Click "Next: Tags" near the bottom right7.png

10. Click "Next: Review" near the bottom right8.png

11. Click "Create Policy" near the bottom right9.png

 

 

Connecting an AWS S3 Bucket Network

1. Go to the Setup Settings

2. Follow the instructions in the article How To Add a Direct Network Connection, selecting "Amazon S3" as the desired network

 

Creating an AWS S3 Mapping

1. If not still there, Go to the Setup Settings

2. Select the Amazon AWS network created in the previous section10.png

3. Click on "More Actions" near the top right

4. Click on "Edit Mapping"11.png

5. Click on "Add Mapping" near the top right or center12.png

6. Fill in the fields with the appropriate info

NinjaNote: Amazon AWS refers to its access credentials as "keys" So, to be more specific the required access credentials may be referred to as "Access Key ID" and "Secret Access Key"

7. Click "Connect" near the bottom right13.png

After clicking connect, two new fields will be revealed at the bottom of the page

8. To fill the "Current Directory" field, click on the drop-down menu labeled "File System"

Select the directory/folder in your S3 Bucket that you want NinjaCat to find your data in

9. Use the "File System" field drop-down to select the CSV file you are going to use to define your columns in the next step of creating a mapping14.png

After clicking on the name of the desired CSV more fields will be revealed at the bottom of the page, AND new form tabs will be revealed at the top of the page

10. Fill in the newly revealed fields with the appropriate information15.png

NinjaNote: Do you see the "Save" button near the bottom left? DON'T CLICK IT YET! You must follow the instructions to create at least one Dimension and one Metric in this Mapping before clicking Save. That button is the final step in creating the Mapping NOT an incremental "save" step.

11. FILL IN THE NEEDED INFORMATION IN EACH OF THE REQUIRED TABS

FILTERING Tab15a.png

 

  • Date Column
  • Account Matching
  • Account Column
  • Download Time
  • Download Type - This selection defines the pattern of how your data file will be updated
    • By File Name - This setting expects that the single selected file will be updated with all new data
    • Latest File In Directory - This selection expects that every single subsequent file will contain new data and any updates to historic data
    • All Updated Files In Directory - This setting expects that multiple files in the directory will continue to be updated with new data and changes to historic data

DIMENSIONS Tab -- Repeat the instructions below for each dimension you need to pull from your sheet

     1. Click the "Add Dimensions" button near the center or top right16.png

 

     2. Fill in the needed fields in the "Dimension Editor" pop up window

  • Name - A title for the Dimension that will be visible in the Template Builder
  • Header Mapping - Select the actual column from your sheet that contains the right data for the Dimension
  • Aggregation Mapping - (If appropriate) The common Dimension set that this data could fall under if being combined with data from other data sources
  • Display As - The specific format that the data from the sheet should be represented as17.png

 

     3. Click the "OK" button near the bottom right of the pop-up window after all the fields are filled

 

METRICS Tab -- Repeat the instructions below for each dimension you need to pull from your sheet

     1. Click the "Add Metrics" button near the center or top right18.png

 

     2. Fill in the needed fields in the "Metrics Editor" pop up window

  • Name - A title for the Metric that will be visible in the Template Builder
  • Header Mapping - Select the actual column from your sheet that contains the right data for the Metric
  • Display As - The specific numeric format that the data from the sheet should be represented as
  • Calculate totals as... - For combining metric values choose Summation or Average
  • Aggregation Mapping - (If appropriate) The common Metric set that this data could fall under if being combined with data from other data sources19.png

 

     3. Click the "OK" button near the bottom right of the pop-up window after all the fields are filled

12. Click "Save" near the bottom right20.png

 

Using AWS S3 Data in the Template Builder

1. Access the NinjaCat Template Builder

2. Add a data widget to the template

3. Click on "Add Data Source" near the top right

12.png

 

4. Click on "Amazon S3" from the list of data sources1.png

 

5. Select the desired Mapping from the field labeled "Mapping - Select a Mapping" in the "Amazon S3" pop up window2.png

 

6. Click the "Close" button near the bottom right of the pop-up window3.png

NinjaNote: Once a mapping is selected in the pop-up window, two buttons will now be available below the "Mapping" field. The "View Mapping" button will give you a view of the Dimensions and Metrics available through the Mapping. The "Edit Mapping" button will take you to the Edit Mappings area and allow you to make changes to the Mapping itself. When done you will be returned to the NinjaCat Template builder.

 

Related articles

Comments

0 comments

Article is closed for comments.