Amazon EC2

Intro

Amazon EC2 is Amazon.com’s cloud computing platform that allows users to rent virtual machines (instances), and host their applications on either Linux or Windows. Here’s how to aggregate app and system logs from these instances.

Setup

Configuring logging on a single EC2 instance is exactly the same as logging from any other virtual or dedicated computing instance. For more information on logging from Windows, Linux, or a specific framework, see the appropriate guide under the configuration section.

Best Practices

For environments where many instances are regularly provisioned and de-provisioned, it’s recommended to group systems in Papertrail to keep them organised. For example, you might have a group for database servers, storage, etc. Here are a few methods that will allow you do that.

Read lots more on Mapping senders to groups.

Automatically add new systems to a group

Papertrail can automatically add systems to a group based on what destination they log to. On the destinations configuration screen, separate destinations can be created for different environments. For example, one for for Production and one for Staging, or for different products, or apps, or anything else.

Add new systems to one or more groups

During the EC2 bootstrap process, systems can join themselves to one or more groups by invoking papertrail-join-group.

Automated Deployments

For the greatest flexibility, a Chef or Puppet deployment automation script can fully control which systems should be part of which groups via Papertrail’s REST API

VPC

Amazon EC2 Virtual Private Cloud configurations use a VPN to reach your datacenter. VPC security policies include a quite restrictive firewall policy. If using VPC, ensure that Papertrail is reachable from your VPC hosts.

While your network architecture will differ, here are two screenshots of an example inbound firewall policy which permits UDP logs inbound to a VPC jump/bastion host, which relays the traffic to the Internet. The port indicated by the orange arrow should match a port shown on Log Destinations. A corresponding outbound rule should also be added.

VPC firewall policy

VPC firewall policy