Amazon EC2 is Amazon.com’s cloud computing platform that allows users to rent virtual machines (instances), and host their applications on either Linux or Windows. Here’s how to aggregate app and system logs from these instances.
Configuring logging on a single EC2 instance is exactly the same as logging from any other virtual or dedicated computing instance. For more information on logging from Windows, Linux, or a specific framework, see the appropriate guide under the configuration section.
For environments where many instances are regularly provisioned and de-provisioned, it’s recommended to group systems in Papertrail to keep them organised. For example, you might have a group for database servers, storage, etc. Here are a few methods that will allow you do that.
Read lots more on Mapping senders to groups.
Papertrail can automatically add systems to a group based on what destination they log to. On the destinations configuration screen, separate destinations can be created for different environments. For example, one for for Production and one for Staging, or for different products, or apps, or anything else.
During the EC2 bootstrap process, systems can join themselves to one or more groups by invoking papertrail-join-group.
For the greatest flexibility, a Chef or Puppet deployment automation script can fully control which systems should be part of which groups via Papertrail’s REST API
Amazon EC2 Virtual Private Cloud configurations use a VPN to reach your datacenter. VPC security policies include a quite restrictive firewall policy. If using VPC, ensure that Papertrail is reachable from your VPC hosts.
While your network architecture will differ, here are two screenshots of an example inbound firewall policy that permits UDP logs inbound to a VPC jump/bastion host, which relays the traffic to the Internet. The port indicated by the orange arrow should match a port shown on Log Destinations. A corresponding outbound rule should also be added.
If traffic is still being blocked after security groups are properly configured, there may be a restrictive network ACL applied to the entire VPC. Find these from the Network ACLs section of the Amazon VPC Console.