Mapping senders to groups

Background

Here are different ways to map senders to groups (like “Production,” “Staging,” “us-west-1,” and “Web apps”) as well as how the methods can apply in various hosting environments.

Papertrail supports a few ways to map senders to groups, depending on how descriptive the senders’ names are, how many senders exist, how frequently they change, and how you prefer to manage configuration information.

Simpler environments

Fewer than 10 senders that rarely change

While we love automation, with a small number of senders that rarely change, it’s probably overkill. Consider spending 5 minutes to create groups and add the systems manually. Revisit or automate when more systems or dynamic provisioning is involved.

More complex environments

Senders with descriptive names

When senders’ names are descriptive enough to indicate group membership, like www32.prod, use Papertrail’s dynamic group membership. Create groups and provide name wildcard(s), like *stg*.

Senders without descriptive names

This boils down to a decision how you prefer to manage configuration information and how much control you want.

I prefer simple

Create a Papertrail log destination per group. On the same page, map each destination to a group (like “Production” and “Staging”). Have the senders log to the correct log destination.

I prefer full-featured

When new senders are provisioned, have them run the papertrail-join-group command-line app.

This works great with automated deployment processes like chef, puppet, and VM instance bootstrap scripts. It also means that senders can easily join multiple groups, and that group information is kept in the same place as your other configuration/deployment information.

Related: Amazon EC2

Let me control everything

Make calls to Papertrail’s HTTP API. While papertrail-cli wraps the API in command-line tools, you can invoke them directly. See systems, especially register and join.

Combine these

Many of the techniques mentioned here can be used together. For example, log destinations might map to groups, but the destinations could still be configured using an automated deployment system like puppet. There’s one other tool to mix in: choosing a more useful hostname.

While most loggers choose to use the system hostname as a log identifier, and Papertrail honors that, you can easily define a name that is different than the system hostname.

If your provisioning process knows more than the hostname, consider having the provisioning process generate log configs, or even system hostnames, with more informative names. Regardless of how you use Papertrail, often this is useful to do on its own.

How are senders named?

Log senders like rsyslog and remote_syslog2 typically set a sender identifier field in each syslog packet to the system hostname, though it can be set to other values (see Override the hostname sent by a logger).

Because Papertrail accepts inbound links which use the sender name, such as https://papertrailapp.com/systems/www42, the sender name must be unique. When Papertrail receives a log message from a new sender and:

Papertrail will append a hyphen and sequence number (-1) to the default sender name shown in Papertrail. For example: www42-1

This display name in Papertrail can still be edited, but it ensures that administrators do not confuse the new sender with an existing sender.

Removing A Sender

Depending on your Destination Settings, Papertrail may not automatically remove senders that have stopped logging and no longer have searchable events. If Automatically Remove Idle Senders (in Log Destination settings) is not checked, you’ll need to manually remove the sender.

To remove a sender:

Sender Settings Button

Delete This System

If Automatically Remove Idle Senders is checked, idle senders will be removed two days after their most recent log message is no longer searchable, or one week after they’ve stopped sending, whichever is longer.