React to log events: when Papertrail receives new log events that match one of your saved searches, it can notify services like Campfire, Librato Metrics, or a HTTP URL that you provide (web hooks).

Sample app

Here's the source to a working example web hook handler on GitHub.

This tiny Sinatra app accepts a web hook request from Papertrail containing log matches. It then submits them to a remote Web service.

The papertrail-services handlers may also be useful.

Here's a tiny client which POSTs example log data on GitHub.

Web hooks operated by Papertrail

Papertrail operates a set of web hooks called papertrail-services for common services like Campfire and email. These are available in Papertrail's Web interface. From the Dashboard, click the "Edit" link for an existing saved search, then click the "Manage Alerts" tab:

Papertrail hosts these alert processors.

They are also available as a standalone open-source Sinatra Web service, which you can run yourself, fork and modify, and even contribute new services which you want Papertrail to run on your behalf.

The papertrail-services README has more.

You can also write your own standalone web hook handler without using papertrail-services. The rest of this page covers how to do that.

Example use

You have a monitoring system running in your datacenter or on another hosted service. New log events with the string segfault are an important failure case, and they should be sent to the monitoring system so it can page administrators.

Papertrail will notify your monitoring system every minute that new matches are found, and will also provide the contents of the new log matches.

Papertrail can notify every hour or day for less-urgent message types, as for generating summaries.

Setup

1. In Events, search for the relevant query (in the example use above, "segfault")
   
2. Click "Save Search" and give the search a name.
   
3. From the Dashboard, edit the search. Define your webhook URL and callback frequency (minute, hour, or day)

Callback

The callback will be a POST request in the format used by the github-services suite of webhook handlers. The POST body will contain a single parameter called payload, which will contain a JSON hash.

The important key in this hash is:

• events: an array of hashes of log events (one hash per event)

Example

For example, here is a POST body containing 2 events:

payload={
  "events":[
    {"hostname":"abc","received_at":"2011-05-18T20:30:02-07:00","severity":"Info","facility":"Cron","source_id":2,"message":"message body","program":"CROND","source_ip":"208.75.57.121","display_received_at":"May 18 20:30:02","id":7711561783320576,"source_name":"abc"},
    {"hostname":"def","received_at":"2011-05-18T20:30:02-07:00","severity":"Info","facility":"Cron","source_id":19,"message":"A short event","program":"CROND","source_ip":"208.75.57.120","display_received_at":"May 18 20:30:02","id":7711562567655424,"source_name":"server1"}
  ],
  "saved_search":{
    "id":42,
    "name":"Important stuff",
    "query":"cron OR server1",
    "html_edit_url":"https://papertrailapp.com/searches/42/edit",
    "html_seach_url":"https://papertrailapp.com/searches/42"
  },
  "max_id":7711582041804800,
  "min_id":7711561783320576
}

Note: For readability, the example above is not URL-encoded. See Encoding below.

The following hash keys are defined for each log event:

• id
• received_at
• generated_at
• source_name
• source_ip
• facility
• severity
• program
• message

HTTP and HTTPS URLs are supported.

Encoding

The payload JSON hash is encoded with application/x-www-form-urlencoded, as is standard for all POST requests. Apps rarely need to consider this because almost all Web frameworks automatically URL-decode the POST contents. In those cases, the app only needs to decode JSON.

Here's a sample test client to simulate a request from Papertrail.

Here's a very minimal example of how the request is generated. This uses the Ruby irb console and the payload event only contains a hostname, but it shows the format of the URL-encoded POST:

>> payload = { :events => [ { :hostname => 'abc' } ] }
=> {:events=>[{:hostname=>"abc"}]}

>> puts payload.to_json
{"events":[{"hostname":"abc"}]}

>> puts URI.escape(payload.to_json)
%7B%22events%22:[%7B%22hostname%22:%22abc%22%7D]%7D

Parsing

In Ruby with the Ruby on Rails framework, this will turn the callback POST params into an Ruby hash called payload:
payload = Yajl::Parser.parse(params[:payload])

Notes

Up to 25,000 events are included in a single callback. That was chosen as a compromise based on:

• the practical payload of a single HTTP POST over the Internet (especially one that could happen every minute)
• the number of events that a typical receiving app might be equipped to handle in an unexpected case
• trying to support apps that do things we didn't expect or plan for

The attribute reached_record_limit is set in the top-level hash when the 25,000 limit is reached. If you need more, just ask and we'll try to make it work.

Papertrail uses 64-bit event IDs, which JSON does not support, so the id column is a string. The other columns are consistently set to the type you would expect. No columns should be null except for program (when none is defined by the message). An empty message body is a blank string ('').