Heroku is a Platform-as-a-Service for deploying and scaling applications. Here’s how to aggregate logs from a Heroku app. New to Papertrail or Heroku? This introduction explains how great log visibility helps.
Heroku apps can log to Papertrail using either of two methods. Both capture all Heroku logs, including router output, app/dyno output, platform errors, Heroku Postgres status, and any other log messages.
Papertrail has a Heroku add-on for one-click hosted log management. Activate it on elements.heroku.com.
To have more than one Heroku app log to Papertrail, determine whether the additional app(s) are managed by, accessed by, and paid for by the same team as the existing app.
For example, a small ops team which maintains
myapp-staging apps might prefer to have 1 Papertrail add-on subscription capture logs from both apps. On the other hand, a larger company (with app-specific access policies or payment methods) or a consultancy (managing apps for different clients) would want each Papertrail subscription to serve only 1 app.
If you don’t already have an account, sign up (free).
$ heroku drains:add syslog+tls://logsN.papertrailapp.com:XXXXX
XXXXX with the hostname and port from the page above. For example:
$ heroku drains:add syslog+tls://logs2.papertrailapp.com:12345 --app my-app-42
Wait 30 seconds for logging to start. Once it starts, messages will appear in Papertrail’s event viewer in realtime. Use
heroku drains:add to add the same drain to any additional apps.
Because Papertrail has no way to obtain the app’s name, its logs will be attributed to a Heroku-generated GUID like
d.1234-5678. Give the app a more meaningful name.
This command will list all drains attached to all Heroku apps, including the drain ID:
$ heroku apps | grep -v "=" | cut -d " " -f 1 | xargs -tL 1 heroku drains --app
On Papertrail’s Dashboard, click the “All Systems” group name link. Scroll down to the new drain ID and click “Settings.” Change the sender’s name from the drain ID to a more useful name (such as the Heroku app name). Repeat for each new app/drain.
Note: A Papertrail sender will only be created after the first log message is received from Heroku.
Papertrail provides Heroku-specific searches when using the add-on, but these aren’t created on standalone accounts. Copy and paste the following commands to add them. Replace
YOUR_TOKEN with your API token.
$ curl -G -v -H "X-Papertrail-Token: YOUR_TOKEN" -X POST https://papertrailapp.com/api/v1/searches.json --data-urlencode 'search[name]=Platform errors' --data-urlencode 'search[query]="error code=H" OR "Error R" OR "Error L"' $ curl -G -v -H "X-Papertrail-Token: YOUR_TOKEN" -X POST https://papertrailapp.com/api/v1/searches.json --data-urlencode 'search[name]=Deploys' --data-urlencode 'search[query]=program:(heroku/api heroku/slug) -scheduler' $ curl -G -v -H "X-Papertrail-Token: YOUR_TOKEN" -X POST https://papertrailapp.com/api/v1/searches.json --data-urlencode 'search[name]=Scheduler jobs' --data-urlencode 'search[query]=program:scheduler' $ curl -G -v -H "X-Papertrail-Token: YOUR_TOKEN" -X POST https://papertrailapp.com/api/v1/searches.json --data-urlencode 'search[name]=Dyno state changes' --data-urlencode 'search[query]=web (Idling OR Unidling OR Cycling OR "State changed" OR "Starting process")' $ curl -G -v -H "X-Papertrail-Token: YOUR_TOKEN" -X POST https://papertrailapp.com/api/v1/searches.json --data-urlencode 'search[name]=Web app output' --data-urlencode 'search[query]="app/web"'
If you have added unicorn to your Heroku app stack, Papertrail’s Unicorn logging explanation may be useful.
Logplex may log the following message:
heroku logplex Error L10 (output buffer overflow): 500 messages dropped since 2013-04-17T19:04:46+00:00.
L10 means that Logplex could not keep up with the volume of logs generated by the application and has been forced to discard some messages to a specific drain. Heroku’s error codes has more.
L10 doesn’t mean that Papertrail is responding slowly. It means that the sending service has overwhelmed logplex with an extremely high volume of messages, on the order of thousands of messages per second (from a single dyno or worker, not app-wide). That volume momentarily overwhelms logplex itself.
Note that the very high log volume may only exist for a moment, like because a dyno tried to log the contents of a POSTed file.
The root cause is almost always unintentionally logging a ton of data, like binary data from a form upload or unanticipated output from a background job. If you don’t see cases of this, contact us and we’ll help track it down.
Papertrail displays the Heroku app name as the sender name. With the Heroku add-on in method 1, this is the actual Heroku app name (such as
electric-fog-9847). With a standalone account in method 2, it is the system name value that you provide, which may be the app name or another string of your choosing.
In Papertrail’s webhooks API, this is passed as the
Papertrail displays the Heroku process name in the Papertrail program name. A few examples are:
heroku ps output for additional process names.
In Papertrail’s webhooks API, this is passed as the
Short lived dynos only log their output to the console so only startup and shutdown messages are displayed in Papertrail. For Rake tasks, the workaround is to run in detached mode using:
$ heroku run:detached rake
$ heroku run rake
--color switch to the app’s Procfile and set it to
always. For example:
web: node index.js --color=always
Heroku’s add-on system pro-rates add-on subscription charges, so an add-on subscription or plan choice may only be active for a few hours or a few days. For that reason, Papertrail’s Heroku add-on measures logs on a daily basis; measuring usage over a month-long period would not work.
In contrast, Papertrail’s standalone service measures logs on a monthly basis. Log data transfer above the plan limit may be purchased by enabling additional usage, which will provide an extra 200% of data on a pay-as-you-go model. For example, with additional usage enabled, an 8 GB plan can provide up to 8 GB + 16 GB = 24 GB.
The cost for a given log volume is intended to be approximately equivalent between Papertrail’s Heroku add-on and a standalone account. Neither is intentionally more or less expensive. The total cost may be different based on your app’s usage patterns. Typically, apps that log approximately the same amount each day are a good fit for Heroku add-on plans, while apps with highly variable log rates over a month may benefit from the flexibility of a standalone account.
Heroku add-on usage resets daily at midnight (UTC). Paid standalone accounts reset each month on the day that service was activated.
To migrate from one type of service to another, such as from a add-ons to a standalone account or vice versa, follow the instructions for the type of service you’ll be migrating to.
After setting up the new service, confirm it’s working. Once you see logs on the new Papertrail service, de-provision the old Papertrail service. This typically means deactivating Heroku add-on(s) with
heroku addons:remove or removing standalone drain(s) with