Search syntax

Full explanation of Papertrail's search query syntax.

Contents:

Background

If you've used Google or GMail, you're familiar with 95% of Papertrail's search syntax. Type what you have, whether it's an IP address, an error string, the name of a program or system, a user's session, or fragments of any of those things.

From there, build up sets of what you do and don't want to see. Here's a few examples for the Ruby on Rails framework to give you an idea.

This came from our experience using Papertrail for our logs, from watching early users try different syntaxes, and most importantly, from seeing what we and others actually wanted to find. By far the most common type of search was a set of desired program names or message strings (important things) combined with a few exclusions (noise).

This Boolean syntax makes queries like that fast to compose and easy to read/update.

Check out Linking to logs to quickly integrate Papertrail searches into your app's internal admin interface.

Example Searches

These 6 searches demonstrate most of the syntax. The sender name, component/file name, and message are all searchable:

"Accepted password" sudo
-CROND
(www42 some.log) OR (nyc-db2 other.log)
(message-id postfix/smtpd) OR -google.com
10.1.2 -("TTL exceeded" OR "1 packet")

Searches can also be constrained by an attribute, so Papertrail considers only that field:

program:(production.log staging.log) '"action"=>"index"' -("count(*)" OR abc)

Quick Reference

While in the event viewer, choose Help -> Search Tips to see these examples without leaving the viewer:

search_tips.png

search_tips_inline.png

Tips

Here's how to get the narrowest set of results as fast as possible.

  • Create groups that represent different sets of systems, then search those groups (instead of searching an all-encompassing group). Aside from making search faster, properly-scoped groups make the Dashboard easier to interpret, since searches are tied to the group they're actually related to rather than being lumped together. As an example, Production Web servers may be far fewer logs than Production servers, and when a search is specific to Web logs, it probably logically belongs there.

  • Use attributes when searching only specific parts of a log message. For example, abc sender:www42 will be faster than abc www42.

  • Provide (or link with) a time/date or rough estimate. Papertrail will start searching from the time you seek to in the event viewer. Seek to a time at the end of or after the incident so that the search starts there.

  • Integrate contextual "Show related logs" links from your app's admin dashboard. Your dashboard can generate links to logs with timestamps (such as a timestamp from a user's last login or when a background job completed) and search queries. The additional search keywords (beyond what you would manually type) may return fewer unrelated matches and it saves typing or copy-and-pasting.

Attributes

Use attributes:

  • to constrain matching to only a single field, either to eliminate false positives or search less data (and increase speed)
  • to search facility or severity, which are not normally examined

Papertrail supports 5 attributes: sender, program, message, severity, facility. All are used in key:value fashion. Here's an example search which incorporates attribute constraints:

("something bad" program:ssh -noise) OR severity:error

Here's another, which includes two valid program values. Messages matching either one are returned (OR):

program:(raid5tools ethtool)

Attributes and attribute-less constraints can be used together, as can negation.

facility and severity support matching multiple values, so this matches any facility value beginning with local:

facility:local

As a result, they also support shorthand, like this. No need to type the full severity values:

severity:(crit emerg)

For more examples, see Papertrail's attribute announcement.

Search Syntax

Papertrail supports combining terms and phrases using AND, OR, and exclusion operators, as well as order of operations (parentheses). The message text, program/sender name, and system name (as shown in Papertrail's dashboard) are searched.

AND

By default, terms in the same search are automatically ANDed. This search:

something somethingelse

is equivalent to this search:

something AND somethingelse

OR

To match one or more of multiple terms, use the OR keyword. For example:

something OR somethingelse

Note: OR must be capitalized.

Phrases

Enclose phrases in single- or double-quotes. For example:

something OR "something else"

Single- and double-quotes are interchangeable, as long as the same type of quote is used to start and end the same phrase. To search for a phrase which contains a quote, use the other type of quote. For example, to match this punctuation-filled phrase:

{"action"=>"index",

enclose it in single-quotes, like this:

'{"action"=>"index",'

Order of operations

To indicate order of operations for matching, use parentheses. For example:

something ("something else" OR "a third thing")

says to match either of the 2 terms inside the parentheses, then AND it with the term outside the parentheses.

Contrast that search with the same search without parentheses:

something "something else" OR "a third thing"

which is ambiguous about whether the OR applies to just the middle term, or to both of the first 2 terms. (Without parentheses, terms are matched left to right).

AND and OR can be freely used inside and outside of parentheses.

Exclusion (negation)

Prepend - to a term, quoted phrase, or parenthesized set. For example:

something -("something else")

Exclude all matches in a parenthesized set:

something -("but not" OR "something else")

Exclusions can be used anywhere in a query. For example:

something -("but not" OR "something else") "but this too"

Case sensitivity

Searches are case insensitive. This query:

Completed in

is equivalent to this query:

completed in

Attributes

Papertrail supports 5 attributes: sender, program, message, severity, and facility.

These attributes can be used nearly interchangeably with attribute-less operators. Negation and phrases work the same.

4 of the 5 attributes - all except message - can only have a single value per message, so AND is never relevant. Because of this, all attributes default to OR. program:(a b) means program:(a OR b).

Additionally, any of the attribute names host, source, or system function as an alias for sender.

Punctuation

Other than quotes, no special adaptations should be needed to use punctuation in a term or phrase. For example, to see all occurrences of IP addresses within 10.20.30.0 to 10.20.30.255, use this query:

10.20.30.

The periods are interpreted as part of the string. Other punctuation works the same way, and no adaptations or escaping is needed.

Similarly, the entire log message is considered for matching. The search:

bcd

will match lines containing any of these:

abcde
123bcd456
456bcd
456abcde

Time/Date Search

To search for a time or date, click the clock icon to the right of the search box and enter a time, date, or combination. The time or date can be in almost any format, and will be parsed and displayed in realtime. Here's a screenshot:

time_date_search.png

This blog post has more.

Timezones

When searching by time in the Web interface, the searched-for timestamp is interpreted in the timezone of the computer you're accessing Papertrail from.

Also, Papertrail tracks the time when a log message was received and automatically converts it to your account timezone when displaying the message. Set an account timezone in (Profile)[papertrailapp.com/user/edit]. Users can have different timezones within the same organization, so no one needs to perform mental timezone conversion.

Sender name and program name

While sender name and program name/filename can be used in searches, they aren't part of the phrase that Papertrail matches against. For example, this log message:

www42 httpd: GET / HTTP/1.0

would be matched by any of these queries:

www42 GET
httpd GET
www42 httpd GET
www42 httpd "GET /"
www42 OR "GET /"

However, that message would not be matched by this phrase query because the program name and separating colon are not included in the phrase:

"httpd: GET"

Linking to Searches

This document has moved to Linking to logs.