Log Analytics with Xplenty

Xplenty is a Hadoop cloud-based platform as a service. Using Xplenty, there is no need to deal with Hadoop infrastructure and there is no need to write Hadoop code. Xplenty provides intuitive and familiar interfaces to accomplish these tasks.

When using Xplenty, a copy of Papertrail log archives must be stored in your S3 Amazon account. Hadoop infrastructure, packages and jobs are managed by Xplenty. You only pay for what you use.

Setup

Enable log archiving

Create an Amazon Web Services account and configure it for Papertrail by following the steps at Automatic S3 archive export.

Every night thereafter, Papertrail will upload a compressed log archive file to your S3 bucket. Log in to the AWS management console, browse to your S3 bucket, and confirm that you see at least one date-stamped item in papertrail/logs/.

Sign up or log in at Xplenty

If you’re not already signed up for Xplenty, visit sign-up to start a free trial. If you have an existing account, log in.

Design a package

Once signed in, go to the My Packages page and click New Package to begin designing an Xplenty package. Start creating your package by choosing the components applicable to your data processing and analysis needs, or select a template package as a basis for creation. To select a template package, click the arrow next to New Package and choose From template….

Create a cluster

To create a Hadoop cluster, go to the My Clusters page, and click New Cluster. Use the slider to determine the size and then click Create Cluster. The cluster will be ready for use within a few minutes.

Xplenty dashboard

Example uses

Basic

This is what the following query will look like on Xplenty’s graphical package designer:

_SELECT source_name, COUNT(source_name) FROM events GROUP BY source_name_

Basic Package with Xplenty

Advanced

The following package inspects Papertrail logs, specifically the message field, to identify log events that contain “source”, then breaks down the message field to its components and aggregates them based on source name. The following screenshots display some of the relevant dialog boxes that make up the package.

Advanced package - Filter component

Advanced package - Projection

Advanced package - Aggregate

What else can you do with Xplenty?

To learn more, visit Xplenty’s knowledge base or check out the API documention here.