Log Analytics with Xplenty

Xplenty is a Hadoop cloud based platform as a service. Using Xplenty, there is no need to deal with Hadoop infrastructure and there is no need to write Hadoop code. Xplenty provides intuitive and familiar interfaces to accomplish these tasks. To get started, sign up for a free trial.

Background

This article covers how to:

Papertrail logs are stored on your S3 Amazon account. Hadoop infrastructure, packages and jobs are managed by Xplenty. You only pay for what you use.

Example uses:

Basic

This is how the following query will look like on Xplenty’s graphical package designer:

_SELECT source_name, COUNT(source_name) FROM events GROUP BY source_name_

Basic Package with Xplenty

Advanced

The following package inspects Papertrail logs, specifically the message field to identify log records that contain “source”, then it breaks down the message field to its components and aggregates them based on source name. The following screenshots display some of the relevant dialog boxes that comprise the package.

Advanced package - Filter component

Advanced package - Projection

Advanced package - Aggregate

Setup

Enable Log Archiving

Create an Amazon Web Services account and configure it for Papertrail by following the steps here, then fill in the details under your account settings.

Every night thereafter, Papertrail will upload a compressed log archive file to your S3 bucket. Login to the AWS management console, browse to your S3 bucket, and confirm that you see at least one date-stamped item in papertrail/logs/.

Login to Xplenty

If you’re not already signed up to Xplenty, click here to get a free trial. If you have an existing account, click here to log in.

Design a Package

Once you’re signed in, go to the “My Packages” page and click on “New Package”. You will then be able to design your Xplenty package. You do not need to define or configure anything in advance. Simply start creating your package by choosing the relevant components applicable to your data processing and analysis needs. You can also choose to begin exploring packages by creating one of the template packages. To do this, click the arrow next to the “New Package” button and choose: “From template…”.

Create a Cluster

In order to create a Hadoop cluster, go to the “My Clusters” page, and hit the “New Cluster” button. Use the slider to determine the size and then click on “Create Cluster”. The cluster will be ready for use within a few minutes.

Xplenty dashboard

What else can you do with Xplenty?

To learn more, visit Xplenty’s knowledge base or check out the API documention here.