Maximizing Value with Cost Management

Introduction

In order to enable our customers to effectively manage their monthly log volumes, DataSet developed the Cost Management feature. Customers can now perform the following actions from a centralized location:

1. Identify and categorize logs with DataSet search queries
2. Set a planned usage quota for any category they define
3. Send email notifications if the planned usage is exceeded
4. Directly discard log segments

This article is designed to provide users with a quick overview of how the Cost Management feature works; however, full details can be found within our documentation: https://app.scalyr.com/help/cost-management

Set up Log Categories

Video Explanation: https://youtu.be/J93TeHktHE0?si=M8J-0gC-iBqUdJxV&t=186

Prior to establishing any categories, there will be an "Other" category with no planned amount. The "Other" category acts as a catch-all for log activity that does not have a category assigned to it.

Click on the "Add New Category" link, and enter the details for the log(s) you wish to add to the category:

DataSet queries can be used in the "Logs to include" field to quickly define files of interest.

Video Explanation: https://youtu.be/J93TeHktHE0?si=enuaVCmCYelH3iTy&t=265

When you click the "Save" button:

1. The new category, its planned usage, and its usage statistics are displayed in the "Cost Management" dialog
   1. 1. Each log source is only counted once
      2. Log categories are processed in the order which they are configured. You may wish to process these categories in the order of most specific to least specific
2. A "Discard Filters" entry for the new category is established beneath the "Cost Management" dialog

Discarding Logs

Video Explanation: https://youtu.be/J93TeHktHE0?si=sruEQUjgHR5Fg-ny&t=394

Log lines which are discarded do not count toward your total log volume. Simply click the "Add New Filter..." link for the category you wish to discard from and enter a DataSet query. For example,

Then, click the "Test Filter" link to confirm that the filter works as expected. Results that match the discard filter will be displayed in a new search tab.

Manage Notifications

Add recipients who will receive a notification when the actual log usage for a category exceeds the planned usage. Usage is calculated as the daily average since the beginning of the month. This is an important characteristic, as it calculations are directly applicable to your billing plan.

Consequently, for the purposes of alerting users about data usage, we recommend use of the "Manage Notifications"

Upcoming Improvements

We strive to continuously improve the support we provide to our valued customers. There are additional refinements in development for this feature, and we welcome any suggestions or comments you may have. Thank you!