Optimizing your Dashboard Configuration

Introduction

DataSet offers a variety of graphs and tables to highlight trends and outliers within your log data. This article supplements the production documentation and includes a number of best practices we recommend to make dashboards load faster and display more relevant results.

Dashboard Configuration

Setting a Minimal Duration

The duration parameter is used to set the time frame displayed by each graph within a dashboard. It is 4 hours by default, however, a smaller range like 30 minutes will speed up the dashboard's load time while providing users with a recent summary of events.

If the dashboard is being used for investigative queries (ex. the user normally enters a specific time range), set the duration to a short(er) period to allow the dashboard to load more quickly.

Filter Options

Dashboard parameters can be configured in several different ways. This includes a textbox (for free entry), a dropdown (for predefined filters), or a dynamically populated dropdown (for selecting host names). In order to support this functionality, the graph queries need to be updated with the field name(s) of the parameters (ex. #host# #platform#). These filters can reduce load times while improving the relevance of graphs displayed within a dashboard.

For example,

...
  parameters: [
    // Add interactivity through free form text input
    { name : "Filter", defaultValue: "state!='WAITING' my_env='production'" },
  ],
...

Creates a text box at the top of the dashboard with a pre-populated value. The user can add additional filters as needed.

Organization and Layout

In order to expedite the rendering of graphs and the relevance of information, we recommend categorizing graphs. Since dashboards behave as containers for related groups of graphs, it makes sense to group graphs that share a corresponding purpose or time frame. When unrelated graphs are included within the same dashboard, additional time is spent loading and rendering information that may not be used. This can either be done with tabs or separate dashboards. 

For example, a dashboard provides up-to-the minute details on system metrics. Graphs that are related to system statistics (like CPU, RAM, network, and processes) would go here, while graphs specific to your server access logs would not.

Similarly, you wouldn't want a dashboard with 50 graphs of everything related to your Kubernetes platform and the applications running on it, as this configuration would be visually overbearing (and may load more slowly). In most cases, we recommend assigning 5-10 graphs to a dashboard, as this improves responsiveness and context.

Tabs

Click the "+" at the top of your dashboard to access:

Tabs are more streamlined than separate dashboards since you can have multiple tabs within the same dashboard. Graphs which are based within tabs are loaded on demand, which reduces the dashboard's load time.

Graphs - Best Practices

Searches

This applies to both PowerQueries and conventional searches.

• When possible, use parsed attributes. For example a search of "value123" will search the raw message, which typically contains more data than an attribute. Searching for keyabc="value123" is a better approach, provided that keyabc is an available attribute.
• See Optimize your Search Queries: Attributes for more information

Conventional Graphs vs PowerQueries

Although PowerQueries offer a lot of versatility, they are rendered "on the fly", which may slightly  extend load times.

Conventional graphs use a histogram that's generated in advance to expedite their rendering. Our Engineering team is presently developing additional enhancements for PowerQuery driven graphs. Consequently, we recommend using conventional graphs when possible (graphs like pie charts and donuts only support PowerQueries) in the meantime.

When PowerQueries are used, they can be optimized by:

• Filtering data. Make the search query (first line) as specific as possible to your use case.
• Reducing the level(s) of aggregation (via the group by statement)
• Minimizing the use of join or union statements
• Removing extraneous sort statements.
• Maximize relevance by using the sort and limit statements for data tables (no point in displaying 1000 rows when only the top 10 are needed)
• See PowerQueries - Best Practices for more information

Conventional Graphs

Our conventional graphs were designed to load quickly and have built-in features for the aggregation of data. Similar to PowerQueries, they use a search query to retrieve the desired log events. Unlike PowerQueries, all aggregation is done via facets which are configured for the graph.

Breakdowns

The breakdown field can be considered as the equivalent of a single-column group by statement. When working with data that has a large number of breakdown values, only the top ~20 breakdown values will be used. As you can see, this is still rather overwhelming!

To avoid this, define a range of desired values in the original search query. For this example,

account in ('blah','test','whee')

In addition to making the results more legible, this prevents the other ~17 breakdown values from being graphed.

Plots

Plots are commonly used to render different queries within the same graph. Since each plot requires its own query to be run, limiting the scope and quantity of queries used with plots can improve the load time of graphs.