Edit Settings

Editing/Defining a Derived Dataset

A Derived Dataset in LityxIQ is one in which one or multiple other datasets can be brought together and modified (joined, aggregated, filtered, transformed, etc) to create a new dataset. In order to define a derived dataset, follow these steps: 1) Select the dataset from the available datasets list, then select Edit Settings from the Selected Dataset menu. If the dataset was just created using the Create New Dataset button, this will open automatically and you can skip to step 2. 2) Th...

Defining QC (Quality Control) Rules for a Derived Dataset

QC, or Quality Control, rules provide a way to do error checking on a dataset after it finishes executing. When no QC rules are set, the dataset execution is considered a success when it completes (of course, other errors may have happened along the way, such as an invalid field definition). If QC rules are defined, each rule is checked when the dataset is otherwise finished executing. If any of the QC rules are evaluated to be be valid, the result of the execution is considered an "error" inste...

Defining Incoming Data for a Derived Dataset

Incoming datasets in a view are those that are stacked together (a SQL-experienced person would call this a UNION) to begin the process of creating the view. After opening the Edit View dialog (see https://support.lityxiq.com/241959-EditingDefining-a-Dataset-View), follow these steps to define the incoming data for a view. If the Incoming Data panel is not already opened, click on that panel header to open it. Then click the Edit Incoming Data button. Note that if this view had previously be...

Define a Single Incoming Dataset

When you click the Add Dataset button in the Incoming Dataset section of a derived dataset, or edit an existing Incoming Dataset, you will have a number of options related to how it is used. Dataset: A list of all datasets available to you appears in the topmost drop down menu. Find the dataset to be used as the basis for this view and select it (it will become checked once selected). Variables to Keep: In this area, you will select which variables will be brought in from the selected dat...

Define Multiple Incoming Datasets for a Derived Dataset

When defining the Incoming Datasets for a derived dataset, you can define mulitple datasets at once using the Multiple Datasets button. Clicking this button, or editing an existing Multiple Datasets definition, provides options described below: All variables from all datasets will be included in the view. Method - currently, two methods for defining multiple incoming datasets are supported: - Manually Select Multiple Datasets - this option allows you to manually click datasets that you w...

Defining Joins for a Derived Dataset

After opening the settings dialog for the derived dataset (see https://support.lityxiq.com/241959-EditingDefining-a-Dataset-View (https://support.lityxiq.com/926307-Using-the-Search-and-Filter-Dialog)), follow these steps to define any joins. It is not required that a derived dataset have any joins defined. 1) If the Joins panel is not already opened, click on that panel header to open it. Then, click the Add New Join button. Note that if this view had previously been defined, you may see inf...

Defining a Transpose Operation in a Derived Dataset

A transpose operation within a derived dataset allows you to put data that spans multiple columns into data in multiple rows. This is sometimes also referred to as a pivot on a dataset. For example, suppose you have a dataset that has a unique field such as StoreID, and has a number of fields containing zip codes. You can think of this perhaps as a set of zip codes that define a footprint for a store. The dataset is currently structured as in the screenshot below, with one unique row per store, ...

Defining New Fields for a Derived Dataset

After opening the settings dialog for the derived dataset (see https://support.lityxiq.com/241959-EditingDefining-a-Dataset-View), follow these steps to define any new fields to be created. It is not required that a derived dataset have any new fields defined. 1) If the New Fields panel is not already opened, click on that panel header to open it. If new fields have already been defined for this view, as in the example below, you will see a list of all fields currently defined. Note that the ...

Create Multiple New Fields at Once

The new fields area of a derived dataset (defined in detail here https://support.lityxiq.com/608058-Defining-New-Fields-for-a-View) has the option to create multiple new fields at once. This functionality is a powerful way to apply the same basic operation to many variables at the same time. Two use case examples are show below. Example 1 - Take Absolute Value of Many Fields Suppose you have many numeric variables, and you want to create a new field related to each of them that contains the...

Defining a Filter for a Derived Dataset

After opening the settings dialog for a derived dataset (see https://support.lityxiq.com/241959-EditingDefining-a-Dataset-View), follow these steps to define a filter for the view. A filter is a method for subsetting the rows of the dataset to match a given criteria. Note that it is not required that a derived dataset have a filter defined. 1) If the Filter panel is not already opened, click on that panel header to open it. Then, click the Edit button. Note that if this view had previously b...

Defining an Aggregation for a Derived Dataset

After opening the settings dialog for a derived dataset (see https://support.lityxiq.com/241959-EditingDefining-a-Dataset-View), follow these steps to define an aggregation step. It is not required that a derived dataset have an aggregation defined. If an aggregation is defined, the resulting dataset will typically have fewer (usually many fewer) records than the original datasets. The number of records will be the total number of different combinations of the selected aggregation variables (see...

Create a New Aggregation Field with Examples

The new aggregation field function will provide an aggregated value for every record in an existing dataset. This is a distinct feature as opposed to the “aggregate” function, which can change the number of fields and number of records in an existing dataset. See https://support.lityxiq.com/192161-New-Field-Aggregations---Concept-and-Comparisons for a more complete overview of New Field Aggregations. 1) In Data Manager select the dataset you wish to edit, and under the Selected Dataset tab clic...

Comparison of Aggregation Types

In LityxIQ, two different types of data aggregations are available, which can be used in conjunction with each other. The two types can be described as: 1) aggregating over all combinations of the levels of the selected variables,. 2) aggregating over all individual levels of the selected variables. This document provides a comparison of these two techniques. Both area available in the Aggregation area of defining a View. Assume we have a very small and simple dataset with three variables: A...

Defining a Pivot within the Aggregation Step

This video demonstrates the steps listed below: https://www.youtube.com/watch?v=fAr6mu81N2g The Aggregation step of a Derived Dataset includes an option to "pivot" (or "transpose") data from rows into columns of the result set. An Aggregation definition may include any number of pivoted variables, restricted only by the limit of 1600 total variables in the result set. As an example, consider the following dataset having variables STATE, domain_name, and ZIP. This is a small snapshot of a da...

Define Finalization Settings for a Derived Dataset

The final step when creating a derived dataset is called the Finalize and QC step. It is optional, but provides important functionality. To edit the finalization settings, follow these steps. 1) Edit the derived dataset, and open the Finalize and QC panel. The panel displays any settings currently defined for the finalization step. Clicking the Delete button will completely remove any current settings. Clicking the Edit button will open the dialog to edit the settings. 2) The Finalize dialo...

Configuration Dataset Settings Options

Configuration Variables (described in more detail here https://support.lityxiq.com/084396-Create-and-Utilize-a-Configuration-Variable), can be created dynamically as the result of either a data import process, or executing a derived dataset. In either case, the same interface options are available, and these are described here. The screenshot below is from the Finalization dialog when editing a derived dataset, but the same options are available in the Define Dataset Source dialog for a Raw Data...