Remove Duplicates

Contents[Hide]

The Remove Duplicates transform lets you remove duplicated records by grouping all of the selected input columns and copying the results to the output.

Transform - Remove Duplicates
Transform - Remove Duplicates

1. Input

The Remove Duplicates transform requires one input transform.

For example, the input could be a SQL Select transform that reads data from the AdventureWorks 2012 table [Sales].[SalesOrderDetail].

Input table
Input table

2. Add the transform

To add this transform to an existing data cube process, first select the connection link between two connected transforms.

Go to the toolbar, click Insert Other, and then select Remove Duplicates.

Select connection link and insert the transform
Select connection link and insert the transform

The Remove Duplicates transform is inserted between the two transforms.

Remove Duplicates transform is inserted
Remove Duplicates transform is inserted

3. Configure the transform

Double-click the Remove Duplicates transform, or select the Configure option from its right-click menu.

In the configuration dialog for the transform, select the input columns to be grouped and sent to the output. For example, if you simply select a ProductID column for the SalesOrderDetail table, the expectation will be that the resulting output will consist of a column of unique product IDs.

Select input columns to be grouped and copied to the output
Select input columns to be grouped and copied to the output

4. Output

The output of the Remove Duplicates transform consists of the column(s) you selected, where the records have been grouped in order to remove duplicates.

Select the Process Result transform and open the Data Preview to see the output of the data cube.

Output of Remove Duplicates transform shows records with unique ProductID values
Output of Remove Duplicates transform shows records with unique ProductID values

5. See also

Dundas Data Visualization, Inc.
400-15 Gervais Drive
Toronto, ON, Canada
M3C 1Y8

North America: 1.800.463.1492
International: 1.416.467.5100

Dundas Support Hours:
Phone: 9am-6pm, ET, Mon-Fri
Email: 7am-6pm, ET, Mon-Fri