In Wyn Enterprise, data is ingested into cached datasets and cached data models by sending queries designed with the dataset designer to the data sources, which possibly included applying transformations on the data fetched from operational sources. Wyn Enterprise 8.1 supports applying transformations to the ingested data in the Wyn Enterprise server before it is loaded into the dataset or data model and then cached in the Wyn Enterprise server. This helps reduce the load on operational systems. Using the new ETL Designer, you can visually design an ETL workflow where data is loaded from data sources and passed through transformation commands and then loaded as a new data table in the dataset or an entity in a data model.

type=danger

The ETL designer is an experimental feature that is not ready for production use. It is provided to get customer feedback and is subject to change.

The ETL designer is a drag-and-drop designer where you drag commands from the command pane to the canvas and connect them to form a data flow or pipeline that form a Directed-Acyclic Graph. An ETL flow consists of one or more Input Sources connected to transformation commands and must end with one Output Target. You connect the output of one or more commands with an arrow to the input of another transformation command to form a flow. Each ETL command must be configured and given a name. The result set of the Output Target is the result set of the flow that is returned to the dataset or data model, which then can be joined to other result sets. The ETL flow cannot be validated or saved unless it has an Output Target.

type=info

If you click anywhere on the canvas and drag around one or more commands, a context menu appears that allows you to remove the selected command(s) or duplicate them.

The ETL designer consists of the following elements:

1. Command Pane: The commands' pane contains the transformation commands that you can drag and drop on the ETL canvas.

2. Toolbar: The toolbar has buttons to save, execute and search for commands in the ETL. You can execute the ETL on all the rows in the input sources, or for speed during design by setting a limit on the number of rows fetched. There is also a search box to search for used commands by name; the first match will open the configuration of the found command.

The Save button is enabled only when the ETL flow is valid. One common reason for an ETL not to be valid is a missing Output Target.

4. Canvas: To create an ETL workflow, you drop ETL commands from the Command Pane to the ETL canvas and connect them to form a transformation flow. When you select a command, its configuration pane gets displayed where you can change its settings. To remove a command or a connector, hover over it and click the Recycle Bin button.

5. MiniMap: is a miniature map of the workflow, which helps you control the visible part of a large workflow. You can move the rectangle area in the minimap to control which part of the workflow is visible on the ETL canvas.

6. Configuration Pane: When an ETL command is selected on the canvas, its configuration pane is displayed, where you can change its settings. The configuration pane differs from one command to the other, but all commands have a name that you can change and a Data tab that displays the result set, which is the output of the selected command.

type=note

The configuration panel for all ETL commands has one configuration tab that is different for each command but all commands can be renamed and the their outputs can be previewed in the **Data** tab.