CloverETL Designer
CloverETL Overview
Family of CloverETL Products
CloverETL Designer
CloverETL Engine
CloverETL Server
Getting Additional Information
Integrating CloverETL Designer with CloverETL Server
Creating CloverETL Server project
Connecting via HTTP
Connecting via HTTPS
Designer has its Own Certificate
Configuring CloverETL Designer
Designer does not have its Own Certificate
Opening CloverETL Server Projects
Installation Instructions
System Requirements for CloverETL Designer
Downloading and Installing Designer/Eclipse 3.5 Bundle
Starting Eclipse
Installing Designer Into Eclipse 3.5
Installing Designer Into Eclipse 3.4
Getting Started
Creating the First CloverETL Project
CloverETL Examples Project
CloverETL Project
CloverETL Server Project
Structure of CloverETL Projects
Standard Structure of All CloverETL Projects
Workspace.prm File
Opening the CloverETL Perspective
Appearance of CloverETL Perspective
CloverETL Designer Panes
Graph Editor with Palette of Components
Navigator Pane
Outline Pane
Tabs Pane
Creating CloverETL Graphs
Creating Empty Graphs
Creating a Simple Graph in a Few Simple Steps
Running CloverETL Graphs
Working with CloverETL Designer
Using Cheat Sheets
Common Dialogs
URL File Dialog
Edit Value Dialog
Open Type Dialog
Import
Import CloverETL Projects
Import from CloverETL Server Sandbox
Import Graphs
Import Metadata
Metadata from XSD
Metadata from DDL
Export
Export Graphs
Export Graphs to HTML
Export Metadata to XSD
Export to CloverETL Server Sandbox
Export Image
Advanced Topics
Program and VM Arguments
Changing Default CloverETL Settings
Enlarging the Font of Displayed Numbers
Setting and Configuring Java
Setting Java Runtime Environment
Installing Java Development Kit
Graph Elements, Structures and Tools
Components
Edges
Connecting Components by the Edges
Types of Edges
Assigning Metadata to the Edges
Propagating Metadata through the Edges
Colors of the Edges
Debugging the Edges
Enabling Debug
Selecting Debug Data
Viewing Debug Data
Turning Off Debug
Metadata
Data Types and Record Types
Internal Metadata
Creating Internal Metadata
Externalizing Internal Metadata
Exporting Internal Metadata
External (Shared) Metadata
Creating External (Shared) Metadata
Linking External (Shared) Metadata
Internalizing External (Shared) Metadata
Creating Metadata
Extracting Metadata from a Flat File
Extracting Metadata from Delimited Files
Extracting Metadata from Fixed Length Files
Extracting Metadata from an XLS(X) File
Extracting Metadata from a Database
Creating Metadata from a DBase File
Creating Metadata by User
Dynamic Metadata
Reading Metadata from Special Sources
Assigning Metadata to an Edge
Creating Database Table from Metadata and Database Connection
Metadata Editor
Basics of Metadata Editor
Record Pane
Details Pane
Record Details
Field Details
Changing and Defining Delimiters
Changing Record Delimiter
Changing Default Delimiter
Defining Non-Default Delimiter for a Field
Editing Metadata in the Source Code
Database Connections
Internal Database Connections
Creating Internal Database Connections
Externalizing Internal Database Connections
Exporting Internal Database Connections
External (Shared) Database Connections
Creating External (Shared) Database Connections
Linking External (Shared) Database Connections
Internalizing External (Shared) Database Connections
Database Connection Wizard
Encrypting the Access Password
Browsing Database and Extracting Metadata from Database Tables
JMS Connections
Internal JMS Connections
Creating Internal JMS Connections
Externalizing Internal JMS Connections
Exporting Internal JMS Connections
External (Shared) JMS Connections
Creating External (Shared) JMS Connections
Linking External (Shared) JMS Connection
Internalizing External (Shared) JMS Connections
Edit JMS Connection Wizard
Encrypting the Authentication Password
QuickBase Connections
Lookup Tables
Internal Lookup Tables
Creating Internal Lookup Tables
Externalizing Internal Lookup Tables
Exporting Internal Lookup Tables
External (Shared) Lookup Tables
Creating External (Shared) Lookup Tables
Linking External (Shared) Lookup Tables
Internalizing External (Shared) Lookup Tables
Types of Lookup Tables
Simple Lookup Table
Database Lookup Table
Range Lookup Table
Persistent Lookup Table
Aspell Lookup Table
Sequences
Internal Sequences
Creating Internal Sequences
Externalizing Internal Sequences
Exporting Internal Sequences
External (Shared) Sequences
Creating External (Shared) Sequences
Linking External (Shared) Sequences
Internalizing External (Shared) Sequences
Editing a Sequence
Parameters
Internal Parameters
Creating Internal Parameters
Externalizing Internal Parameters
Exporting Internal Parameters
External (Shared) Parameters
Creating External (Shared) Parameters
Linking External (Shared) Parameters
Internalizing External (Shared) Parameters
Parameters Wizard
Parameters with CTL Expressions
Using Parameters
Internal/External Graph Elements
Dictionary
Specifying the Dictionary Contents
Using the Dictionary in a Graph
Accessing the Dictionary from Readers and Writers
Accessing the Dictionary with CTL
Accessing the Dictionary with Java
Notes in the Graphs
Search Functionality
Transformations
Components Overview
Introduction to Components
Palette of Components
Common Properties of All Components
Edit Component Dialog
Component Name
Phases
Enable/Disable Component
PassThrough Mode
Common Properties of Most Components
Group Key
Sort Key
Defining Transformations
Return Values of Transformations
Error Actions and Error Log
Transform Editor
Common Properties of Readers
Viewing Data on Readers
Input Port Reading
Incremental Reading
Selecting Input Records
Data Policy
Examples of File URL Used In Readers
XML Features
CTL Templates for Readers
Java Interfaces for Readers
Common Properties of Writers
Viewing Data on Writers
Output Port Writing
How and Where Data Should Be Written
Selecting Output Records
Examples of File URL Used In Writers
Partitioning Output into Different Output Files
Java Interfaces for Writers
Common Properties of Transformers
CTL Templates for Transformers
Java Interfaces for Transformers
Common Properties of Joiners
Join Types
Slave Duplicates
CTL Templates for Joiners
Java Interfaces for Joiners
Common Properties of Others
Custom Components
Component Reference
Readers
DataGenerator
UniversalDataReader
ParallelReader
CloverDataReader
XLSDataReader
DBFDataReader
DBInputTable
XMLExtract
XMLXPathReader
JMSReader
LDAPReader
MultiLevelReader
QuickBaseRecordReader
QuickBaseQueryReader
Writers
Trash
UniversalDataWriter
CloverDataWriter
XLSDataWriter
StructuredDataWriter
EmailSender
DBOutputTable
DB2DataWriter
InfobrightDataWriter
InformixDataWriter
MSSQLDataWriter
MySQLDataWriter
OracleDataWriter
PostgreSQLDataWriter
XMLWriter
JMSWriter
LDAPWriter
QuickBaseRecordWriter
QuickBaseImportCSV
Transformers
SimpleCopy
ExtSort
FastSort
SortWithinGroups
Dedup
ExtFilter
EmailFilter
Concatenate
SimpleGather
Merge
Partition
DataIntersection
Aggregate
Reformat
Denormalizer
Normalizer
Rollup
XSLTransformer
Joiners
ApproximativeJoin
ExtHashJoin
ExtMergeJoin
LookupJoin
DBJoin
RelationalJoin
Others
SystemExecute
JavaExecute
DBExecute
RunGraph
HTTPConnector
WebServiceClient
CheckForeignKey
SequenceChecker
LookupTableReaderWriter
SpeedLimiter
Clover Transformation Language
Transformation Language Overview
Language Reference
Program Structure
Comments
Import
Data Types in CTL
Literals
Date and Time Format
Numeric Format
Locale
Variables
Operators
Arithmetic Operators
Relational Operators
Logical Operators
Simple Statement and Block of Statements
Control Statements
Conditional Statements
If Statement
Switch Statement
Iteration Statements
For Loop
Do-While Loop
While Loop
For-Each Loop
Jump Statements
Break Statement
Continue Statement
Return Statement
Error Handling
Functions
Message Function
Eval
Accessing Data Records and Fields
Mapping
Lookup Tables
Sequences
Parameters
Functions Reference
Conversion Functions
Date Functions
Mathematical Functions
String Functions
Container Functions
Miscellaneous Functions
Custom CTL Functions
Tutorial
Creating a Project and a Graph
Creating a New Data File
Pasting Components in the Graph Editor Pane
Connecting Components by the Edges
Extracting Metadata from the Input File
Assigning Metadata to the Edges
Propagating Metadata
Setting Up Readers (UniversalDataReader)
Setting Up the ExtFilter Component
Setting Up the ExtSort Component
Setting Up Writers (UniversalDataWriter)
Running the Graph
Refreshing the Data-Out Folder
Opening the Output File
Custom Component
Creating a Custom Component
Integrating a Custom Component in CloverETL Designer
CloverETL Overview
Family of CloverETL Products
CloverETL Designer
CloverETL Engine
CloverETL Server
Getting Additional Information
Integrating CloverETL Designer with CloverETL Server
Creating CloverETL Server project
Connecting via HTTP
Connecting via HTTPS
Designer has its Own Certificate
Configuring CloverETL Designer
Designer does not have its Own Certificate
Opening CloverETL Server Projects
Installation Instructions
System Requirements for CloverETL Designer
Downloading and Installing Designer/Eclipse 3.5 Bundle
Starting Eclipse
Installing Designer Into Eclipse 3.5
Installing Designer Into Eclipse 3.4
Getting Started
Creating the First CloverETL Project
CloverETL Examples Project
CloverETL Project
CloverETL Server Project
Structure of CloverETL Projects
Standard Structure of All CloverETL Projects
Workspace.prm File
Opening the CloverETL Perspective
Appearance of CloverETL Perspective
CloverETL Designer Panes
Graph Editor with Palette of Components
Navigator Pane
Outline Pane
Tabs Pane
Creating CloverETL Graphs
Creating Empty Graphs
Creating a Simple Graph in a Few Simple Steps
Running CloverETL Graphs
Working with CloverETL Designer
Using Cheat Sheets
Common Dialogs
URL File Dialog
Edit Value Dialog
Open Type Dialog
Import
Import CloverETL Projects
Import from CloverETL Server Sandbox
Import Graphs
Import Metadata
Metadata from XSD
Metadata from DDL
Export
Export Graphs
Export Graphs to HTML
Export Metadata to XSD
Export to CloverETL Server Sandbox
Export Image
Advanced Topics
Program and VM Arguments
Changing Default CloverETL Settings
Enlarging the Font of Displayed Numbers
Setting and Configuring Java
Setting Java Runtime Environment
Installing Java Development Kit
Graph Elements, Structures and Tools
Components
Edges
Connecting Components by the Edges
Types of Edges
Assigning Metadata to the Edges
Propagating Metadata through the Edges
Colors of the Edges
Debugging the Edges
Enabling Debug
Selecting Debug Data
Viewing Debug Data
Turning Off Debug
Metadata
Data Types and Record Types
Internal Metadata
Creating Internal Metadata
Externalizing Internal Metadata
Exporting Internal Metadata
External (Shared) Metadata
Creating External (Shared) Metadata
Linking External (Shared) Metadata
Internalizing External (Shared) Metadata
Creating Metadata
Extracting Metadata from a Flat File
Extracting Metadata from Delimited Files
Extracting Metadata from Fixed Length Files
Extracting Metadata from an XLS(X) File
Extracting Metadata from a Database
Creating Metadata from a DBase File
Creating Metadata by User
Dynamic Metadata
Reading Metadata from Special Sources
Assigning Metadata to an Edge
Creating Database Table from Metadata and Database Connection
Metadata Editor
Basics of Metadata Editor
Record Pane
Details Pane
Record Details
Field Details
Changing and Defining Delimiters
Changing Record Delimiter
Changing Default Delimiter
Defining Non-Default Delimiter for a Field
Editing Metadata in the Source Code
Database Connections
Internal Database Connections
Creating Internal Database Connections
Externalizing Internal Database Connections
Exporting Internal Database Connections
External (Shared) Database Connections
Creating External (Shared) Database Connections
Linking External (Shared) Database Connections
Internalizing External (Shared) Database Connections
Database Connection Wizard
Encrypting the Access Password
Browsing Database and Extracting Metadata from Database Tables
JMS Connections
Internal JMS Connections
Creating Internal JMS Connections
Externalizing Internal JMS Connections
Exporting Internal JMS Connections
External (Shared) JMS Connections
Creating External (Shared) JMS Connections
Linking External (Shared) JMS Connection
Internalizing External (Shared) JMS Connections
Edit JMS Connection Wizard
Encrypting the Authentication Password
QuickBase Connections
Lookup Tables
Internal Lookup Tables
Creating Internal Lookup Tables
Externalizing Internal Lookup Tables
Exporting Internal Lookup Tables
External (Shared) Lookup Tables
Creating External (Shared) Lookup Tables
Linking External (Shared) Lookup Tables
Internalizing External (Shared) Lookup Tables
Types of Lookup Tables
Simple Lookup Table
Database Lookup Table
Range Lookup Table
Persistent Lookup Table
Aspell Lookup Table
Sequences
Internal Sequences
Creating Internal Sequences
Externalizing Internal Sequences
Exporting Internal Sequences
External (Shared) Sequences
Creating External (Shared) Sequences
Linking External (Shared) Sequences
Internalizing External (Shared) Sequences
Editing a Sequence
Parameters
Internal Parameters
Creating Internal Parameters
Externalizing Internal Parameters
Exporting Internal Parameters
External (Shared) Parameters
Creating External (Shared) Parameters
Linking External (Shared) Parameters
Internalizing External (Shared) Parameters
Parameters Wizard
Parameters with CTL Expressions
Using Parameters
Internal/External Graph Elements
Dictionary
Specifying the Dictionary Contents
Using the Dictionary in a Graph
Accessing the Dictionary from Readers and Writers
Accessing the Dictionary with CTL
Accessing the Dictionary with Java
Notes in the Graphs
Search Functionality
Transformations
Components Overview
Introduction to Components
Palette of Components
Common Properties of All Components
Edit Component Dialog
Component Name
Phases
Enable/Disable Component
PassThrough Mode
Common Properties of Most Components
Group Key
Sort Key
Defining Transformations
Return Values of Transformations
Error Actions and Error Log
Transform Editor
Common Properties of Readers
Viewing Data on Readers
Input Port Reading
Incremental Reading
Selecting Input Records
Data Policy
Examples of File URL Used In Readers
XML Features
CTL Templates for Readers
Java Interfaces for Readers
Common Properties of Writers
Viewing Data on Writers
Output Port Writing
How and Where Data Should Be Written
Selecting Output Records
Examples of File URL Used In Writers
Partitioning Output into Different Output Files
Java Interfaces for Writers
Common Properties of Transformers
CTL Templates for Transformers
Java Interfaces for Transformers
Common Properties of Joiners
Join Types
Slave Duplicates
CTL Templates for Joiners
Java Interfaces for Joiners
Common Properties of Others
Custom Components
Component Reference
Readers
DataGenerator
UniversalDataReader
ParallelReader
CloverDataReader
XLSDataReader
DBFDataReader
DBInputTable
XMLExtract
XMLXPathReader
JMSReader
LDAPReader
MultiLevelReader
QuickBaseRecordReader
QuickBaseQueryReader
Writers
Trash
UniversalDataWriter
CloverDataWriter
XLSDataWriter
StructuredDataWriter
EmailSender
DBOutputTable
DB2DataWriter
InfobrightDataWriter
InformixDataWriter
MSSQLDataWriter
MySQLDataWriter
OracleDataWriter
PostgreSQLDataWriter
XMLWriter
JMSWriter
LDAPWriter
QuickBaseRecordWriter
QuickBaseImportCSV
Transformers
SimpleCopy
ExtSort
FastSort
SortWithinGroups
Dedup
ExtFilter
EmailFilter
Concatenate
SimpleGather
Merge
Partition
DataIntersection
Aggregate
Reformat
Denormalizer
Normalizer
Rollup
XSLTransformer
Joiners
ApproximativeJoin
ExtHashJoin
ExtMergeJoin
LookupJoin
DBJoin
RelationalJoin
Others
SystemExecute
JavaExecute
DBExecute
RunGraph
HTTPConnector
WebServiceClient
CheckForeignKey
SequenceChecker
LookupTableReaderWriter
SpeedLimiter
Clover Transformation Language
Transformation Language Overview
Language Reference
Program Structure
Comments
Import
Data Types in CTL
Literals
Date and Time Format
Numeric Format
Locale
Variables
Operators
Arithmetic Operators
Relational Operators
Logical Operators
Simple Statement and Block of Statements
Control Statements
Conditional Statements
If Statement
Switch Statement
Iteration Statements
For Loop
Do-While Loop
While Loop
For-Each Loop
Jump Statements
Break Statement
Continue Statement
Return Statement
Error Handling
Functions
Message Function
Eval
Accessing Data Records and Fields
Mapping
Lookup Tables
Sequences
Parameters
Functions Reference
Conversion Functions
Date Functions
Mathematical Functions
String Functions
Container Functions
Miscellaneous Functions
Custom CTL Functions
Tutorial
Creating a Project and a Graph
Creating a New Data File
Pasting Components in the Graph Editor Pane
Connecting Components by the Edges
Extracting Metadata from the Input File
Assigning Metadata to the Edges
Propagating Metadata
Setting Up Readers (UniversalDataReader)
Setting Up the ExtFilter Component
Setting Up the ExtSort Component
Setting Up Writers (UniversalDataWriter)
Running the Graph
Refreshing the Data-Out Folder
Opening the Output File
Custom Component
Creating a Custom Component
Integrating a Custom Component in CloverETL Designer