In this lesson you will
- Read multiple files using a wildcard (filename*.csv)
- Enable “debug mode” on edges to inspect data passing through (Data Inspector)
- Use Reformat to manipulate data (to remove dashes ‘-‘ from account numbers)
- Learn to use CTL – CloverETL Transformation Language (if-then and basic string functions)
- Write rejected records to a file
You can use Data Inspector to view data on components and edges. Typically, you’d use it to check a reader or data flow through edges. Remember that you need to turn on “Debug” for edges to collect data while the transformation is running before you can view the data.
Right-click the edge you want to debug.
Select “Enable Debug“.
Run (Don’t forget the transformation needs to re-run in order to collect debug data!).
Finally, switch to Data tab and click on edges or components you want to inspect.
Many components (Reformat being one of the most common) support adding pieces of code to add business logic.
In our example, we’ll be checking Account_number for dashes (‘-‘). If we find them, we will remove them by concatenating parts of the string to the left and to the right of the dash and finally we’ll convert the result from string to integer.
Basics of CTL
- Please refer to CTL – Language Reference