Launch Services – ETL Transformation as a WebService

launch-services-glossary

Transformations on CloverETL Server can be run by users in a very simple way by using just a web browser and a correct link. When a user click on the link, a transformation is triggered on the Server and requested data are generated. These data can be in several different formats as Excel spreadsheet, CSV file, XML, or HTML. One can even place a form on the web page that serves as a source for parameters for the transformation.[Continue reading]

Introducing Rollup

Simple rollup transform

Rollup is a general transformation component introduced in CloverETL version 2.8. It serves as an executor of rollup transformations written in Java or CTL. Basically, rollup transformations are used to process groups of data records. Each group consisting of M data records may be used to output N different data records. The number of output data records is determined at runtime within the rollup transformation. If no group key is defined, all input data records belong to a single group.[Continue reading]

Sorting Data: ExtSort vs. FastSort – which one is better for me? (Part 2)

extsort-screen

In my  previous post I have focused on tips for tweaking the FastSort component – performance sort component available in all commercial CloverETL editions. Today, I would like to touch the original ExtSort component which has been in CloverETL for a while and is available in both commercial and free (Community, opensource engine) editions.[Continue reading]

Sending E-mails from Data Transformation (Part 2 – Attachements)

Attachments attribute editor

In my previous post I talked about using the EmailSender component featured in CloverETL 2.8 and later to send messages from inside a running transformation graph. EmailSender is used in cases when you need to compose a message from the data that you process in your graph. For example, bulk mailing computed reports, reporting faulty data to administrators, etc. Read my previous post on EmailSender to learn the basics.[Continue reading]

Loop Execution of Data Transformation

ISIR_graph

Case study description

Czech Insolvency Registry (http://isir.justice.cz) basically contains data about economic subjects that entered insolvency and have financial difficulties with paying off their debts. The registry allows everybody to download data using public SOAP Web Service. It can be done manually or automatically with the right software.[Continue reading]

Sending E-mails from Data Transformation (Part 1)

EmailSender attributes window

If you ever have a situation where you need to send an email from your data transformation, CloverETL gives you several options to do it. If you need to monitor your transformation's health and status, you would be better off with CloverETL Server, which offers plenty of monitoring and reporting features. These can be hooked to an “e-mail” action which sends you e-mail alerts based on predefined rules. This is very useful feature in an enterprise environment. We will hopefully cover these options in some later posts.[Continue reading]

Data Profiling with CloverETL

BasicStatistic

Before you start to develop any data transformation you should explore your data (make data profiling). There are a lot of tools on the market that can help you. But why to install and learn another software when you can use the tool you are familiar with? CloverETL is mainly data transformation tool but it can be easily used for data profiling as well (as I will show you in this blog post).[Continue reading]

Joining Data with RelationalJoin

Example graph with RelationJoin.

CloverETL version 2.8 offered a brand new commercial  component called RelationalJoin. It extends the CloverETL pallete of joiner components with new functionality - joining of records with relational operators different from equal (=). This component has two input ports for master and slave data records, and a single output port for joined data records. Master and slave data records are joined and sent to the output port if they are in a specific relation to each other. In other words, it's just another joiner that uses a relational operator to drive the joining process.[Continue reading]