CloverETL as a High-throughput XML Processor

Mapping definition

XML is a markup language that has been around for some years now. Originally, it comes from the world of documents - used in web hypertext, word processors and other representations. Today, it is very popular in many areas, including the world of data exchange. The reasons are simple - the format is straightforward, well defined, and easily transferable accross platforms. XML can be easily read and modified by users in contrast to proprietary and binary formats. It also represents structured hierarchical data, which can be very difficult to express in plain CSV format. XML is self-descriptive, which heavily increases the user's ability to understand data and eliminates the need of data format description and parsing instructions.[Continue reading]

ParallelReader Component: Performance Boost in Data Processing

In October release 2.8.1 of Clover we introduced a new component which definitely should attract your attention – the Parallel Reader. The name itself already suggests the goal of the component – improve reading speed by going parallel. The component is very similar to Universal Data Reader in function – it reads delimited flat files like CSV, tab delimited, etc. - much hasn't changed here. But the real difference comes from under the hood.[Continue reading]

Hidden Features: Using Mutable Delimiter for Data Parsing

CloverETL provides a very useful feature: mutable delimiter. When you parse a delimited file (eg. CSV) you can specify different delimiter for each field. This isn't surprising for daily CloverETL users however for users of other ETL tools it can be. It might not be very well known that in CloverETL you can even define more delimiters for one field (so called "mutable delimiter") and CloverETL chooses the right one. It reveals new ways of file processing with irregular structure in CloverETL. I believe this functionality isn't provided by any other ETL tool on the market. If I am wrong you can leave me a message in comments. I'm always happy to find "hidden features" of other ETL tools.[Continue reading]