Building Data Warehouse with CloverETL: Slowly Changing Dimension Type 2

D_CUSTOMER_SCD2

In the last part of our data warehouse (DWH) tutorial, I showed you how to load a dimension table that stores historical data according to the Slowly Changing Dimension Type 1 (SCD1). In today‚Äôs post, I will focus on a Slowly Changing Dimension Type 2 (SCD2) dimension table. I think that SCD2 is the most challenging sub-task of ETL part of DWH design and each ETL architect should be able to deal with it.[Continue reading]

CloverETL as a High-throughput XML Processor

Mapping definition

XML is a markup language that has been around for some years now. Originally, it comes from the world of documents - used in web hypertext, word processors and other representations. Today, it is very popular in many areas, including the world of data exchange. The reasons are simple - the format is straightforward, well defined, and easily transferable accross platforms. XML can be easily read and modified by users in contrast to proprietary and binary formats. It also represents structured hierarchical data, which can be very difficult to express in plain CSV format. XML is self-descriptive, which heavily increases the user's ability to understand data and eliminates the need of data format description and parsing instructions.[Continue reading]

Sending E-mails from Data Transformation (Part 2 – Attachements)

Attachments attribute editor

In my previous post I talked about using the EmailSender component featured in CloverETL 2.8 and later to send messages from inside a running transformation graph. EmailSender is used in cases when you need to compose a message from the data that you process in your graph. For example, bulk mailing computed reports, reporting faulty data to administrators, etc. Read my previous post on EmailSender to learn the basics.[Continue reading]