CloverETL Data Quality – An Introduction to Validator

validator_available_rules_active_rules

Before your data enters the ETL process, it's in your best interest to only work with "good" data – that is, data that conforms to its respective domain rules, ranges, allowed values, and perhaps other unusual restrictions. If it doesn't, you'll want to log and remove all erroneous records so as not to pollute your transformation, as well as to have means to report and fix the data later on.[Continue reading]

Creating Custom Component in CloverETL Step By Step

component-04a

CloverETL is designed to be an extensible system. Although it already contains a lot of components, there can be cases where you need to create your own – be it a specific and often-used task, use of an external library, your own connector, etc. The full guide for creating a custom component is described in the documentation, but for those of you who want a quick look at the process without the lengthy details, I’ve prepared some brief instructions. If you follow this guide step by step, you’ll be able to create custom components in no time.[Continue reading]

Take a Break from Big Data

David Pavlis

Do you really have a Big Data problem? Frankly, for most data needs, the answer is no. You may have a lot of data, but in terms of what the industry's been pitching as Big Data, I wouldn’t worry. There are many more pressing problems for data management and integration. However, if you’re looking for a more elaborate answer, keep reading to find out what all the buzz is about.[Continue reading]

CloverETL 3.5 ­– What to Expect?

It’s been few months since we released CloverETL 3.4, our most recent production version. As we make significant progress on the next version, I’d like to share a few of the features available in CloverETL 3.5. In fact, some of these were released a few days ago in the milestone version. Want to try CloverETL for yourself? Download it here.[Continue reading]

CloverETL Designer User Interface Improvements in 3.4

UImprovements_4

Here at CloverETL we not only work to create new and exciting features, but also have a workgroup specifically dedicated to improving the “little things”. I’m talking about those little things that can help you avoid aimlessly clicking around while trying to do work, or make you think in a quick, intuitive, and productive manner. Listed below are some of the most important tweaks and improvements that CloverETL 3.4 can provide. We think you will find them useful in that they will make working with the CloverETL Designer not only more comfortable, but also more effective.[Continue reading]

Transforming Coordinate Reference Systems using CloverETL – A Use Case

view-data

A common task an ETL tool has to deal with nowadays is the emergence of data containing a form of geographical information. Just like any other type of data – monetary values, times and dates, etc. – geographical data pose interesting challenges to an ETL developer. Working with different Coordinate Reference Systems (CRS) within a project is a common difficulty. Typically, two or more software systems need to exchange geographical data, but each one of them uses a different CRS.[Continue reading]