It’s been few months since we released CloverETL 3.4, our most recent production version. As we make significant progress on the next version, I’d like to share a few of the features available in CloverETL 3.5. In fact, some of these were released a few days ago in the milestone version. Want to try CloverETL for yourself? Download it here.
Server Changes – Monitoring
I’ll start the list off with some changes we’ve made to the Server. In version 3.5, we’ve redesigned and updated the monitoring part of the Server user interface. It will now provide users with more information about the current state of the Server and Cluster. Tuning the Server settings will now be much easier, with the ability to export and import the Server configuration. This change will make tuning your configuration on a testing installation of the Server, then transporting it to a production one more comfortable.
Version Control Systems Support
In CloverETL 3.5, we’ve made a concerted effort to improve version control systems support for CloverETL Server projects. Mostly, changes in Server-Designer integration will simplify the usage of the version control systems. In the current version, to keep a Server project in a version control system, you have to export the project from the Server as well as work with a local copy. In 3.5, this will no longer be the case. We’ve changed the implementation of Server projects in the Designer, so that users will be able to put them into the version control system directly. To do this, just use the regular Eclipse™ plugin for SVN, CVS, Git and more.
The release will also introduce some new components – the first being Validator. It’s a complex, yet easy-to-use tool for checking the validity of data being processed. With Validator, you can avoid graph failures by having better validation. Configurable and customizable via our user friendly UI, Validator allows you to specify a set of rules and check the validity of incoming data based on these rules. It comes with predefined rules for common checks such as empty field, email address, data type, date field format, etc., but you can also define your own custom rules using CTL or Java.
The next set of new components focuses on MongoDB – a modern NoSQL, JSON-oriented database. We’ve extended CloverETL with components such as MongoDBReader, MongoDBWriter and MongoDBExectue. Another JSON-related component we’re introducing is the JSONExtract component, designed for stream processing of JSON data. The relationship between JSONExtract and JSONReader is similar to that of XMLExtract and XMLReader. Extract components are stream-oriented, allowing you to parse larger files, while their Reader counterparts offer more precise mapping capabilities, at the cost of performance and resource effectiveness.
In 3.5, you’ll also find a new jobflow component called Loop. As the name suggests, it introduces loops (for-loop, while-loop) to CloverETL jobflows. This will simplify solving tasks like paging or repeated checks (retries).
There are many more improvements offered in CloverETL 3.5, such as graph bottleneck detection, better extraction of metadata from flat files, and support for Window shares (SMB) in readers and writers. Also, you’ll experience improved graph parameters management with the introduction of required and secure parameters, or changes in time zone handling in CTL.
The production release of 3.5 is planned for early December of this year. As previously mentioned, a milestone release with a few of these features is already available for download. The milestone offers a preview of the MongoDB components, import/export of Server configuration, support for Windows shares, and the Loop component for jobflows. Feel free to experiment with them. And as always, we value any feedback you may have for us during development, so let us know what you think.