CloverETL 4.2.0 Milestone

CloverETL 4.2 aims at greatly improving developer productivity. We have redesigned the internals of how Designer connects to Server projects so that you can put any project under version control now.

About this release:
This release is now deprecated. Please upgrade to production release CloverETL 4.2.0 or newer.

Released May 12, 2016
  • Milestone
Recent Releases
4.9.1 Aug 14, 2018
4.9.0 Jul 23, 2018
4.8.3 Jul 17, 2018
4.7.1 Nov 28, 2017
4.6.1 July 19, 2017
4.5.0 Mar 14, 2017

See all releases

Try it yourself

Download CloverETL

New features in 4.2.0-M1

Version control (Git, SVN, ...) now functions with Server projects

Designer now mirrors connected remote Server projects on your local disk, allowing you to put the local copy under version control.

With this you can have remote Server projects directly under version control, you no longer need to keep a versioned local copy of the Server sandbox separately.

Version control operations such as pull, push, commit, merge, etc. work directly with the files in the local copy of the project, so you can use any standard version control tool, including built-in Eclipse plugins for Git and SVN. Designer then transparently synchronizes all changes to the Server sandbox.

Remote Server files available locally

As an effect of the previous feature, you can open and save files originating from a Server sandbox with any tool you need.

CloverETL - remote files locally

Convert projects between local and Server

You can easily deploy a project to Server from your workspace by using Convert To Server Project shortcut. Conversely, you can disconnect a Server project to keep just the local copy or connect and sync it with another Server sandbox.

CloverETL - convert to server project

Source editor Syntax highlighting

The Source tab in the main editor now supports syntax highlighting.

CloverETL - source syntax hightlight

Draw edge from input

You can now draw new edge from an input port to create a new upstream component. Just click an input port, drag to a blank area and then use the Add Component dialog to create the component.

CloverETL - create edge from input port

Detailed list of improvements and fixes

Productivity / Version Control for Server Projects [ See All | Hide All ]

Placeholders for large files in Designer New Feature CLO-7899 4.2.0-M1
Large files (over 10MB by default) are not automatically synchronized to the local copy. You can download the files manually or change the threshold in Preferences > CloverETL
Resolving Conflicts between local and server copy New Feature CLO-7836 4.2.0-M1
When Designer synchronizes with Server, you can manually resolve conflicts between incoming and local changes.
Ignore files in server project synchronization New Feature CLO-7604 4.2.0-M1
Unify various clover project natures Improvement CLO-7741 4.2.0-M1
Convert local project to server project (and back) New Feature CLO-8290 4.2.0-M1
In Designer, you can now easily convert a local project to a Server one and vice versa. No need to create new sandbox and then export/copy the files.
Handling of connection outages Improvement CLO-7613 CLO-8207 4.2.0-M1
When working with with Server projects in Designer, connection problems do not affect the synchronization process. Designer will queue the changes until you're back online. Also, Designer reconnects automatically.
Graph header (.grf/.jbf XML files) is now not modified when not needed Improvement CLO-8192 4.2.0-M1
This caused inconveniences when someone resaved a graph without making any changes, yet it appeared as modified in version control, because Designer would change the header in the XML source.

Platform Support [ See All | Hide All ]

Hadoop integration now supports versions CDH 4.3, 5.4 Improvement CLO-7254 4.2.0-M1
Dropped support for Eclipse 3.X Improvement CLO-6003 4.2.0-M1
We no longer support installations of Designer plugin into 3.x Eclipse environments

Packaging / Versions [ See All | Hide All ]

Designer is now bundled with Java 8 Improvement CLO-8558 4.2.0-M1
Designer now comes with Java sources in the bundled JDK Improvement CLO-7042 4.2.0-M1
When developing Java code/transformations, you have out-of-the-box access to full Java sources
Server bundle upgraded to Tomcat 8 Improvement CLO-7764 4.2.0-M1
The Server bundle download is now built on top of Tomcat 8. We don't expect any major changes to setup and configuration

Engine Improvements [ See All | Hide All ]

Amazon S3 support: Replaced JetS3t with Amazon SDK Improvement CLO-7170 4.2.0-M1
S3 connectivity now uses the latest native Amazon SDK
Validator email address rule has new option "Allow addresses with no TLD". Formerly, this was a default behaviour, now you have to explicitly turn it on. Compatibility New Features CLO-6916 4.2.0-M1
FlatFileReader and FlatFileWriter aliases for UniversalDataReader and UniversalDataWriter New Feature CLO-8266 4.2.0-M1
Improved temp file management in Spreadsheet components Improvement CLO-7118 4.2.0-M1
JSONWriter and XMLWriter: New attribute "Write raw value" enables you to take raw snippets of JSON/XML and send them to its output without escaping it. Improvement CLO-6786 4.2.0-M1
New second output port for unmatched data in ExtMergeJoin, ExtHashJoin Improvement CLO-6588 4.2.0-M1
You can now capture unmatched records in ExtMergeJoin and ExtHashJoin
New metadata templates for port writing outputs in XMLWriter, UniversalDataWriter, CloverDataWriter, SpreadsheetDataWriter, StructuredDataWriter, JSONWriter Improvement CLO-4728 4.2.0-M1
Predefined metadata templates on writers, when you want to write to ouput port. Connect output edge to the writer, right the edge and select New Metadata From Template

Designer Improvements [ See All | Hide All ]

Syntax highlighting in graph source editor Improvement CLO-6465 4.2.0-M1
Improved searching for dependencies of graph elements Improvement CLO-5588 4.2.0-M1
More precise algorithm that detects and highlights unused metadata and other graph elements in Outline
Removed legacy options from CloverETL Preferences page Improvement CLO-7962 4.2.0-M1
iif() and lookup() CTL functions were missing in function list in Transform and Expression editors Improvement CLO-7533 4.2.0-M1
Draging & dropping a job into jobflow now names the newly created component based on the name of the job Improvement CLO-6972 4.2.0-M1
For example, if you take TruncateTables.grf and drop it into a jobflow, it will now create an ExecuteGraph component named "TruncateTables"
Drawing an edge from input port: You can now click an input port, draw a new edge and then pick the source component form the Add Component dialog New Feature CLO-6050 4.2.0-M1

Server Improvements [ See All | Hide All ]

Users can now pick JNDI names from JNDI tree whenever JNDI configuration option is present New Feature CLO-7160 CLO-7159 4.2.0-M1
New option to create a heap dump directly from Server Console New Feature CLO-7151 4.2.0-M1
Heap (and thread) dump can be downloaded in Configuration > System Info > Diagnostics.

Fixes [ See All | Hide All ]

Shift+space to insert a subgraph should use ${SUBGRAPH_DIR} instead of sandbox:// Fix CLO-7683 4.2.0-M1
SimpleGather is blocked in fast propagate mode Fix CLO-7211 4.2.0-M1
Opening file though File menu is throwing a null pointer exception Fix CLO-7297 4.2.0-M1
SpreadSheetWriter: mapping by order does not work with hyperlinks Fix CLO-7104 4.2.0-M1
Transform Editor does not open at all when metadata missing Fix CLO-3549 4.2.0-M1

Compatibility [ See All | Hide All ]

S3 connectivity: Replaced JetS3t with Amazon SDK Compatibility Improvement CLO-7170 4.2.0-M1
Designer as eclipse plugin: Dropped support of Eclipse 3.x Compatibility CLO-6003 4.2.0-M1
XMLExtract: wrong attribute value is parsed Compatibility Fix CLO-7126 4.2.0-M1
DBJoin should propagate metadata Compatibility Improvement CLO-8477 4.2.0-M1
Rollup - preExecute() is never called Compatibility Fix CLO-7202 4.2.0-M1
SpreadSheetWriter: mapping by order does not work with hyperlinks Compatibility Fix CLO-7104 4.2.0-M1
Sequences in compiled mode skip values Compatibility Fix CLO-5762 4.2.0-M1
Validator email address rule has new option "Allow addresses with no TLD". Formerly, this was a default behaviour, now you have to explicitly turn it on. Compatibility Fix CLO-6916 4.2.0-M1

Under the hood [ See All | Hide All ]

Metadata propagation algorithm moved from Designer to Server Improvement CLO-6437 4.2.0-M1
Allow more hyperlinks in graph run log Improvement CLO-7023 4.2.0-M1

ETL Developers (This item is most useful for developers; it either brings new functionality for transforming data or brings optimizations)

Administrators (This item is an improvement or feature that will help setup, install, administer and manage the application)

Support (This item helps staff supporting the production operation to identify potential problems or avoid such)