CloverETL 3.4.3

Get Big Data Under Control

Crunching huge volumes of data and resolving big data problems is a new step in the CloverETL evolution. A strong Cluster toolset to grasp parallel data processing with precise control and monitoring, together with the integration of popular Hadoop storage and MapReduce provides users a new view on robust and rigid data integration – with a modern analytics ad-hoc approach.

Updated Nov 28, 2013
First release Jul 3, 2013
  • Production
Recent Releases
4.7.1 Nov 28, 2017
4.8.0-M1 Nov 14, 2017
4.7.0 Oct 10, 2017
4.6.1 July 19, 2017
4.5.0 Mar 14, 2017
4.4.1 Feb 14, 2016
4.3.1 Nov 10, 2016
4.2.1 Aug 18, 2016
4.1.4 Jun 1, 2016

See all releases

Interested in trying it out? Learn how to get started or sign in to download.

Release 3.4 Contents

Related releases

  • Major New Feature

Hadoop Support

Let CloverETL oversee all your data flows while Hadoop does the heavy-lifting for specific big data tasks.

CloverETL fits into a Hadoop-based setup through HDFS and Hive data storage connectors and MapReduce jobs integration.


  • Designer
  • Server
  • New Feature

Load Balancing

Optimize data throughput by using automatic load balancing of data flows.

  • Designer
  • Server
  • Improvement

Database Connections Proxying from the Designer Through the Server

All database operations during development are carried out on the Server.
No need for the Designer to have the database box connectivity.

  • Profiler
  • Server

Data Profiler Reporting and Automation

Use profiling metrics as data quality inputs in data integration jobs, share profiling results with the Reporting Console, or build your own data quality reports on top of the Reporting Console API.


  • New Feature

HTTPConnector, a full REST client for cloud/SaaS integration

Introducing full support for all HTTP operations in HTTPConnector, making a step towards combining data and application integration and workflow control.

  • Server Cluster
  • Major New Feature

Cluster Control in Your Hands

Take precise control over parallel data flows – spread the load for resource-intensive components, route data based on proximity of resources, and dynamically change the level of parallelism for execution.

  • Server
  • Improvement

Server User Interface Redesign

We're introducing a new look & feel to the Server Console user interface. It's geared towards continuous improvements for support personnel – and better insight into the Server processes and a simpler configuration.

 
  • Improvement

Usability Improvements

We care about users who need a tool that not only does things right, but also makes life easier. We continually add features and tweaks to do just that.
  • Designer
  • Feature
Insert components onto an edge
Drag and drop components from the Palette onto an edge to insert it in between the two existing components, automatically linking it to both sides.
Hint: You can also use the Find component dialog (Shift-Space) and then place the selected component onto an edge.
Insert Components onto an Edge

Additional Features & Fixes

New in 3.4.3 bugfix release (Nov 27, 2013)

No new features in 3.4.3. Please scroll down to Fixed issues section below.

New in 3.4.2 bugfix release (Oct 4, 2013)

  • Server
  • New
User IP address is logged in the userAction log – Server is now logging IP address of the session in the userAction log.
  • Engine
  • Improvement
SpreadsheetDataReader now reports better error message when encoutering corrupt .xls files with custom formatted cells (CLO-822)

New in 3.4.1 bugfix release (Jul 3, 2013)

  • Designer
  • New
Export graph to HTML improved – added graph description, updated to a better look, preserved newlines for better code legibility, and added new print style
  • Server
  • New
Server Console - Date/Time form fields are now keyboard-enabled

Fixed issues in 3.4.3

  • Engine
  • Fix
Null date/time values incorrectly typed when saving to "TIME" database fields (CLO-953)
  • Engine
  • Fix
Fixed display of status and "blue record numbers" on edges after joiner that had input edges in different phases (CLO-2297)
  • Engine
  • Fix
Fixed EmailReader not properly failing on error (CLO-2095)
  • Engine
  • Fix
Fixed insufficient error messages when reading specific CSV files (CLO-1744, CLO-1744)
  • Engine
  • Fix
HTTPConnector retains correct EOL characters when returning response. Formerly it converted it to \n in all cases. (CLO-1031)
  • Designer
  • Fix
Filter editor dialog: in() operator is now inserted correctly, avoiding previous confusing and incorrect syntax (CLO-2024)
  • Designer
  • Fix
Dragging notes with 'Alt' ('Command') on Mac OS fixed. Previously it was behaving oddly compared to Windows (CLO-1482)
  • Designer
  • Fix
Hadoop engine plugin was registered twice in Designer (CLO-2045)
  • Designer
  • Fix
Fixed sequences in Transform Editor being falsely reported as invalid (CLO-1922)
  • Designer
  • Fix
Components in graph editor can now overlap (CLO-1698)
  • Designer
  • Fix
SpreadsheetDataReader mapping editor: Fixed occasional java.lang.IndexOutOfBoundsException
  • Designer
  • Fix
Fixed increase/decrease phase feature not always assigning correct phase numbers (CLO-1632)
  • Server
  • Fix
Fixed clover.war deployment failing on Oracle WebLogic Server 11g 64-bit. See linked issue for details (CLO-2013)
  • Server
  • Fix
Fixed "Test" button in Launch Services when running on Weblogic (CLO-2190)

Fixed issues in 3.4.2

  • Engine
  • Fix
EmailSender now properly initializes connection when it needs it (at phase start) (CLO-1137)
  • Designer
  • Fix
Export graph to HTML - fixed missing international characters; XHTML validation minor fixes; file not getting closed on Windows (CLO-1159, CLO-1148)
  • Profiler/Server
  • Fix
Fixed "Closed connection" problem with long running Profiler jobs on the Server (CLO-1613)
  • Designer
  • Fix
Fixed Edit component freeze when editing values (Linux-gtk) (CLO-1605)
  • Designer
  • Fix
Fixed "stop view data" action - sometimes freezing Designer on Mac OS (CLO-1454)
  • Designer
  • Fix
Fixed enhanced tooltips producing errors to the Error log on Linux (CLO-1239)

Fixed issues in 3.4.1

  • Server
  • Fix
Fixed jobs containing ExecuteGraph/Jobflow that would occasionally hang and could not be aborted. (CLO-976)
  • Designer
  • Fix
In the Transform editor, you can switch back to the visual three-column mapping editor after removing an invalid code from the Source tab (CLO-947)
  • Engine
  • Fix
Fixed DataIntersection dropping some records if Allow key duplicates = false
  • Engine
  • Fix
Fixed NPE occuring when data flowed from JSONReader to ExtHashJoin and both components were in the same phase (CLO-1005)
  • Engine
  • Fix
Fixed SortWithingGroups outputting more records than were on input (CLO-980)
  • Engine
  • Fix
Fixed lookup table not being able to read quoted data (CLO-937)
  • Engine
  • Fix
Fixed: A graph with DBLookupTable used in a CTL code is not restartable in batch mode (CLO-715)

New in 3.4.0 production release

  • Server
  • New
Only one cluster node needs a connection to the Server database – other nodes will connect through it (CLS-1252)
  • Cluster
  • New
New component ClusterSimpleCopy (CLO-329)
  • Cluster
  • Improvement
Improved visualization of errors in graph log (CLO-293)

Released in previous milestones

  • Server
  • Jobflow
ExecuteGraph/Jobflow – specify the number of parallel running jobs in synchronous mode ("Number of executors")
  • Engine
  • New
CTL2 wrapper for returning capturing groups from regular expressions
  • Engine
  • New
CTL function for writing data into lookup tables
  • Engine
  • New
SystemExecute – new attribute "Ignore exit value"
  • Engine
  • New
Write binary data without padding with 0
  • Server
  • New
CloverETL Server – support for Weblogic 12c application server platform
  • Profiler
  • Server
  • New
ExecuteProfilerJob – "source" and "encoding" can now be mapped from input edge
  • Designer
  • Improvement
Metadata name is now displayed in the Join key editor
  • Server
  • Fix
Inconsistent input validation in "send an email" task (CLS-1007)
  • Server
  • Fix
ExecuteGraph starting clustered job fails with "RunRecord not found" (CLS-988)
  • Server
  • Fix
When Sandbox is referred from Event Listener or Scheduler and cannot be removed, there is exception instead of message (CLS-932)
  • Server
  • Fix
Task Archivator: log files are not properly cleared on the Cluster (CLS-806)
  • Designer
  • Fix
Shared sandbox must be open in a Server project to use its resources (not needed anymore) (CLDOC-502)
  • Doc
  • Fix
The documentation to ExecuteGraph component does not contain any information about graph instantiation before the jobflow run (not needed anymore – graph is loaded lazily) (CLDOC-487)
  • Designer
  • Server
  • Fix
Transform editor mapping tab does not work with CTL import in a Server project (CLD-4281)
  • Designer
  • Fix
Set initial component ID based on its name (CLD-3866)
  • Designer
  • Fix
Mapping URL property in XMLExtract does not escape special characters (CL-2604)
  • Engine
  • Fix
ExecuteGraph component should not construct the whole TransformationGraph instance in initialization time (CL-2551)
  • Server
  • Fix
Slave gather has output connected, but it's not allowed to write to the output port! (CLS-984)
  • Server
  • Fix
Error reading zip file when no names of compressed files are given (CLS-537)
  • Designer
  • Fix
Cannot connect to two sandboxes with the same id on different nodes in non-clustered environment (CLD-3753)
  • Profiler
  • Fix
Fixed occasional crash in histogram view (DP-954)
  • Server
  • Fix
"Suspend" sandbox button fixed (CLS-1098)

Known Issues & Compatibility

  • Engine
  • Compatibility
Changed internal record counters and sequence counters to "long" instead of "integer". Jobflows that use ExecuteGraph/Jobflow and MonitorGraph/Jobflow components and map the tracking data will need to be updated. To fix this, just change "integer" to "long" in your metadata. Note that this may also affect those who implement Engine extensions, custom components, or functions. (CL-1825, CL-2652)
  • Engine
  • Compatibility
DBInputTable now distinguishes between empty strings and NULL values (CL-2748)
  • Server
  • Designer
  • Compatibility
Database connections refactored to be proxied through the Server (see above and CL-2682). This may affect those who've implemented their own connection and use the Engine's Connection interface.
  • Server
  • Compatibility
The attribute "redirectErrorOutput" in ExecuteScript can no longer be mapped from input port – it affects graph topology and thus does not make sense to be changed in runtime. (CLO-236)
  • Engine
  • Server
  • Compatibility
Changed structure of error and exception messages. This may affect those who parse the graph output logs in a certain automated way, so please check your matching rules. (CL-2710)
  • Designer
  • Compatibility
The Designer no longer supports Dictionary entries with empty names (CLD-4709)