CloverETL 3.3.0-M1 - Release Notes

CloverETL Designer

New Excel Spreadsheet Components (COMMERCIAL)

CloverETL 3.3.0-M1 - Spreadsheet components
  • New components SpreadsheetDataReader/Writer effectively replacing now obsolete XLSReader/Writer (still kept as deprecated for backward compatibility)
  • Support for XLS and XLSX formats
  • Simple template support (duplicate&insert)
  • Writing: Insert/overwrite/append modes
    • Use overwrite smartly in multiple phases to create non-homogenous outputs
  • Powerful mapping
    • Arbitrary column/row mappings
    • Fixed mapping
    • Dynamic (by name, by order) mapping for varying data sources
    • Multiline records
  • Vertical/horizontal reading/writing
  • "Form" reading/filling
  • Working with cell formats support (reading, writing)
  • Merged cells support
  • Multifile/multisheet reading/writing (partitioning)
  • Streaming mode for best performance and low memory consumption

Graph Elements Locking

CloverETL 3.3.0-M1 - Graph Element Locking
  • Graph elements like metadata, connections, parameter files, etc. can be locked
  • A lock is a simple mechanism for cooperative environments to indicate that a shared resource is not to be modified
  • Locks are not security – anyone user can release the lock
  • Locks can be augmented with a descriptive message

Metadata Grouping in Outline

CloverETL 3.3.0-M1 - Metadata Groups
  • Metadata can now be organized into groups (kind of like folders on the file system)
  • Basic bulk operations supported

Data Access: Introducing New Generic ODBC

  • There exists cases when a JDBC driver for a particular database does not exist or is inconvenient to use – most notably for MS Access or some other legacy or discontinued databases. CloverETL now enhances its support for a JDBC-ODBC bridge, further broadening the list of supported databases by those for which ODBC compatible driver exists.

Data access: Introducing MS Access Support

CloverETL 3.3.0-M1 - MS Access Support
  • Over built-in JdbcOdbc bridge

ComplexDataReader – default Selector improved with regular expression functionality (COMMERCIAL)

CloverETL 3.3.0-M1 - Regular Expressions
  • Selector now works on regular expressions
  • Rules are now ordered – top to bottom, allowing for customized rule preference
  • Compatibility with old graphs maintained by automated conversion

View Data Improvements

CloverETL 3.3.0-M1 - View Data
  • Automatic paging – no need to specify range up front
  • Truncate long fields
  • Information about more input available or end-of-data reached
  • Mac: multiline values now display correctly

Improved Log Messages

  • Identification of offending value during parsing or any other processing is now more legible and contains better identification of the field, record, etc.
  • Components with incompatible metadata on error ports now fail with an information message about how to set the metadata correctly
    • Possible incompatibility when users rely on incorrect metadata
  • Better error message for Aggregate on unsorted inputs

Tracking View

  • Can now be copied/exported using Ctrl-C

Engine Fixes

  • CTL2: Fixed datediff() function to work with milliseconds
  • FastSort: Fixed rare synchronization problem during long runs causing a deadlock (COMMERCIAL)
  • UniversalDataReader: Fixed buffer size problems when verbose mode is on with big records
  • UniversalDataReader/Writer, ComplexDataReader and ParallelReader: Explicit Quoted string and Quote character have now higher priority than setting these values in metadata (i.e. can now be properly overridden in component setting)
  • Analyzedb.bat: Paths to Clover-bundled JDBC drivers are now properly set
  • UniversalDataWriter and partitioned output – locking of file pattern is now more finely grained so that multiple graphs can run at the same time

Designer Fixes

  • If a tooltip displayed a text too long, it would hang the whole designer for unpredictable or long amount of time – now fixed
  • Fixed exception when deleting projects/refactoring
  • Fixed field size limit of 32768 bytes removed
  • DBOutputTable field mapping editor – now supports schema.table format

CloverETL Server

Support for WebLogic 10.3.5

  • Although CloverETL Server is an enterprise-class application built according to JEE specification, differences in various application servers, libraries and environment require CloverETL to be built for specific application Server. With 3.3.0.M1 we introduce the build for Weblogic 10.3.5

Web GUI - disabled users may be enabled again

  • Users are disabled-- not deleted, so they can't login, but their records in history are still visible. Since 3.3.0-M1 the user with permission to disable users may re-enable them.

Designer-server Integration - Cluster Improvements

  • Better logging of partitioned execution (nodeID in master worker log)
  • Fix of a rare exception in the Designer log. This includes a couple of improvements and bug fixes in Graph tracking view and Logs view in the Designer. In the log view, the cluster nodeID of the partitioned worker is always visible.
  • Also, tracking data is displayed properly until the graph is definitely finished.
  • And finally, when the error occurs on one or more partition workers in cluster, all of them are displayed in log view.