CloverETL 4.4.0 (Milestone 2)

For the latest production release, please visit CloverETL 4.3 release notes

This is a second milestone. You'll find more details about some of the features here in CloverETL 4.4.0 Milestone 1


CloverETL 4.4 Milestone 2 further expands Salesforce connectivity with a new SalesforceReader component that brings support for complex SOQL queries (function calls, joins, subqueries) and improves the visual SOQL query builder.

In this update, we also continued working on Remote File Listeners (the ability to watch FTP, SFTP, Samba and S3 folders) and introduced UI shortcuts for managing Task Failure Listeners.

Don’t forget to pay attention to the Compatibility section. We’re changing some defaults that potentially affect many core components. However, you shouldn’t see any negative impact, as the changes we’re making are aimed towards more predictable, and thus error-proof, behavior.



About this release: This is a milestone release that we publish as a preview of the upcoming 4.4 version. You can expect the production-ready release in mid-December 2016. Milestone versions are not recommended for a production upgrade. However, some clients successfully enjoy running our milestone releases without trouble.

Released Oct 18, 2016
  • Milestone
Recent Important Releases
4.4.0-M2 Nov 16, 2016
4.3.1 Nov 10, 2016
4.4.0-M1 Oct 18, 2015
4.3.0 Sep 8, 2016
4.2.1 Aug 18, 2016
4.2.0 Jun 9, 2016
4.1.4 Jun 1, 2016

See all releases

Try it yourself

TRY NOW


New features


Salesforce - full SOQL support

We've expanded the Salesforce component family with another reader. The new SOAP API-based SalesforceReader supports complex SOQL queries - aggregation functions, joins, aliases, and subqueries. Each read with the new component takes only a single API call (API limits) and returns results faster. However, SOAP API is limited in numbers of returned records. For large data sets, we recommend using SalesforceBulkReader - for lookups and complex queries you can now use the new SalesforceReader.

Task Failure Listeners

CloverETL Server Corporate and Cluster only

Task Failure listeners have been introduced in Milestone 1 and allow you to configure a reaction that is triggered when a "parent" listener (File Event Listener, Graph Event Listener, ...) fails for whatever reason.

This milestone simplifies the creation and management of these by integrating their controls directly into all Listeners modules (look for Failure Notification).

Compatibility

We recommend reviewing the Compatibility section for this release. We're introducing some changes to defaults that could affect your production data (only if you are relying on some strangely behaved defaults).


Installation/Upgrade Instructions for CloverETL 4.4

To help you install or upgrade to this version, we've prepared a simple checklist:

Before You Upgrade

  • Be sure to check the "Compatibility" notes for ALL intermediary releases. We mark all changes that can potentially alter the function of your existing transformations by a "Compatibility" label to make it easier for you. You can safely ignore most of them, as we try hard to keep as much backwards compatibility as we can. There's a comprehensive list of all releases that will help you get the information quickly.
  • Upgrade Designer and Server together. We always release Designer and Server together under one version. It's highly recommended to upgrade Server and Designer at the same time. Although using different version of Designer to connect to Server might work, it is not generally supported.
  • There are no incremental patches. We don't release incremental patches. Every upgrade is in fact a full installation that, if installed over older version, will update whatever necessary in your workspaces, sandboxes, and databases as needed automatically .
  • Don't forget to backup. Although none of the above upgrade steps requires explicit backup, we recommend you always back up your work. The upgrade will keep all your transformations, jobflows, and configurations safe. However, as a good word of advice - it never hurts to have a backup.

Designer Upgrade

  • Download the latest version by logging into your customer account. If you no longer have access there, no problem. Click here to recover your password or contact our CloverCARE Support.
  • Install the new version of Designer. You can install Designer over your existing installation. The process will automatically clean up the old version. Don't worry, you will NOT lose your workspaces, graphs, and transformations. However, if you installed some additional plugins to Designer (Eclipse plugins) you might need to reinstall them. Eclipse should automatically help you do that. When you start the application, point it to your existing workspace directory. With some major releases, we may notify you about upgrading the workspace to the latest version. In such cases, you won't be able to use the workspace with previous versions. Thus, be sure to upgrade all Designers if you're sharing the workspace.
  • Activate the product on first start. We issue new license keys with every major version (i.e. from 4.2 to 4.3). We automatically renew the keys for you. Just go to the download area again and copy/paste the license key from there. If you can't find the latest keys, you will need to renew your product maintenance.

Server Upgrade

  • Download the latest version by logging into your customer account. You'll find Server in the same list as Designer downloads. If you no longer have access there, no problem. Click here to recover your password or contact our CloverCARE Support.
  • Plan for downtime. Upgrading Server requires downtime, so plan your upgrades in advance. If you're running multiple environments, upgrade the non-production installation first and run all your checks there first.
  • Follow step-by-step Server Upgrade Guide. We've prepared detailed instructions on how to properly shut down Server and install a new one. Server will upgrade its database and sandboxes from any previous version automatically.
  • Activate the product on the login screen of Server Console. The license key changes with every major version (i.e. from 4.2 to 4.3) and we automatically renew the keys for you. Just go to the download area again and copy/paste the license key from there.




Detailed list of improvements and fixes in 4.4.0-M2

Salesforce Connector [ See All | Hide All ]

New Salesforce SOAP Reader (Full SOQL support, saves API calls) New Feature CLO-9171 4.4.0-M2
A new Salesforce reader component based on SOAP API.Brings full SOQL support (functions, joins, subqueries) and saves API calls limit (only 1 call per query)

Task Failure Listeners [ See All | Hide All ]

New shortcut in Event Listeners view for creating and managing failure notifications (Task Failure Listeners) Improvement CLO-9659 4.4.0-M2
On Detail tab of a Task Failure Listener, you can now see a listing of all parameters available to the graph executed as the reaction to the failure (applies only if "Task" is set to "Start a graph") Improvement CLO-9715 4.4.0-M2

Remote File Event Listeners [ See All | Hide All ]

Improved URL entry in File Listeners. A new Paste URL option will automatically parse and populate the form fields (host, port, …). Validate Accessibility button can be used to verify that CloverETL can connect to the specified remote location. Improvement CLO-9723 4.4.0-M2

UI Improvements [ See All | Hide All ]

Record counts on edges are now highlighted on selection for better visibility Improvement CLO-9569 4.4.0-M2

Fixes [ See All | Hide All ]

Fixes in Cluster error handling (nodes not respecting forced suspend, ignoring heartbeat failures, deadlocks on network failures) Fix CLO-9591 4.4.0-M2
Fixed WebServiceClient causing NullPointerException when Response Mapping was used as a bound parameter in a subgraph Fix CLO-9804 4.4.0-M2

Compatibility [ See All | Hide All ]

Changed default encoding to UTF-8 (formerly ISO-8851-1). Affects UniversalDataReader/Writer, FlatFileReader/Writer, etc. Compatibility CLO-9676 4.4.0-M2
See related ticket to see the list of affected components
SpreadsheetDataWriter no longer allows String field as a target when writing the resulting Excel (binary) data to an output port Compatibility CLO-9726 4.4.0-M2
Unified handling of decimal type in implicit vs. explicit mappings. Implicit mapping (i.e. a component without a "Mapping" attribute specified) formerly produced NULL values when source and destination DECIMAL precisions didn't match. In other words, the component treated the two fields as different (types) and didn't map them. From now on, decimal fields are always implicitly mapped despite their potentially different precisions. This prevents losing data (i.e. fields aren't silently ignored in implicit mapping anymore) but can produce runtime errors (only) for values that are larger than available destination precision. (Work in progress and logic might change for production release) Compatibility Fix CLO-9831 4.4.0-M2
Please note this applies ONLY TO IMPLICIT mappings. If you've defined any explicit mapping ($out.decimalField = $in.0.sourceDecimal) or star mapping ($out.0.* = $in.0.*), the behavior was already there - you would get a warning in the mapping dialog and you would get runtime errors for values that didn't fit.Affected components using the implicit mapping: SalesforceBulkWriter, SalesforceWriter, SalesforceReader, SalesforceBulkReader, EmailFilter, EmailSender, DBExecute, DBOutputTable, HttpConnector, TokenGather, Barrier, Profiler, Validator, MongoDBReader, MongoDBWriter, MongoDBExecute and jobflow components with mappings


ETL Developers (This item is most useful for developers; it either brings new functionality for transforming data or brings optimizations)

Administrators (This item is an improvement or feature that will help setup, install, administer, and manage the application)

Support (This item helps staff supporting the production operation to identify potential problems or avoid such)