CloverETL 4.8.0

There is a newer version available. Please visit CloverETL 4.8.1.

CloverETL 4.8.0 adds numerous improvements to Data Services (released in 4.7) and introduces an assortment of minor useful improvements.

We focused on securing Data Services, adding management of SSL contexts for individual REST jobs and adding a "sensitive" option to parameters, preventing them from showing up in Server logs, Execution History, etc. We also simplified configuration of reading input data as byte or character streams.

Expanding the security theme, we're introducing a Server login "lockout" to prevent brute-force password guessing and a new option to disable HTTP Server API (and keep only HTTPS). Even more on the security front, there's now support for Kerberos in Hadoop connection (HDFS + YARN) and Hive JDBC connections.


About this release: This is a production release. We recommend upgrading to this version from all other previous versions. Please check Installation/Upgrade and especially the Compatibility section for details.


Released Dec 13, 2017
  • Production
What are production
and milestone tracks?
Recent Releases
4.9.0-M2 Mar 29, 2018
4.8.1 Feb 26, 2018
4.7.1 Nov 28, 2017
4.6.1 July 19, 2017
4.5.0 Mar 14, 2017

See all releases

Related releases


New features in 4.8


Data Services—SSL Context Configuration

CloverETL - Data Services SSL Contexts

There's a new more accessible configuration for your Data Services running over SSL. You can deploy multiple contexts using different keys and options. And each Data Service can be either running in the default Server context or one of the custom ones.

Data Services—Sensitive parameters

CloverETL - Data Services Sensitive parameters

Input parameters of a Data Service can be marked as sensitive. They work just like normal parameters with one exception— their values are masked with an asterisk in Server logs, Executions History, etc.

Data Services—Failure notifications

CloverETL - Data Services failure notifications

Alerts and notifications can be configured for each Data Service. You can choose ranges of HTTP codes that are reported as failure, as well as failure rates to alert you only in serious cases and not bother in occasional hiccups.

Data Services—Streaming Request Body

CloverETL - Data Services Direct Body streaming

An improvement from last version, the binary (or string) body of a request can be directly read using an edge coming from Body of the input panel connected to a component with a "port:" File URL.

Server Security updates

New password lockout mechanism can be configured for the Server to prevent brute-force attacks. You can configure the number of attempts before the account gets locked and a timeout for next try.

To further secure the Server, you can now completely disable Server's HTTP API to eliminate potential API attacks.

Kerberos Supported in Hadoop Connection

Hadoop connections (HDFS and YARN) and Hive JDBC connection now support Kerberos authentication.

MongoDB—Improved Bulk Write and new Bulk Update

MongoDB 3.2 brought an improved bulk write operation and also introduced new bulk update function. We've updated CloverETL MongoDB implementation to support these new features.

Compatibility Notes

We dropped support for Apache Tomcat 6.x, added support for Tomcat 9.x and updated the Server Bundle to Tomcat 8.0.47.


Installation/Upgrade Instructions for CloverETL 4.8

To help you install or upgrade to this version, we've prepared a simple checklist:

Before You Upgrade

  • Be sure to check the "Compatibility" notes for ALL intermediary releases. We mark all changes that can potentially alter the function of your existing transformations with a "Compatibility" label. Typically, you can safely ignore most of them, as we try hard to keep as much backwards compatibility as possible. There's a comprehensive list of all releases that will help you get the information quickly.
  • Upgrade Designer and Server together. We always release Designer and Server together under a single version. It's highly recommended to upgrade Server and all Designers at the same time. Although using different versions of Designer to connect to Server might work, it is not generally supported.
  • There are no incremental patches. We don't release incremental patches. Every upgrade is in fact a full installation that, if installed over the older version, will automatically update whatever is necessary in your workspaces, sandboxes, and Server databases as needed, no user data will get overwritten.
  • Don't forget to backup. Although none of the above upgrade steps requires explicit backup, we recommend you always back up your work. The upgrade will keep all your transformations, jobflows, and configurations safe. However, as a good word of advice, it never hurts to have a backup.

Designer Upgrade

  • Download the latest version by logging into your customer account. If you lost your credentials or no longer have access there, click here to recover your password or contact our CloverCARE Support.
  • Install the new version of Designer. You can install Designer over your existing installation. The process will automatically clean up the old version. Don't worry, you will NOT lose your workspaces, graphs, and transformations. However, if you installed some additional plugins to Designer (Eclipse plugins) you might need to reinstall them. Eclipse should automatically help you do that. When you start the application, point it to your existing workspace directory. With some major releases, we may notify you about upgrading the workspace to the latest version. In such cases, you won't be able to use the workspace with previous versions—be sure to upgrade ALL Designers at once if you're sharing the workspace.
  • Activate the product on first start. You will need a new key as we issue new license keys for every new major version (e.g. from 4.6 to 4.7). If you're on our maintenance program, we automatically renew the keys for you. Just go to the download area again and copy/paste the license keys from there. If you can't find the latest keys, please contact us to renew your product maintenance.

Server Upgrade

  • Download the latest version by logging into your customer account. You'll find Server in the same list as Designer downloads. If you no longer have access there, click here to recover your password or contact our CloverCARE Support.
  • Plan for downtime. Upgrading Server requires downtime, so plan your upgrades in advance. If you're running multiple environments, upgrade the non-production installation first and run all your tests there first.
  • Follow our step-by-step Server Upgrade Guide. We've prepared detailed instructions on how to properly shut down Server and install a new one. Server will upgrade its database and sandboxes from any previous version automatically.
  • Activate the product on the login screen of Server Console. Server requires new license keys with every major version (e.g. from 4.6 to 4.7) and we automatically renew the keys for you. Just go to the download area again and copy/paste the license key from there.



Detailed list of improvements and fixes in 4.8

Data Services

Read request body as stream using an edge connected to "Body" port on the Data Service REST job Input panel		 Improvement CLO-11788 CLO-11981 4.8.0-M1
Manage multiple SSL contexts for Data Services, with different keys, URLs, etc.		 New feature CLO-11380 4.8.0-M1
Mark REST job parameters as "sensitive" to hide them from Server logs, Executions History, etc.		 New feature CLO-11780 4.8.0-M1
New filter widget in Data Services module for quickly finding a particular service		 New feature CLO-11714 4.8.0-M1
Job test page can now be used send HTTP request payloads, testing services using a "body" parameter.		 New CLO-12145 4.8.0
"Never mark endpoint as failing"—New option in Alerts and notification for ignoring a particular Data Service failures		 New CLO-12077 4.8.0
HTTP code or range of codes can be used to identify what is and what's not considered a failure in Alerts and Notificatons		 New CLO-12075 4.8.0
Executing a data service from its testing page or from Designer now stores a record in Executions History		 Improvement CLO-12045 4.8.0
State and History tab now shows a cause of a failure		 Improvement CLO-12040 4.8.0

Connectors

We now allow SELECT queries in DBExecute for specific purposes		 New feature CLO-11969 4.8.0-M1
MS Access driver updated, fixing some connecting issues ("TABLES: user lacks privileges or object not found")		 Compatibility Improvement CLO-12162 4.8.0
MongoDBWriter: Added batch number option to Ouput and Error mappings		 New CLO-12059 4.8.0
Connections to S3 now support proxy using File URL syntax for proxies		 New CLO-10185 4.8.0
MongoDBWriter: Adding support for improved Bulk Write and Bulk Update in MongoDB 3.2		 New feature CLO-11637 4.8.0-M1

Server Security

User lock-out can be configured after predefined number of failed attempts to log into Server Console to prevent brute-force attacks.		 New feature CLO-11925 4.8.0-M1
Server's HTTP API can now be disabled to allow only HTTPS connection and prevents abuse through plain HTTP.		 New feature CLO-11887 4.8.0-M1
Data Services job configured with POST method fails when explicitly calling getRequestBody() CTL function inside the job (e.g. in Mapping of GetJobInput component). This problem only appears when when data is POSTed from a web form that causes the receiving Data Service job to parse the POST body as parameters, making the body no longer available. In this case, use getRequestParameters() function instead.		 Known Issue CLO-11309 4.8.0-M1

Server UI

Improved performance of Executions History view for large amounts of records		 Improvement CLO-10347 4.8.0
Added a convenient search field to Users and Groups administration modules		 New CLO-12091 4.8.0

Hadoop Security

Support for Kerberos authentication in Hadoop HDFS, YARN connections and Hive JDBC connection		 New feature CLO-11837 4.8.0-M1

Fixes

Data Services: An error is thrown when multiple inputs are connected and response format is set to "CSV"		 Compatibility Fix CLO-11936 4.8.0-M1
Fixed Server Console showing the plain values of secure parameters under certain conditions		 Fix CLO-11931 4.8.0-M1
Fixed REST Job failing if there were both enabled and disabled components connected to the Output panel		 Fix CLO-11399 4.8.0
Fixed event listener manual task execution params not being passed		 Fix CLO-12110 4.8.0

Compatibility

Dropped support for Server on Apache Tomcat 6.x		 Compatibility New CLO-11922 4.8.0-M1
Add support for Apache Tomcat 9		 Compatibility New CLO-11924 4.8.0-M1
Updated Server All-in-one bundle with Tomcat 8.0.47		 Compatibility New CLO-11923 4.8.0-M1
Data Services: An error is thrown when multiple inputs are connected and response format is set to "CSV"		 Compatibility Fix CLO-11936 4.8.0-M1
Updated underlying smbj library (from 0.3.0 to 0.5.1)		 Compatibility Improvement CLO-12123 4.8.0
MS Access driver updated, fixing some connecting issues ("TABLES: user lacks privileges or object not found")		 Compatibility Improvement CLO-12162 4.8.0


ETL Developers (This item is most useful for developers; it either brings new functionality for transforming data or brings optimizations)

Administrators (This item is an improvement or feature that will help setup, install, administer and manage the application)

Support (This item helps staff supporting the production operation to identify potential problems or avoid such)