CloverETL 4.8.0 (Milestone 1)

This milestone release is no longer supported.
Please go to CloverETL Full 4.8.0 Release Notes.

CloverETL 4.8.0 Milestone 1 adds improvements to Data Services (released in 4.7) and introduces an assortment of minor useful improvements.

We focused on securing Data Services, adding management of SSL contexts for individual REST jobs and adding a "sensitive" option to parameters, preventing them from showing up in Server logs, Execution History, etc. We also simplified configuration of reading input data as byte or character streams.

Expanding the security theme, we're introducing a Server login "lockout" to prevent brute-force password guessing and a new option to disable HTTP Server API (and keep only HTTPS). Even more on the security front, there's now support for Kerberos in Hadoop connection (HDFS + YARN) and Hive JDBC connections.


About this release:

This is a milestone release that we publish as a preview of the upcoming CloverETL 4.8 version. You can expect the production-ready release at the end of 2017. Milestone versions are not recommended as a production upgrade. However, some clients successfully run applications using the milestone releases without trouble.


Released Nov 14, 2017
  • Milestone
Recent Releases
4.8.0 Dec 13, 2017
4.7.1 Nov 28, 2017
4.7.0 Oct 10, 2017
4.6.1 July 19, 2017
4.5.0 Mar 14, 2017
4.4.1 Feb 14, 2016
4.3.1 Nov 10, 2016

See all releases

New features in 4.8 Milestone 1


Data Services—SSL Context Configuration

CloverETL - Data Services SSL Contexts

There's a new more accessible configuration for your Data Services running over SSL. You can deploy multiple contexts using different keys and options. And each Data Service can be either running in the default Server context or one of the custom ones.

Data Services—Sensitive parameters

CloverETL - Data Services Sensitive parameters

Input parameters of a Data Service can be marked as sensitive. They work just like normal parameters with one exception— their values are masked with an asterisk in Server logs, Executions History, etc.

Data Services—Streaming Request Body

CloverETL - Data Services Direct Body streaming

An improvement from last version, the binary (or string) body of a request can be directly read using an edge coming from Body of the input panel connected to a component with a "port:" File URL.

Server Security updates

New password lockout mechanism can be configured for the Server to prevent brute-force attacks. You can configure the number of attempts before the account gets locked and a timeout for next try.

To further secure the Server, you can now completely disable Server's HTTP API to eliminate potential API attacks.

Kerberos Supported in Hadoop Connection

Hadoop connections (HDFS and YARN) and Hive JDBC connection now support Kerberos authentication.

MongoDB—Improved Bulk Write and new Bulk Update

MongoDB 3.2 brought an improved bulk write operation and also introduced new bulk update function. We've updated CloverETL MongoDB implementation to support these new features.

Compatibility Notes

We dropped support for Apache Tomcat 6.x, added support for Tomcat 9.x and updated the Server Bundle to Tomcat 8.0.47.


Installation/Upgrade Instructions for CloverETL 4.8 Milestone 1

To help you install or upgrade to this version, we've prepared a simple checklist:

Before You Upgrade

  • This is a milestone release. We recommend you install this version on a test Server first and use it as a preview of upcoming features and for development. Milestone releases are not supported for production use.

  • Be sure to check the "Compatibility" notes for ALL intermediary releases. We mark all changes that can potentially alter the function of your existing transformations with a "Compatibility" label. Typically, you can safely ignore most of them, as we try hard to keep as much backwards compatibility as possible. There's a comprehensive list of all releases that will help you get the information quickly.
  • Upgrade Designer and Server together. We always release Designer and Server together under a single version. It's highly recommended to upgrade Server and all Designers at the same time. Although using different versions of Designer to connect to Server might work, it is not generally supported.
  • There are no incremental patches. We don't release incremental patches. Every upgrade is in fact a full installation that, if installed over the older version, will automatically update whatever is necessary in your workspaces, sandboxes, and Server databases as needed, no user data will get overwritten.
  • Don't forget to backup. Although none of the above upgrade steps requires explicit backup, we recommend you always back up your work. The upgrade will keep all your transformations, jobflows, and configurations safe. However, as a good word of advice, it never hurts to have a backup.

Designer Upgrade

  • Download the latest version by logging into your customer account. If you lost your credentials or no longer have access there, click here to recover your password or contact our CloverCARE Support.
  • Install the new version of Designer. You can install Designer over your existing installation. The process will automatically clean up the old version. Don't worry, you will NOT lose your workspaces, graphs, and transformations. However, if you installed some additional plugins to Designer (Eclipse plugins) you might need to reinstall them. Eclipse should automatically help you do that. When you start the application, point it to your existing workspace directory. With some major releases, we may notify you about upgrading the workspace to the latest version. In such cases, you won't be able to use the workspace with previous versions—be sure to upgrade ALL Designers at once if you're sharing the workspace.
  • Activate the product on first start. You will need a new key as we issue new license keys for every new major version (e.g. from 4.6 to 4.7). If you're on our maintenance program, we automatically renew the keys for you. Just go to the download area again and copy/paste the license keys from there. If you can't find the latest keys, please contact us to renew your product maintenance.

Server Upgrade

  • Download the latest version by logging into your customer account. You'll find Server in the same list as Designer downloads. If you no longer have access there, click here to recover your password or contact our CloverCARE Support.
  • Plan for downtime. Upgrading Server requires downtime, so plan your upgrades in advance. If you're running multiple environments, upgrade the non-production installation first and run all your tests there first.
  • Follow our step-by-step Server Upgrade Guide. We've prepared detailed instructions on how to properly shut down Server and install a new one. Server will upgrade its database and sandboxes from any previous version automatically.
  • Activate the product on the login screen of Server Console. Server requires new license keys with every major version (e.g. from 4.6 to 4.7) and we automatically renew the keys for you. Just go to the download area again and copy/paste the license key from there.




Detailed list of improvements and fixes in 4.8

Data Services

Read request body as stream using an edge connected to "Body" port on the Data Service REST job Input panel
Improvement CLO-11788 CLO-11981 4.8.0-M1
Manage multiple SSL conexts for Data Services, with different keys, URLs, etc.
New feature CLO-11380 4.8.0-M1
Mark REST job parameters as "sensitive" to hide them from Server logs, Executions History, etc.
New feature CLO-11780 4.8.0-M1
Filtering in Data Services module allows to quickly find particular service
New feature CLO-11714 4.8.0-M1

DBExecute updates

We now allow SELECT queries in DBExecute for specific purposes
New feature CLO-11969 4.8.0-M1

MongoDBWriter updates

Adding support for improved Bulk Write and Bulk Update in MongoDB 3.2
New feature CLO-11637 4.8.0-M1

Server Security

User lock-out can be configured after predefined number of failed attempts to log into Server Console to prevent brute-force attacks.
New feature CLO-11925 4.8.0-M1
Server's HTTP API can now be disabled to allow only HTTPS connection and prevents abuse through plain HTTP.
New feature CLO-11887 4.8.0-M1
Data Services job configured with POST method fails when explicitly calling getRequestBody() CTL function inside the job (e.g. in Mapping of GetJobInput component). This problem only appears when when data is POSTed from a web form that causes the receiving Data Service job to parse the POST body as parameters, making the body no longer available. In this case, use getRequestParameters() function instead.
Known Issue CLO-11309 4.8.0-M1

Hadoop Security

Support for Kerberos authentication in Hadoop HDFS, YARN connections and Hive JDBC connection
New feature CLO-11837 4.8.0-M1

Fixes

Data Services: An error is thrown when multiple inputs are connected and response format is set to "CSV"
Compatibility Fix CLO-11936 4.8.0-M1
Fixed Server Console showing the plain values of secure parameters under certain conditions
Fix CLO-11931 4.8.0-M1

Compatibility

Dropped support for Server on Apache Tomcat 6.x
Compatibility New CLO-11922 4.8.0-M1
Add support for Apache Tomcat 9
Compatibility New CLO-11924 4.8.0-M1
Updated Server All-in-one bundle with Tomcat 8.0.47
Compatibility New CLO-11923 4.8.0-M1
Data Services: An error is thrown when multiple inputs are connected and response format is set to "CSV"
Compatibility Fix CLO-11936 4.8.0-M1


ETL Developers (This item is most useful for developers; it either brings new functionality for transforming data or brings optimizations)

Administrators (This item is an improvement or feature that will help setup, install, administer and manage the application)

Support (This item helps staff supporting the production operation to identify potential problems or avoid such)