More fancy things : Getting file names from a file
There may be occasions when you do not know the name of an input file when setting up the graph and so you need to get a list of file names from another file. This file could be created by earlier components in your graph.
Let's say that a UniversalDataWriter had written the following file names into a file called "FilesToProcess.csv".
We can get the reader to read this list of files using the UniversalDataReader. We have to set up the File URL in a special way to get this to process correctly.
The image below shows a simple graph.
The Data Generator simple generates a filename (FilesToProcess.csv) that gets sent to the output port. The Read File(s) component gets this name and then opens up that file. It then reads each of the 3 files and loads them in before passing them to the next component, which is Thrash.
You will need to have a metadata definition containing a string value with the FileName. You will also have a Customer metadata record to pass on the read customer data from the files.