Navigation:  Advanced Topics > Reading and writing XML > Reading XML >

With XPathReader

You may need to steel yourself a bit now. If you do not have any intention of processing XML using XPath statements then save yourself some head-scratching and skip this section. If you know a bit about XPath then this is tricky the first time you tackle it (it was for me anyway). However, as with all things in Clover, behind it all is a very clear logic.


You will see all of the stuff described below in action in the video lower down. After digesting this, try to reproduce it yourself from scratch.


The SOAP example in the next chapter "Web Services" contains another good example of this component.


The Brief

We need to read in the CustOrders.xml file we wrote in the Writing XML section and then extract just enough data to fuel a report that should contain the following information, sorted by Company Name.


Company Name
Total Orders Placed



The Graph

This is the graph that will fulfill the above brief.




This graph will


1.Read in the XML and split it into customers and orders.
2.Join Customers and Order as on Clover dataset.
3.Aggregate to get the total orders placed by each company.
4.Sort the final dataset by company name.
5.Throw the results away (to keep things simple).




Let's start off by thinking about what Metadata we want. The XMLXPathReader will read in the XML and spit out data on one or more ports. For each record type that is being spit out  we will need one port. In our example, we have Customer data and Orders data. As this is a one to many relationship it is hopefully clear that we have to have 2 stream of data (Customers and Orders) and therefore 2 Ports.


So, we need to create Metadata for each Port.


CustomerFields : CompanyName, Country, IndexKey (more on this field in a moment)
OrderFields : OrderID (we can use any field really as we are going to do a Count on it to generate the total number of orders), CustIndexKey




You will notice that we have an IndexKey/CustIndexKey field in the respective Metadata definitions. The reason for this is that you will need to be able to tie up the Parent-Child relationships once the data has been extracted and split into the 2 ports. As it happens, we do have a link field in our data (CustomerID in the Orders) but you may very well not have such a key in the XML and so Clover can generate it for us. When we get to the Mapping in a moment you will see how this is set up.



Naming Your Metadata Fields

You can actually save yourself a lot of typing the Mapping part if you give your Metadata fields the exact same names as the elements in your XML (case sensitive, too).




This is where the work is done and the complexity lies. The Mapping property in the Component properties dialog.


[1] <Context xpath="/root/customer" outPort="0" sequenceField="IndexKey">

[2]    <Mapping xpath="./CompanyName" cloverField="CompanyName"/>

[3]    <Mapping xpath="./Country" cloverField="Country"/>

[4]        <Context xpath="orders" parentKey="IndexKey" generatedKey="CustIndexKey" outPort="1">

[5]            <Mapping xpath="OrderID" cloverField="OrderID"/>

[6]        </Context>



Let's start off by look at the two main elements here


Context - this tells Clover where to find data. The first Context element can reference as deep as you like within the XML but you then cannot extract data at any level higher than this. Subsequent Context elements can be relative to the one above it. So line 1's xpath is telling Clover to grab data from "/root/customer" within the XML. Line 4 is effectively getting data from "/root/customer/orders" as the xpath="orders" is relative.
Mapping - this tells Clover which fields to extract and then map to specific Metadata fields on the output port indicated in the Context it belongs to (the one above it). So, on Line  5, we are extracting the OrderID field and mapping it to the OrderID metadata field on port 1.



Letting it sink in

Take a while and make sure you understand every part of the Mapping. Once you do, you have mastered pretty well all you need to know. In summary


1.We are extracting the CompanyName and Country from /root/customer and mapping them to the CompanyName and Country metadata fields on port 0.
2.We then get the OrderID from each and every order for a given customer and map it to the OrderID metadata field on port 1.



The final bit of the puzzle - IndexKeys

As we mentioned earlier, all the customers will be spat out onto port 0 and lots more orders onto port 1. These are then completely separate datasets that will need to be matched up again for our report. In order to do this, Clover can generate key fields.


[1] <Context xpath="/root/customer" outPort="0" sequenceField="IndexKey"> The thing to note here is sequenceField="IndexKey". This is telling Clover to generate a sequential key and map it to the metadata field "IndexKey" in the metadata on Port 0. Each customer will be automatically assigned a unique, automatically incrementing number.
[4] <Context xpath="orders" parentKey="IndexKey" generatedKey="CustIndexKey" outPort="1"> Orders is a child of customers (there can be many orders for one customer) and so "parentKey" tells Clover to assign the "sequenceKey" of its parent (the customer that the order belongs to) to the metadata field "CustIndexKey" on port 1. If Orders itself had children, we would also use the "sequenceField" attribute here so those children could tie up with the orders.


I know that is a bit of a mouthful, but it is a) very logical and b) very powerful.



Finally, a timesaving tip

If you are careful to choose metadata field names that exactly match the XML, you can actually leave out the Mapping tags entirely. This means that we could get away with the following Mapping XML


[1] <Context xpath="/root/customer" outPort="0" sequenceField="IndexKey">

[2]    <Context xpath="orders" parentKey="IndexKey" generatedKey="CustIndexKey" outPort="1"/>




A Complex Example

If you want to see a more complex example that also has to cope with Namespaces, please jump to this section.




This video will hopefully help see this in context.





XML XPath Reading

Extract selected fields from an XML file using the XMLXpathReader component




Here is the Clover documentation if you want to refer to it at any point


User Guide (detailed)