The W3C (the World Wide Web Consortium, http://www.w3.org) published the XML 1.0 specification on February 10th, 1998. The XML 1.1 specification was published six years later, on February 4th, 2004. In these six years, XML has taken the industry by storm. XML has become the standard for how to describe and exchange data.
The current primary development platforms, .NET and J2EE, support XML natively. All modern enterprise applications—may it be a SQL Server or Oracle database, a BizTalk Server, an Office suite or any of the other thousands of applications—support XML to various degrees. You will be pretty hard pressed to find an application that does not support or use XML.
There is more to XML then just a way of describing data. Over the years, a number of XML-based standards have emerged. Among the most fundamental ones are XSD (XML Schema Definition), XPath Query, XSLT (Extensible Stylesheet Language Transformation), SOAP (Simple Object Access Protocol), and WSDL (Web Services Description Language). All build on top of the XML syntax.
This article begins a series. The first three articles explain the fundamentals of XPath queries, XSD schemas, and XSLT. The following articles look at how well these standards are supported by the .NET framework and what are the most important namespaces and types. This series of articles is not intended as a comprehensive description of all the .NET types around XML. The goal, rather, is to provide a good introduction so you understand the XML capabilities of the .NET framework and can start leveraging them for your current .NET projects.
The Sample XML Document for the Series of Articles
This series of articles assumes that you are familiar with XML itself. The sample XML document used throughout the articles is a list of employees, which must have for each employee the first name, last name, phone number, and e-mail address, and also can provide the job title and a Web address.
<?xml version="1.0" encoding="utf-8"?> <Employees> <Employee ID="1"> <FirstName>Klaus</FirstName> <LastName>Salchner</LastName> <PhoneNumber>410-727-5112</PhoneNumber> <EmailAddress>klaus_salchner@hotmail.com</EmailAddress> <WebAddress>http://www.enterprise-minds.com</WebAddress> <JobTitle>Sr. Enterprise Architect</JobTitle> </Employee> <Employee ID="2"> <FirstName>Peter</FirstName> <LastName>Pan</LastName> <PhoneNumber>604-111-1111</PhoneNumber> <EmailAddress>peter.pan@fiction.com</EmailAddress> <JobTitle>Sr. Developer</JobTitle> </Employee> </Employees>
A quick note to editing XML documents in Visual Studio .NET 2003: When you open up a XML document, you see at the bottom of the window an "XML" and "Data" tab. The XML tab allows you to edit the XML. It also formats the XML nicely for you with indentions and all that. The Data tab parses the XML and shows a data grid. You can add new nodes by adding new rows to the data grid and entering corresponding values. Using the example data, this would be new Employees.
You also can show a document outline through the "View | Other Windows | Document Outline" menu, which shows you a hierarchy of all the nodes in the XML document. Selecting a node in the "document outline" will automatically select it in the XML document too.
The Fundamentals of XPath Queries
Once you have data in XML format, you will want to be able to navigate and search its nodes. You don't want to have to parse the whole XML document to find that there are two Employee nodes. That would be terribly inefficient. You want to apply an XPath query, which then gives you all the matching nodes. To find all Employee nodes, you would run the following XPath query:
//Employee
If you use the table above, this query will return two nodes, each representing an employee. This makes it very easy to find matching nodes and walk through the result set.
XPath is a standard (XPath 1.0) and can be found at http://www.w3.org/TR/xpath/. The working draft of XPath 2.0 can be found at http://www.w3.org/TR/2001/WD-xpath20-20011220/. Let's look more closely at some of the most common XPath operators.
| Operator | Description |
| / (child operator) | Refers to the root of the XML document when used at the beginning of the XPath expression. The child operator is used to specify the next child to select. The expression "/Employees/Employee", for, example says, start at the root of the XML document, select the Employees node and then select all the Employee child nodes within the Employees node. This will return the two Employee nodes in the sample XML document. |
| // (recursive descendant operator) | The recursive descendant operator indicates to include all descendant nodes in the search. Using the operator at the beginning of the XPath expression means you start from the root of the XML document. The expression "//LastName" starts at the root and finds any LastName node. The expression "/Employees//LastName" selects the Employees node and then, within that node, finds any LastName node. It yields the same result, but searches in a different way. |
| * (wildcard operator) | The wildcard operator finds any node. The expression "/*" finds any node under the root, which in our case is Employees. The expression "/Employees/*" means find any node under the Employees node, which in our case results with the two Employee nodes. Now what is the difference between the "/Employees" and "/Employees/*" expression? The first expression returns the Employees node but the second node finds any node under the Employees node, meaning it returns the two Employee nodes. The expression "//*" means to select any node including descendant nodes, so it will effectively list every single node in the complete XML document. |
| . (current context operator) | The current context operator refers to the current context. For example, you have written some code that selected the Employees node and then from there you run the expression "./Employee", which means it starts out from the currently selected Employees node and then selects the two Employee nodes. The expression "Employee" would yield the same result because it also starts out from the current context. Similar the expression ".//LastName" means start from the current context, the Employees node, and find any LastName node including any descendant nodes. |
| .. (parent operator) | The parent operator refers to the parent. For example, the expression "/Employees/Employee/.." returns the Employees node because you navigate down to the Employee nodes and then tell it to return its parent, which is the Employees node. |
| @ (attribute operator) | The attribute operator refers to an attribute instead of an element. The expression "/Employees//@ID" selects any ID attribute it finds under the Employees node. Now, keep in mind that the XPath query always returns the selected node. In the case of an attribute, the node below it is its value. So, the expression really two returns nodes, each with the value of each selected attribute. Furthermore, you can use the wildcard operator with attributes, so "/Employees//@*" means any attribute underneath the Employees node. |
| [ ] (filter operator) | You can apply a filter operator to filter the selected nodes. This works with attributes and with elements. The expression "/Employees/Employee[@ID=1]" returns any Employee node under the Employees node that has an ID attribute with the value one. You also can apply filters that just say that an attribute or element with that name needs to be present. For example, the expression "/Employees/Employee[WebAddress]" returns Employee nodes that have a WebAddress node as child. The expression "/Employees/Employee[FirstName='Klaus']" returns the Employee node that has a FirstName node with the value Klaus. |
| text() function | The "text()" function refers to the text of the selected node or attribute. The expression "//Employee//text()" does not list all the descendant nodes of all Employee nodes but rather the value for each descendant node. The expression "//Employee/FirstName[text()='Klaus']" lists all FirstName nodes which have a value of Klaus. |
| [ ] (collection operator) | When your expression returns more then one node with the same name, you have a collection returned. The expression "//Employee" returns two Employee nodes, which is nothing more than a collection of Employee nodes. You can apply a collection operator and specify which item from the collection you want to select. Keep in mind that the index starts at one. The expression "//Employee[2]" returns the second Employee node. The order of the selected nodes is the same order as in the XML document. You can use the collection operator in any blend, such as "//Employee[1]/LastName", which selects the first Employee node and then from there the LastName node. |
| ( ) (group operator) | The collection operator can sometimes have some odd side effects. Assume you have two Employee nodes and each has two Job nodes. What does the expression "//Employee/Job[1]" return? It returns the first Job node for each selected Employee node. But, using the group operator allows you to apply explicit precedence to selections. The expression "(//Employee/Job)[4]" first selects all Job nodes for all Employee nodes and from that collection it returns the fourth node. The group operator can only be applied to the top level expression; for example, "//Employees/(Employee/FirstName)" is invalid. |
| comment() function | Returns a comment node. The expression "//comment()" returns any comment node in the XML. The expression "/Employees/comment()" returns the comment nodes under the Employees node. |
| node() function | XML documents consist of elements, attributes, and their values, each being a node. So, in XPath expressions you can use a node() function instead of a node name or the text() function. It is a generic way to address a node. The expressions "//Employee/JobTitle/node()" and "//Employee/JobTitle/text()" return the same result, the value of both JobTitle nodes. But, "//Employee//node()" will not just return the elements but also the values of each element, because both are nodes. |
| | (union or set operator) | Returns the union of one or more location paths. The expression "//LastName | //FirstName" returns all the LastName and FirstName nodes. It preserves the order of the elements as in the XML and does not return any duplicates. The two location paths "//Employee[@ID=1] | //Employee[FirstName='Klaus']" return the same nodes but the union of these two returns just the one unique node. |
The table does not represent a complete list, but it lists the most basic operators and functions. As you can see from the samples, this already enables you to build fairly complex XPath queries. Keep in mind that the precedence of the operators is the group operator, followed by the filter operator, the child operator, and recursive descendant operator followed by the rest.
Let's spend a few minutes on a few basic terms commonly used. You have seen that I use the term XPath expression and XPath query interchangeably. An XPath expression is also called a "Location Path," because you are selecting one or more locations in your XML document and, to be able to do so, you specify the path to it. The expression "//Employees/Employee" is a path to certain locations in your XML document you want to select. It is the path to these locations. The "Location Path" consists of "Location Steps."
Your path has two steps: First, go to the Employees node and then, from there, go to all the Employee nodes. Each "Location Step" consists of an axis, a node-test, and a predicate or filter. The axis defines the relationship between your current location and the next location you specify. When no location is specified (see table below), child is assumed. The node-test is the node you want to select. You provide the name of the node or the wildcard operator for any node. The predicate or filter is any filter you apply by using the filter operator. The expression //child::Employees/child::Employee" is the same as "//Employees/Employee". Let's look closer at some of the most common axes:
| Axes | Description |
| ancestor:: | This returns all ancestors of the selected node. The expression "/Employees/Employee/JobTitle/ancestor::*" selects the JobTitle of each Employee node and then returns all its ancestors up to the root, so it returns the Employee node with the ID one, the Employee node with the ID two, and the Employees node itself. This makes it easy to find all ancestors of your nodes. But, you also can look for a specific ancestor; for example, with the expression "//JobTitle[ancestor::Employee/@ID=1]", you select all JobTitle nodes that have an ancestor node Employee with an ID attribute of the value one. |
| child:: | This axis is the default if none is specified and has the same result as the child operator. For example, the expression "/child::Employees/child::Employee" selects all the Employee nodes in the Employees node. |
| self:: | This has the same result as the current context operator. The expression "//*/self::JobTitle" first selects all nodes in the XML document and then refers to the current context and from there selects only the nodes of the type JobTitle. The expression "//*/./JobTitle" returns the same result using the current context operator. |
| ancestor-or-self:: | Same as the ancestor axis, but it includes the current context. The expression "//JobTitle/ancestor-or-self::*" lists both Employee nodes, both JobTitle nodes, and the Employees node. But, on the other hand, the expression "//JobTitle/ancestor::*" does not return the two JobTitle nodes. |
| attribute:: | The attribute axis returns the same as the attribute operator; it walks the attributes instead of the elements. The expression "//attribute::*" as well as "//@*" return all the attributes in your XML document. |
| descendant:: | The descendant operator returns the same as the recursive descendant operator. It returns all descendant nodes. The expression "//Employee/descendant::*" as well as "//Employee//*" return all descendant nodes under the two Employee nodes. |
| descendant-or-self:: | Same as the descendant axis but it includes the current context. So, the expression "//Employee/descendant-or-self::*" lists both Employee nodes, including all its descendant nodes. But, on the other hand, the expression "//Employee/descendant::*" does not return the two Employee nodes. |
| following:: | Selects all the nodes following the current context node, which includes all siblings (nodes at the same hierarchy level) and its children but not their descendants. The expression "//Employee[@ID=1]/following::*" selects the Employee node with the attribute ID value of one and then returns all its following Employee nodes, in our case only the one with the attribute ID value of two, and its children, but not the descendants of the children. |
| following-sibling:: | Returns the same as the following axis but includes only siblings and no children and not their descendants. So, the expression "//Employee[@ID=1]/following-sibling::*" returns only the Employee node with the attribute ID value of two. |
| preceding:: | Selects all the nodes preceding the current context, which includes all siblings (nodes at the same hierarchy level), and its children but not its descendants. The expression "//Employee[@ID=2]/preceding::*" selects the Employee node with the attribute ID value of two and then returns all its preceding Employee nodes—in our case, only the one with the attribute ID value of one, and its children but not the descendants of the children. |
| preceding-sibling:: | Returns the same as the preceding axis but includes only siblings and no children and not its descendants. So, the expression "//Employee[@ID=2]/preceding-sibling::*" returns only the Employee node with the attribute ID value of one. |
The axes give you more control over which nodes you want to select. It makes it easy to find ancestors, descendants, following, and preceding nodes. Unfortunately, you cannot not run XPath queries against an XML document from within the Visual Studio .NET 2003 IDE. The overall support of XPath is rather weak in Visual Studio.
You can, however, use commercial tools such as XML Spy from Altova (http://www.altova.com), Stylus Studio from Sonic Software (http://www.stylusstudio.com), or as I use a simple freeware tool called "Visual XPath" (http://weblogs.asp.net/nleghari/articles/27951.aspx). It is very useful to use such tools so you can experiment with XPath and easily create the right XPath query you need plus visually verify that you got the expected result. Here are a few more examples to study:
Sample 1
/descendant::JobTitle[ancestor::Employee/@ID=2]
Selects all descendant JobTitle nodes that have as an ancestor an Employee node with the attribute ID value of two. The result will be as follows:
<JobTitle> Sr. Developer </JobTitle>
Sample 2
//Employee[WebAddress and LastName='Salchner']/attribute::ID
Selects all descendant Employee nodes that have a WebAddress child node and a LastName child node with the value Salchner. From the result, it selects the ID attribute, which returns the values of the selected attributes. The result returns 1, the value of the selected ID attribute.
Sample 3
//JobTitle[ancestor::*/@ID=1]
Selects all descendant JobTitle nodes that have any ancestor node with an attribute ID value of one. The result is as follows:
<JobTitle> Sr. Enterprise Architect </JobTitle>
Understanding the base operators, functions, and axes allows you to build very powerful XPath queries and easily select the XML elements, attributes, or nodes you require. In your filter, you can use a number of boolean and comparison expressions as well as string functions. Here is a list of the most common ones:
| Expression or Function | Description |
| and (logical and) | Allows you to have two or more logical 'and' conditions in your filter. For example, the expression "//Employee[FirstName='Klaus' and LastName='Salchner']" checks for both the FirstName and LastName. |
| or (logical or) | Allows you to have two or more logical 'or' conditions in your filter. The expression "//Employee[FirstName='Klaus' or FirstName='Peter']" checks for nodes that have as FirstName either Klaus or Peter. |
| = (equal operator) | For example, "//FirstName[.='Klaus']". |
| != (unequal operator) | For example, "//FirstName[.!='Klaus']". |
| not() (not operator) | For example, "//Employee[not(FirstName='Klaus')]". |
| "<=" or "<=" (less then equal operator) | For example, "//Employee[@ID<=1]". |
| "<" or "<" (less then operator) | For example, "//Employee[@ID<2]". |
| ">=" or ">=" (greater equal operator) | For example, "//Employee[@ID>=2]". |
| ">" or ">" (greater operator) | For example, "//Employee[@ID>2]". |
| position() | Returns the position or index relative to all the selected nodes. The expression "//Employee[position()=1]" returns the first Employee node. |
| last() | Returns the position or index of the last of all selected nodes. The expression "//Employee[position()=last()]" returns the last Employee node. |
| concat() | Concatenates two strings together. You can specify two string literals or actual node names. The following expression, //Employee[concat(FirstName,LastName)=concat('Klaus','Salchner')], concatenates the FirstName and LastName together and compares them to the concatenated string literals 'Klaus' and 'Salchner'. All matching nodes are returned. |
| contains() | Checks whether the first string contains the second string. You can specify node names or string literals. The following expression, "//Employee[contains(FirstName,'Kl')]", returns all nodes containing the string 'Kl' in the FirstName node. |
| starts-with() | Checks whether the first string starts with the second string. You can specify node names or string literals. The following expression, "//Employee[starts-with(FirstName,'Kl')]", returns all nodes starting with the string 'Kl' in the FirstName node. |
| substring-after() | Checks whether the first string contains the second string and returns the remaining string after the first occurrence. The expression "//Employee[substring-after(FirstName,'Kl')='aus']" first looks whether the string 'Kl is included in the FirstName node and returns the remaining string after the first occurrence. The FirstName node containing 'Klaus' contains the string 'Kl' and the remaining string returned is 'aus'. It selects all nodes that match this criteria. |
| substring-before() | Checks whether the first string contains the second string and returns the preceding string before the first occurrence. The expression "//Employee[substring-before(FirstName,'aus')='Kl']" first looks whether the string 'aus is included in the FirstName node and returns the preceding string before the first occurrence. The FirstName node containing 'Klaus' contains the string 'aus' and the preceding string returned is 'Kl'. It selects all nodes which match this criteria. |
| substring() | Returns a sub-string starting at the position specified and with the number of characters specified. This allows you to cut out a sub-string from the specified string. You can specify a node or string literal. The expression "//Employee[substring(FirstName,2,3)='lau']" takes the node FirstName and returns a sub-string starting at position two for three characters in length. The FirstName node containing 'Klaus' returns the sub-string 'lau'. It selects all nodes that match this criteria. |
| string-length() | Returns the length of a string. You can give a node name or string literal. The expression "//Employee[string-length(LastName)>=8]" returns all nodes that have a LastName node of greater or equal of eight characters. |
The table does not contain a complete list but explains the most common expression and functions. For a complete list, please refer to the XPath standard.
Summary
XPath queries are a very efficient and powerful way to query and navigate your XML documents. The operators, axes, and functions provided by XPath allow you to create very simple and complex queries. This will always be more efficient then navigating the XML documents on your own. XPath queries are also very important in writing XSLT transformations so you can select the right data to output and format. For a complete reference, refer to the W3C Web site or your MSDN library.
The next article in this series will explain the fundamentals of XSD schemas. If you have comments to this article, please contact me at klaus_salchner@hotmail.com or post at the bottom of this article. I want to hear if you learned something new. Contact me if you have questions about this topic or article.
About the Author
Klaus Salchner has worked for 14 years in the industry, nine years in Europe and another five years in North America. As a Senior Enterprise Architect with solid experience in enterprise software development, Klaus spends considerable time on performance, scalability, availability, maintainability, globalization/localization, and security. The projects he has been involved in are used by more than a million users in 50 countries on three continents.
Klaus calls Vancouver, British Columbia his home at the moment. His next big goal is running in the New York marathon in 2005. Klaus is interested in guest speaking opportunities or as an author for .NET magazines or Web sites. He can be contacted at klaus_salchner@hotmail.com or http://www.enterprise-minds.com.
