We begin our discussion of the Domlette API by describing how to obtain a model of your XML documents to manipulate further. Because XML documents offer such rich functionality and exist in such varied environments, there can be a surprising amount of work that you must do to simply load your XML documents. We begin by providing a short-cut for easy access. We will then dive into the full suite of document loading utilities.

XML = """
<ham>
<eggs n='1'/>
This is the string content with <em>emphasized text</em> text
</ham>"""

from Ft.Xml import Parse

doc = Parse(XML)
# If the above XML document were located in the file
# "target.xml", we could have used `Parse("target.xml")`.
print doc.xpath('string(ham//em[1])')

import  Ft.Xml.Domlette
import warnings
def disable_warnings(*args): pass

warnings.filterwarnings("ignore", category=Warning)
warnings.showwarning = disable_warnings

XML = "<spam/>"
doc  = Ft.Xml.Domlette.NonvalidatingReader.parseString(XML)
Ft.Xml.Domlette.Print(doc)

from Ft.Xml.Domlette import NonvalidatingReader
doc = NonvalidatingReader.parseUri(
  "http://www.w3.org/2000/08/w3c-synd/home.rss")

from Ft.Xml.Domlette import NonvalidatingReader
doc = NonvalidatingReader.parseUri("file:///tmp/spam.xml")

from Ft.Xml.Domlette import NonvalidatingReader
from Ft.Lib import Uri
file_uri = Uri.OsPathToUri('spam.xml')
doc = NonvalidatingReader.parseUri(file_uri)

from Ft.Xml.Domlette import NonvalidatingReader
doc = NonvalidatingReader.parseString("<spam>eggs</spam>")

from Ft.Xml.Domlette import NonvalidatingReader
s = """<!DOCTYPE spam [ <!ENTITY eggs "eggs.xml"> ]>
<spam>&eggs;</spam>"""
doc = NonvalidatingReader.parseString(s, 'http://foo/test/spam.xml')
# during parsing, the replacement text for &eggs;
# will be obtained from http://foo/test/eggs.xml

from Ft.Xml.Domlette import EntityReader
s = """
<spam1>eggs</spam1>
<spam2>more eggs</spam2>
"""
docfrag = EntityReader.parseString(s, 'http://foo/test/spam.xml')

# ValidatingReader is a global instance
from Ft.Xml.Domlette import ValidatingReader

XML = """<!DOCTYPE a [
  <!ELEMENT a (b, b)>
  <!ELEMENT b EMPTY>
]>
<a><b/><b/></a>"""

doc = ValidatingReader.parseString(XML, "urn:x-example:valid-a")
# And of course, as with other readers, you can use `parse`, `parseUri`, and
# `parseStream` as well.

# The following document, however, is invalid because an `a` element can only
# have two `b` children according to its DTD.
XML = """<!DOCTYPE a [
  <!ELEMENT a (b, b)>
  <!ELEMENT b EMPTY>
]>
<a><b/><b/><b/></a>"""

# This throws a `Ft.Xml.ReaderException` when it encounters invalid structure,
# and does not finish parsing the document into `doc`.
doc = ValidatingReader.parseString(XML, "urn:x-example:invalid-a")

from Ft.Xml.Domlette import NonvalidatingReaderBase
reader = NonvalidatingReaderBase()
doc = reader.parseUri("http://xmlhack.com/read.php?item=1560")

from Ft.Xml import InputSource, CreateInputSource
from Ft.Xml.Domlette import NonvalidatingReader

#
# Use CreateInputSource to parse a URL:
#
isrc = CreateInputSource("http://xmlhack.com/read.php?item=1560")
doc1 = NonvalidatingReader.parse(isrc)
#
# Or a string:
#
isrc = CreateInputSource("<spam>eggs</spam>", "http://spam.com/base")
doc2 = NonvalidatingReader.parse(isrc)
#
# InputSource is a file-like object, so you can treat it as such:
#
isrc = CreateInputSource("http://xmlhack.com/read.php?item=1560")
raw_text = isrc.read()
#
# The uri/system ID you used for it is maintained
#
print isrc.uri
#
# You can also create other InputSources from URIs relative to this one
#
isrc2 = isrc.resolve("read.php?item=1703")

from Ft.Xml import InputSource
from Ft.Xml.Domlette import NonvalidatingReader

factory = InputSource.DefaultFactory
isrc = factory.fromUri("http://xmlhack.com/read.php?item=1560")
doc1 = NonvalidatingReader.parse(isrc)
#
# The factory is reusable. Here we also parse a string:
#
isrc = factory.fromString("<spam>eggs</spam>", "http://spam.com/base")
doc2 = NonvalidatingReader.parse(isrc)
#
# InputSource is a file-like object, so you can treat it as such:
#
isrc = factory.fromUri("http://xmlhack.com/read.php?item=1560")
raw_text = isrc.read()
#
# The uri/system ID you used for it is maintained
#
print isrc.uri
#
# You can also create other InputSources from URIs relative to this one
#
isrc2 = isrc.resolve("read.php?item=1703")

from Ft.Xml.Domlette import ConvertDocument
converted_document = ConvertDocument(oldDocument, documentURI=u'http://www.example.org/')

You will use a large part of the Domlette API to interact with the model of your XML documents. The implementation of this part of the API is found in the Ft.Xml.cDomlette module. This part of the API allows you to navigate around a document and modify the content of that document. It is very similar to the DOM Level 2 specification and follows some of the DOM Level 3 specification; feel free to refer to those specifications and the 4Suite API documentation for details about the intended behavior of this API. You can find brief descriptions of the methods and attributes provided by this API listed below. This API is also nearly the same as the API for xml.dom, which is bundled with Python. The node type constants are inherited directly from xml.dom.Node.

Many objects that you will work with in the Domlette API are descendents of the Domlette Node class. Documents, document fragments (of class DocumentFragment), Elements, attributes (class Attr), text (class Text), processing instructions (class ProcessingInstruction), and comments (class Comment) are all nodes; any node operations are defined on objects of these types, as well. Some operations do not make sense on some objects, however. For example, it does not make sense to add children to an attribute node.

In the DOM model of XML documents, there is a Document node which represents the starting point for the other pieces of the document. This node is not the root element of the document; rather, the Document node contains the root element as its only element child. The Document node may have other children, though, such as processing instructions and comments.

You can easily access properties of a node directly. The following properties are available on any node. These properties generally store information about the structure of the document in the near "vicinity" of the target node.

In addition to accessing the structure relative to a node, there are also a set of operations that we can perform on these structures, including a variety of operations for modifying the document. Some of these methods allow you to add new nodes in various places; note that in the DOM, only Document nodes can create new nodes. See “Methods available to Document objects” for details. The following methods are available on any node.

In addition to their behavior as nodes, Document nodes are uniquely responsible for a number of tasks. For example, only Document nodes can create other nodes. The following methods are availble only to Document nodes.

Document nodes also have a number of properties that are not found on other nodes. These properties are summarized in the following list.

Attributes (Attr objects) do not have any special methods, but they do have a few additional properties. These properties are summarized in the following list.

Since attributes can only be attached to elements, Element objects have a set of special methods for managing which attributes are attached to them. We describe these methods below.

Elements also have several properties above and beyond what they get from being Nodes. See the list below for details.

Both Text and Comment nodes are also more general CharacterData nodes in the DOM. CharacterData nodes have several additional properties and methods for managing the string data that they contain. The individual Text and Comment nodes, however, do not add any functionality to their general CharacterData parent class. You can find descriptions of the properties and methods offered by CharacterData objects below.

A few DOM actions are not "owned" by any individual document. In effect, they are general-purpose operations. They can be found in DOMImplementation objects. One such precreated instance can be conveniently found at and used from Ft.Xml.Domlette.implementation. The general methods that such a DOMImplementation object offers are listed below.

doc.xpath(u"//tagname")

Domlette comes with a couple of very fast printer functions which also go to great pains to correctly handle character encoding issues: Print and PrettyPrint. Here are some serialization examples using the Domlette printers, given a node 'node' (it doesn't have to be a document node).

from Ft.Xml.Domlette import Print, PrettyPrint

# basic serialization to sys.stdout
Print(node)

# ... with extra whitespace (indenting)
PrettyPrint(node)

# ... using a single tab, rather than 2 spaces, to indent at each level
PrettyPrint(node, indent='\t')

# serializing to a utf-8 encoded file
f = open('output.xml','w')
Print(node, stream=f)
f.close()

# ... to an iso-8859-1 encoded file
f = open('output.xml','w')
Print(node, stream=f, encoding='iso-8859-1')
f.close()

# ... to an ascii encoded string
import cStringIO
buf = cStringIO.StringIO()
Print(node, stream=buf, encoding='us-ascii')
buf.close()
s = buf.getvalue()

# Normally, output syntax (XML or HTML) is chosen based on the DOM type,
# which is automatically detected. A Domlette or XML DOM can be output in
# HTML syntax if the asHtml=1 argument is given.
PrettyPrint(node, asHtml=1)

See also: Serializing XML from DOM or Domlette documents

As an alternative to parsing a preexisting XML document, you can also build a document model, with certain limitations, from the ground up. W3C and Python DOM facilities for doing this are intended mainly for creating a temporary document whose nodes will be imported into an existing document, and while Domlette does offer a more convenient document creation method, it has many of the same limitations. However, for most documents, its capabilities should be sufficient.

The Ft.Xml.Domlette module contains a DOMImplementation instance named implementation which provides a set of methods for initializing new Documents. The implementation.createRootNode method takes a base URI argument and provides a natural approach for creating an XPath model root node. This is similar to the DOM idea of a document node and even closer to a DOM document fragment (multiple element children are allowed). The implementation.createDocument method, on the other hand, is designed to come close to the DOM interface, although its doctype argument must be None.

doc = implementation.createRootNode('file:///article.xml')

is the equivalent of

from Ft.Xml import EMPTY_NAMESPACE
doc = implementation.createDocument(EMPTY_NAMESPACE, None, None)

with the added advantage of doc.baseURI being set to 'file:///article.xml', which is not possible to set via standard DOM interfaces (the baseURI attribute is read-only).

Similarly,

from Ft.Xml import EMPTY_NAMESPACE
doc = implementation.createRootNode('file:///article.xml')
docelement = doc.createElementNS(EMPTY_NAMESPACE, 'article')
doc.appendChild(docelement)

is the equivalent of

from Ft.Xml import EMPTY_NAMESPACE
doc = implementation.createDocument(EMPTY_NAMESPACE, 'article', None)

plus doc.baseURI being set to 'file:///article.xml'.

If you want as much fidelity to the DOM API as Domlette offers, use implementation.createDocument. If you just want to create a document or other such root-level node, and never mind the strange parameters, use implementation.createRootNode.

You can easily perform XPath queries by use the xpath method for cDomlette nodes as follows:

from Ft.Xml.Domlette import NonvalidatingReader
doc = NonvalidatingReader.parseString("<spam>eggs<a/><a/></spam>")
print doc.xpath(u'//a')
print doc.xpath(u'string(/spam)')

Notice: this is nothing like W3C DOM's XPath query module. The emphasis, as usual with Domlette, is on speed, simplicity and pythonic-ness.

The API, in brief:

node.xpath(expr[, explicitNss])

• node - will be used as core of the context for evaluating the XPath

• expr - XPath expression in string or compiled form

• explicitNss - (optional) any additional or overriding namespace mappings in the form of a dictionary that maps prefixes to namespace URIs. The base namespace mappings are taken from in-scope declarations on the given node. This explicit dictionary is superimposed on the base mappings.

For additional details, see “XPath queries”.

For some users, always specifying a base URI feels like an inconvenience. Perhaps they always generate XML sources from text or streams without naturally associated URIs, and they have to figure out schemes to come up with base URIs for the parse. But there is good reason for this pickiness. Just ask one of the users who got bitten by carelessness with base URIs in practice. It's better to always put some amount of thought into base URIs when processing XML, and 4Suite encourages this.

Note that 4Suite only enforces the requirement for base URIs in cases where they are needed to make sense of a requested operation. Your document must have a valid base URI if you use external entities, XInclude, xsl:import, xsl:include, the XSLT document() function, the EXSLT exsl:document element, or any other operations that require access to an external resource. If your main use for URI resolution is XSLT import and includes, you can avoid having to give valid base URIs by using XSLT include paths.

A valid base URI starts with a scheme, such as http:. A simple name, such as "spam" is a valid relative URI reference, but not a valid base URI. Without a base URI, a relative reference is no more useful than an apartment number given without the address of the entire apartment building. Merging a base URI with a relative reference is a string operation that is undertaken in a standard manner, and is generally only useful when the base URI is hierarchical; that is, it is a URL using one of the common schemes that have slashes as path separators (e.g., http:, ftp:, gopher:, and most file: URLs). The built-in 4Suite URI resolver Ft.Lib.Uri.BASIC_RESOLVER knows how to perform such resolution.

Domlette is not a complete or fully conformant DOM implementation, but it does provide an interface very close to W3C DOM Level 2 and the corresponding Python mapping as laid out in the xml.dom API docs.

The areas of divergence are inconsequential for most users, and generally reflect decisions made in the interest of eliminating redundancy, inefficiency, and, to some degree, un-Pythonic design. Also, one of the important design principles for Domlette is that where DOM and XPath disagree, XPath wins; aside from making things more efficient to implement, this behavior is generally what people want in an XML document model.

It is also worth noting that in the interest of usability, all DOM implementations exhibit some degree of variation from the specs. Coding a completely implementation-agnostic DOM application is difficult and usually unnecessary.

To enable validation of your documents while otherwise parsing them normally with SAX, set the xml.sax.handler.feature_validation feature to True on your parser using a line similar to parser.setFeature(xml.sax.handler.feature_validation, True). The parser will then throw an xml.sax._exceptions.SAXParseException exception if it determines that the document is invalid, and it will stop parsing the document. Handlers for document components that have been parsed will be called, however. The following example illustrates these concepts.

from Ft.Xml import InputSource, Sax
factory = InputSource.DefaultFactory

XML = """<!DOCTYPE a [
  <!ELEMENT a (b, b)>
  <!ELEMENT b EMPTY>
]>
<a><b/><b/></a>"""

isrc = factory.fromString(XML, 'urn:x-example:valid-a')

class element_counter:
    def startDocument(self):
        self.scount = 0
        self.ecount = 0

    def startElementNS(self, name, qname, attribs):
        self.scount += 1

    def endElementNS(self, name, qname):
        self.ecount += 1

parser = Sax.CreateParser()
handler = element_counter()
parser.setContentHandler(handler)
# And now, to enable validation...
import xml
parser.setFeature(xml.sax.handler.feature_validation, True)
parser.parse(isrc)
print "Saw", handler.scount, "start tags"
print "Saw", handler.ecount, "end tags"

# And now we show what happens on an invalid document:
XML = """<!DOCTYPE a [
  <!ELEMENT a (b, b)>
  <!ELEMENT b EMPTY>
]>
<a><b/><b/><b/></a>"""

isrc = factory.fromString(XML, 'urn:x-example:invalid-a')
parser.parse(isrc)
print "Saw", handler.scount, "start tags"
print "Saw", handler.ecount, "end tags"
# The above document is invalid; it has one more `b` element than is
# allowed by the DTD.  The handlers have still been called for those
# parts of the document that have been parsed.

Saxlette has the ability to walk a Domlette tree, firing off events to a handler as if from a source document parse. This ability used to be too well, hidden, though, and I made an API addition to make it more readily available. This is the new Ft.Xml.Domlette.SaxWalker. The following example should show how easy it is to use:

from Ft.Xml.Domlette import SaxWalker
from Ft.Xml import Parse

XML = "<a><b/><b/></a>"

class element_counter:
    def startDocument(self):
        self.ecount = 0

    def startElementNS(self, name, qname, attribs):
        self.ecount += 1

#First get a Domlette document node
doc = Parse(XML)
#Then SAX "parse" it
parser = SaxWalker(doc)
handler = element_counter()
parser.setContentHandler(handler)
#You can set any properties or features, or do whatever
#you would to a regular SAX2 parser instance here
parser.parse() #called without any argument
print "Elements counted:", handler.ecount

Saxlette includes a convenience ContentHandler (Ft.Xml.Sax.DomBuilder) which listens for SAX events and constructs Domlette Documents.

Python's generators are special functions that can produce a series of partial results within the course of running. The calling program can start up a generator, which is suspended when a partial result is yielded, and resumed explicitly by the program when the next result is required. This capability is mirrored in the Expat parser that is the basis of Saxlette. Saxlette has a feature, FEATURE_GENERATOR which you can set on a parser object to enable generator semantics. If this feature is set, the parse() method returns an iterator. This iterator yields results set by the the SAX handlers. The handlers specify the partial results by setting the property PROPERTY_YIELD_RESULT with the value to be yielded. As an example, the following code reports the name of all attributes used in the document.

class report_attributes:
    def __init__(self, parser):
        self.parser = parser
        return

    def startElementNS(self, name, qname, attribs):
        self.parser.setProperty(Sax.PROPERTY_YIELD_RESULT, attribs)
        return

from Ft.Xml import Sax, CreateInputSource

parser = Sax.CreateParser()
parser.setFeature(Sax.FEATURE_GENERATOR, True)
handler = report_attributes(parser)
parser.setContentHandler(handler)
attribs_iterator = parser.parse(CreateInputSource('test.xhtml'))
for attribs in attribs_iterator:
     for name in attribs.keys(): print name

In SAX processing, the parser passes to the application a stream of events that represents the XML content. An important aspect of SAX is the user's ability to create SAX filters, which accept a stream of SAX events and pass on a modified stream. For example, you might use a SAX filter to take look for DOcbook sect1, sect2 etc. elements, and rename them to section elements before passing them on for further processing (presumably by a SAX handler that only understands how to deal with the latter form). You can chain SAX filters as well, and the idea behind SAX filters is usually reuse across a broad array of applications, focusing each filter they on a single task that can be cleanly separated from upstream and downstream processing. SAX filters can thus be useful building blocks for XML pipelines.

from xml import sax
from xml.sax.saxutils import XMLFilterBase
from Ft.Xml import CreateInputSource, XML_NAMESPACE as XMLNS
from Ft.Xml.Sax import SaxPrinter

XML = """<?xml version="1.0" encoding="utf-8"?>
<menu>
  <item id="A" xml:lang="en">Orange juice</item>
  <item id="A" xml:lang="es">Jugo de naranja</item>
  <item id="B" xml:lang="en">Toast</item>
  <item id="B" xml:lang="es">Pan tostada
    <note xml:lang="en">Wheat bread only, please</note>
  </item>
</menu>
"""

#Define constants for the two states we care about
ALLOW_CONTENT = 1
SUPPRESS_CONTENT = 2

class english_only_filter(XMLFilterBase):
    def __init__(self, downstream):
        XMLFilterBase.__init__(self, downstream)
        return

    def startDocument(self):
        #Set the initial state, and set up the stack of states
        self._state_stack = [ALLOW_CONTENT]
        XMLFilterBase.startDocument(self)
        return

    def startElementNS(self, name, qname, attrs):
        #Check if there is any language attribute
        lang = attrs.get((XMLNS, 'lang'))
        if lang:
            #Set the state as appropriate
            if lang[:2] == 'en':
	        self._state_stack.append(ALLOW_CONTENT)
            else:
	        self._state_stack.append(SUPPRESS_CONTENT)
        #Always update the stack with the current state
        #Even if it has not changed
        
        #Only forward the event if the state warrants it
        if self._state_stack[-1] == ALLOW_CONTENT:
            XMLFilterBase.startElementNS(self, name, qname, attrs)
        return

    def endElementNS(self, name, qname):
        self._state_stack.pop()
        #Only forward the event if the state warrants it
        if self._state_stack[-1] == ALLOW_CONTENT:
            XMLFilterBase.endElementNS(self, name, qname)
        return

    def characters(self, content):
        #Only forward the event if the state warrants it
        if self._state_stack[-1] == ALLOW_CONTENT:
            XMLFilterBase.characters(self, content)
        return

if __name__ == "__main__":
    parser = sax.make_parser(['Ft.Xml.Sax'])
    #SaxPrinter is a special SAX handler that merely writes
    #SAX events back into an XML document
    filtered_parser = english_only_filter(parser)
    handler = SaxPrinter()
    filtered_parser.setContentHandler(handler)
    filtered_parser.parse(CreateInputSource(XML))

Most SAX handlers operate as state machines, meaning they manage some variables based on the stream of events that come in, and change behavior based on these variables. english_only_filter is set up to be in one of two states: one in which content is passed on to the downstream handler, and one in which content is suppressed. This state is marked in the self._state_stack. The state is initially set to ALLOW_CONTENT, and changed to SUPPRESS_CONTENT if the filter encounters an xml:lang attribute that represents a language other than English (which can be done by checking the first two characters of the value, according to the rules of standard language codes). It has to be a stack because XML language specifications are scoped, so that in the example XML at the top of the listing the string "Pan tostada" is within the scope of the element with the attribute xml:lang="es", and so it is marked as being in Spanish. The entire note element, however, is marked as being in English by an overriding xml:lang="en" attribute.

The SAX handler is set to Ft.Xml.SaxPrinter, which channels the final SAX evenis onto a 4Suite printer which creates a serialized XML document. It's quite easy to chain filters. If you wanted the parser to send events to a filter of class some_other_filter which then passed on events to english_only_filter the relevant line would look as follows:

    filtered_parser = english_only_filter(some_other_filter(parser))

The combination of streaming parsing using Saxlette and streaming serialization using Ft.Xml.Lib.CanonicalXmlPrinter allows for very efficient XML canonicalization (c14n).

import sys
from xml import sax
from Ft.Xml import CreateInputSource
from Ft.Xml.Sax import SaxPrinter
from Ft.Xml.Lib.XmlPrinter import CanonicalXmlPrinter

parser = sax.make_parser(['Ft.Xml.Sax'])
handler = SaxPrinter(CanonicalXmlPrinter(sys.stdout))
parser.setContentHandler(handler)
parser.parse(CreateInputSource('   <a><b b="1" a="2"/></a>   '))

If you are using Domlette, as described above, the quickest and easiest way to use the XPath facility in 4Suite is the xpath() method, which any Domlette Node supports:

from Ft.Xml.Domlette import NonvalidatingReader
doc = NonvalidatingReader.parseString("<spam>eggs<a/><a/></spam>")
doc2 = NonvalidatingReader.parseString("<spam>eggs<eggs n='1'> and ham</eggs></spam>")
print doc.xpath(u'(//a)[1]')
print doc.xpath(u'string(/spam)')
print doc2.xpath(u'string(//eggs/@n)')

The line

print doc.xpath(u'(//a)[1]')

Is actually a shortcut for the following more involved construct, which is described in detail in the next section:

from Ft.Xml.XPath import Evaluate
print Evaluate(u'(//a)[1]', contextNode=doc)

This example prints three lines. The first line shows a string representation of a list containing a single element. As we see from this line, an XPath selection of nodes returns a Python list. In this case, it is a list containing a single element—the first element with a local name of a, which has no attributes and no children. The second line shows the correct string value of the selected spam element, and the third line shows the correct string value of the n attribute.

[<Element at 0xb7d10bb4: name u'a', 0 attributes, 0 children>]
eggs
1

4Suite XPath functions return results with Python types that depend on the XPath data model type of the query result. The following list shows how the five XPath result types (String, number, boolean, node-set and object) are mapped to Python types:

• XPath string: Python unicode type

• XPath number: Python float type (int or long also accepted), or instance of Ft.Lib.number.nan (for NaN) or Ft.Lib.number.inf (for Infinity)

• XPath boolean: Ft.Lib.boolean instance

• XPath node-set: Python list of Domlette nodes, in document order, with no duplicates

• XPath foreign object: any other Python object (you will very rarely encounter this case)

XPath expressions can refer to both variables and qualified names (QNames) that must be defined by the environment that is executing the XPath expression. This section describes how to use these advanced features of XPath using the 4Suite interface.

4Suite's XPath implementation uses a Domlette node as the context node for XPath operations. The following example demonstrates the use of XPath to extract content from an XML document. The document must be parsed before Xpath can be used to access it. The following example parses the XML document and explicitly sets up an XPath context to run an XPath query.

XML = """
<ham>
<eggs n='1'/>
This is the string content with <em>emphasized text</em> text
</ham>"""

from Ft.Xml import Parse
from Ft.Xml.XPath.Context import Context
from Ft.Xml.XPath import Evaluate

doc = Parse(XML)
ctx = Context(doc)
nodes = Evaluate(u'//em', ctx)

# The return value, a node set, comes back as a Python list of nodes
# which may be accessed using an iterator
for n in nodes:
    # print dir(n)
    print n.tagName
    print n.firstChild.nodeValue

XPath always requires a context for execution; a common XPath context is the root of the target document, such as we did in the above example. Think about an XPath query being executed from some location in an XML document. This location in the document is a necessary component of using XPath.

There is more to an XPath context than just the context node, but if your needs are as straightforward as that of the above example, there is an abbreviated version of the Evaluate method for this purpose. For example, the following fragment is equivalent to the two lines creating a context and evaluating the expression in the above example.

# No need to create a context object
Evaluate(u'//em', contextNode=doc)

If your source document uses XML Namespaces you will likely need to use QNames in your XPath expressions. For this to work, you'll need to introduce namespace mappings into your XPath context. For example, if the elements of our XML document above are in an XML namespace, then we must set up our context slightly differently.

XML = """<ham xmlns="http://example.com/ns#">
<eggs n='1'/>
This is the string content with <em type='bold'>emphasized Namespaced Text</em> text
</ham>"""

from Ft.Xml import Parse
from Ft.Xml.XPath.Context import Context
from Ft.Xml.XPath import Evaluate

NSS = {u'ex': u'http://example.com/ns#'}
doc = Parse(XML)
ctx = Context(doc, processorNss=NSS)
nodes = Evaluate(u'//ex:em', ctx)
for n in nodes:
    # print dir(n)
    print n.tagName
    print n.firstChild.nodeValue

You define XPath namespace prefixes through a Python dictionary (NSS in the above example) which maps these prefixes, such as 'ex' in the above example, to the appropriate namespace URI, such as 'http://example.com/ns#' in the above example. This prefix mapping is added to your XPath context using the processorNss parameter to the Context function.

In a similar way, you can also pass in variable bindings which may be used as values later in your XPath expressions. In this case, however, variables are Python tuples containing the namespace URI and local name of the variable.

ctx = Context(node, varBindings=
  {(EMPTY_NAMESPACE, u'date'): u'2003-06-20'})
Evaluate('event[@date = $date]', context=ctx)

This creates a variable in the default namespace named 'date', with a value of '2003-06-20'; this is then used for comparison with the date attribute in the Xpath expression.

XPath variables are Qnames, so you pass in variable names as namespace/local name tuples. The values can be numbers, unicode objects or boolean objects:

from Ft.Xml.XPath import boolean
ctx = Context(node, varBindings={(EMPTY_NAMESPACE, u'test'): boolean.true})

This sets the variable 'test' to the boolean value true (remember that this is for the XPath environment, not the Python one), and again this may be used as in any XSLT stylesheet.

If you only want a value once, you may of course still use string constants, as in

nodes=Evaluate(u'//testPrefix:em[@type="bold"]',ctx)

Note the quotes used? These must be balanced, hence the literal value uses double quotes.

Sometimes you want to re-use an XPath expression and namespace mapping multiple times, for efficiency and convenience. The following example shows an example of this:

from Ft.Xml.XPath.Context import Context
from Ft.Xml.XPath import Compile, Evaluate
from Ft.Xml import Parse

DOCS = ["<spam xmlns='http://spam.com'>eggs</spam>",
        "<spam xmlns='http://spam.com'>grail</spam>",
        "<spam xmlns='http://spam.com'>nicht</spam>",
       ]

# Pre-compile for efficiency and convenience
expr = Compile(u"/a:spam[contains(., 'i')]")
ctx = Context(None, processorNss={u"a": u"http://spam.com"})

i = 1
for doc in DOCS:
    doc = NonvalidatingReader.parseString(doc.encode('UTF-8'),
                                          "http://spam.com/base")
    retval = Evaluate(expr, doc, ctx)
    if len(retval):
        print "Document", i, "meets our criteria"
    i += 1

Which should display:

Document 2 meets our criteria
Document 3 meets our criteria

There is a usable XPath module in PyXML (warning: PyXML's XSLT implementation is not usable: use 4Suite if you need XSLT), but there are a lot of updates and improvements in the XPath library version in 4Suite.

If you are familiar with PyXML, you may have used a different form of imports to load in XPath and XSLT features. The imports are different under 4Suite.

Usage example:

1. PyXML usage (do not use with 4Suite):

   import xml.xslt
   import xml.xpath

2. 4Suite usage (use these imports):

   import Ft.Xml.XPath
   import Ft.Xml.Xslt

For basic XSLT transform needs, or to get started quickly, the Ft.Xml.Xslt module offers a quick way to apply transforms XML documents and get back the simple string result. Within this module, the function of interest is Transform.

XML = """
<ham>
<eggs n='1'/>
This is the string content with <em>emphasized text</em> text
</ham>"""

from Ft.Xml.Xslt import Transform
# URL for the identity transform: reproduces the input XML in the result
ID_TRANSFORM = 'http://cvs.4suite.org/viewcvs/*checkout*/4Suite/Ft/Data/identity.xslt'

result = Transform(XML, ID_TRANSFORM)
print result

# If the above XML document were located in the file
# "target.xml", we could have used `Transform("target.xml", ID_TRANSFORM)`.

#It's more efficient to redirect the processor output to an output stream.  The following does so:
import sys
result = Transform(XML, ID_TRANSFORM, output=sys.stdout)
print result

Here is the general procedure for using the Python API for XSLT processing:

1. Create an Ft.Xml.Xslt.Processor.Processor instance.

2. Prepare Ft.Xml.InputSource instances (via their factory) for the source XML and stylesheet.

3. Call the Processor's appendStylesheet method, passing it the stylesheet's InputSource.

4. Call the Processor's run method, passing it the source document's InputSource.

For input to our transform, we will use the namespaced example as in the last section.

$ cat testNS.xml
<ham xmlns="http://example.com/ns#">
<eggs n='1'/>
This is the string content with
 <em type='bold' f='2'>emphasized Namespaced Text</em>
text
</ham>

For our stylesheet, we will again use one of the simplest useful examples, the identity stylesheet.

$ cat identity.xsl
<?xml version="1.0" encoding="utf-8"?>
<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="1.0">

  <xsl:template match="@*|node()">
    <xsl:copy>
      <xsl:apply-templates select="@*|node()"/>
    </xsl:copy>
  </xsl:template>

</xsl:stylesheet>

The code below follows the processing outline, having converted the input file and stylesheet to the URI format.

from Ft.Xml.Xslt import Processor
# We use the InputSource architecture
from Ft.Xml import InputSource
from Ft.Lib.Uri import OsPathToUri  # path to URI conversions

processor = Processor.Processor()

# Prepare an InputSource for the source document
# Convert from local file to uri
srcAsUri = OsPathToUri('testNS.xml')
source = InputSource.DefaultFactory.fromUri(srcAsUri)

# Prepare an InputSource for the stylesheet
# Convert from local file to uri
ssAsUri = OsPathToUri('identity.xsl')
transform = InputSource.DefaultFactory.fromUri(ssAsUri)

processor.appendStylesheet(transform)
result = processor.run(source)

# result is a string with the serialized transform result
print result

You can call run multiple times on different InputSources. When you're done, the processor's reset method can be used to restore a clean slate (at which point you would have to append stylesheets to the processor again).

The following example uses our processor from the previous example to transform a new XML document, this one constructed manually.

XML = """<foo><bar/></foo>"""
source = InputSource.DefaultFactory.fromString(XML, 'http://example.org/foo')

result = processor.run(source)

# result is a string with the serialized transform result
print result

This code continues from the previous example to process the second document, using the same processor and stylesheet. This is a useful form when there is a requirement for server side processing of multiple input documents with a common stylesheet.

In the example below, strings are used as the source of the transform (stylesheet) and source documents, and we are careful to pass in a URI to identify each of them. In the source document, the URI is needed for resolving external entity references and XIncludes. In the stylesheet, the URI is needed for resolving document function calls, xsl:includes and xsl:imports.

If you do not provide a URI and you attempt to use any of these features, you may get an exception.

# The identity transform: duplicates the input to output
TRANSFORM = """
<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="1.0">

  <xsl:template match="@*|node()">
    <xsl:copy>
      <xsl:apply-templates select="@*|node()"/>
    </xsl:copy>
  </xsl:template>

</xsl:stylesheet>"""

SOURCE = """<spam id="eggs">I don't like spam</spam>"""

# The processor class is the core of the XSLT API
from Ft.Xml.Xslt import Processor
processor = Processor.Processor()

# We use the InputSource architecture
from Ft.Xml import InputSource

# Prepare an InputSource for the transform
transform = InputSource.DefaultFactory.fromString(TRANSFORM,
  "http://spam.com/identity.xslt")

# Prepare an InputSource for the source document
source = InputSource.DefaultFactory.fromString(SOURCE,
  "http://spam.com/doc.xml")
processor.appendStylesheet(transform)
result = processor.run(source)

# result is a string with the serialized transform result
print result

If your documents are already in the form of Domlette documents, you don't need to create InputSources for them; you can just use the Processor's appendStylesheetNode and runNode methods instead of appendStylesheet and run, respectively.

# The identity transform: duplicates the input to output
TRANSFORM = """
<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="1.0">

  <xsl:template match="@*|node()">
    <xsl:copy>
      <xsl:apply-templates select="@*|node()"/>
    </xsl:copy>
  </xsl:template>

</xsl:stylesheet>"""

SOURCE = """<spam id="eggs">I don't like spam</spam>"""

from Ft.Xml.Xslt import Processor
processor = Processor.Processor()
from Ft.Xml.Domlette import NonvalidatingReader

# Create a DOM for the transform
transform = NonvalidatingReader.parseString(TRANSFORM,
  "http://spam.com/identity.xslt")

# Create a DOM for the source document
source = NonvalidatingReader.parseString(SOURCE, "http://spam.com/doc.xml")
processor.appendStylesheetNode(transform, "http://spam.com/identity.xslt")
result = processor.runNode(source, "http://spam.com/doc.xml")
print result

If you have objects from another DOM library, you can first convert them to Domlette objects as shown in “Converting from other DOM libraries”.

You can pass in stylesheet parameters as a Python dictionary. Use the parameter names for keys. Values use the 4Suite XPath library's standard type mappings, which are described in “Type mappings”.

Parameter and variable names in XPath/XSLT are actually expanded-names, which we represent as (namespaceURI, localName) tuples. If your parameter name is in a namespace, you have to use a tuple as the mapping key. Otherwise, you may simply use a unicode string that represents the local-name part only (Ft.Xml.EMPTY_NAMESPACE is the default namespace).

Here is an example, which passes in the computed "date" parameter to the stylesheet from the program:

SRC = """<?xml version="1.0"?><dummy/>"""

STY = """<?xml version="1.0"?>
<xsl:stylesheet version="1.0"
  xmlns:xsl="http://www.w3.org/1999/XSL/Transform">

  <xsl:param name="date" select="'unknown'"/>

  <xsl:output method="xml" indent="yes" encoding="us-ascii"/>

    <xsl:template match="/">
      <result>
        <xsl:value-of select="$date"/>
      </result>
    </xsl:template>

</xsl:stylesheet>"""

from Ft.Xml import InputSource
from Ft.Xml.Xslt import Processor
import time
src_isrc = InputSource.DefaultFactory.fromString(SRC, 'http://foo/dummy.xml')
sty_isrc = InputSource.DefaultFactory.fromString(STY, 'http://foo/dummy.xsl')

proc = Processor.Processor()
proc.appendStylesheet(sty_isrc)
params = {u'date': unicode(time.asctime())}
result = proc.run(src_isrc, topLevelParams=params)
print result

4Suite honors the Associating Stylesheets with XML Documents W3C Recommendation and RFC 3023: XML Media Types. Instead of (or in addition to) using the processor's explicit APIs to establish the stylesheet to be used for the transformation, the source document may contain an xml-stylesheet processing instruction (PI) that refers to a stylesheet via a URI reference.

The xml-stylesheet PI must meet the following criteria:

• It must appear in the document prolog.

• It must contain a "type" pseudo-attribute having one of the following values:

  • application/xslt+xml

  • application/xslt

  • text/xml

  • application/xml

• It must contain an "href" pseudo-attribute that is a URI reference for the stylesheet. It will be resolved relative to the base URI of the source document that contains the xml-stylesheet PI.

This example shows a PI being used to refer to the identity stylesheet mentioned earlier

<?xml-stylesheet type="application/xslt" href="identity.xsl"?>

If you need to add to the supported media types, e.g., to add the nonstandard "text/xsl", then follow the example given in this mailing list message.

If the PI contains "alternate" and "media" pseudo-attributes, the package will do its best to handle them. See this message for details and examples.

Normally, the processor buffers all output, then returns it as a byte string. If you want to write directly to some other stream (any Python file-like object that has a write method), you can supply the stream as the optional outputStream argument to the Processor's run method. When you supply your own output stream, the run method will return None. Here is an example that writes directly to stdout:

SRC = """<?xml version="1.0"?><dummy/>"""

STY = """<?xml version="1.0"?>
<xsl:stylesheet version="1.0"
  xmlns:xsl="http://www.w3.org/1999/XSL/Transform">

  <xsl:output method="xml" indent="yes" encoding="us-ascii"/>

  <xsl:template match="/">
    <result>hello world</result>
  </xsl:template>

</xsl:stylesheet>"""

import sys
from Ft.Xml import InputSource
from Ft.Xml.Xslt import Processor

src_isrc = InputSource.DefaultFactory.fromString(SRC, 'http://foo/dummy.xml')
sty_isrc = InputSource.DefaultFactory.fromString(STY, 'http://foo/dummy.xsl')

proc = Processor.Processor()
proc.appendStylesheet(sty_isrc)
result = proc.run(src_isrc, outputStream=sys.stdout)

You also have the option of other kinds of output. Just set the writer argument of the processor's run method to an instance of an XSLT output writer, which is a handler of SAX-like events coming from the processor as it generates the result tree. 4Suite provides several writer classes for alternative output:

• If you want the XSLT output as SAX events, use an instance of Ft.Xml.Xslt.SaxWriter.SaxWriter. Give its constructor a saxHandler keyword argument that is your own PyXML SAX2 event handler.

• If you want the XSLT output as a Domlette document, use an instance of Ft.Xml.Xslt.RtfWriter.RtfWriter. Give its constructor a second argument: the base URI of the document to create. Obtain the document by calling the writer's getResult method after XSLT processing is finished.

• If you want the XSLT output as any other kind of Python DOM document, use an instance of Ft.Xml.Xslt.DomWriter.DomWriter. Give its constructor an implementation keyword argument that is your desired DOM implementation. Also try to set the ownerDoc to an existing Document node (from the same implementation) from which a base URI for the new document can be obtained.

• If you want the XSLT output in a regular file, open a file for writing then pass this file object to the proc.run as the outputStream parameter value, in the same way as the example above which used the sys.stdout file object. An example is shown below.

• If you want to make a custom output writer, just make your class extend Ft.Xml.Xslt.NullWriter.NullWriter. If it needs access to the XSLT output parameters, then the constructor should take an instance of Ft.Xml.Xslt.OutputParameters.OutputParameters, which will have the data attributes method, version, encoding, omitXmlDeclaration, standalone, doctypeSystem, doctypePublic, mediaType, cdataSectionElements, and indent, which your writer can act upon, if appropriate. See the NullWriter API documentation for further info.

Here is an example of writing to a regular Domlette document:

SRC = """<?xml version="1.0"?><dummy/>"""

STY = """<?xml version="1.0"?>
<xsl:stylesheet version="1.0"
  xmlns:xsl="http://www.w3.org/1999/XSL/Transform">

  <xsl:output method="xml" indent="yes" encoding="us-ascii"/>

  <xsl:template match="/">
    <result>hello world</result>
  </xsl:template>

</xsl:stylesheet>"""

import sys
from Ft.Xml import InputSource
from Ft.Xml.Xslt import Processor
from Ft.Xml.Xslt.DomWriter import DomWriter
from Ft.Xml.Domlette import PrettyPrint

src_isrc = InputSource.DefaultFactory.fromString(SRC, 'http://foo/dummy.xml')
sty_isrc = InputSource.DefaultFactory.fromString(STY, 'http://foo/dummy.xsl')

from Ft.Xml.Domlette import implementation as impl
domlette_writer = DomWriter(implementation=impl)

proc = Processor.Processor()
proc.appendStylesheet(sty_isrc)
proc.run(src_isrc, writer=domlette_writer)

result_doc = domlette_writer.getResult()
PrettyPrint(result_doc)

This example writes the transform output to a file. This is a variant of the earlier one. Output is written to tmp.xml.

SRC = """<?xml version="1.0"?><dummy/>"""

STY = """<?xml version="1.0"?>
<xsl:stylesheet version="1.0"
  xmlns:xsl="http://www.w3.org/1999/XSL/Transform">

  <xsl:output method="xml" indent="yes" encoding="us-ascii"/>

  <xsl:template match="/">
    <result>hello world</result>
  </xsl:template>

</xsl:stylesheet>"""

import sys
from Ft.Xml import InputSource
from Ft.Xml.Xslt import Processor

src_isrc = InputSource.DefaultFactory.fromString(SRC, 'http://foo/dummy.xml')
sty_isrc = InputSource.DefaultFactory.fromString(STY, 'http://foo/dummy.xsl')

proc = Processor.Processor()
proc.appendStylesheet(sty_isrc)

f = open('tmp.xml', mode='w')
result = proc.run(src_isrc, outputStream=f)
f.close()

There are many more options available for customizing XSLT development; see the Processor module documentation for details:

>>> from Ft.Xml.Xslt import Processor
>>> help(Processor)

4Suite provides some hooks for scenarios where the output from one transform becomes the source document for another. This is called transform chaining. The user still has to write the sequence of transform invocations in the Python API (the 4xslt command can perform chaining for the user). This section shows how.

In the following example the next transform in the chain is set from within XSLT.

# The first transform: just reproduces all para elements within a wrapper
TRANSFORM = """
<xsl:stylesheet version="1.0"
  xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
  xmlns:f="http://xmlns.4suite.org/ext"
  extension-element-prefixes="f"
>

<!-- Top level param so that user can pass in the next transform in the
     chain.  By default, use the identity transform -->
<xsl:param name="next-xslt"/>

<!-- grab just the first paras for the output -->
<xsl:template match="/">
  <parawrapper>
    <xsl:apply-templates select="//para"/>
  </parawrapper>
  <!-- Set the next transform in the chain.  You can also set to a
       hard-coded string -->
  <!-- notice that this is within a template, for instantiation -->
  <f:chain-to href="{$next-xslt}"/>
</xsl:template>

<xsl:template match="para">
  <xsl:copy-of select="."/>
</xsl:template>

</xsl:stylesheet>"""

DOC = """<doc>a<para>1</para>b<para>2</para>c</doc>"""

from Ft.Xml.Xslt import Processor
from Ft.Xml import InputSource

transform = InputSource.DefaultFactory.fromString(TRANSFORM, "urn:x-bogus:main.xslt")

IDT = u'http://cvs.4suite.org/viewcvs/*checkout*/4Suite/Ft/Data/identity.xslt'

processor = Processor.Processor()
processor.appendStylesheet(transform)
source = InputSource.DefaultFactory.fromString(DOC, "urn:x-bogus:doc.xml")
result = processor.run(source, topLevelParams={(None, 'next-xslt'): IDT})
print result

# processor.chainTo is the fully-resolved absolute URI of the next transform,
# or None if there was no f:chain-to element instantiated in the transform that
# the processor last processed.
next = processor.chainTo

processor = Processor.Processor()
processor.appendStylesheet(InputSource.DefaultFactory.fromUri(next))
source = InputSource.DefaultFactory.fromString(DOC, "urn:x-bogus:doc.xml")
result = processor.run(source)
print result

next = processor.chainTo                      # Should now be None
print "chainTo:", processor.chainTo

Note: There is not yet an API for automating the transform chain loop above. Ideas were discussed and an experiment was conducted here. If you have ideas for a good API, please submit them to the mailing list.

XSLT defines a pattern language based on XPath which is used to declare rules for matching patterns in the XML source against which to fire XSLT templates. The pattern implementation that 4Suite's XSLT library uses is also exposed as a library of its own. XSLT patterns are useful when your task is not so much to compute arbitrary information from a given node but, rather, to choose quickly from a collection of nodes the ones that meet some basic rules. This might seem a subtle difference. The following example might help illustrate it.

• XPath task: extract the class attribute from all the child elements of the context node

• XSLT pattern task: given a list of nodes, sort them into piles of those that have a class attribute and those that have a title child

The main API for pattern processing in 4Suite is Ft.Xml.Xslt.PatternList. The following is a code snippet that takes a node and returns a list of patterns it matches.

from Ft.Xml.Xslt import PatternList
from Ft.Xml.Domlette import NonvalidatingReader

# first pattern matches nodes with an href attribute
# the second matches elements with a title child
PATTERNS = ["*[@class]", "*[title]"]

# Second parameter is a dictionary of prefix to namespace mappings
plist = PatternList(PATTERNS, {})

DOC = """
<spam>
  <e1 class="1"/>
  <e2><title>A</title></e2>
  <e3 class="2"><title>B</title></e3>
</spam>"""

doc = NonvalidatingReader.parseString(DOC, "file:foo.xml")
for node in doc.documentElement.childNodes:
    # Don't forget that the white space text nodes before and after
    # e1, e2 and e3 elements are also child nodes of the spam element
    if node.nodeName[0] == "e":
        print plist.lookup(node)

The PatternList initializer takes my list of strings, which it conveniently converts to a list of compiled pattern objects. Such objects have a match method that returns a boolean value, but I don't use these methods directly in this example. The PatternList initializer also takes a dictionary that makes up the namespace mapping. In this example, we use no namespaces, so the dictionary is empty. The lookup method is applied to a selection of the children of the spam element (all the nodes whose name starts with "e", which happens to be all the element nodes). The output of listing 4 follows:

[*[attribute::class]]
[*[child::title]]
[*[attribute::class], *[child::title]]

The output is a list of the representations of the pattern objects that matched each node. Notice how the axis abbreviations have been expanded in the pattern object representation.

To define your own extension functions for XPath and XSLT, you write corresponding Python function in a module, and provide a mapping from the desired XPath function names to Python function objects (or any callables). Start with a simple example. The following is a complete module which defines a single XPath function, unichr(s) a simple example that takes a string and returns the Unicode code point number for the first character in that string.

#ord.py
from Ft.Xml.XPath import Conversions

def Ord(context, s):
    '''
    Available in XPath as ord() as defined by ExtFunctions mapping below
    Takes an object, which is coerced to string
    Returns the Unicode code point number for the first character in that string    Or returns -1 if it's an empty string
    '''
    s = Conversions.StringValue(s)  #Coerce the passed object to string
    if s:
        return ord(s[0])
    else:
        return -1

ExtFunctions = {
    (u'urn:x-4suite:x', u'ord'): Ord,
}

As this simple example illustrates, The basic way to map XPath function names to Python function objects is in dictionary named "ExtFunctions", global to the module in which the extension function is defined. The XPath/XSLT extension names are expressed as a Python tuple of two Unicode objects. If you're familiar with XPath, this is just a Python representation of an expanded name. The first item in the expanded name tuple is the namespace URI for the element, and the second is the local name. The namespace URI cannot be an empty string.

You have to actually tell the processor to load your extension modules. There are several ways to do so.

1. From Python code you can register them in a context object used for XPath processing by using the optional extModuleList to pass in a list of module objects.

2. You can also register particular functions rather than a complete module in a XPath context object using the optional extFunctionMap argument. It takes a mapping dictionary similar to the ExtFunctions dictionary shown in the above sample module.

3. If you are using the XSLT processor you can register extension functions on a processor object using the registerExtensionModules() method.

4. When using the XSLT processor you can also register individual extension functions on a processor object using registerExtentionFunction() method. It takes the namespace and localName for the extension function and the callable object that implements it).

5. In some cases the user can list extension modules using the environment variable "EXTMODULES". "EXTMODULES" is a colon-separated list of Python modules names. This works for the 4xslt command line and for Ft.Xml.XPath.Evaluate. For other APIs, use one of the other two methods, which can easily be extended to read the "EXTMODULES" variable. In general the other methods for registering extensions are preferable.

Note that extension modules will automatically be searched for XSLT extension elements as well as functions.

The following is a longer example, a module that implements two functions are. One returns the current time and the other creates a hash of the context node name:

# demo.py
import time, urlparse
from Ft.Xml.XPath import Conversions

def GetCurrentTime(context):
    '''available in XPath as get-current-time()'''
    return time.asctime(time.localtime())

def HashContextName(context, maxkey):
    '''
    available in XPath as hash-context-name(maxkey),
    where maxkey is an object converted to number
    '''
    # It is a good idea to use the appropriate core function to coerce
    # arguments to the expected type
    maxkey = Conversions.NumberValue(maxkey)
    key = reduce(lambda a, b: a + b, context.node.nodeName)
    return key % maxkey

ExtFunctions = {
    ('urn:x-4suite:x', 'get-current-time'): GetCurrentTime,
    ('urn:x-4suite:x', 'hash-context-name'): HashContextName
}

You can use this in plain XPath as follows:

from Ft.Xml.XPath.Context import Context
from Ft.Xml.XPath import Compile, Evaluate
from Ft.Xml.Domlette import NonvalidatingReader

DOC = "<spam xmlns='http://spam.com'>eggs</spam>"

ctx = Context(None, extFunctionMap=demo.ExtFunctions,
              processorNss={"a": "http://spam.com"})
expr = Compile("get-current-time()")

doc = NonvalidatingReader.parseString(DOC, "http://spam.com/base")
print Evaluate(expr, doc, ctx)

Notice that you might choose to use None for the extension function namespaces. If so, you don't need to specify the processorNss context attribute, but you might want to watch out for clashes with other extenstion function names, including the built-in library. Again, if you plan to use an extension function from within XSLT, its namespace URI must not be None.

You can use this in XSLT just as easily:

# useextfunc.py

TRANSFORM = """<?xml version="1.0"?>
<xsl:stylesheet
  xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
  xmlns:s="urn:x-4suite:x"
  version="1.0">

  <xsl:template match="/">
    <xsl:value-of select="s:get-current-time()"/>
  </xsl:template>

</xsl:stylesheet>
"""

SOURCE = """<dummy/>"""

from Ft.Xml.Xslt import Processor
processor = Processor.Processor()

# Register the extension function using method (3)
processor.registerExtensionModules(['demo'])
from Ft.Xml import InputSource
transform = InputSource.DefaultFactory.fromString(TRANSFORM, "http://foo.com")
source = InputSource.DefaultFactory.fromString(SOURCE, "http://foo.com")
processor.appendStylesheet(transform)
result = processor.run(source)
print result

For good examples of modules with extension elements, see the source code for the modules Ft.Xml.XPath.BuiltInExtFunctions, Ft.Xml.Xslt.BuiltInExtFunctions and the modules in Ft.Xml.Xslt.Exslt. The latter are especially good examples given their diversity and detailed specifications at exslt.org.

To define your own extension elements, define a class derived from Ft.Xml.Xslt.XsltElement. The module in which it is defined should have a global dictionary named "ExtElements" mapping element expanded names to element class objects.

Finally, modules containing any extension elements used must be indicated as such to the processor in one of several ways.

1. You can register all extension functions and elements in a module by using a processor object's registerExtensionModules() method.

2. You can also register individual extension elements on a processor object using registerExtensionElement() method. It takes the namespace and localName for the extension function and the callable object that implements it).

3. In some cases the user can list extension modules using the environment variable "EXTMODULES". "EXTMODULES" is a colon-separated list of Python modules names. This works for the 4xslt command line and for Ft.Xml.XPath.Evaluate. For other APIs, use one of the other two methods, which can easily be extended to read the "EXTMODULES" variable. In general the other methods for registering extensions are preferable.

Note that extension modules will automatically be searched for XPath extension functions as well as Extension elements.

There are several aspects of the extension element API worth discussing in more detail.

The class-level "content" variable specifies a content model to be enforced by the XSLT processor. If the element is used in a way that doesn't meet the specified content model, the user will get an error message. The content model is a structure that uses certain special classes, including:

• ContentInfo.Empty - matches no content at all (empty element)

• ContentInfo.Text - matches plain text content

• ContentInfo.Seq - matches the given sequence of sub-patterns

• ContentInfo.Alt - matches one of the given choice of sub-patterns

• ContentInfo.Rep - matches 0 or more repeated instances of the given sub-pattern

• ContentInfo.Rep1 - matches 0 or more repeated instances of the given sub-pattern

• ContentInfo.Opt - matches zero or one of the given sub-pattern

• ContentInfo.ResultElements - matches elements not in the XSL namespace

• ContentInfo.Instructions - matches any sequence of XSLT elements categorized as instructions in the spec

• ContentInfo.Template - matches an XSLT template body according to the spec

• ContentInfo.TopLevelElements - matches any sequence of XSLT elements categorized as top level in the spec

• ContentInfo.QName - matches a particular element by giving its namespace and node name (the prefix in the node name is only used for documentation and error messages)

So, for instance, the xsl:choose element would be described as

content = ContentInfo.Seq(
    ContentInfo.Rep1(ContentInfo.QName(XSL_NAMESPACE, 'xsl:when')),
    ContentInfo.Opt(ContentInfo.QName(XSL_NAMESPACE, 'xsl:otherwise')),
    )

The class-level "legalAttrs" variable specifies the attributes allowed or required on the element. It is a Python dictionary mapping attribute name to its specification. The specification is a class according o the type of attribute.

The following are the supported attribute classes. The parameters specified are for the initializer. Note that most general patterns have a plain variant and an attribute value template (AVT) variant:

• AttributeInfo.String - any XPath string

• AttributeInfo.StringAvt - an AVT yielding any string

• AttributeInfo.Char - any XPath string of length 1

• AttributeInfo.CharAvt - AVT version of Char

• AttributeInfo.Choice - a string which must be one of a number of given values. The values are given by a list of strings with is the first parameter

• AttributeInfo.ChoiceAvt - AVT version of Choice

• AttributeInfo.YesNo - Abbreviation for AttributeInfo.Choice ( See Oasis web site)

• AttributeInfo.YesNoAvt - AVT version of YesNo

• AttributeInfo.Number - any XPath number

• AttributeInfo.NumberAvt - AVT version of Number

• AttributeInfo.UriReference - XPath string that is syntactically a URI reference

• AttributeInfo.UriReferenceAvt - AVT version of UriReference

• AttributeInfo.Id - XPath string that is syntactically an XML ID

• AttributeInfo.IdAvt - AVT version of Id

• AttributeInfo.QName - XPath string that is syntactically an XML namespaces qualified name

• AttributeInfo.QNameAvt - AVT version of QName

• AttributeInfo.NCName - XPath string that is syntactically an XML namespaces "no colon" name

• AttributeInfo.NCNameAvt - AVT version of NCName

• AttributeInfo.Prefix - Same as NCName

• AttributeInfo.PrefixAvt - Same as NCNameAvt

• AttributeInfo.NMToken - XPath string that is syntactically an XML Name token

• AttributeInfo.NMTokenAvt - AVT version of NMToken

• AttributeInfo.QNameButNotNCName - A QName that contains a colon

• AttributeInfo.QNameButNotNCNameAvt - AVT version of QNameButNotNCName

• AttributeInfo.Token - XPath string that is syntactically an XPath name test (i.e. "foo", "ns:foo", ns:" or "")

• AttributeInfo.TokenAvt - AVT version of Token

• AttributeInfo.Expression - XPath string that is syntactically an XPath expression

• AttributeInfo.ExpressionAvt - AVT version of Expression

• AttributeInfo.StringExpression - XPath string that is syntactically an XPath expression, which would be expected to return a string value

• AttributeInfo.StringExpressionAvt - AVT version of StringExpression

• AttributeInfo.NodeSetExpression - XPath string that is syntactically an XPath expression, which would be expected to return a node set value

• AttributeInfo.NodeSetExpressionAvt - AVT version of NodeSetExpression

• AttributeInfo.NumberExpression - XPath string that is syntactically an XPath expression, which would be expected to return a number value

• AttributeInfo.NumberExpressionAvt - AVT version of NumberExpression

• AttributeInfo.BooleanExpression - XPath string that is syntactically an XPath expression, which would be expected to return a boolean value

• AttributeInfo.BooleanExpressionAvt - AVT version of BooleanExpression

• AttributeInfo.Pattern - XPath string that is syntactically an XSLY pattern

• AttributeInfo.PatternAvt - AVT version of Pattern

• AttributeInfo.Tokens - XPath string that is syntactically a space-delimited series of tokens

• AttributeInfo.TokensAvt - AVT version of Tokens

• AttributeInfo.QNames - XPath string that is syntactically a space-delimited series of QNames

• AttributeInfo.QNamesAvt - AVT version of QNames

• AttributeInfo.Prefixes - XPath string that is syntactically a space-delimited series of NCNames

• AttributeInfo.PrefixesAvt - AVT version of Prefixes

All of these classes take the following optional keyword parameters:

• description - for documentation

• default - the default value of the attribute to be used if omitted

Some examples from the XSLT spec:

xsl:output

content = ContentInfo.Empty
legalAttrs = {
    'method' : AttributeInfo.QName(),
    'version' : AttributeInfo.NMToken(),
    'encoding' : AttributeInfo.String(),
    'omit-xml-declaration' : AttributeInfo.YesNo(),
    'standalone' : AttributeInfo.YesNo(),
    'doctype-public' : AttributeInfo.String(),
    'doctype-system' : AttributeInfo.String(),
    'cdata-section-elements' : AttributeInfo.QNames(),
    'indent' : AttributeInfo.YesNo(),
    'media-type' : AttributeInfo.String(),
    }

xsl:sort

content = ContentInfo.Empty
legalAttrs = {
    'select' : AttributeInfo.StringExpression(default='.'),
    'lang' : AttributeInfo.NMTokenAvt(),
    # We don't support any additional data-types, hence no
    # AttributeInfo.QNameButNotNCName()
    'data-type' : AttributeInfo.ChoiceAvt(['text', 'number'],
                                          default='text'),
    'order' : AttributeInfo.ChoiceAvt(['ascending', 'descending'],
                                      default='ascending'),
    'case-order' : AttributeInfo.ChoiceAvt(['upper-first', 'lower-first']),
    }

xsl:number

content = ContentInfo.Empty
legalAttrs = {
    'level' : AttributeInfo.Choice(['single', 'multiple', 'any'],
                                   default='single'),
    'count' : AttributeInfo.Pattern(),
    'from' : AttributeInfo.Pattern(),
    'value' : AttributeInfo.Expression(),
    'format' : AttributeInfo.StringAvt(default='1'),
    'lang' : AttributeInfo.NMToken(),
    'letter-value' : AttributeInfo.ChoiceAvt(['alphabetic', 'traditional']),
    'grouping-separator' : AttributeInfo.CharAvt(),
    'grouping-size' : AttributeInfo.NumberAvt(default=0),
    }

Of course, it's always a good idea to use descriptions, which the above do not.

For good examples of modules with extension elements, see the source code for the modules Ft.Xml.Xslt.BuiltInExtElements and Ft.Xml.Xslt.Exslt.Common . The various modules in Ft.Xml.Xslt.Exslt have a strong diversity and make good examples, especially given their detailed specifications at exslt.org

class SpamElement(XsltElement):
    legalAttrs = {}
    def instantiate(self, context, processor):
        processor.output().startElement('title')
        processor.output().text('Life of Brian'))
        processor.output().endElement('title')
        return (context,)

def Spam(context):
    context.processor.output().startElement('title')
    context.processor.output().text('Life of Brian'))
    context.processor.output().endElement('title')
    return

stream = cStringIO.StringIO()

# Clone the current outputparameters
op = processor.writers[-1]._outputParams.clone()

# Force XML output method with XML declaration
# Output method is a qualified name, so must flag null ns
# to use standard xml method
op.method = (EMPTY_NAMESPACE, 'xml')
op.omitXmlDeclaration = "yes"

# Push the new handler to the top of the writer stack
processor.addHandler(op, stream)
processor.output().startElement('title')
processor.output().text('Life of Brian'))
processor.output().endElement('title')

# Pop back to the previous handler stream.getvalue()
# now contains the new  output
processor.removeHandler()

try:
    # Set the output to an RTF riter, which wll create an RTF for us
    processor.pushResultTree(self.baseUri)

    # The template is manifested as children of the extension element
    # node.  Instantiate each in turn
    for child in self.children:
        child.instantiate(context, processor)
# You want to be sure you re-balance the stack even in case of error
finally:
    # Retrieve the resulting RTF
    result_rtf = processor.popResult()

# Extension parameters have fully qualified names, so you must come up
# with a namespace to set them
processor.extensionParams[(SPAM_NAMESPACE, 'tstamp')] = time.time()

After importing the MarkupWriter class, you have to create a MarkupWriter object instance and then start the new Document. (See below for output options of MarkupWriter.) Remember that you are working with a streaming API. You must decide what features you want your output to have before you start to write that output.

>>> from Ft.Xml import MarkupWriter
>>> writer = MarkupWriter()
>>> writer.startDocument() 

You are now ready to add data to the new document.

There are two ways to add new elements as children of other document or element nodes.

1. When you want to add a new element that will itself have child elements, you can use the startElement/endElement method combination to signal the beginning and the ending of an element, respectively.

   writer.startElement(u'xsa')
   # other document content can be output here
   writer.endElement(u'xsa')

2. Alternatively, you can use the simpleElement method, which is a shortcut for the startElement/endElement combination and produces an element with no content or with text content (if you specify the content parameter).

   writer.simpleElement(u'xsa')

There are two ways to add attributes to elements:

1. First, you can use the attributes parameter of the startElement method. This parameter is a dictionary which maps each attribute name to the value of that attribute. If an attribute's name is in a namespace, then you must specify the name as a Python tuple, with the attribute's QName as the first member of the tuple, and the namespace URI as the second member of the tuple. For an example of this advanced syntax, see “Writing XHTML with MarkupWriter”.

   writer.startElement(u'product', attributes={u'id': u"100\u00B0"}

2. Alternatively, you can use a distinct attribute method with two parameters: the attribute's name and the attribute's value. As with the dictionary approach above, if the attribute's name is in a namespace, then the whole name should be a Python tuple.

   writer.startElement(u'product')
   writer.attribute(u'id', u"100\u00B0")

Similarly, there are two ways to add text nodes to elements.

1. First, the simpleElement method takes a content parameter, which can be used to create a single text node child of the node with the specified name.

   writer.simpleElement(u'name', content=u'Centigrade systems')

2. Alternatively, instances of the MarkupWriter class, such as writer, have a text method that inserts a single text node as the next child of the element which was last started with the startElement method and which has not yet been closed with the endElement method.

   writer.startElement(u'product')
   writer.text(u'Centigrade systems')
   writer.endElement(u'product')

MarkupWriter also allows you to insert well-formed XML entities as complete chunks in the output. This is a very convenient way to emit boilerplate XML without breaking it down into all the separate element/attribute/content bits. As such the lines:

writer.simpleElement(u'name', content=u"100\u00B0 Server")
writer.simpleElement(u'version', content=u"1.0")
writer.simpleElement(u'last-release', content=u"20030401")

Could instead be written:

writer.xmlFragment("""
<name>100° Server</name>
<version>1.0</version>
<last-release>20030401</last-release>""")

The API provides the comment and processingInstruction methods for inserting processing instructions and comments. The comment method takes a unicode string, which is the intended value of the comment. The processingInstruction method takes two unicode strings. The first is the name of the processing instruction, and the second is the value of the processing instruction. For example, the following code:

writer.comment(u"This is a processing instruction")
writer.processingInstruction(u'xml-stylesheet', u'type="text/xsl" href="akara.xsl"')

<!--This is a processing instruction-->
<?xml-stylesheet type="text/xsl" href="akara.xsl"?>

When you create a new element or an attribute, you can use namespaces. See the next program:

from Ft.Xml import MarkupWriter

writer = MarkupWriter(indent=u'yes')
writer.startDocument()

RDFNS = u"http://www.w3.org/1999/02/22-rdf-syntax-ns#"

writer.startElement(u"rdf:RDF", RDFNS)
writer.startElement(u"rdf:Description", RDFNS,
    attributes={(u'rdf:about', RDFNS): u'http://media.example.com/audio/guide.ra'})
writer.endElement(u'rdf:Description', RDFNS)
writer.endElement(u'rdf:RDF', RDFNS)

And this is the output:

<?xml version="1.0" encoding="UTF-8"?>
<rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#">
    <rdf:Description rdf:about="http://media.example.com/audio/guide.ra"/>
</rdf:RDF>

In the above example, you can see how parameters that control the output are passed into the MarkupWriter initializer, including document type info and whether to indent (pretty print).

You can pass any of the usual controls for XSLT output into the initializer this way.

The XSLT spec also defines a method parameter to choose between XML, HTML or plain text output rules, but for MarkupWriter at the moment you should stick to XML. The result of changing the method is undefined. We'll probably relax this restriction in later releases.

In order to show how to use XUpdate to make namespace-aware modifications, The following tasks will be demonstrated:

1. Add a new element in the products namespace, but using no prefix.

2. Add a new element with a prefix and in the products namespace.

3. Add a new element that is not in any namespace.

4. Add a new global attribute in the XHTML namespace.

5. Add a new global attribute in the special XML namespace.

6. Add a new attribute in no namespace.

7. Remove only the code element in the XHTML namespace

8. Remove a global attribute

9. Remove an attribute that is not in any namespace

Modification in place can always be simulated with an addition and then a removal. The following code shows how these tasks can be performed in XUpdate.

<xup:modifications version="1.0"
  xmlns:xup="http://www.xmldb.org/xupdate"
  xmlns:p="http://example.com/product-info"
  xmlns:html="http://www.w3.org/1999/xhtml"
  xmlns:xl="http://www.w3.org/1999/xlink"
>

  <!-- Task 1 -->
  <xup:append select="/products/p:product[1]">
    <xup:element
      name="launch-date"
      namespace="http://example.com/product-info"/>
  </xup:append>

  <!-- Task 2 -->
  <xup:append select="/products/p:product[1]">
    <xup:element
      name="p:launch-date"
      namespace="http://example.com/product-info"/>
  </xup:append>

  <!-- Can also be accomplished using literal result elements:
  <xup:append select="/products/p:product[1]">
    <p:launch-date/>
  </xup:append>
  -->

  <!-- Task 3 -->
  <xup:append select="/products/p:product[1]">
    <xup:element name="island"/>
  </xup:append>

  <!-- Can also be accomplished using literal result elements:
  <xup:append select="/products/p:product[1]">
    <island/>
  </xup:append>
  -->

  <!-- Task 4 -->
  <xup:append select="/products/p:product/p:description/html:div">
    <xup:attribute name="global"
      namespace="http://www.w3.org/1999/xhtml">spam</xup:attribute>
  </xup:append>

  <!-- Task 5 -->
  <xup:append select="/products/p:product/p:description/html:div">
    <xup:attribute name="xml:lang">en</xup:attribute>
  </xup:append>

  <!-- Task 6 -->
  <xup:append select="/products/p:product/p:description/html:div">
    <xup:attribute name="class">eggs</xup:attribute>
  </xup:append>

  <!-- Task 7 -->
  <xup:remove select="//html:code"/>

  <!-- Task 8 -->
  <xup:remove select="/products/p:product/p:description/html:div/ref/@xl:href"/>

  <!-- Task 9 -->
  <xup:remove select="/products/p:product[1]/@id"/>

</xup:modifications>

If you're familiar with XSLT, then you'll see the resemblance of XUpdate at first glance. The envelope element for modifications expressed in XUpdate is xup:modifications, similar to xsl:transform or xsl:stylesheet. The namespace declarations on this element assign prefixes for use in the XUpdate script and have no connection to the prefixes used in the document being modified (the source document), even though they happen to be the same. If you want to access elements in a namespace declared as the default in the source document, then just as in XSLT you must declare and use a prefix for the namespace in the XUpdate script.

Each modification request is expressed as an XUpdate instruction. This example demonstrates xup:append and xup:remove. There are other instructions providing types of modification such as xup:insert-before xup:update and there are also control constructs such as xup:if, which is similar to xsl:if. Instructions usually have a select attribute containing an XPath expression that specifies the node to be used as a reference for modification. In the case of xup:append, select specifies a node after which some new XML will be appended. In the case of xup:remove, select identifies nodes to be removed. When an instruction needs to specify a chunk of XML to be used in the modification it is expressed as the content of the instructions in a similar fashion to XSLT templates. In the case of xup:append this template expresses the chunk of XML to be inserted into the document. In order to generate elements and attributes XUpdate provides output instructions such as xup:element and xup:attribute, which are very similar to their XSLT equivalents. In another idea borrowed from XSLT, XUpdate allows you to create element by placing literal result elements in the templates. If you'd like to get a closer look at XUpdate, the best way is by browsing the very clear examples in the XUpdate Use Cases compiled by Kimbro Staken. The following listing is a Python code that can be used to apply an XUpdate script. It's a simplified version of the code for the 4xupdate command line.

import sys
from Ft.Xml import XUpdate
from Ft.Xml import Domlette, InputSource
from Ft.Lib import Uri

# Set up reader objects for parsing the XML files
reader = Domlette.NonvalidatingReader
xureader = XUpdate.Reader()

# Parse the source file
source_uri = Uri.OsPathToUri(sys.argv[1], attemptAbsolute=1)
source = reader.parseUri(source_uri)

# Parse the XUpdate file
xupdate_uri = Uri.OsPathToUri(sys.argv[2], attemptAbsolute=1)
isrc = InputSource.DefaultFactory.fromUri(xupdate_uri)
xupdate = xureader.fromSrc(isrc)

# Set up the XUpdate processor and run against the source file
# The Domlette for the source is modified in place
processor = XUpdate.Processor()
processor.execute(source, xupdate)

# Print the updated DOM node to standard output
Domlette.Print(source)

Notice the use of Uri.OsPathToUri to convert file system paths to proper URIs for use in 4Suite. I strongly recommend this convention as one way to help minimize confusion between file specifications and URIs -- the basis of many frequently asked questions. The XUpdate.Processor class defines the environment for running XUpdate commands and execute() is the method that actually kicks off the processing. It operates on a Domlette instance, modifying it in place (so be careful when using using XUpdate in this way). The updated document object is printed to standard output using Domlette.Print.

The following snippet illustrates how to run the test script, and the output result.

$ python listing4.py products.xml listing3.xup
<?xml version="1.0" encoding="UTF-8"?>
<products xmlns:p="http://example.com/product-info"
xmlns:html="http://www.w3.org/1999/xhtml"
xmlns:xl="http://www.w3.org/1999/xlink"
>
  <product xmlns="http://example.com/product-info">
    <name xml:lang="en">Python Perfect IDE</name>
    <description>
      Uses mind-reading technology to anticipate and accommodate
      all user needs in Python development.  Implements all
       features though
      the year 3000.  Works well with <code>1166</code>.
    </description>
  <launch-date/><p:launch-date/><island/></product>
  <p:product id="1166">
    <p:name>XSLT Perfect IDE</p:name>
    <p:description>
      <p:code>red</p:code>
      <html:code>blue</html:code>
      <html:div global="spam" class="eggs" xml:lang="en">
        <ref xl:type="simple">A link</ref>
      </html:div>
    </p:description>
  </p:product>
</products>

XML Inclusions (XInclude) is a W3C Recommendation that provides XML document authors with a robust way of supporting document modularity via the use of transclusions (inclusions by reference). Such modularity would otherwise require using references to external entities declared in a DTD, a system which has various limitations inherited from SGML.

Unlike XML's built-in entity-reference system, the processing of XIncludes is, fundamentally, an XML Infoset transformation, not strictly an operation performed on the serialized (textual) form of a document. Therefore, there is no requirement for when and where XInclude processing should occur; it could happen at parse time if the parser supports it, or could occur on an already-parsed document.

XInclude references consist of two special elements that are placed in the XML document into which external content is to be included: <include> and <fallback>, both in the namespace http://www.w3.org/2001/XInclude. When processed, these elements are replaced with the content they reference, which can be XML or any other text.

4Suite supports XInclude processing only at parse time, as an optional feature of the Domlette readers. It is turned on by default, so if you want to suppress it, you must use the full parsing API — not the Ft.Xml.Parse and Ft.Xml.CreateInputSource convenience functions — and set the parameter processIncludes to False either when creating an InputSource or when calling the parseString, parseUri, or parseStream method of the Domlette reader.

The following example includes one section stub into a larger article but has to use the fallback for the second section stub, where resolution fails. “Document using XInclude” lists the contents of the file article.xml, which references two sections using XInclude and provides a fallback for each in case they fail to load. “Section to be included” lists the contents of section1.xml, but this example purposefully does not provide a section2.xml in order to illustrate the fallback behaviour. “Loading the document” lists the Python code used to parse and print this document; note that XInclude processing is done automatically by default.

<article>
  <title>My important article</title>
  <xi:include href="section1.xml" xmlns:xi="http://www.w3.org/2001/XInclude">
    <xi:fallback><!-- Section 1 failed to load! --></xi:fallback>
  </xi:include>
  <xi:include href="section2.xml" xmlns:xi="http://www.w3.org/2001/XInclude">
    <xi:fallback><!-- Section 2 failed to load! --></xi:fallback>
  </xi:include>
</article>

<section>
  <title>Section 1</title>
  <!-- Write me! -->
</section>

from Ft.Xml import Parse
from Ft.Xml.Domlette import PrettyPrint
doc = Parse("article.xml")
PrettyPrint(doc)

“Self-contained example” is very similar to the above example, only this version is self-contained; the resources are stored in Python strings and resolved using a custom resolver.

article = """<article><title>My important article</title>
<xi:include href="ex:section" xmlns:xi="http://www.w3.org/2001/XInclude">
  <xi:fallback><!-- Section 1 failed to load! --></xi:fallback>
</xi:include>
<xi:include href="ex:section2" xmlns:xi="http://www.w3.org/2001/XInclude">
  <xi:fallback><!-- Section 2 failed to load! --></xi:fallback>
</xi:include>
</article>"""

section = "<section><title>Section 1</title><!-- Write me! --></section>"

from Ft.Lib.Uri import FtUriResolver, Absolutize
from Ft.Lib import UriException
from cStringIO import StringIO
class MyResolver (FtUriResolver):
  def normalize(self, uriRef, baseUri):
    return Absolutize(uriRef, baseUri)
  def resolve(self, uri):
    if uri == "ex:article":
      return StringIO(article)
    elif uri == "ex:section":
      return StringIO(section)
    else:
      raise UriException(UriException.RESOURCE_ERROR,
                         loc=uri, msg="not found, sorry")

myResolver = MyResolver()

from Ft.Xml.InputSource import InputSourceFactory
from Ft.Xml.Domlette import NonvalidatingReader, PrettyPrint
factory = InputSourceFactory(resolver=myResolver)
isrc = factory.fromUri("ex:article")
doc = NonvalidatingReader.parse(isrc)
PrettyPrint(doc)

To turn off XInclude behavior in “Self-contained example”, replace the last three lines with these three lines:

isrc = factory.fromUri("ex:article", processIncludes=False)
doc = NonvalidatingReader.parse(isrc)
PrettyPrint(doc)

“Loading the document” uses the "super simple" parsing API; we need to use the full parsing API in order to disable XInclude expansion (which, paradoxically, takes one less line):

from Ft.Xml.Domlette import NonvalidatingReader, PrettyPrint
doc = NonvalidatingReader.parseStream(file("article.xml"), processIncludes=False)
PrettyPrint(doc)

XPointer is a set of W3C specifications (one part of which is, as of 2006, still a Working Draft) that provide a means of identifying and referring to a portion of an XML document. The portion being referenced need not be contiguous, and need not constitute a well-formed general entity. XPointers were originally intended to be used in the fragment component of a URI or IRI (the fragment being the part after "#"), but the specifications actually place no restrictions on where they can be used.

An example of an XPointer embedded in a URI would be

http://example.com/inventory.xml#xpointer(//part%5Bstarts-with(sku,%20'999')%5D)

The XPointer in that example is actually

xpointer(//part[starts-with(sku, '999')])

but the URI syntax requires further encoding of some data. The result of evaluating this XPointer would be the same as evaluating the XPath expression //part[starts-with(sku, '999')] against the document identified by the URI http://example.com/inventory.xml.

XPointer syntax is simple: a is just a name, and refers to the element with that ID (as determined by a DTD or other schema, typically), much like the XPath 1.0 expression id(somename), but with a little more flexibility, since id() is limited to DTD-based data typing.

A consists of a series of one or more , separated by optional whitespace, with each part looking like a function call. What appear to be function names are actually syntactic and semantic , of which the most common is the ID-oriented element scheme, and of which the most versatile is the XPath-oriented xpointer scheme.

If a scheme-based XPointer contains more than one part, then the parts are evaluated from left to right, skipping any unsupported/unrecognized schemes, until one is found that identifies something that exists in the document. Some schemes, like the namespace/prefix-binding xmlns, identify nothing (by design), and instead just influence the interpretation of subsequent parts. It's possible for an XPointer to produce different results with different processors, if the author doesn't take care to ensure each part identifies the same thing.

Here are some more examples:

The XPath 1.0 expression id(somename) means the same thing as the XPointer xpointer(id(somename)), and nearly the same thing as the XPointers element(somename) and somename, which just have more flexibility in where the ID can be drawn from.

The XPointer element(somename/3/1) means nearly the same thing as the XPath expression id(somename)/*[3]/*[1].

The XPointer xmlns(xhtml=http://www.w3.org/1999/xhtml)xpointer(//xhtml:a[@href]) could be used to refer to all of the links in an XHTML 1.0 document.

4Suite's XPointer implementation, sometimes called 4XPointer, has no command-line interface, but can be used within Python applications. It supports XPointers to different degrees, depending on the circumstances:

1. When an XML document is being parsed into a Domlette with XInclude processing enabled, any XPointer encountered in an xi:include element is automatically evaluated when the included document is parsed. In this mode the XPointer must use an XPath LocationPath that only uses steps along the child axis. Furthermore, any predicates must be literal numbers, or must be of the specific form [@attname='attvalue']. For example, /foo[3] and /foo[@bar='baz'] will work, but ../foo and foo/[.='bar'] will not. Function calls are not allowed.

2. If you have not yet parsed an XML document, but have a URI for it, then you can use Ft.Xml.XPointer.SelectUri() to parse the document and evaluate an XPointer embedded in the URI's fragment component. The parsing is performed with Domlette's default NonvalidatingReader instance. There are some implementation gaps to note when using the xpointer scheme: the only additional function fully supported is here(), and the following functions always return empty location-sets: string-range(), range-to(), start-point(), end-point(), and origin(). origin is illegal to use outside of extended XLinks, anyway.

3. If you have already parsed the document into a Domlette, then you can evaluate an arbitrary XPointer against it by using Ft.Xml.XPointer.SelectNode(). The same implementation gaps as noted in the description of Ft.Xml.XPointer.SelectUri() apply.

Ranges are not supported because Domlette does not support DOM Level 2 Ranges. Uche Ogbuji posted some thoughts about this topic a while back. Also note that although the element scheme is streamable, it is not yet supported in XIncludes due to ID-related limitations in Domlette. Since element and shorthand pointer support are requirements for full XInclude conformance, they will probably be implemented in the future.

In 4Suite 1.0b1 and earlier, the implementation was based on older versions of the specs, and several additional restrictions were in effect: the element scheme was not even an option, XPointers in XIncludes had to be given via URIs (not attributes) and couldn't contain NameTests involving "*", and all other XPointers were only allowed to identify a single node.

The following example uses XInclude with XPointer references to include various sections from one document into another document. “article.xml: Document using XInclude with XPointer references” lists the contents of the file article.xml, which references one section using a shorthand pointer and then references any sections that have their condition attribute set to unfinished. “article2.xml: Document with content referenced from article.xml” lists the contents of the file article2.xml, which is referenced from article.xml. “Loading the document” lists the Python code used to parse and print this document; note that XPointer processing is driven from XInclude processing, which is done automatically by default.

<article>
  <title>My important article</title>
  <xi:include href="article2.xml"
              xpointer="woo"
              xmlns:xi="http://www.w3.org/2001/XInclude"/>
  <xi:include href="article2.xml"
              xpointer="xpointer(article/section[@condition='unfinished'])"
              xmlns:xi="http://www.w3.org/2001/XInclude"/>
</article>

<article>
  <section condition="unfinished">
    <title>Section 1</title>
    <!-- Write me! -->
  </section>
  <section xml:id="woo">
    <title>Section 2</title>
    <para>Yeah, content.</para>
  </section>
  <section condition="unfinished">
    <title>Section 3</title>
    <!-- Write me, too! -->
  </section>
</article>

from Ft.Xml import Parse
from Ft.Xml.Domlette import PrettyPrint
doc = Parse("article.xml")
PrettyPrint(doc)

As mentioned earlier, XPointer is most commonly used along with XInclude, but 4Suite provides an API for using XPointer directly from Python. Using article2.xml as listed above in “article2.xml: Document with content referenced from article.xml”, “Using XPointer directly from Python” loads two of the nodes loaded previously with XInclude. Note that when using the standalone interface, the code is able to take advantage of more of the XPointer syntax.

from Ft.Xml import Parse
from Ft.Xml.Domlette import PrettyPrint
from Ft.Xml.XPointer import SelectNode

article2 = Parse("article2.xml")
# Shorthand XPointer syntax
node = SelectNode(article2, "woo")[0]
PrettyPrint(node)
# Scheme-based XPointer syntax
node = SelectNode(article2,
                  "xpointer(//section[@condition='unfinished'][2])")[0]
PrettyPrint(node)

“Self-contained example” is very similar to the examples above, only this version is self-contained; the resources are stored in Python strings and resolved using a custom resolver.

article = """<article><title>My important article</title>
<xi:include href="ex:article2"
            xpointer="woo"
            xmlns:xi="http://www.w3.org/2001/XInclude"/>
<xi:include href="ex:article2"
            xpointer="xpointer(article/section[@condition='unfinished'])"
            xmlns:xi="http://www.w3.org/2001/XInclude"/>
</article>"""

article2 = """<article>
<section condition="unfinished"><title>Section 1</title><!-- Write me! --></section>
<section xml:id="woo"><title>Section 2</title><para>Yeah, content.</para></section>
<section condition="unfinished"><title>Section 3</title><!-- Write me, too! --></section>
</article>"""

from Ft.Lib.Uri import FtUriResolver, Absolutize
from Ft.Lib import UriException
from cStringIO import StringIO
class MyResolver (FtUriResolver):
  def normalize(self, uriRef, baseUri):
    return Absolutize(uriRef, baseUri)
  def resolve(self, uri):
    if uri == "ex:article":
      return StringIO(article)
    elif uri == "ex:article2":
      return StringIO(article2)
    else:
      raise UriException(UriException.RESOURCE_ERROR,
                         loc=uri, msg="not found, sorry")

myResolver = MyResolver()

from Ft.Xml.InputSource import InputSourceFactory
from Ft.Xml.Domlette import NonvalidatingReader, PrettyPrint
factory = InputSourceFactory(resolver=myResolver)
isrc = factory.fromUri("ex:article")
doc = NonvalidatingReader.parse(isrc)
PrettyPrint(doc)

from Ft.Xml.XPointer import SelectNode

isrc = factory.fromUri("ex:article2")
article2 = NonvalidatingReader.parse(isrc)
node = SelectNode(article2, "woo")[0]
PrettyPrint(node)
node = SelectNode(article2,
                  "xpointer(//section[@condition='unfinished'][2])")[0]
PrettyPrint(node)

In the XML universe, one common use-case is converting DocBook (a common XML application) to various output formats for publishing using the DocBook XSL set of XSLT scripts. If you have the DocBook XSL distribution installed (or if you have an Internet connection), you can transform DocBook files completely within the 4Suite XML API. The following example illustrates how this can be done, and in the process this example touches on a wide variety of 4Suite XML techniques. These are listed below.

• Building a Domlette XML model manually

• Parsing XML into a Domlette XML model

• Using XSLT in 4Suite XML

• Using InputSources with automatic XML Catalog resolution

• Managing URIs

• Writing XML from a Domlette XML model

• And a bonus feature unrelated to 4Suite: i18n with the DocBook XSL scripts!

from Ft.Xml.Domlette import implementation, PrettyPrint, NonvalidatingReader
from Ft.Xml.Xslt import Processor
from Ft.Xml import Catalog, InputSource, EMPTY_NAMESPACE
from Ft.Lib import Uri, UriException

# New processor
processor = Processor.Processor()

# If you have the DocBook XSL scripts installed in your system, then they are likely
# integrated into the system catalog, which is often at `/etc/xml/catalog` on
# Unix-like systems.  If you have a catalog which resolves the DocBook XSL URIs
# located in a different filename, you can change this filename below.  Otherwise,
# this example will access the DocBook XSL scripts directly (i.e. over the network).
catalog_filename = '/etc/xml/catalog'
# Turn the catalog filename into the corresponding `file` URI.
catalog_URI = Uri.OsPathToUri(catalog_filename)
# Try to load the catalog, moving right along if it doesn't exist.
theCatalog = None
try:
  theCatalog = Catalog.Catalog(catalog_URI)
except UriException, e:
  pass

# Create a new `InputSourceFactory` object to use our catalog.
inputSourceFactory = InputSource.InputSourceFactory(catalog = theCatalog)
docbook_xsl_URI = 'http://docbook.sourceforge.net/release/xsl/current/html/docbook.xsl'
# Set up an `InputSource` for the DocBook XSL stylesheets.
docbook_xsl_source = inputSourceFactory.fromUri(docbook_xsl_URI)
# Build a DOM of our stylesheet, then load the stylesheet into the XSLT processor.
transform = NonvalidatingReader.parse(docbook_xsl_source)
processor.appendStylesheetNode(transform, docbook_xsl_URI)

# Now we build our DocBook DOM, with a document root of myDoc.
myDoc = implementation.createRootNode('file:///article.xml')
article = myDoc.createElementNS(EMPTY_NAMESPACE,  'article')
myDoc.appendChild(article)
article.setAttributeNS(None, 'lang', "es")
myDoc.publicId="-//OASIS//DTD DocBook XML V4.2//EN"
myDoc.systemId="http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"

element = myDoc.createElementNS(EMPTY_NAMESPACE, 'title')
element.appendChild(myDoc.createTextNode('Title of article'))
article.appendChild(element)

section = myDoc.createElementNS(EMPTY_NAMESPACE, 'section')
article.appendChild(section)

element = myDoc.createElementNS(EMPTY_NAMESPACE, 'title')
element.appendChild(myDoc.createTextNode('Title of section'))
section.appendChild(element)

element = myDoc.createElementNS(EMPTY_NAMESPACE, 'para')
element.appendChild(myDoc.createTextNode('paragraph of section'))
section.appendChild(element)

print '************************ xml *******************************'
# Serialize the source document as XML.
PrettyPrint(myDoc)

print '************************ html *******************************'
# Print the result of transforming the document.
result = processor.runNode(myDoc)
print result