Chapter 3. Extensions

1. System property extensions

XML Calabash supports several additional property names in p:system-property:

cx:transparent-json: Returns “true” if the transparent JSON extension is enabled, “false” otherwise.
cx:json-flavor: Returns the flavor of JSON returned by the transparent JSON extension, if it's enabled.
cx:general-values: Returns “true” if the general values extension is enabled, “false” otherwise.
cx:xpointer-on-text: Returns “true” if the XPointer on text extension is enabled, “false” otherwise.
cx:saxon-version: Returns the version of the underlying Saxon processor.
cx:saxon-edition: Returns the edition of the underlying Saxon processor (“he”, “pe”, or “ee”).

2. Extension Attributes

XML Calabash supports an extension attribute cx:depends-on to provide finer grained control over the dependencies between steps. Consider the following example:

Because step “C” depends on steps “A” and “B”, you can be sure that step “C” will run after “A” and “B”. (If Calabash was a multi-threaded, streaming implementation, the constraint would be weaker.) But what about steps “A” and “B”?

Because neither step depends (directly or indirectly) on the output of the other, there is no guarantee about the order in which they will be evaluated.

If step “B” relies on some side effect of step “A” (perhaps “A” calls a web service or writes a document to disk and “B” relies on this having already happened for its correct operation), then you need a way to force the ordering.

It's best if this can be accomplished by establishing an input/output dependency, reading an output of “A” in the evaluation of “B”.

However, if you can't easily arrange an input/output dependency and you're not concerned about introducing an implementation dependency into your pipeline, you can establish the relationship directly by adding

  1 cx:depends="A"

as an extension attribute on “B”. That informs XML Calabash that the evaluation of “A” must precede the evaluation of “B”. The value of cx:depends is a space-separated list of step names.

It is an error if the use of cx:depends introduces a circular dependency into the pipeline.

FIXME: Also document cx:logstyle.

3. Language Extensions

XML Calabash has three language extensions: “general values”, “XPointer on text”, and “transparent JSON”. Enabling any of these extensions makes the processor non-conformant. Pipelines that rely on these extensions are not likely to be interoperable with other implementations. You're encouraged to avoid them if possible. That said, they can be very useful.

3.1. General values extension

If the general values extension is enabled, variables, options, and parameters are not limited to strings. If the select expression that establishes the value of a variable, option, or parameter selects nodes from a tree, then those node values will be passed along as the value.

Consider the pipeline in Example 3.1, “Pipeline that exploits the general values extension”:

Example 3.1. Pipeline that exploits the general values extension

  1 <p:declare-step xmlns:p="http://www.w3.org/ns/xproc" version="1.0">
  2 <p:input port="parameters" kind="parameter"/>
    <p:output port="result"/>
  4 <p:serialization port="result" indent="true"/>
    
  6 <p:identity>
      <p:input port="source">
  8     <p:inline>
          <doc>
 10         <para>Some <emph>text</emph> in a paragraph.</para>
          </doc>
 12     </p:inline>
      </p:input>
 14 </p:identity>
    
 16 <p:xslt template-name="root">
      <p:input port="stylesheet">
 18     <p:inline>
          <xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
 20                       version="2.0">
    
 22         <xsl:param name="text" required="yes"/>
            <xsl:param name="gv" required="yes"/>
 24 
            <xsl:template name="root">
 26           <root general-values="{$gv}">
                <xsl:sequence select="$text"/>
 28           </root>
            </xsl:template>
 30       </xsl:stylesheet>
        </p:inline>
 32   </p:input>
      <p:with-param name="text" select="/doc/para"/>
 34   <p:with-param name="gv" select="p:system-property('cx:general-values')"
                    xmlns:cx="http://xmlcalabash.com/ns/extensions"/>
 36 </p:xslt>
    
 38 </p:declare-step>

In a conformant processor, the text parameter passed to the p:xslt step is a string:

  1 <root general-values="false">Some text in a paragraph.</root>

If the general values extension is enabled, the value contains the selected subtree:

  1 <root general-values="true">
  2    <para>Some <emph>text</emph> in a paragraph.</para>
    </root>

The general values extension can be enabled in three ways:

With the -Xgeneral-values command-line option.
With the com.xmlcalabash.general-values Java system property.
Or with the <extension name="general-values" value="true"/> configuration option.

A pipeline can test whether the general values extension is enabled or not with the p:system-property function using the argument cx:general-values.

3.2. XPointer on text

If the XPointer on text extension is enabled, the xpointer attribute on an XInclude element can be used when parse="text". The XPointer must use the text() scheme and must contain an RFC 5147 fragment identifier. Consider the following DocBook XML fragment:

  1 <programlisting>
  2 <xi:include href="examples/general-values.xpl" parse="text"
                xpointer="text(line=6,15;length=1081)"/>
  4 </programlisting>

If processed with the XPointer on text extension enabled, it would produce effectively the following result:

  1 <programlisting><![CDATA[
  2 <p:identity>
      <p:input port="source">
  4     <p:inline>
          <doc>
  6         <para>Some <emph>text</emph> in a paragraph.</para>
          </doc>
  8     </p:inline>
      </p:input>
 10 </p:identity>
    ]]></programlisting>

Only the “length” integrity check is supported.

The XPointer on text extension can be enabled in three ways:

With the -Xxpointer-on-text command-line option.
With the com.xmlcalabash.xpointer-on-text Java system property.
Or with the <extension name="xpointer-on-text" value="true"/> configuration option.

A pipeline can test whether the XPointer on text extension is enabled or not with the p:system-property function using the argument cx:xpointer-on-text.

3.3. Transparent JSON

If the transparent JSON extension is enabled, p:http-request, p:store, p:document, and p:data will translate automatically between JSON and XML. If JSON is returned by a web service or loaded, it will be turned into XML. If a JSON-in-XML document is sent or stored, it will be turned into textual JSON first.

Consider this JSON object:

  1 {
  2   "id": "1234",
      "": "empty string",
  4   "foo$bar": "foo$bar",
      "1foo$bar": "1foo$bar",
  6   "x:html": "x:html",
      "array": [1,2,3,false,null],
  8   "bool": false,
      "isnull": null,
 10   "image": {
        "url": "http://example.com/image.jpg",
 12     "width": 500,
        "height": 500
 14   }
    }

It can be translated into any one of five XML representations:

marklogic

  1 <j:json xmlns:j="http://marklogic.com/json" type="object">
  2    <j:_ type="string">empty string</j:_>
       <j:id type="string">1234</j:id>
  4    <j:_0031foo_0024bar type="string">1foo$bar</j:_0031foo_0024bar>
       <j:x_003ahtml type="string">x:html</j:x_003ahtml>
  6    <j:isnull type="null"/>
       <j:foo_0024bar type="string">foo$bar</j:foo_0024bar>
  8    <j:image type="object">
          <j:height type="number">500</j:height>
 10       <j:width type="number">500</j:width>
          <j:url type="string">http://example.com/image.jpg</j:url>
 12    </j:image>
       <j:bool type="boolean">false</j:bool>
 14    <j:array type="array">
          <j:item type="number">1</j:item>
 16       <j:item type="number">2</j:item>
          <j:item type="number">3</j:item>
 18       <j:item type="boolean">false</j:item>
          <j:item type="null"/>
 20    </j:array>
    </j:json>

jsonx

  1 <j:object xmlns:j="http://www.ibm.com/xmlns/prod/2009/jsonx">
  2    <j:string name="">empty string</j:string>
       <j:string name="id">1234</j:string>
  4    <j:string name="1foo$bar">1foo$bar</j:string>
       <j:string name="x:html">x:html</j:string>
  6    <j:null name="isnull"/>
       <j:string name="foo$bar">foo$bar</j:string>
  8    <j:object name="image">
          <j:number name="height">500</j:number>
 10       <j:number name="width">500</j:number>
          <j:string name="url">http://example.com/image.jpg</j:string>
 12    </j:object>
       <j:boolean name="bool">false</j:boolean>
 14    <j:array name="array">
          <j:number>1</j:number>
 16       <j:number>2</j:number>
          <j:number>3</j:number>
 18       <j:boolean>false</j:boolean>
          <j:null/>
 20    </j:array>
    </j:object>

jxml

  1 <j:object xmlns:j="http://www.xmlsh.org/jxml">
  2    <j:member name="">
          <j:string>empty string</j:string>
  4    </j:member>
       <j:member name="id">
  6       <j:string>1234</j:string>
       </j:member>
  8    <j:member name="1foo$bar">
          <j:string>1foo$bar</j:string>
 10    </j:member>
       <j:member name="x:html">
 12       <j:string>x:html</j:string>
       </j:member>
 14    <j:member name="isnull">
          <j:null/>
 16    </j:member>
       <j:member name="foo$bar">
 18       <j:string>foo$bar</j:string>
       </j:member>
 20    <j:member name="image">
          <j:object>
 22          <j:member name="height">
                <j:number>500</j:number>
 24          </j:member>
             <j:member name="width">
 26             <j:number>500</j:number>
             </j:member>
 28          <j:member name="url">
                <j:string>http://example.com/image.jpg</j:string>
 30          </j:member>
          </j:object>
 32    </j:member>
       <j:member name="bool">
 34       <j:boolean>false</j:boolean>
       </j:member>
 36    <j:member name="array">
          <j:array>
 38          <j:number>1</j:number>
             <j:number>2</j:number>
 40          <j:number>3</j:number>
             <j:boolean>false</j:boolean>
 42          <j:null/>
          </j:array>
 44    </j:member>
    </j:object>

calabash

  1 <c:json xmlns:c="http://www.w3.org/ns/xproc-step" type="object">
  2    <c:pair name="" type="string">empty string</c:pair>
       <c:pair name="id" type="string">1234</c:pair>
  4    <c:pair name="1foo$bar" type="string">1foo$bar</c:pair>
       <c:pair name="x:html" type="string">x:html</c:pair>
  6    <c:pair name="isnull" type="null"/>
       <c:pair name="foo$bar" type="string">foo$bar</c:pair>
  8    <c:pair name="image" type="object">
          <c:pair name="height" type="number">500</c:pair>
 10       <c:pair name="width" type="number">500</c:pair>
          <c:pair name="url" type="string">http://example.com/image.jpg</c:pair>
 12    </c:pair>
       <c:pair name="bool" type="boolean">false</c:pair>
 14    <c:pair name="array" type="array">
          <c:item type="number">1</c:item>
 16       <c:item type="number">2</c:item>
          <c:item type="number">3</c:item>
 18       <c:item type="boolean">false</c:item>
          <c:item type="null"/>
 20    </c:pair>
    </c:json>

calabash-deprecated

  1 <json type="object">
  2    <pair name="" type="string">empty string</pair>
       <pair name="id" type="string">1234</pair>
  4    <pair name="1foo$bar" type="string">1foo$bar</pair>
       <pair name="x:html" type="string">x:html</pair>
  6    <pair name="isnull" type="null"/>
       <pair name="foo$bar" type="string">foo$bar</pair>
  8    <pair name="image" type="object">
          <pair name="height" type="number">500</pair>
 10       <pair name="width" type="number">500</pair>
          <pair name="url" type="string">http://example.com/image.jpg</pair>
 12    </pair>
       <pair name="bool" type="boolean">false</pair>
 14    <pair name="array" type="array">
          <item type="number">1</item>
 16       <item type="number">2</item>
          <item type="number">3</item>
 18       <item type="boolean">false</item>
          <item type="null"/>
 20    </pair>
    </json>

This was the original format introduced when JSON support was added to p:unescape-markup. Don't use it. Automatic conversion from XML to JSON is not supported for this format.

The transparent JSON extension can be enabled in three ways:

With the -Xtransparent-json command-line option.
With the com.xmlcalabash.transparent-json Java system property.
Or with the <extension name="transparent-json" value="true"/> configuration option.

A pipeline can test whether the transparent JSON extension is enabled or not with the p:system-property function using the argument cx:transparent-json.

The desired XML flavor can be selected using the same mechanisms:

With the -Xjson-flavor=flavor command-line option.
With the com.xmlcalabash.json-flavor Java system property.
Or with the <extension name="json-flavor" value="flavor"/> configuration option.

A pipeline can determine the current flavor with the p:system-property function using the argument cx:json-flavor.

4. Extension functions

XML Calabash supports one extension function.

pxf:cwd()

The cwd function returns the “current working directory” of the processor. This function takes no arguments and does not depend on the context.

There are no XML Calabash steps that change the working directory, so this function is likely to return the same value every time it is called. However, there is nothing which prevents an extension step from being defined which changes the current working directory, so it is not necessarily the case that the same value will always be returned.

5. Extension steps

XML Calabash supports a repertoire of extension steps (EXProc and implementation-specific). You can declare all of them in one fell swoop by importing the following library:

  1 <p:import href="http://xmlcalabash.com/extension/steps/library-1.0.xpl"/>

Importing this library does not require network access. An internal version of the library is automatically imported when this URI is referenced in a p:import statement.


Chapter 2 . Configuration		Standard XProc Steps