Home Prev Up Next XML Calabash Reference (Version 0.28)

2. Using Piperack

Running pipelines with Piperack is performed through a RESTful web services API. The examples that follow use curl, but any tool that supports the ability to GET and POST resources to a web service will work.

Important

When running, Piperack has all of the privileges of the user who started it. Just because you could allow the public to access it, doesn't mean you should. Your gun, your bullet, your foot.

When started, Piperack listens for requests on the configured port. In the examples that follow, it is running on “localhost” on port “8088”. If you're running it on a different machine or port, adjust the examples accordingly.

For most endpoints, content negotiation is used to determine how the results should be returned. XML, HTML, plain text, and JSON are all supported. Endpoints that return the outputs of a pipeline after it has been run always return XML.

Our example pipeline for this section is pipe.xpl:

<p:declare-step xmlns:p="http://www.w3.org/ns/xproc"
                xmlns:ex="http://example.com/ns/example"
                name="main" version="1.0" exclude-inline-prefixes="ex">
   <p:input port="source" sequence="true"/>
   <p:input port="parameters" kind="parameter"/>
   <p:output port="result" sequence="true" primary="true"/>
   <p:output port="result2" sequence="true">
     <p:pipe step="id" port="result"/>
   </p:output>
   <p:option name="opt1" select="'default'"/>
   <p:option name="ex:opt2" select="'default'"/>
   <p:serialization port="result" indent="true"/>

   <p:identity name="id"/>
</p:declare-step>

It doesn't do anything useful, it simply copies its input to two different output ports.

2.1. /status

The /status endpoint just returns information about the status of the server. Only GET is supported.

$ curl -H "Accept: text/plain" http://localhost:8088/status
XML Calabash version 1.0.14a, an XProc processor.
Running on Saxon version 9.5.1.1, EE edition.
Copyright © 2007-2013 Norman Walsh
See docs/notices/NOTICES in the distribution for licensing.
See also http://xmlcalabash.com/ for more information.

2.2. /help (or just /)

The /help endpoint returns a summary of the valid endpoints only GET is supported.

2.3. /pipelines

A new pipeline is added to the server by POSTing it to the /pipelines endpoint. The user can suggest an ID with the name parameter. (If the requested name is in use, a different, unique name will be returned.)

$ curl -X POST -d@pipe.xpl http://localhost:8088/pipelines?name=mypipe
<response xmlns="http://xmlcalabash.com/ns/piperack">
   <code>201</code>
   <expires>2013-10-25T15:49:04Z</expires>
   <uri>http://localhost:8088/pipelines/mypipe</uri>
   <message>Created http://localhost:8088/pipelines/mypipe</message>
</response>

The response also includes the HTTP Location header, pointing to the newly minted URI for the pipeline.

All pipelines get an expiration time when they are created. Periodically, the server will delete any pipelines that are past their expiration time.

The default expiration time can be set in several ways, see Section 34, “Piperack configuration”. The expires parameter can be used to specify a different expiration time for any particular pipeline when it is created. If -1 is specified, the pipeline will never expire.

Performing a GET on /pipelines returns a list of the pipelines available.

$ curl -H "Accept: application/json" http://localhost:8088/pipelines
{"pipelines":["http://localhost:8088/pipelines/mypipe"]}

2.4. /pipelines/{id}

Performing a GET on /pipelines/{id} returns information about the pipeline:

$ curl http://localhost:8088/pipelines/mypipe
<pipeline xmlns="http://xmlcalabash.com/ns/piperack">
   <uri>http://localhost:8088/pipelines/mypipe</uri>
   <has-run>false</has-run>
   <input documents="0">source</input>
   <input primary="true" documents="0">parameters</input>
   <option>
      <name>opt1</name>
      <value default="true"/>
   </option>
   <option>
      <name xmlns:ex="http://example.com/ns/example">ex:opt2</name>
      <value default="true"/>
   </option>
</pipeline>

The exact information returned varies depending on the state of the pipeline (before or after it has been run) and on the state of the various inputs, outputs, options, and parameters.

As we shall see, there are individual endpoints for sending documents to the pipeline, setting options, setting parameters, and running the pipeline. However, you can post directly to the pipeline URI.

POSTing multipart/form-data according to RFC 2388 to /pipelines/{id} has the following effects:

  1. Each uploaded document is delievered to the named input port. It is an error if the pipeline has no such port.

  2. Any POSTed data that isn't an uploaded document is treated as the value of a named option or parameter.

POSTing a single document to /pipelines/{id} has the following effects:

  1. The POSTed document is delivered to the pipeline's primary input port. It is an error if the pipeline does not have a primary input port.

  2. Any parameters passed as part of the URI are delivered to the pipeline as the values of named options or parameters.

By default simple name=value pairs posted to the pipeline are treated as named options. To specify a parameter, use a name of the form -pparameter-name. By default the parameter will be written to the primary parameter input port. To specify a particular port explicitly, use a field name of the form -pport@parameter-name.

Use fields of the form xmlns:prefix to specify namespace bindings, if necessary.

After the POSTed data is processed, the pipeline is run.

If the pipeline runs successfully, the first document on the pipeline's primary output port is returned. If the pipeline has no primary output port, nothing is returned.

For example:

$ curl -X POST -F "source=@doc.xml;type=application/xml" -F "opt1=somevalue" http://localhost:8088/pipelines/mypipe
<doc/>

Or

$ curl -X POST -H "Content-type: application/xml" -d @doc.xml "http://localhost:8088/pipelines/mypipe&opt1=somevalue"
<doc/>

2.5. /pipelines/{id}/inputs/{port}

POSTing to /pipelines/{id}/inputs/{port} delivers the POSTed document to the specified port.

2.6. /pipelines/{id}/options

POSTing to /pipelines/{id}/options delivers the POSTed parameters (from the application/x-www-form-urlencoded body or from the URI) to the pipeline as options.

$ curl -X POST "http://localhost:8088/pipelines/mypipe/options?xmlns:x=http://example.com/ns/example&x:opt2=two&opt1=one"
<response xmlns="http://xmlcalabash.com/ns/piperack">
   <code>202</code>
   <message>Options added: opt1, x:opt2</message>
</response>

The pipeline information endpoint will now show that those values have been specified.

$ curl http://localhost:8088/pipelines/mypipe
<pipeline xmlns="http://xmlcalabash.com/ns/piperack">
   <uri>http://localhost:8088/pipelines/mypipe</uri>
   <has-run>false</has-run>
   <input primary="true" documents="0">source</input>
   <input documents="0">parameters</input>
   <option>
      <name>opt1</name>
      <value>one</value>
   </option>
   <option>
      <name xmlns:ex="http://example.com/ns/example">ex:opt2</name>
      <value>two</value>
   </option>
</pipeline>

2.7. /pipelines/{id}/options/{option}

POSTing to /pipelines/{id}/options/{option} sets the value of the option named “{option}” to the POSTed content. This option exists to support the “general values” extension; it's otherwise not necessary.

If the option name is in a namespace, use a parameter of the form xmlns:prefix on the URI to specify the namespace binding.

2.8. /pipelines/{id}/parameters

POSTing to /pipelines/{id}/parameters delivers the POSTed parameters (from the application/x-www-form-urlencoded body or from the URI) to the pipeline as parameters.

This endpoint uses the primary parameter input port.

2.9. /pipelines/{id}/parameters/{port}

POSTing to /pipelines/{id}/parameters/{port} delivers the POSTed parameters (from the application/x-www-form-urlencoded body or from the URI) to the pipeline as parameters on the “{port}” parameter input port.

2.10. /pipelines/{id}/parameters/{port}/{param}

POSTing to /pipelines/{id}/parameters/{port}/{param} sets the value of the option named “{param}” to the POSTed content. This option exists to support the “general values” extension; it's otherwise not necessary.

If the parameter name is in a namespace, use a parameter of the form xmlns:prefix on the URI to specify the namespace binding.

2.11. /pipelines/{id}/run

POSTing to the /pipelines/{id}/run endpoint runs the pipeline and returns the first document on the primary output port.

The pipeline information endpoint will now show outputs instead of inputs.

$ curl http://localhost:8088/pipelines/mypipe
<pipeline xmlns="http://xmlcalabash.com/ns/piperack">
   <uri>http://localhost:8088/pipelines/mypipe</uri>
   <has-run>true</has-run>
   <output primary="true" documents="0">result</output>
   <output documents="1">result2</output>
</pipeline>

2.12. /pipelines/{id}/outputs/{port}

After a pipeline has been run, a GET request to /pipelines/{id}/outputs/{port} returns the next document from the specified port.

$ curl http://localhost:8088/pipelines/mypipe/outputs/result2
<doc/>

2.13. /pipelines/{id}/reset

POSTing to /pipelines/{id}/reset resets the pipeline: deletes all inputs, outputs, options, and parameters.

2.14. /stop

POSTing to /stop stops Piperack.

Prev  Home  Next
1 .  Running Piperack  Up  Standard XProc Steps

Copyright © 2011–2020 Norman Walsh.