Simple SciDB HTTP API

The SciDB HTTP Service implements a very simple API that clients may use to interact with SciDB. The API consists of seven URIs (described in detail below): /new_session, /release_session, /execute_query, /cancel, /read_lines, /read_bytes, and /upload_file.

Clients always begin by requesting a session ID from the service, then running a query and releasing the session ID when done. Note that session IDs are distinct from SciDB query IDs--a session ID groups a SciDB query together with server resources for input and output to the client.


Example API Workflow

  1. /new_session
  2. /execute_query
  3. /read_lines or /read_bytes
  4. /release_session

API Reference

We use the example URL http://scidb:8080 below. Parameters are required unless marked optional.

/new_session

DESCRIPTION Request a new HTTP session from the service.
METHODGET
PARAMETERS
RESPONSE Success: HTTP 200 and text session ID value in text/plain payload
Failure (out of resources): HTTP 503
EXAMPLE http://scidb:8080/new_session

HTTP/1.0 200 OK
Content-Length: 3
Content-Type: text/plain

0

/release_session

DESCRIPTION Release an HTTP session
METHODGET
PARAMETERS id (an HTTP session ID)
RESPONSE Success: HTTP 200
Failure (Session not found): HTTP 404
Failure (invalid http query): HTTP 400
EXAMPLE http://scidb:8080/release_session?id=0

HTTP/1.0 200 OK
Content-Length: 0
Content-Type: text/plain


/execute_query

DESCRIPTION Execute a SciDB query
METHODGET
PARAMETERS id (an HTTP session ID)
query (query string, encoded for use in URL as required)
save optional (format string) Save the query output in the specified format for subsequent by read_lines or read_bytes. If the save parameter is not specified, don't save the query output.
release optional 0 or 1: if 1 then release_session as soon as query completes. The default value is 0 if not specified.
RESPONSE Success: HTTP 200 text/plain (SciDB Query ID)
Failure (SciDB connect error): HTTP 503 text/plain (SCIDB ERROR TEXT)
Failure (SciDB query error): HTTP 500 text/plain (SCIDB ERROR TEXT)
Failure (Invalid session): HTTP 404
Failure (invalid http query): HTTP 400
NOTES 500 and 503 errors result in removal of the web session ID and related resources (thus, release_session does not have to be called after such an error). This method blocks until the query completes. Do not specifiy the option relase=1 when the save option is also set, or output will not be available to read_bytes or read_lines. Instead, explicitly call release_session after reading is complete.
EXAMPLE http://scidb:8080/execute_query?id=0&query=remove(x)&release=1

HTTP/1.0 200 OK
Content-Length: 13
Content-Type: text/plain

1100993821834
EXAMPLE (ERROR) http://scidb:8080/execute_query?id=0&query=remove(x)&release=1

HTTP/1.0 500 ERROR
Content-Length: 286
Content-Type: text/plain

UserQueryException in file: src/query/parser/ALTranslator.cpp function: createArrayReferenceParam line: 863
Error id: scidb::SCIDB_SE_QPROC::SCIDB_LE_ARRAY_DOESNT_EXIST
Error description: Query processor error. Array 'x' does not exist.
remove(x)
       ^
Failed query id: 1100994052246

/cancel

DESCRIPTION Cancel a SciDB query associated with a session
METHODGET
PARAMETERS id (an HTTP session ID)
RESPONSE Success: HTTP 200
Failure (session not found): HTTP 404
Failure (invalid http query): HTTP 400
Failure (could not connect to SciDB): HTTP 503
EXAMPLE http://scidb:8080/cancel?id=0

HTTP/1.0 200 OK
Content-Length: 0
Content-Type: text/plain


/read_lines

DESCRIPTION Read text lines from a query that saves its output
METHODGET
PARAMETERS id (an HTTP session ID)
n (maximum number of lines to read and return)
RESPONSE Success: HTTP 200 followed by text/plain query result (up to n lines)
Failure (invalid HTTP query string): HTTP 400
Failure (session not found): HTTP 404
Failure (end of file): HTTP 410
Failure (invalid request): HTTP 414
Failure (SciDB server error): HTTP 500
Failure (could not connect to SciDB server error): HTTP 503
Failure (server out of memory): HTTP 507
EXAMPLE http://scidb:8080/new_session
http://scidb:8080/execute_query?id=0&query=list('functions')&save=dcsv
http://scidb:8080/read_lines?id=0&n=20

HTTP/1.0 200 OK
Content-Length: 903
Content-Type: text/plain

{No} name,profile,deterministic,library
{0} "%","double %(double,double)",true,"scidb"
{1} "%","int16 %(int16,int16)",true,"scidb"
{2} "%","int32 %(int32,int32)",true,"scidb"
{3} "%","int64 %(int64,int64)",true,"scidb"
{4} "%","int8 %(int8,int8)",true,"scidb"
{5} "%","uint16 %(uint16,uint16)",true,"scidb"
{6} "%","uint32 %(uint32,uint32)",true,"scidb"
{7} "%","uint64 %(uint64,uint64)",true,"scidb"
{8} "%","uint8 %(uint8,uint8)",true,"scidb"
{9} "*","double *(double,double)",true,"scidb"
{10} "*","float *(float,float)",true,"scidb"
{11} "*","int16 *(int16,int16)",true,"scidb"
{12} "*","int32 *(int32,int32)",true,"scidb"
{13} "*","int64 *(int64,int64)",true,"scidb"
{14} "*","int8 *(int8,int8)",true,"scidb"
{15} "*","uint16 *(uint16,uint16)",true,"scidb"
{16} "*","uint32 *(uint32,uint32)",true,"scidb"
{17} "*","uint64 *(uint64,uint64)",true,"scidb"
{18} "*","uint8 *(uint8,uint8)",true,"scidb"
NOTES
  1. Set n=0 to download the entire output buffer.
  2. Be sure to properly url-encode special characters like the plus sign (+) in the request.
  3. When n>0, iterative requests to read_lines are allowed, and will return at most the next n lines of output. Use the 410 error code to detect end of file output.

/read_bytes

DESCRIPTION Read bytes from a query that saves its output
METHODGET
PARAMETERS id (an HTTP session ID)
n (maximum number of bytes to read and return)
RESPONSE Success: HTTP 200 followed by application/octet-stream binary query result (up to n bytes)
Failure (invalid HTTP query string): HTTP 400
Failure (session not found): HTTP 404
Failure (end of file): HTTP 410
Failure (invalid request): HTTP 414
Failure (SciDB server error): HTTP 500
Failure (could not connect to SciDB server error): HTTP 503
Failure (server out of memory): HTTP 507
EXAMPLE http://scidb:8080/new_session
http://scidb:8080/execute_query?id=0&query=build(%3Cx:double%3E%5Bi=1:10,10,0%5D,random())&save=(double)
http://scidb:8080/read_bytes?id=0&n=20

HTTP/1.0 200 OK
Content-Length: 20
Content-Type: application/octet-stream

Š/�A�}��A�
NOTES
  1. Iterative requests to read_lines are allowed, and will print at most the next n bytes of output. Use the 410 error code to detect the end of output.
  2. Be sure to properly url-encode special characters in the request.

/upload_file

DESCRIPTION Upload a file to the HTTP service
METHODPOST/GET
PARAMETERS id (an HTTP session ID)
A valid file-upload HTTP POST message.
RESPONSE Success: HTTP 200 and the name of the file uploaded to the server in a text/plain response.
Failure (invalid HTTP query string): HTTP 400
Failure (Session not found): HTTP 404
Failure (Server error): HTTP 500
EXAMPLE Example POST to session id=0:
POST /upload_file?id=0 HTTP/1.1
Host: scidb:8080
Accept: */*
Content-Length: 526
Expect: 100-continue
Content-Type: multipart/form-data; boundary=----------------------------d1f47951faa4

------------------------------d1f47951faa4
Content-Disposition: form-data; name="file"; filename="data.csv"
Content-Type: application/octet-stream

"","Sepal.Length","Sepal.Width","Petal.Length","Petal.Width","Species"
"1",5.1,3.5,1.4,0.2,"setosa"
"2",4.9,3,1.4,0.2,"setosa"
"3",4.7,3.2,1.3,0.2,"setosa"
"4",4.6,3.1,1.5,0.2,"setosa"
"5",5,3.6,1.4,0.2,"setosa"
"6",5.4,3.9,1.7,0.4,"setosa"
"7",4.6,3.4,1.4,0.3,"setosa"
"8",5,3.4,1.5,0.2,"setosa"
"9",4.4,2.9,1.4,0.2,"setosa"

------------------------------d1f47951faa4--


Example response:
HTTP/1.0 200 OK
Content-Length: 23
Content-Type: text/plain

/tmp/shim_file_Hrloh9
NOTES The file to upload can be binary. Use the returned server-side file name in a subsequent SciDB load query, for example. The file does not persist after the HTTP session is released.