NED VO Table Access Protocol (TAP) Service

This page describes NED's implementation of the VO Table Access Protocol, or TAP. TAP is an Application Program Interface (API) supporting Astronomical Data Query Language (ADQL) queries. Currently, the service allows users to query the NED Object Directory ("objdir") from computer programs and scripts without the need to go through a browser.

Users can perform a wide variety of parameter-constrained searches, including regions of the sky (cone, box, polygon, etc.). The output can be a VO Table, a FITS table, JSON, or several other formats. It also provides the option of selecting output columns, and performing functions on the results.

Here is an example of how to do a cone search of the NED Object Directory with the commonly available curl command. The units of RA, Dec, and search radius are decimal degrees. While the default output format is a "VOTable" (XML with binary encoding of the data), we are generating the output using "TEXT" format for readability.

curl -o out.txt "https://ned.ipac.caltech.edu/tap/sync?query=SELECT+*+FROM+objdir+WHERE+CONTAINS(POINT('J2000',ra,dec),CIRCLE('J2000',66.76957,26.10453,0.01))=1&LANG=ADQL&REQUEST=doQuery&FORMAT=text"

The "+" signs here indicate spaces within a URL. Special characters (">", "<", etc.) within the query will need to be urlencoded. One can refer to https://www.urlencoder.org/ for the encoding. These will be omitted in the descriptions below for clarity.

Constructing a TAP Query

One can make either a synchronous or asynchronous query. A synchronous query runs until it completes and streams the results back -- use this for small-area searches. An asynchronous query will run in the background and give you a place to check whether your job is done, rather than streaming results back directly. The main query parameters are QUERY, LANG, REQUEST, FORMAT (optional), PHASE (optional, async query only), EXECUTIONDURATION (optional), and MAXREC (optional), described in details below.

For synchronous calls (small-area searches):

  • https://ned.ipac.caltech.edu/tap/sync?QUERY={QUERY}&LANG=ADQL&REQUEST=doQuery&FORMAT={FORMAT}&MAXREC={MAXREC}

For asynchronous calls (large searches):

  • https://ned.ipac.caltech.edu/tap/async?QUERY={QUERY}&LANG=ADQL&REQUEST=doQuery&FORMAT={FORMAT}&PHASE={PHASE}&EXECUTIONDURATION={DURATION}&MAXREC={MAXREC}
Note, asynchronous calls should be made only using HTTP POST (see examples below). In NED's implementation of the asynchronous query, PHASE=RUN is required. You will be given a web address that you can use to check on your job. For example, if the address is:
  • https://ned.ipac.caltech.edu/tap/async/35

Then you can check the progress of your job by checking the "phase" associated with this job:

  • https://ned.ipac.caltech.edu/tap/async/35/phase

This will return either QUEUED, EXECUTING, COMPLETED, ERROR, or ABORT. When the job is completed, you will be directed to the results page, which will look similar to this:

  • https://ned.ipac.caltech.edu/tap/async/35/results/result

QUERY:

An ADQL Query.
QUERY=SELECT {columns} FROM {table} WHERE {geometric constraint} AND ({sql constraint}) {order by}{group by} {having}

keyword description
columns

Comma-separated list of column names, or functions of columns to be returned.

Supported functions are those that are both defined in ADQL 2.0 and implemented by database software providers PostgreSQL.

Examples (for illustration only -- don't run without constraints):

1. Output all columns.

  • SELECT *

2. Output selected columns.

  • SELECT ra,dec

3. Output the ra, dec, sine of ra and cosine of dec from the NED Object Direcotry.

  • SELECT ra,dec,sin(ra),cos(dec) FROM objdir

4. Rename the output column returned by "count(ra)" to ra_count for NED objects with redshift greater than 10.

  • SELECT count(ra) as ra_count FROM objdir WHERE z > 10

5. Prefix the ra with the table name. This is useful when performing operations on multiple tables.

  • SELECT objdir.ra FROM objdir WHERE z > 10

Note on Column Names: To get the available column names for the NED Object Directory:

curl -o out.txt "https://ned.ipac.caltech.edu/tap/sync?QUERY=SELECT+*+FROM+TAP_SCHEMA.columns+WHERE+table_name='NEDTAP.objdir'&REQUEST=doQuery&LANG=ADQL&FORMAT=text"

table

Specifies the table to search.

Example (for illustration only):

Select all columns from the NED object direcotry.

  • SELECT * FROM objdir

Note on Table Names: To obtain the string needed for the "table" parameter, you can download a text file of NED's available tables. The string needed is in the column labeled "table_name". There is also a useful column called "description".

curl -o out.txt "https://ned.ipac.caltech.edu/tap/sync?QUERY=SELECT+*+FROM+TAP_SCHEMA.tables&REQUEST=doQuery&LANG=ADQL&FORMAT=text"

table_name Description
TAP_SCHEMA.schemas List of schemas published in this TAP service
TAP_SCHEMA.tables List of tables published in this TAP service
TAP_SCHEMA.columns List of columns of all tables listed in TAP_SCHEMA.TABLES and published in this TAP service
TAP_SCHEMA.keys List all foreign keys but provides just the tables linked by the foreign key. To know which columns of these tables are linked, see in TAP_SCHEMA.key_columns using the key_id
TAP_SCHEMA.key_columns List all foreign keys but provides just the columns linked by the foreign key. To know the table of these columns, see in TAP_SCHEMA.keys using the key_id
NEDTAP.objdir NED object directory

geometric contraint

Geometric constraints are typically a CONTAINS function (= 1 for true) operating on a POINT and a shape. There are three supported shape functions. For the shape functions, the first argument, the coordinate system, can be one of 'ICRS', 'GALACTIC', 'J2000', 'FK4' and 'FK5'. Coordinates will be interpreted in terms of that coordinate system. All coordinates and angular sizes are in decimal degrees.

Circle

This requires the coordinate system, coordinates of the center, and radius. A 0.01 degree cone search around M101 would be:

  • SELECT * FROM objdir WHERE CONTAINS(POINT('J2000',ra,dec),CIRCLE('J2000',210.80225,54.34894,0.01))=1

Box

This requires the coordinate system, coordinates of the center of the box, the width, and the height. The box will be aligned with the coordinate system. A 0.1 degree by 0.1 degree box around M101 would be:

  • SELECT * FROM objdir WHERE CONTAINS(POINT('J2000',ra,dec),BOX('J2000',210.80225,54.34894,0.1,0.1))=1

Polygon

This requires the coordinate system, and then a series of coordinates of the vertices. A triangle search to return the number of objects around M101 would be:

  • SELECT count(*) FROM objdir WHERE CONTAINS(POINT('J2000',ra,dec),POLYGON('J2000',209.80225,53.34894,209.80225,55.34894,211.80225,54.34894))=1
sql constraint

SQL constraints can be any constraint expressible in ADQL 2.0, with the restriction that functions must be supported by the PostgreSQL backend.

Example (for illustration only):

Query the NED object directory, and return number of objects that fall within a certain redshift range.

  • SELECT count(*) FROM objdir WHERE (z > 9 and z < 10)
order by

A comma-separated list of column names specifying which to use for sorting the output table rows, in order of priority.

Example:

Query the NED object directory. Return ra, dec, z for records within a specified redshift range, and order the results by the redshift. If multiple records have identical redshift, then order by right ascension.

  • SELECT ra, dec, z FROM objdir WHERE (z > 9 and z < 10) order by z, ra
group by Groups the returned records by the specified columns, which allows you to perform functions on that group.

Example:

Query the NED Object Directory. Return prefer object type and their corresponding count for records within a specified redshift range, and group the records by preferred object type.

  • SELECT pretype, count(pretype) from objdir where (z > 8 and z < 10) group by pretype order by pretype
having Constrains selections after an aggregate function such as "group by". Quicker than a WHERE since it acts on the smaller group.

Example:

Query the NED Object Directory. Return prefer object type and their corresponding count for records within a specified redshift range, and group the records by preferred object type, then select those in the group that occurs more than 5 times.

  • SELECT pretype, count(pretype) from objdir where (z > 8 and z < 10) group by pretype having count(pretype) > 5

Note: Other ADQL 2.0 functions may also be implemented, but this page is not meant to be a complete description of SQL functions.

LANG:

Must be the string 'ADQL'.

REQUEST:

Must be the string 'doQuery' or 'getCapabilities'.

FORMAT:

Optional. The TAP service will by default return a VOTable. Other formats are also possible by setting the FORMAT keyword. The supported output formats are:

keyword description
VOTABLE VO Table - application/x-votable+xml. This is the default output format, which is binary encoding.
VOTABLE/B2 VO Table - application/x-votable+xml;serialization=BINARY2
VOTABLE/TD VO Table - application/x-votable+xml;serialization=TABLEDATA. This format is better for human readibility, but the size of the file is larger than the binary encoded VO tables.
VOTABLE/FITS VO Table - application/x-votable+xml;serialization=FITS
FITS Flexible Image Transport System Binary Table
JSON JavaScript Object Notation
CSV Comma Separated Value table
TSV Tab Separated Value table
TEXT Plain Text
HTML HyperText Markup Language

PHASE:

Optional, for Async calls only, please refer to section 5.2 Running the query in the TAP Version 1.0 document.

EXECUTIONDURATION:

Optional. Max execution duration in seconds, default is set to 4 minutes (240 seconds). If a query takes longer to run, it will be aborted and a warning message will be sent to the user suggesting to switch to the async mode. One can increase the default execution duration by passing a "EXECUTIONDURATION=number of seconds" parameter in the http request.

Note that a query will also be restricted by the proxy server timeout limit. The proxy server times out at 5 minutes (300 seconds). This would only affect sync query as async query is designed to return with the job number immediately. If a sync query takes longer than 5 minutes to run, i.e., nothing is returned within the 5-minute time frame, the https server will abort the session and the query will be terminated with a warning message. To avoid this, please always submit a request via async mode and increase the default execution duration if needed.

MAXREC:

Optional. Result limit in terms of rows, the default is set to 1,000,000 rows. If the output has more rows, only the first 1,000,000 rows will be returned. The query will then be aborted with no error or warning message. One can increase the output default limit by passing a "MAXREC=number of rows" parameter in the http request.

Note: The returned result of a query can vary depending on which limit it reaches first among EXECUTIONDURATION, proxy server timeout limit, and MAXREC.

Examples

  1. Cone Search

    • curl "https://ned.ipac.caltech.edu/tap/sync?query=SELECT+count(*)+FROM+objdir+WHERE+CONTAINS(POINT('J2000',ra,dec),CIRCLE('J2000',66.76957,26.10453,0.01))=1&LANG=ADQL&REQUEST=doQuery&FORMAT=text"

  2. Cone Search returning a JSON table file

    • curl "https://ned.ipac.caltech.edu/tap/sync?query=SELECT+count(*)+FROM+objdir+WHERE+CONTAINS(POINT('J2000',ra,dec),CIRCLE('J2000',66.76957,26.10453,0.01))=1&LANG=ADQL&REQUEST=doQuery&FORMAT=json"

  3. Cone Search with only ra, dec, sin(ra) and cos(dec) and return only the first 5 rows.
    • curl "https://ned.ipac.caltech.edu/tap/sync?query=SELECT+TOP+5+ra,dec,sin(ra),cos(dec)+FROM+objdir+WHERE+CONTAINS(POINT('J2000',ra,dec),CIRCLE('J2000',66.76957,26.10453,0.1))=1&LANG=ADQL&REQUEST=doQuery&FORMAT=text"

  4. Cone Search with result ordered by ra

    • curl "https://ned.ipac.caltech.edu/tap/sync?query=SELECT+TOP+5+ra,dec,sin(ra),cos(dec)+FROM+objdir+WHERE+CONTAINS(POINT('J2000',ra,dec),CIRCLE('J2000',66.76957,26.10453,0.1))=1+order+by+ra&LANG=ADQL&REQUEST=doQuery&FORMAT=text"

  5. Box Search

    • curl "https://ned.ipac.caltech.edu/tap/sync?query=SELECT+count(*)+FROM+objdir+WHERE+CONTAINS(POINT('J2000',ra,dec),BOX('J2000',66.76957,26.101,1.0,1.0))=1&LANG=ADQL&REQUEST=doQuery&FORMAT=text"

  6. Polygon Search

    • curl "https://ned.ipac.caltech.edu/tap/sync?query=SELECT+count(*)+FROM+objdir+WHERE+CONTAINS(POINT('J2000',ra,dec),POLYGON('J2000',209.80225,53.34894,209.80225,55.34894,211.80225,54.34894))=1&LANG=ADQL&REQUEST=doQuery&FORMAT=text"

  7. Asynchronous Cone Search

    Submit the query.

    • curl -k -i -Ls -o /dev/null -w "%{url_effective}" -H Content-Type:application/x-www-form-urlencoded -XPOST --data "query=SELECT+count(*)+FROM+objdir+WHERE+CONTAINS(POINT('J2000',ra,dec),CIRCLE('J2000',66.76957,26.101,1))=1&LANG=ADQL&PHASE=RUN&REQUEST=doQuery&EXECUTIONDURATION=100&FORMAT=text" https://ned.ipac.caltech.edu/tap/async

    The output should include a line like:

    • https://ned.ipac.caltech.edu/tap/async/1518723190466A

    Check the progress of the job.

    • curl https://ned.ipac.caltech.edu/tap/async/1518723190466A/phase

    This query completes fairly quickly, so you should see "COMPLETED". To get the results:

    • curl https://ned.ipac.caltech.edu/tap/async/1518723190466A/results/result