www.archive-org-2014.com » ORG » O » OPENDAP

Choose link from "Titles, links and description words view":

Or switch to "Titles and links view".

    Archived pages: 580 . Archive date: 2014-01.

  • Title: DODS Programmers Guide -- 4.1 Data Servers
    Descriptive info: Go backward to.. 4 Using the Toolkit.. Go up to.. Go forward to.. 4.. 1.. 1 The Dispatch CGI.. 1 Data Servers.. The DODS data server consists of a.. dispatch.. program.. and a set of.. filter.. programs.. The dispatch program reads the incoming URL and decides which of the filter programs to run based on the URL suffix.. A.. typical DODS data request uses three filters: one to return the.. DAS.. (.. das.. ), one for the.. DDS.. dds.. ), and the third for the data (.. dods.. ).. A client can also request ASCII data (.. asc.. or.. ascii.. ), usage information about the server (.. info.. ), or version information about the server and the data (.. ver.. The task of building a DODS server can then be separated into the following steps:.. Create concrete classes of the entire.. BaseType.. hierarchy, with.. read.. functions for each data type.. Certain APIs cannot handle certain DODS types.. For these types, there must still be  ...   is between you and your data.. DODS makes no demands on how these structures are created.. That is, for example, if all the data to be served has the same DDS, feel free to cheat.. The only thing that is important is that the structures accurately reflect the relationships of the data.. Create filter programs to return the.. ,.. , data, and server usage and version information.. Create a dispatch program to parse an incoming URL and invoke the correct filter program.. To install the finished server, put the filter programs into a web server's CGI directory, and put the datasets to be served somewhere they can be seen by those filter programs.. Refer to the.. The OPeNDAP User Guide.. for more details about installing a server.. 2 The.. filter programs.. Caching.. Objects.. 3 The Data filter.. The ASCII Data Filter.. 4 The Usage Filter.. 5 Documenting Your Work.. The README File.. The ERRORS File.. Installation Notes.. Information Files.. Tom Sgouros, July 2, 2004..

    Original link path: /api/pguide-html/pguide_24.html
    Open archive

  • Title: DODS Programmers Guide -- 4.1.1 The Dispatch CGI
    Descriptive info: The DODS dispatch CGI program receives a data request from the DODS client, and dispatches the request to one of several filter programs.. The dispatch CGI is stored in a CGI directory on the host machine.. Its name is an important detail of its operation.. The name should begin with.. nph-.. , and end with the letters that distinguish data files containing data formatted with that API from other files.. 8.. So, for example,.. NetCDF.. data files are called.. foo.. nc.. , so the.. dispatch CGI is called.. nph-nc.. The dispatch CGI's job is to parse the incoming URL and execute the appropriate filter programs with the arguments enclosed in the URL.. The dispatch CGI is also be responsible for the first level of error information that must be returned to the user.. These tasks are easily accomplished in any scripting language.. On the off chance you wish to use Perl, DODS provides a Perl class designed to make writing the CGI a simple task.. The.. file.. DODS Dispatch.. pm.. contains the definitions of the.. class.. This class provides several methods used to parse the incoming URL, and one method for delivering error messages to the client.. The.. provides the following methods:.. command().. Returns the command string implied  ...   will be.. das.. dds.. dods.. info.. , or.. ver.. cgi-dir().. Returns the absolute pathname of the directory in which the dispatch CGI is stored.. This is generally the same as the directory in which the DODS filter programs are stored.. script().. Returns the name of the dispatch CGI, minus the.. , and any suffixes used for a secure server.. print error message(.. This returns an error message to the client, explaining how to use the server.. argument should be a string containing the version of the server software.. The error message returned is encoded in the.. file.. print help message().. This returns a help message to the client.. This can be issued in response to a confusing or inadequate URL.. The help message returned is encoded in the.. sample (simple) DODS dispatch CGI is shown in.. This is a Perl script using the.. methods.. This script assumes that all data is rooted in the http document directory subtree.. 9.. #!/usr/local/bin/perl use Env; use DODS_Dispatch; $dispatch = new DODS_Dispatch; $command = $dispatch- command(); if ($command ne "") { # if no error.. exec($command); } else { my $script_rev = '$Revision: 11906 $ '; $script_rev =~ s@\$([A-z]*): (.. *) \$@$2@; $dispatch- print_error_msg($script_rev); }.. A simple DODS data server dispatch CGI..

    Original link path: /api/pguide-html/pguide_25.html
    Open archive

  • Title: DODS Programmers Guide -- 4.1.2 The DAS and DDS filter programs
    Descriptive info: The simplest way to learn about creating a new filter program to return a dataset's.. is to examine the existing filter programs.. In this section, we will examine the.. servers.. The source code for the.. filter program distributed with the.. server software is shown in.. 2.. filters are very similar, so only the.. filter will be discussed here.. The important differences between the two will be pointed out.. The CGI dispatch program makes heavy use of commonly used functions collected in the.. In the same way, the.. DODSFilter.. class collects several commonly used functions for the construction of filter programs.. The example program uses several methods of that class.. Other useful utility functions are in the.. cgi-util.. collection.. The filter program in.. can be separated into the following steps:.. line 16.. Step 1: The.. class provides a constructor that parses the argument list to create the data.. You can use the.. OK.. method to check that the list was parsed properly.. Any errors here indicate a mistake in the dispatch CGI itself.. This is why the.. print usage.. function prints its message to the WWW server log file when it returns an error object to the client.. line 21.. Step 2: If the user has only requested version information from the server, it is provided here.. line 26.. Step 3: The.. read variables.. function performs the real work of this program.. This involves scanning the dataset itself for data variable attributes and using the.. method functions to assemble the corresponding.. This operation is specific to the data access API in use, so does not make a good example.. line 29.. Step 4: Each of the filter programs must create a.. Multipurpose Internet Mail Extensions.. document to hold its return value.. filters return a text MIME document; they set up the MIME headers using the utility function.. set mime text.. line 34.. Step 5: Once the data set has been read and the attribute table built, the.. ancillary file is loaded.. The example filter looks for a file with the same root name as the data set and an extension of.. If such a  ...   1; return 0; }.. The DAS filter program.. Note that the example filter in.. does not use any caching.. It is possible to build a more sophisticated filter program that saves the generated DAS to a text file and then uses that file without first interrogating the data set, thus saving on access.. It is also possible to write a DAS by hand and.. always.. use that if the data set does not contain any of the type of information that the DAS has.. Because the construction of the DAS and DDS objects requires that an entire data set be scanned, it can become very inefficient to continually rebuild these objects.. Because the.. filter programs use a text representation for transmission from the server to the client, it is simple to store both the.. objects once they have been created.. Subsequent accesses to these objects can be accomplished by reading and transmitting the textual representation without actually building the binary data object.. When taking advantage of this optimization, it is important that the server check the date stamp of the.. /.. text objects and compare it to the latest modification date of the data set.. For any dataset to which new data is periodically added, the.. text object must clearly be updated so that the cached text object matches exactly the object that would be created if the object were built by querying the data set.. The update of the.. text object can itself be optimized significantly.. It is not actually necessary to completely re-read the entire data set.. Because the software used to build both the.. and the.. binary objects work incrementally, it is possible to read text version of the.. object, and then read only the new parts of the data set.. The binary object will be added to as needed.. software may not properly update.. changed.. data (data that was present in a previous version of the data set, but is now different) nor is it straightforward to remove data which is no longer present in the data set.. In these cases it is usually better to regenerate the.. from scratch..

    Original link path: /api/pguide-html/pguide_26.html
    Open archive

  • Title: DODS Programmers Guide -- 4.1.3 The Data filter
    Descriptive info: The data filter program is structured similarly to both the.. filters except that it returns a binary MIME document rather than text and that it takes two arguments instead of just one.. In addition to the data set or file name (argument 1) it also takes the DODS constraint expression (argument 2, which was enclosed in the URL's.. query.. data filter is all but identical to the.. The only difference is that it calls the.. send data.. method of.. to send the binary data over the network.. This function calls the.. send.. method.. If for some reason you cannot use the.. member function of.. , then you must ensure that the the.. CE evaluation.. serialize.. operations are all carried out in the correct order.. Furthermore, you must ensure that the return  ...   the projection clauses of the constraint expression.. The text part is separated from the data by the keyword "Data:" at the start of the line.. 10.. DODS is packaged with a filter to translate a DODS data stream into an ASCII data file.. Clients can request ASCII data by appending.. to their URL instead of.. asciival.. program is useful as a standalone client (see.. ), but may also be used by a server to provide ASCII data.. A request for ASCII data is processed as any other request for data, but the final output of the data filter is piped into the.. program and the result returned to the client:.. nc_dods Data.. nc | asciival -m -- -.. class takes care of this step automatically, when it encounters a request using..

    Original link path: /api/pguide-html/pguide_27.html
    Open archive

  • Title: DODS Programmers Guide -- 4.1.4 The Usage Filter
    Descriptive info: Client requests containing a.. suffix should return to the client HTML text containing documentation of both the server usage and the dataset named in the query.. DODS provides a.. usage.. filter that can be used for this purpose.. class invokes this filter.. The DODS-provided.. filter accepts two arguments, the data file name requested and the name of the CGI script (the dispatch CGI) in use:.. CGI-name.. filter looks in the dataset directory for a file called.. html.. , and in the  ...   without the.. html.. head.. body.. tags.. For example, suppose a dispatch CGI using the.. class receives a URL like this:.. http://dods/cgi-bin/nph-nc/data.. info.. In this case, the.. filter looks for two files:.. cgi-bin/nph-nc.. data.. (the htdocs directory is assumed in the second case).. The contents of these two files are concatenated with an HTML representation of the.. for the.. file, and the whole thing is returned to the client.. If the HTML files are not found, the returned document contains only the..

    Original link path: /api/pguide-html/pguide_28.html
    Open archive

  • Title: DODS Programmers Guide -- 4.1.5 Documenting Your Work
    Descriptive info: 2 Client Libraries.. If you do write a server, and intend to circulate it beyond your own site, here are some guidelines for documenting that server that will help others use it.. Since there are two sets of "users" for a data server program, there are two sets of instructions that need to be prepared for a given server.. One set will be read by the person who installs and maintains the server on the host platform.. The other set is designed to be read by people who intend to request data from that server.. These users will get this documentation by submitting queries to the Info Service, in rather the same way that many UNIX commands have a.. -usage.. option.. In addition to these two documents, all servers should include a set of text files in their distribution directory.. README.. file should contain the following information:.. A brief overview that describes the purpose and method of operation of the server.. The revision level of the server.. Any features the local.. daemon must support to use this server.. Any data translations that this data server can do.. If any are done, they should be described in detail, so that users can know what data they get.. ERRORS.. file should contain a complete list of the error messages and explanations that might ever be issued by the server.. These instructions should be included in a file called.. INSTALL.. which is to be included with the server distribution.. At a minimum, they should cover the following topics:.. Configuring and compiling the server code.. Ideally, there should be a.. configure.. script included, but detailed instructions on editing the.. Makefile.. will often suffice.. Remember to install the usage data file somewhere the server can find it.. Are there any environment variables that must be defined in order to run the server? Are there other programs (e.. g.. gzip.. that must be installed on the host machine?.. What configuration options are there for the installed server? This covers issues like enabling data compression, ancillary data caching, and choosing the GUI manager program with which the server will communicate.. If there are performance trade-offs associated with each option, note them here.. Ancillary data files:.. Must the installer prepare ancillary data files by hand, or are these created automatically and cached?.. If they must be created, where ought they be put?.. If they are cached, where are they kept?.. Also, if the ancillary data files are cached, what implications are there for updating the data sets served by this server? (i.. must the ancillary data files be updated also? Deleted and recreated?).. What temporary files will be created by the server? Where will they be stored? Under what conditions  ...   are to be controlled by the user.. 11.. A list of the programs a user should have to use certain features of the server.. For example, note here that the server expects that the GUI manager is running a Tcl interpreter.. A list of the error messages that the user is apt to see.. Include explanations of the conditions that may have caused them, and any steps the user may take to recover from them.. The answers to any questions you are frequently asked about this server or its usage.. The usage data file need not be any more elaborate than any man page.. To create information for a server, write an HTML fragment using the format given below, and put the HTML file in the same directory as teh server.. Name it using the base name of the server; for example, the HTML file that describes the netCDF server (made up of.. , and.. nc das.. nc dods.. and so on) is called.. nc.. This example shows the correct HTML tagging for server information:.. h3 Server Function: /h3 dl dt geolocate(variable, lat1, lat2, lon1, lon2) /dt dd Returns the elements of em variable /em that fall within the box created by ( em lat1 /em , em lon1 /em ) and ( em lat2 /em , em lon2 /em ).. /dd p dt time(variable, start_time, stop_time) /dt dd Returns the elements of em variable /em that fall within the time interval em start_time /em and em stop_time /em.. /dd /dl p.. For datasets, put the HTML file, tagged using the format given below, in the same directory as the datasets.. Name it using the base name of the datasets; for example, the HTML file for.. fnoc1.. fnoc2.. fnoc3.. might be called.. fnoc.. This example shows the correct HTML for a dataset information file:.. h3 About the dataset /h3 This is where the server administrator would supply information about the dataset.. And so on.. p.. You may prefer to override this method of creating documentation and simply provide a single, complete HTML document that contains general information for the server or for a group of datasets.. For example, to force the info server to return a particular HTML document for all its datasets, you would create a complete HTML document and give it the name.. dataset.. ovr.. , where.. is the dataset name.. The HTML file in this case would look like this:.. html head title Override document /title /head body h2 Test dataset /h2 This is where the server administrator would supply information about the dataset(s) and what-have-you.. /body /html.. Remember to ensure that the installation instructions cover installing the usage data file in a place where the server can find it..

    Original link path: /api/pguide-html/pguide_29.html
    Open archive

  • Title: DODS Programmers Guide -- 4.2 Client Libraries
    Descriptive info: 1 Rewriting the Open and Close Functions.. The goal of building a client library is to provide a drop-in replacement for an existing API so that user programs written for that API can switch to the DODS version and access remote DODS data.. The user programs should not require any modification to change over to the DODS client library version of the API.. However, the  ...   the DODS client library for a particular API, it is useful to divide the API to be re-implemented into five categories of functions:.. Open or connect.. Variable information read.. Data read.. Write.. Close or disconnect.. 2 Getting Information about Variables.. 3 Reading the Values of Variables from a Dataset.. Translation.. 4 Functions that Write to Data Sets.. 5 Adding Local Access to a DODS Client Library..

    Original link path: /api/pguide-html/pguide_30.html
    Open archive

  • Title: DODS Programmers Guide -- 4.2.1 Rewriting the Open and Close Functions
    Descriptive info: The functions that perform the dataset "open" and "close" operations must be implemented so that information about the data set can be retrieved from the data server.. These functions must store the necessary state information so that subsequent accesses for variable information or data reads can be satisfied.. This state information will, in almost every case, be the dataset's.. The open function for a DODS client library version of a given API must first determine if the data object (typically a file) is local to the user program making the open call or is a remote data object to be accessed through DODS.. It is possible to access DODS objects which are local to a user program, but there is little reason to do so if the data object can also be accessed through the original API.. In any case, the distinction of local or remote is made on the basis whether a URL is used to reference the data  ...   be passed back to the user program as the return value of the open function.. You add this data to the.. objects when you sub-class them for a particular API.. Subsequent accesses to the data set will include this identifier (or pointer), and each function that is a member of the API can be modified to use it to gain access to the state information stored by the open function.. The close function should use the state information accessible with the file identifier or pointer returned by the open function to determine if the dataset is local or remote.. In the case of a local data set, the original implementation's close function must be called.. In the case of a remote data set, the locally stored state information must be freed.. You can do this by destroying the.. object.. Section.. 3.. for an example of a recoded open function and a description of its use.. (The example uses the.. API..

    Original link path: /api/pguide-html/pguide_31.html
    Open archive

  • Title: DODS Programmers Guide -- 4.2.2 Getting Information about Variables
    Descriptive info: Most APIs for self-describing data sets include functions which return information about the variables that comprise a data set.. These functions return information about the type and shape of variables in a form that can be used by a program as well as attribute information about the variables that is more often than not intended for use by humans.. Each of these functions must be rewritten so that to the extent possible, information present in the.. is used to satisfy them.. While many `self-describing' APIs may have dozens of these functions, the basic structure of the  ...   the locally stored state information (.. ) to answer the request for data.. Rewriting these functions can be the most labor intensive part of re-implementing a given API.. This is typically the largest group of functions in the API and the information stored in the.. must often be `massaged' before it fulfills the specifications of the API.. Thus the rewritten functions must not only get the necessary information from the.. objects, but they must also transform the types of the objects used to return that information to the user program into the data types the program expects..

    Original link path: /api/pguide-html/pguide_32.html
    Open archive

  • Title: DODS Programmers Guide -- 4.2.3 Reading the Values of Variables from a Dataset
    Descriptive info: To read data values from a dataset using a typical data access API, a user would submit to some API function the name of the variable to be read.. The DODS client library version of this same function must take that variable name and use it to construct a constraint expression.. (See Section.. 3.. for more information on using constraint expressions to access data.. ) The constraint expression must then be appended to the dataset URL (with the suffix.. ), and the resulting URL sent out into the internet.. For example, to get a variable called.. var.. from a dataset at:.. http://blah/cgi-bin/nph-nc/weekly.. dods.. you would use the URL:.. dods?var.. class contains a member function,.. request data.. that performs this task.. It takes the constraint expression and the suffix to use for requesting data, appends them to the.. URL, and sends the entire string off to retrieve its corresponding data.. function returns a pointer to a.. object, which contains the data as well as the structure description corresponding to the data request.. Once.. the.. member function has returned, the client library must still call the.. deserialize.. member function (which is part of the DODS Type Classes) for each returned variable.. The client library should use the variable objects contained in the.. object returned  ...   buf2val.. member function for the cardinal and vector types and by accessing the values of fields for constructor types.. For a DODS client library to be robust, it may have to be equipped to deal with data types it was not designed to use.. For example, the.. software cannot manipulate a DODS Sequence.. But a user can use the DODS version of the.. library to request data from a server that provides Sequence data.. When cases like this arise (and they arise farily often), the author of the client library must choose an appropriate data type into which the served data is to be translated, and implement functions to do that translation.. Often, translation from one data type to another is a simple task.. Translating an Array into Sequence format is fairly straightforward, although there are several ways to do it.. (The author of the client library should choose one, and document that choice in a README file.. Other translations are more complex, and may even require that the client library violate the semantics of the original API, or of one of the DODS data types.. For example, translating a Sequence to an Array in.. requires that the client know in advance the length of the Sequence, which is not necessarily known..

    Original link path: /api/pguide-html/pguide_33.html
    Open archive

  • Title: DODS Programmers Guide -- 4.2.4 Functions that Write to Data Sets
    Descriptive info: DODS is a read-only data system.. While it is not technically inconceivable, a system which allows modification of remote data sets would be operationally much more complex than DODS.. Thus, functions that write data are rewritten so that they call the original implementation in the case of a local access or return an error code in the case of a remote access.. The error code should indicate a recoverable error so that programs which perform both reads and writes can recover if their logic permits..

    Original link path: /api/pguide-html/pguide_34.html
    Open archive

  • Archived pages: 580