### RedTitan Enterprise Data Processor: Reference semantics (version 1.40)

 Apply ApplyFDD Assert AskWizard Attr AttrData Buffer Capture Clean Concat Copy Count CSVparse DataAttr DataBase Date Def Demote Dispose Edit EncodeCSV EncodeXML Eval Execute External FDDdata FDDparse FileName Files FileWatch Format Group HostInfo HTML Install Internal Join LookUp LookUpS MakeDir Match Merge Modify MsDOS OpenWizard Pages Print Promote Readfile Rename Revise SaveWizard Select Separate Sort SQL Substitute Summary Sync Table TaskList TCP TCPserver Text Timer Unwrap View Wrap WriteFile XMLparse Appendix

### Execute

This function runs a program each time a target element is encountered. The command line is constructed by means of a script contained in the content of the options. Option attributes specify the program name, input target and whether processing will wait until the program terminates execution.

 Port Direction Description DATA IN The data stream that triggers program execution. The program is run for each targetted element in the data stream. DATA OUT The output consists of a copy of the input data stream. The target element content is deleted, and an additional attribute value COMMAND is inserted into the header being the actual command line executed. If the WAIT option is TRUE (default) then the target element is not output until the specified program exits.

 Property Default Description TARGET 0 The target element, either the level number (defaults to 0 - the outermost level), or the element tag name. TAG all tags are used The tag name used in the script for replacable elements. PROGRAM REQUIRED Specifies the name of the program to be run. WAIT TRUE A Boolean value to specify if processing should wait until the program terminates. The default is TRUE - i.e. that processing is suspended until the program exits.
 Content The command line (note that line-breaks are treated as single spaces). This may contain elements, which will be replaced by same-named attribute values from the target element. See Appendix

#### Example:

 Port Value DATA (in) DATA (out)

### MsDOS

This function runs an input stream as an MsDOS BAT script and awaits its completion. The standard output is collected and made available as the LOG output stream.

 Port Direction Description Script IN The BAT script to be executed. The data content of each outer element is made into a BAT file and executed. LOG OUT The LOG output from the execution of each BAT job.

#### Example:

 Port Value Script (in) dir c:\windows\*.sys [EOF] LOG (out) C:\NDP>dir c:\windows\*.sys Volume in drive C is DISK1_VOL1 Volume Serial Number is 8AA0-5D38 Directory of C:\WINDOWS HIMEM SYS 33,191 11/05/98 20:01 HIMEM.SYS RAMDRIVE SYS 12,663 11/05/98 20:01 RAMDRIVE.SYS LOGOS SYS 129,078 11/05/98 20:01 LOGOS.SYS LOGOW SYS 129,080 11/05/98 20:01 LOGOW.SYS ASPI2HLP SYS 1,105 11/05/98 20:01 ASPI2HLP.SYS CMD640X SYS 24,626 11/05/98 20:01 CMD640X.SYS CMD640X2 SYS 20,901 11/05/98 20:01 CMD640X2.SYS DBLBUFF SYS 2,614 11/05/98 20:01 DBLBUFF.SYS IFSHLP SYS 3,708 11/05/98 20:01 IFSHLP.SYS NDISHLP SYS 6,140 11/05/98 20:01 NDISHLP.SYS 10 file(s) 363,106 bytes 0 dir(s) 1,030.54 MB free C:\NDP> [EOF]

### Print

This function makes a Windows Shell-services PRINT request on a document.

 Port Direction Description DATA IN The data stream that triggers the print request. LOG OUT The output consists of a copy of the input data stream. The target element content is deleted. If the WAIT option is TRUE (default) then the target element is not output until the specified program exits.

 Property Default Description TARGET 0 The target element, either the level number (defaults to 0 - the outermost level), or the element tag name. USING REQUIRED The attribute name in the target element that will supply the document filename to be printed. WAIT false A Boolean value to specify if processing should wait until the program printing the document terminates. The default is false - i.e. that processing continues without waiting.

### HTML

This function displays the data stream that flows to/through it in a window for the user to examine but which is assumed to be XHTML data. It appears directly in the nQ remote access display and so permits an arbitary HTML display to be available for remote browsing.
Note that the window will not appear until some actual data has been read. An empty data stream does not cause the window to be created.

 Port Direction Description DATA IN The data input stream. DATA OUT For future use, currently no data is generated.

 Property Default Description CAPTION The task name The title for the window.

### Table

This function displays data attributes in tabular form, both in a local window and also in the remote browser display. Each element at the targetted level generates one table row.

 Port Direction Description DATA IN The data input stream. DATA OUT Not specified yet - DO NOT USE

 Property Default Description CAPTION The task name The title for the tabular display. TARGET 0 The table row elements, either the level number (defaults to 0 - the outermost level), or the element tag name.

### Edit

A diagnostic function that displays data as it flows through the block, but with the possibility that the user can edit it before it is output.

 Port Direction Description DATA IN The input data. Each element is loaded into the display dialogue and then available for manual editing. DATA OUT The output data.

### View

This function displays the data stream that flows to/through it in a window for the user to examine. Useful as a diagnostic aid. Note that the window will not appear until some actual data has been read. An empty data stream does not cause the window to be created.

 Port Direction Description DATA IN The data input stream. DATA OUT The output stream which is just a copy of the input stream after it has been displayed.

 Property Default Description CAPTION The task name Supplies a title for the view window. FORMAT true Specifies that the display is to be formatted to aid readability. Specify FALSE to see the data displayed without added line-breaks. LIMIT infinite Specifies the maximum continuous number (approximately) of data characters to be displayed. When data display stops the text "[data]" is added to the display line to indicate that it is truncated. This is very useful when dealing with data streams containing huge quantities of character data, especially as the display window creation is quite a slow process.

This function displays a wizard panel to ask the user for an input value. The main text area of this dialogue is filled in from the data content of the properties after substituting values from the attributes of the current input element (if any). The output element is modified by adding an attribute whose value is the filename specified. The original content is retained.

 Port Direction Description Source IN An optional input stream that supplies data that can be used to initialise the wizard. Response OUT An output stream that is either a copy of the input or a new element.

 Property Default Description CAPTION "Open file" Supplies a title for the wizard window. REPLACE input element Supplies a name for the generated element tag. By default the input element is retained with its attributes. Where no input exists the default name is INFO CREATE REPLY Supplies the attribute name to be used for the resulting attribute. DEFAULT Supplies a default value to be written in to the input field on the dialogue. MINCHARS 0 Specifies the minimum number of characters acceptable for the response value. MAXCHARS 255 Specifies the maximum number of characters acceptable for the response value.

A dialogue box is displayed to the user when a level 0 element is received, or at startup if there is no input source. When the user enters a response and answers OK then the result is output along with its associated content. If the user answer CANCEL then no output is generated (and the input element is lost).

 Content This is used to fill the message box in the user dialogue. This may contain elements, which will be replaced by same-named attribute values from the target element. See Appendix

#### Example1:

(with no options specified)
 Port Value Response (out) [EOF]

### OpenWizard

This function displays the file-open wizard panel to the user - and awaits a filename specification. The main text area of this dialogue is filled in from the data content of the properties after substituting values from the attributes of the current input element (if any). The output element is modified by adding an attribute whose value is the filename specified. The original content is retained.

 Port Direction Description Source IN An optional input stream that supplies data that can be used to initialise the wizard. FileName OUT An output stream that is either a copy of the input or a new element.

 Property Default Description CAPTION "Open file" Supplies a title for the wizard window. FILTER All files Supplies filters for the file-open dialogue. These are currently a list of description|*.xxx items which are additional to the "All Files" specification which is always present. DIRECTORY Current directory Supplies a default directory path for the file-open dialogue. REPLACE input element Supplies a name for the generated element tag. By default the input element is retained with its attributes. Where no input exists the default name is FILE CREATE NAME Supplies the attribute name to be used for the resulting filename. DEFAULT Supplies a default filename to be written in to the filename field on the dialogue. NAME *.* Specifies the filename template. This can be the complete filename, or a wildcard specification to be applied to the selected attribute of the input stream. Use this to set the default filename and directory from the input data stream. USING NAME Specifies the name of the attribute to be used from the input stream for construction of a filename.

A dialogue box is displayed to the user when a level 0 element is received, or at startup if there is no input source. When the user chooses an existing filename and answers OK then the result is output along with its associated content. If the user answer CANCEL then no output is generated (and the input element is lost).

 Content This is used to fill the message box in the user dialogue. This may contain elements, which will be replaced by same-named attribute values from the target element. See Appendix

#### Example1:

(with no options specified)
 Port Value FileName (out) [EOF]

### SaveWizard

This function displays the file-save wizard panel to the user - and awaits a filename specification. The main text area of this dialogue is filled in from the data content of the properties after substituting values from the attributes of the current input element (if any). The output element is modified by adding an attribute whose value is the filename specified. The original content is retained.

 Port Direction Description Source IN An optional input stream that supplies a data that can be used to initialise the wizard. FileName OUT An output stream that is either a copy of the input or a new element.

 Property Default Description CAPTION "Save file" Supplies a title for the wizard window. FILTER All files Supplies filters for the file-open dialogue. These are currently a list of description|*.xxx items which are additional to the "All Files" specification which is always present. DIRECTORY Current directory Supplies a default directory path for the file-open dialogue. REPLACE input element Supplies a name for the generated element tag. By default the input element is retained with its attributes. Where no input exists the default name is FILE CREATE NAME Supplies the attribute name to be used for the resulting filename. DEFAULT Supplies a default filename to be written in to the filename field on the dialogue. NAME *.* Specifies the filename template. This can be the complete filename, or a wildcard specification to be applied to the selected attribute of the input stream. Use this to set the default filename and directory from the input data stream. USING NAME Specifies the name of the attribute to be used from the input stream for construction of a filename.
 Content This is used to fill the message box in the user dialogue. This may contain elements, which will be replaced by same-named attribute values from the target element. See Appendix

#### Example1:

(with no options specified)
 Port Value Source (in) Data content [EOF] FileName (out) Data content [EOF]

### External

This function converts a data stream to a compact textual representation suitable for short-term storage in external media or transmission across a communications channel. Note that such a data stream can only definitely be read back by the same version of the program and so should not be archived. This function is used by the spooling and database functions. The data stream can optionally be encrypted for security in transmission.

 Port Direction Description DATA IN The data stream to be encoded. Each outermost element is encoded seperately. DATA OUT The resulting output data stream, which now consists of a stream of elements whose content is pure data. Use the Internal function to decode this data format.

 Property Default Description KEY No encryption Supplies a key to be used encrypt the data stream. The same key must be specified to Internal to decrypt the data.

#### Example1:

(with no options specified)
 Port Value DATA (in) The quick brown fox. [EOF] DATA (out) nDPx1���ª�The quick brown fox. � [EOF]

with option:
 Port Value DATA (in) The quick brown fox. [EOF] DATA (out) nDPx1èÿÿÿü$ÁÝèÕÓ|®G¥l)||å|³¸¶ -t˜ [EOF] See also: Internal ### Internal This function converts the compact textual representation created by External back to the original data stream. This function is used by the spooling and database functions.  Port Direction Description DATA IN The data stream to be decoded. Each outermost element is decoded seperately and should consist of pure data. DATA OUT The resulting output data stream, which now recreates the original data stream encoded by External  Property Default Description KEY No encryption Supplies a key to be used decrypt the data stream. This must be the same key as was specified to the External function that created the data stream. #### Example1: (with no options specified)  Port Value DATA (in) nDPx1���ª�The quick brown fox. � [EOF] DATA (out) The quick brown fox. [EOF] #### Example2: with option: KEY="password"  Port Value DATA (in) nDPx1èÿÿÿ|>Ö¸|/|£| G|Ì|ð|Ç/ [EOF] DATA (out) The quick brown fox. [EOF] See also: External ### CSVparse Read elements of raw data in CSV format and convert to a data stream. The raw data content of each element is converted to a list of CSV elements, each of which contains a list of elements (named FLD1, FLD2 etc.) with data content being the CSV fields.  Port Direction Description DATA IN The raw CSV data. DATA OUT The encoded CSV data stream.  Property Default Description QUOTE " Specifies the character to be treated as a quoation mark. The default is the double-quote character. SEPARATOR , Specifies the character that is used to separate the data fields. The default is the comma. RECORD CSV Supplies a tag name to be used for the record elements. FIELD FLD# Supplies a tag name to be used for the data field elements. If "#" characters are appended to the name then the field number within the record is appended to the tag name with a minimum number of digits specified by the "#" characters. #### Example: (with no options specified)  Port Value DATA (in) item1,item2,"item3" another item,,last one DATA (out) item1 item2 item3 another item last one See also: EncodeCSV, ApplyFDD, FDDparse, EncodeXML, XMLparse, External, Internal ### EncodeCSV Write a data stream as CSV data. The data values are taken first from the attributes, and then from the element content.  Port Direction Description DATA IN The data stream to be written as CSV. DATA OUT Elements of pure data content that represent the data stream in CSV form.  Property Default Description QUOTE " Specifies the character to be used as a quoation mark. The default is the double-quote character. SEPARATOR , Specifies the character that is used to separate the data fields. The default is the comma. NAMES omit field names Specifies a list of field names to be included in the output. If this parameter is specified (and a blank specification is treated as "all names") then the first record contains the field names. Note that using this option also controls the order of output of the data values, which by default will be determined by the data stream itself. Any values that are coming from attributes will, by default, occur in alphabetical order. #### Example: (with no options specified)  Port Value DATA (in) SIZE="45"> item1 item2 item3 another item last one DATA (out) "item1","item2","item3" "another item",,"last one" See also: CSVparse, ApplyFDD, FDDparse, EncodeXML, XMLparse, Internal, External ### EncodeXML Write a data stream as XML data.  Port Direction Description DATA IN The data stream to be written as XML. XML OUT Elements of pure data content that represent the data stream in XML syntax.  Property Default Description FORMAT true This option being true causes extra line-breaks to be added to the XML output so as to make the resulting file more readable and editable. Unless specific action has been taken, data streams do not tend to contain line-breaks and therefore the unformatted XML output would probably appear as a single long line to some text editors. WRAP false - subject to change When true, this option results the root element being retained as wrapping for the resuting data stream. This is convenient if the next function is WriteFile #### Example: (with no options specified)  Port Value DATA (in) Test XML First line. Last line. [EOF] XML (out) Test XML First line. Last line. [EOF] See also: XMLparse, EncodeCSV, CSVparse, ApplyFDD, FDDparse, Internal, External ### XMLparse Parses the raw data stream as XML data. This parser supports UTF-8, ISO-8859-1 and ASCII encodings at present - and also UCS-2 and UCS-4 but without auto-recognition. Internally, the parser is an SGML parser and is slightly forgiving of some of the more strict syntax rules in XML. It will also parse a sequence of multiple documents sourced from a single data stream.  Port Direction Description XML IN The data to be processed as XML data. DATA OUT The parsed XML data stream.  Property Default Description TARGET outer The target level number or element name. By default, the outermost level is processed and also the element header is removed leaving the parsed XML document in place of the original element. If a target is specified then the original element is always retained with its content being replaced by the output of the XML parser. Note that this means that specifying a target level of 0 does not produce the same results as leaving the target unspecified. BAD abort on error Supples a tag to be used as the root element if it turns out that the XML data is not valid. This prevents an error-halt for the task. Note that the use of this option requires the function to store all the XML data before forwarding since an error could occur at any time so this is most appropriate for short XML sequences. #### Example:  Port Value XML (in) Test XML First line. Last line. [EOF] DATA (out) Test XML First line. Last line. [EOF] See also: EncodeXML, EncodeCSV, CSVparse, ApplyFDD, FDDparse, Internal, External ### FDDparse This function reads an FDD data description (as created by the RedTitan Page Designer) and converts it to a data stream suitable for use by ApplyFDD.  Port Direction Description FDD IN A data stream supplying the contents of an FDD file (but see the FILENAME option below. DATA OUT The FDD data description for feeding to ApplyFDD  Property Default Description FILENAME Use input data Specifies the name of the FDD file. This option causes the function to read the specified file rather than processing the data from the input stream. #### Example: (with no options specified)  Port Value FDD (in) DATA NAME PNUM 4 WIDE AT 1 8; DATA NAME PIPE 7 WIDE AT 1 1; [EOF] DATA (out) [EOF] See also: ApplyFDD, EncodeXML, XMLparse, EncodeCSV, CSVparse, Internal, External ### ApplyFDD This function decodes raw data using an FDD specification, which means that it processes row+column data descriptions and also tagged data. The FDD specification would normally be generated by FDDparse  Port Direction Description DATA IN The raw data stream to be processed. FDD IN The data description, ususally generated by FDDparse. DATA OUT The resulting data stream.  Property Default Description TARGET 1 This can either be a level number (0 being the outermost level) or the name of the elements to be selected. The targeted elements supply raw text data from which fields are identified and output. PRESERVE false If "true" is specified then the original data is retained. The extracted fields are appended to the raw data. FIXEDWIDTH variable If an integer value is specified then lines are broken after the fixed number of characters specified. A zero value requests that the fixed line length is calculated from the data specification. By default lines are terminated by a line-end marker. FIXEDHEIGHT variable If an integer value is specified then a record is complete after the specified number of lines have been read. A zero value requests that the line count be calculated from the data specification. By default the record terminates at the end of the data content. See also: FDDdata, FDDparse, EncodeXML, XMLparse, EncodeCSV, CSVparse, Internal, External ### FDDdata This function constructs row/column data from attribute values in the input data stream according to an FDD specification. The FDD specification would normally be generated by FDDparse  Port Direction Description DATA IN The data stream to be processed. FDD IN The data description, ususally generated by FDDparse. DATA OUT The resulting raw data stream.  Property Default Description TARGET 1 This can either be a level number (0 being the outermost level) or the name of the elements to be selected. The targeted elements supply attribute fields from which the raw data is constructed. See also: ApplyFDD, FDDparse, EncodeXML, XMLparse, EncodeCSV, CSVparse, Internal, External ### Match This function cuts up data content by searching for specified strings of characters.  Port Direction Description DATA IN Elements containing raw data content. DATA OUT The resulting stream with elements marking out the matched data.  Property Default Description TARGET 1 The target element, either the level number, or the element tag name. The data content of the target element is examined and modified if matched. BEGIN REQUIRED if END not specified Specifies a string that denotes the start of a section of data. If no END is specified then a section ends whenever another match of this string occurs. END REQUIRED if BEGIN not specified Specifies a string that denotes the end of a section of data. If no BEGIN is specified then a section begins at the first data character. TAG MATCH Specifies the name for the element that will be created to mark out the matched data section. Instead of an element start and end bracketing the selected data, data strings can be specified. In this case, the tag is written as a pair of quoted strings,e.g. "{","}" DATA unchanged Specifies the name for the element that will be created to mark out data that is not in the matched section. By default the unmatched data will remain as it was. If this option is explicitly specified as an empty then the unmatched data will be deleted from the stream. NEST false Specify true if the BEGIN and END strings can be nested. DISPOSITION delete Specifies what happens to the BEGIN/END strings that mark out the match data. (default) results in the strings being deleted. 0 for strings to be included within inside single start/end element. 1 causes the strings to be included inside the new TAG element. -1 causes the strings to be included outside the new TAG element, i.e. at the end/start of the outer content. LIMIT infinite Specifies the number of data characters to be passed before matching terminates. There are many different overall modes of operation for this function. The simple way to understand the possibilities is to consider its operation as a step-by-step process. • A matched region starts when the string specified by BEGIN is encountered. If not specified then the region starts with the first data character. • The matched region finishes when the string specified by END is encountered or when the target level ends. If not specified then the BEGIN string is used and the region finishes before the next BEGIN is encountered. • If NEST is true then an inner region can be started by a further instance of the BEGIN string. If false then the region must finish before another start is considered. • Where both the BEGIN and END matches at the same position in the data then the longest match will be used. If BEGIN and END are the same string then the region both starts and finishes at the same place. The effect of this depends on the state of NEST, where false results in the match string alternately starting and ending a region, and true results in only the match string itself becoming the region. • The DISPOSITION setting now deterines what happens to the match string itself. The default action is to not include it in the output. Other possibilities are to have it included either inside or outside the matched region thereby making it part of the outlying data or as part of a region. The special (0) option puts it outside either of these, and in the special case of the BEGIN and END strings being the same in the nested mode then the match string appears as the regions only content. • Data that is not included in any region can be optionally packaged as specified by the DATA option, which includes the possibility of having this outlying data removed from the output. • The matched region is packaged according to the TAG specification. This allows for it becoming the content of a new element, or being prefixed and suffixed by specified character data. See also: Text ### Text This cuts up data into lines by looking for [RET] and/or [LF] characters and replacing them with the EOLN marker. It will optionally package the lines into pages either delimited by a FORMFEED ($0C) character occurring anywhere in the data, by a specified TAG character occurring at the start if a line, and by a line count being exceeded.

 Port Direction Description DATA IN The raw data input. TEXT OUT The output data with line-breaks encoded as EOLN markers, and enclosed inside page elements if pagination was specified.

 Property Default Description TARGET All levels. The target element, either the level number, or the element tag name. PAGE No pagination. The tag name to be used for page records. If this is specified then pagination will be done. The output consists of a series of elements with this tag - the content being the text lines. If the LINES property is not specified then pages end on the ascii Form-Feed character. LINES Infinite The number of lines per page. TAG None A character occurring in the first column of a line that indicates that the line is the first of a page. Use this to handle data using line-printer style PCC characters. OMIT [RET] A character to be ignored. This is used to obtain the required behaviour where [RET] or [LF] characters are to be ignored.

#### Example:

(with no options specified):
 Port Value DATA (in) RedTitan Ltd., Main Road, Walter's Ash. [EOF] TEXT(out) RedTitan Ltd.,Main Road,Walter's Ash.[EOF]

### Pages

This makes the specified target elements into ASCII data pages by removing the element header and replacing the end tag with a FormFeed (0C) character. It is effectively the inverse of the "Text" function when the "PAGE" option had been specified.  Port Direction Description TEXT IN The lines of text. DATA OUT The resulting stream of ASCII pages.  Property Default Description TARGET 1 The target element, either the level number, or the element tag name. PRESERVE false Specify true to retain the element wrapping thus only adding the formfeed characters to the data stream. See also: Text ### Timer This function generates an element at repeated intervals, providing a convenient way of prompting a routine process.  Port Direction Description DATA OUT The output data, a stream of EVENT elements.  Property Default Description EVERY 1M Time between events. The default unit is seconds, but you can specify the time in other units by means of a suffix letter: S for seconds (default) M for minutes H for hours D for days FIRST =EVERY Time from startup to the first event. If not specified then this will be the same as the EVENT property. #### Example:  Port Value DATA (out) See also: Date ### DataBase This function provides information about ODBC databases. It will list the available ODBC database names, the table names within a database, or column names within a table. The function can be used with out without an input stream. Note that ODBC database access requires that your system has the 32-bit ODBC access mechanism installed and configured appropriately and also that your nDP system has the nDPODBC.DLL installed.  Port Direction Description DATA IN Optional input stream. When present, the target elements can be a source some of the parameters. DATA OUT A copy of the input stream, but with the target elements containing the specified list of names.  Property Default Description TARGET 0 The target element, either the level number (defaults to 0 - the outermost level), or the element tag name. DB The name of the database to be opened. DB-FROM The name of the attribute that specifies the database name to be opened. TABLE The name of the table within the specified database to be opened. TABLE-FROM The name of the attribute that specifies the name of the table within the specified database to be opened. USER The user name to be supplied on attempting to access the database. PASSWORD The password required to obtain access to the database. See also: SQL ### SQL This function makes an SQL query to the specified database and reads the results. By default a query is constructed to read the whole of the specified table. Note that ODBC database access requires that your system has the 32-bit ODBC access mechanism installed and configured appropriately and also that your nDP system has the nDPODBC.DLL installed.  Port Direction Description DATA IN The optional input data stream which can supply data to be used to construct the query. DATA OUT A copy of the input data stream with the query results inserted as content of the target element. The query results consists of a series of RECORD elements containing elements which are either: Named using the field name where this is a valid name for an element (spaces are changed to "_" characters). Named FIELD with attribute NAME being the original name.  Property Default Description TARGET 0 The target element, either the level number (defaults to 0 - the outermost level), or the element tag name. DB The name of the database to be opened. DB-FROM The name of the attribute that specifies the database name to be opened. TABLE The name of the table within the specified database to be opened. Use this if you are not supplying an SQL query. TABLE-FROM The name of the attribute that specifies the name of the table within the specified database to be opened. Use this if you are not supplying an SQL query. USER The user name to be supplied on attempting to access the database. PASSWORD The password required to obtain access to the database.  Content This is used to create the (optional) SQL query. The default query is to read the whole speified table. This may contain elements, which will be replaced by same-named attribute values from the target element. See Appendix See also: DataBase ### LookUp This function uses one stream to construct a database and then performs relational lookups on it merging the resulting data into the key stream. The database stream (upper) is read to construct a database for the start of each new bacth on key records. If the stream terminates then the last loaded database is reused. A new batch of key records is deemed to start whenever a containing element of the last used key ends. This means that you can use the same database to process many files of data records, or have each file of keys looked up in a corresponding database.  Port Direction Description DATA IN The database. Keys IN The stream that supplies the keys for look-up in the database. DATA OUT The resulting output data stream.  Property Default Description TARGET 1 The target element, either the level number, or the element tag name. This specifies which elements are used as keys to be looked-up in the database. LOOKUP same as KEY Specifies the key attribute name that selects data from the database. KEY REQUIRED Selects the field attribute to be used as the database key. RANGE exact This option specifies the mode of access. By default the lookup keys must be exactly found. Other modes are to accept a nearby entry by either moving UP (towards higher valued key) or DOWN (towards the start). NOTE: be careful about the ordering of the keys when using the non-exact modes. The order will depend on the data type of the key values so if a numeric colating sequence is required then ensure that the key values are converted to numeric form (e.g. using Eval) otherwise the textual ordering of the keys will be used. #### Example: Options: KEY="LOCATION" LOOKUP=KEY  Port Value DATA (in) KEYS(in) [EOF] DATA(out) [EOF] See also: LookUpS, Readfile, WriteFile, Rename, Files, FileWatch, FileName ### LookUpS This function perfoms the same database lookup function as the LookUp function, but has the extra feature of recording statistics about the database use which become available when the database is closed. The database stream (upper) is read to construct a database for the start of each new bacth on key records. If the stream terminates then the last loaded database is reused. A new batch of key records is deemed to start whenever a containing element of the last used key ends, unless the ONCE option is specified as true. This means that you can use the same database to process many files of data records, or have each file of keys looked up in a corresponding database. The first output stream is exacly the same as for the LookUp function, being a copy of the second input stream but potentially with extra data merged from the selected database record. The second output stream provides the usage statistics. It is a copy of the database stream (albeit with any duplications removed) in key value order with an additional attribute added to each record element giving the number of times that the record was transmitted to the output stream. Note: if you do not want the record access information then use LookUp as recording the usage information makes this function slower.  Port Direction Description DATA IN The database. Keys IN The stream that supplies the keys for look-up in the database. DATA OUT The resulting output data stream. Stats OUT The statistics information in the form of the keys stream (sorted) with added attribute.  Property Default Description TARGET 1 The target element, either the level number, or the element tag name. This specifies which elements are used as keys to be looked-up in the database. LOOKUP same as KEY Specifies the key attribute name that selects data from the database. KEY REQUIRED Selects the field attribute to be used as the database key. RANGE exact This option specifies the mode of access. By default the lookup keys must be exactly found. Other modes are to accept a nearby entry by either moving UP (towards higher valued key) or DOWN (towards the start). NOTE: be careful about the ordering of the keys when using the non-exact modes. The order will depend on the data type of the key values so if a numeric colating sequence is required then ensure that the key values are converted to numeric form (e.g. using Eval) otherwise the textual ordering of the keys will be used. COUNT COUNT Specifies the attribute name to be used for the usage counter in the statistics output stream. ONCE false A true value requests that the database is only loaded once. The effect of this is that only one statistics copy of the database is generated when the key stream terminates. See also: LookUp, Readfile, WriteFile, Rename, Files, FileWatch, FileName ### MakeDir This function creates file directories to receive files. The directory is extracted from the specified filename and will be built if it does not already exist. Use this function immediately before writing a file or renaming a file to ensure that the destination directory exists.  Port Direction Description DATA IN A stream of elements (at the outer level only) that each name a file that needs a directory building. DATA OUT A copy of the input stream.  Property Default Description NAME *.* Specifies the filename template. This can be the complete filename, or a wildcard specification to be applied to the selected attribute of the input stream. USING NAME Specifies the name of the attribute to be used from the input stream for construction of a filename. See also: WriteFile, Rename ### ReadFile Reads one or more disc files.  Port Direction Description FileName IN Selected element attribute (default NAME) supplies filename details DATA OUT For each file successfully opened, a single FILE element consisting of its entire contents (as unparsed data) is generated. It has a NAME attribute which conatins the full-path filename and a SIZE attribute giving the file size in bytes. Other attributes are copied from the input element.  Property Default Description NAME *.* Specifies the filename template. This can be the complete filename, or a wildcard specification to be applied to the selected attribute of the input stream. USING NAME Specifies the name of the attribute to be used from the input stream for construction of a filename. #### Example1: (with no options specified)  Port Value FileName Some content that will be discarded DATA The entire contents of file c:\data\file1.dat........... The entire contents of file c:\data\file2.dat........... #### Example2: with options: USING="OTHER" NAME="*.txt"  Port Value FileName DATA The entire contents of file c:\data\file1.txt (not file1.dat) ........... See also: LookUp, WriteFile, Rename, Files, FileWatch, FileName ### WriteFile Writes data to one or more disc files.  Port Direction Description DATA IN The data content of each element builds a file. The content should ideally be pure data but EOLN markers are written as ASCII [RET][LF]. LOG OUT For each file successfully created, a FILE element is created with NAME attribute being the filename used and a SIZE attribute being the number of bytes written to the file. Other attributes are copied from the input element.  Property Default Description NAME *.* Specifies the filename template. This can be the complete filename, or a wildcard specification to be applied to the selected attribute of the input stream. USING NAME Specifies the name of the attribute to be used from the input stream for construction of a filename. APPEND false Either an expression to be evaluated or a constant Boolean value. If true then an existing file will be appended with the data. If false then an existing file will be overwritten. WAIT 0 Specifies the time to wait for access to the file. This is useful for appending to a log file which might be open by some other task or user. The default unit is seconds, but you can specify the time in other units by means of a suffix letter: S for seconds (default) M for minutes H for hours D for days #### Example: (with no options specified)  Port Value DATA (in) Data to be written to a file called c:\data\file1.dat ............. LOG (out) SIZE=67/> See also: Readfile, LookUp, Rename, Files, FileWatch, FileName ### Rename Renames or deletes files.  Port Direction Description FileName IN A stream of elements (at the outer level only) that each name file to be renamed or deleted. LOG OUT For each file successfully renamed or deleted, a FILE element is created with ORIGINAL attribute being the filename used and a NAME attribute being the new name (or null string for delete).  Property Default Description NAME *.* Specifies the filename template. This can be the complete filename, or a wildcard specification to be applied to the selected attribute of the input stream to make the new name for the file. USING NAME Specifies the name of the attribute to be used from the input stream for construction of a filename. RENAME REQUIRED Specifies the new name for the file. This can use a wildcard specification to derive a name from the original name. If it is an empty string then the file will be deleted. #### Example: RENAME="*.new"  Port Value FileName (in) Some content that will be discarded LOG (out) ORIGINAL="c:\data\file1.dat"/> ORIGINAL="c:\data\file2.dat"/> See also: Readfile, WriteFile, LookUp, Files, FileWatch, FileName ### FileWatch Watches for files that match the specified name and details, renames them to a 'work in progress' filename, and finally renames them or flags them as archived when required to supply the next one. If a file is found that cannot be opened for update or is of zero length then it will be ignored (as it is assumed that it is still being created). There are two different modes of operation: • If no input stream is used then a continuous (never ending) search is made for files that match the WATCH specification. Files are then checked for full exclusive access before an output element is generated. • If there is an input stream then the input elements supply filenames which (after applying the WATCH specification) are individually awaited. Only when the file exists, and is accessible will any output be generated and the next input element read.  Port Direction Description NAMES IN This stream supplies filenames (specified by the attribute named by USING) to be awaited. When not specified, the filenames are found by expanding the wildcard WATCH specification. FileName OUT For each file matched, a FILE element is generated with NAME attribute being the working filename that the file has now, and ORIGIN attribute being the name the file had when matched.  Property Default Description WATCH REQUIRED Specifies the file specification. RENAME No renaming Specifies that the file is to be ultimately renamed to the specification given. Take care using this feature as the asynchronous nature of the dataflow processing could mean that the file renaming could be done too soon. WORKNAME No renaming Specifies that the file is to be renamed to a temporary work name while it is being processed. ARCHIVE false A Boolean value ("true" or "false") which if true specifies that only updated files are to be matched - and set to the archived state when processed. USING NAME Supplies the input attribute name to be used as the basis for creating the target filenames. NB: This option is illegal if there is no input stream. ORDER Directory order A sort key specification (list of attribute names with optional negative prefix) which specifies the order in which filenames should be delivered when there is more than one file that matches the specification. The names select file attributes to sort: SIZE, NAME or WDATE. A positive specification results in the lowest values being delivered first, e.g. SIZE gives priority to smaller files, WDATE gives priority to older files. See also: Readfile, WriteFile, Rename, Files, LookUp, FileName ### Files Generates a list of filenames from a wild-card specification.  Port Direction Description NAMES IN Optional stream to provide filename specifications. If an input stream is connected then the filename search is run for each level 0 element, otherwise the file list is generated once only. FileName OUT For each file found a FILE element is generated with attribute NAME being the full filename. For files that were opened: a SIZE numeric attribute supplies the file size and a WDATE attribute supplies the last-updated date in ISO format. For files that gave an error on opening: an ERROR attribute gives the error code. If an input stream was connected then the output is a copy of the input with the filename list as its content (followed by any original input content).  Property Default Description NAME REQUIRED Specifies the file specification. USING Specifies the attribute name to be used to generate the filename specification (when names are derrived from an input stream). UPDATED All files given A Boolean value which when "true" means that only updated files are to be given, and "false" gives only archived files. BAD false A Boolean value which when "true" means that filenames will be supplied even if the file did not seem to be accessible. OPEN true A value that specifies that the file is to be opened to check that this will be possible. Possible values are "true" for read access check, "false" for no check to be made, or "update" for access to be checked in update mode. #### Example: NAME="c:\ndp\examples\*.csv"  Port Value FileName (out) [EOF] See also: Readfile, WriteFile, Rename, LookUp, FileWatch, FileName ### FileName This function adds a new attribute value to element headers by applying wildcard filename rules to an attribute that names a file.  Port Direction Description DATA IN The input data stream. Element headers at the outer level are processed. DATA OUT A copy of the input stream with the level-0 element headers changed by having a new or modified attribute specified by the CREATE option being the filename obtained by applying the NAME pattern to the attribute value named by USING  Property Default Description NAME *.* Specifies the filename template. This can be the complete filename, or a wildcard specification to be applied to the selected attribute of the input stream. USING NAME Specifies the name of the attribute to be used from the input stream for construction of a filename. CREATE REQUIRED Specifies the name of the attribute to be created. TARGET 0 The target element, either the level number (defaults to 0 - the outermost level), or the element tag name. GETEXT If specified, names an attribute to be created with the value of the file extension of the specified filename. GETNAME If specified, names an attribute to be created with the value of the file name of the specified filename. GETPATH If specified, names an attribute to be created with the value of the file path of the specified filename. CASE Specifies that the case of the filename is to be changed. "UC" to convert filename to uppercase, "LC" for lowercase. #### Example: Options: CREATE="opfile" NAME="*.txt"  Port Value DATA (in) [EOF] DATA (out) [EOF] See also: Readfile, WriteFile, Rename, LookUp, FileWatch, Files ### Install Install tasks using a description encoded as a data stream.  Port Direction Description DATA IN The data stream that specifies the tasks to be installed. LOG OUT A list of TASK elements with NAME attribute being the task identifier.  Property Default Description EDIT 0 An integer value that determines which of a multi-task stream is to be installed. The default (0) value means that all tasks are installed. RUN false A Boolean value that if set to TRUE will cause the task to start running as soon as it is installed. See also: TaskList ### TaskList List a task's composition. The result could be used to install another copy.  Port Direction Description TaskName IN A data stream where NAME attributes specify the name of the task to be listed. DATA OUT The task information in a form that the installer would recognise. See also: Install ### Concat Concatenates two entire data streams.  Port Direction Description Start IN The first part of the resultant data stream. This entire stream is first copied to the output stream. Finish IN The last part of the resultant data stream. This entire stream is copied to the output stream when the first stream is exhausted. DATA OUT The combined data from the two inputs. #### Example:  Port Value START (in) The quick brown fox. [EOF] FINISH (in) Jumped over the lazy dog. [EOF] DATA (out) The quick brown fox. Jumped over the lazy dog. [EOF] See also: Join, Merge ### Merge This concatenates corresponding pairs of elements from up to 3 data streams.  Port Direction Description DATA1 DATA2 DATA3 IN The 3 inputs streams to be combined. DATA OUT The combined data from the various inputs. The data is obtained from each input in turn. The output takes the element header from the named stream - or a combination of the streams according to the options. If a single stream is specified to provide the headers then when that stream becomes exhaused then the output terminates - otherwise output continues until all input is used up.  Property Default Description HEADER 0 An integer value that specifies which stream is to be used to supply the element headers (and associated attributes). The default (0) means that all streams are combined. REPEAT No repeats An integer value specifies a stream whose final element value will be repeated if other streams are not exhausted. #### Example1: (with no options specified)  Port Value DATA 1(in) The quick brown fox. [EOF] DATA 2(in) Jumped over. [EOF] DATA 3(in) The lazy dog. [EOF] DATA(out) The quick brown fox.Jumped over.The lazy dog. [EOF] #### Example2: Option: HEADER="2"  Port Value DATA 1(in) The quick brown fox. [EOF] DATA 2(in) Jumped over. [EOF] DATA 3(in) The lazy dog. [EOF] DATA(out) The quick brown fox.Jumped over.The lazy dog. [EOF] See also: Join, Concat ### Join This function joins up to three data streams on a first-come first-served basis. Each input is polled and the first element to arrive will be copied to the output. If more than one input stream is active simultaneously then elements are taken from each stream in turn.  Port Direction Description DATA1 DATA2 DATA3 IN The input data streams. DATA OUT The output data stream which combines all the input data. #### Example:  Port Value DATA 1(in) The quick brown fox. [EOF] DATA 2(in) The lazy dog. [EOF] DATA 3(in) Jumped over. [EOF] DATA(out) The quick brown fox. The lazy dog. Jumped over.[EOF] See also: Merge, Concat ### DataAttr This folds element data content back into the enclosing element's header making them into attribute definitions.  Port Direction Description DATA IN The data input stream. DATA OUT The output is a copy of the input stream but with elements of the specified level being used to created attributes with the same name.  Property Default Description TARGET 1 The target element, either the level number (0 being the outermost level), or the element tag name. DATA Data content remains. This specifies the tag to be used for data content of the targetted level. The data is normally removed from the element, but will remain if the DATA element name appears in the COPY list (see below). DATALINE 0 This specifies that a single line of the data content is to be processed by the DATA feature (above). It allows you to copy a particular line of data to an attribute or cut the line from the data content. Note that the first line is numbered 1 and that the lines must have been previously marked out using the EOLN marker e.g. by Text, and a zero specification is the default action of using all the data content. FIELDS All elements selected. This optional list of tag names is used to select specific elements for transfer to attributes of the target element. If the list is preceded by a "-" then all names except those specified are processed. COPY This optional list of tag names is used to select specific elements for copying to attributes of the target element. The original element remains in the content. #### Example: (with no options specified):  Port Value DATA (in) 4145 Nile Mediterranean Sea 3900 Amazon Atlantic Ocean 3100 Yangtze Kiang Pacific Ocean 2800 Lena Arctic Ocean 2718 Congo Atlantic Ocean [EOF] DATA (out) [EOF] See also: AttrData, Attr, Substitute, Wrap, UnWrap ### AttrData This copies attribute values into the element content, moving the attribute value to the content of an element with the same name. Non-text attribute values are converted to the standard text representaion.  Port Direction Description DATA IN The input stream containing the element attributes to be made into data. DATA OUT The output stream with the selected attributes copied into the content.  Property Default Description TARGET 1 The target element, either the level number, or the element tag name. DATA Data is not changed. The attribute name to be used to construct the data content of the targetted elements. The value of this one attribute is inserted into the element's content. FIELDS All attributes processed. A list of attribute names (separated by commas ","). The list either specifies the only attributes to be processed, or if the list starts with a minus "-" then it lists the attribute names to NOT be processed. #### Example: (with no options specified):  Port Value DATA (in) [EOF] DATA (out) 4145 Nile Mediterranean Sea 3900 Amazon Atlantic Ocean 3100 Yangtze Kiang Pacific Ocean 2800 Lena Arctic Ocean 2718 Congo Atlantic Ocean [EOF] See also: DataAttr, Attr, Substitute, Wrap, UnWrap ### Substitute This uses attribute values from the main data stream to fill in parameters specified by elements in two other streams (often set up as constants). One of these determines the output attributes and the other determines the content.  Port Direction Description DATA IN The data input stream. For each element, the substitution process if performed on the next outer elements arriving at the other two inputs. Header IN The header script - a data stream that determines the output headers. Script IN The script - a data stream that determines the output content. DATA OUT The output stream. It consists of copies of the scripts after parameters have been substituted using attributes of the DATA stream.  Property Default Description TARGET 0 This is either the level number or element name that identifies the elements to be substituted. TAG This names the element tag that is used in the script to represent a parameter for substitution. If specified, the parameters are elements with this tag which name the source attribute using a NAME attribute. If not specified then any empty tag will be treated as a possible parameter. ATTRMERGE false When true, this option specifies that the original attributes are to be merged with the generated attributes. Generated attributes will replace the originals. #### Example: (with no options specified)  Port Value DATA (in) This is some content [EOF] Header (in) The original NAME was "" Content (in) The changed content is made from ths. The original value of SIZE was DATA (out) The changed content is made from ths. The original value of SIZE was 20 [EOF] See also: Attr, AttrData, DataAttr, Wrap, UnWrap ### Attr This function creates a new attribute for a element, the value being made by the substitution of a script.  Port Direction Description DATA IN The input stream. DATA OUT The output stream, a copy of the input stream but with the target elements having an additional attribute.  Property Default Description TARGET 0 This is either the level number or element name that identifies the elements to be substituted. TAG This names the element tag that is used in the script to represent a parameter for substitution. If specified, the parameters are elements with this tag which name the source attribute using a NAME attribute. If not specified then any element will be treated as a possible parameter. CREATE REQUIRED The name of the new attribute to be created. It can be the same as an existing attribute - which would then be replaced by the new value.  Content The script that defines the value of the resulting attribute. This may contain elements, which will be replaced by same-named attribute values from the target element. See Appendix #### Example: with option: CREATE="NEWONE"  Port Value Property content The size was bytes. DATA (in) [EOF] DATA (out) [EOF] See also: DataAttr, Substitute, Promote ### Capture This function captures data following the target element and appends it to the target element content.  Port Direction Description DATA IN The input stream. DATA OUT A copy of the input stream captured content included in the target elements.  Property Default Description TARGET 1 The target element to be modified, either the level number (defaults to 0 - the outermost level), or the element tag name. #### Example: (with no options specified)  Port Value DATA (in) item 1 item 2 item 3 [EOF] DATA (out) item1 item2 item3 [EOF] See also: Promote, Attr ### Promote This function copies attribute values from the first element of a level to the parent element header.  Port Direction Description DATA IN The input stream. DATA OUT A copy of the input stream with attribute copies added to the target elements.  Property Default Description TARGET 0 The target element to be modified, either the level number (defaults to 0 - the outermost level), or the element tag name. COPY All attributes copied A list of attribute names to be copied. See also: Attr, Capture, Demote ### Demote This function copies attribute values from target elements to specified inner levels.  Port Direction Description DATA IN The input stream. DATA OUT A copy of the input stream with attribute copies added to the specified child elements.  Property Default Description TARGET 0 The target element to be the source of attributes, either the level number (defaults to 0 - the outermost level), or the element tag name. DESTINATION 0 The destination for the selected attributes from the target element, either the relative level number (defaults to 0 - the immediate content of the target element), or the element tag name. COPY All attributes copied as defaults A list of attribute names to be copied. You can indicate whether an attribute is always to be copied by preceding the name with "+" or only to provide a default value if that attribute is not present in the child element by preceding with "-". If absent then all atributes are copied as defaults, and if specified as an empty value then all atributes are always copied. See also: Attr, Capture, Promote ### Clean This function deletes attributes from element headers.  Port Direction Description DATA IN The input stream. DATA OUT A copy of the input data stream with only the specified attributes retained.  Property Default Description TARGET 0 The target element to be modified, either the level number (defaults to 0 - the outermost level), or the element tag name. FIELDS All attributes deleted A list of attribute names to be retained in the target elements. See also: Dispose, Attr ### Dispose This function deletes attributes from element headers. It can also remove specified content from within the target element.  Port Direction Description DATA IN The input stream. DATA OUT A copy of the input data stream with some attributes and possibly other content removed.  Property Default Description TARGET 0 The target element to be modified, either the level number (defaults to 0 - the outermost level), or the element tag name. FIELDS All attributes deleted A list of attribute names to be deleted from the target elements. Specify this as blank for no fields to be deleted. ELEMENT false Specify true to have all elements and all their content within the target completely removed from the stream. PI false Specify true to have processing instructions within the target element completely removed from the stream. COMMENT false Specify true to have comments within the target element completely removed from the stream. DATA false Specify true to have character data within the target element completely removed from the stream. See also: Clean, Attr ### Wrap This function wraps an entire data stream with a header, and thus will convert a list of data objects into a single object containing the list. Note that this function can be used inside the APPLY function. In that situation, each APPLY substream is individually wrapped - but make sure that the wrapper source stream repeats so as to keep supplying fresh wrappers.  Port Direction Description LIST IN The data stream to be wrapped. Header IN The wrapper for the data stream. This is usually a single element. For more complex data wrapping you will need to specify the TAG property. DATA OUT The resulting output data stream, which now consists of a single element.  Property Default Description TAG When specified, this names an empty element (from the wrapper) that will be replaced by the content of the input. When not specified the content is copied immediately before the end of the wrapper. #### Example:  Port Value LIST (in) item1,item2,"item3" another item,,last one [EOF] Header (in) DATA (out) item1,item2,"item3" another item,,last one [EOF] See also: UnWrap, Substitute, Attr, DataAttr ### Unwrap This function removes the outer element headers from a data stream.  Port Direction Description DATA IN The input data stream. DATA OUT The contents of the input elements. Wrap OUT The element headers that have been removed from the main data stream.  Property Default Description TARGET 0 The target element, either the level number (defaults to 0 - the outermost level), or the element tag name. NAMES The specified attributes from the target level (i.e. the element header being removed) are copied into all the next level element headers. If the NAMES list is specified as null (as opposed to being omitted altogether) then all atributes are copied. #### Example: (with no options specified):  Port Value DATA (in) item1 item2 item3 another item last one [EOF] DATA (out) item1 item2 item3 another item last one [EOF] Wrap (out) [EOF] See also: Wrap, Substitute, Attr, DataAttr ### Summary This function summarises a group or potential group of elements.  Port Direction Description DATA IN The data input stream. DATA OUT The summary of the input data stream. This is a copy of the input stream with elements that would be grouped replaced by a single element containing the summary of the attributes of the group.  Property Default Description TARGET 0 The target elements to be considered for grouping and summary, either the level number (defaults to 0 - the outermost level), or the element tag name. GROUP compare tag names A list of names identifying the attributes that must be the same to identify a group of elements to be summarised. Content This contains elements that define how to summarise the data. The element tag names specify the summary type and the attribute NAME specifies the attribute name being summarised. Any attribute values not selected by these options are summarised by retaining them if they have the same value for all elements in the group. Options are:  SUM Compute the arithmetic sum of the specified attribute for all group members. FIRST Supply the first value of the specified attribute in the group. LAST Supply the last value of the specified attribute in the group. LAST Supply the last value of the specified attribute in the group. MIN Supply the lowest value of the specified attribute in the group. MAX Supply the highest value of the specified attribute in the group. AND Compute the bitwise or Booolean AND of the specified attribute for all group members. OR Compute the bitwise or Booolean OR of the specified attribute for all group members. See also: Group ### Group This function adds a wrapper around groups of elements. There are a number of ways that a sequence of elements can be identified to be made into a group. • One or more attirbutes can be identified as having to be of the same value within the group, i.e. a group terminates before an element with different attribute values is encountered. When this grouping method is active the wrapper elements inherit the attributes that are the same within the group. • A specific test can identify an element to start a new group. • A specific test can identify an element to be the last in the group. • A minimum and maximum group size can be specified to group elements by number.  Port Direction Description DATA IN The data input stream. DATA OUT The data output stream.  Property Default Description TARGET 0 The target elements, either the level number (defaults to 0 - the outermost level), or the element tag name. GROUP compare tag names A list of names identifying the attributes that must be the same within a group of elements, or an expression which evaluates as TRUE for the first element of a group. CREATE REQUIRED The element name to be used as wrappers for the groups of data elements. MIN 0 The minimum number of elements per group. Specifying this results in groups that may contain different values of the selected attributes in that adjacent groups will be concatenated if the result contains insufficient elements. Note: by specifying a non-zero minimum, no attributes are included in the wrapper element headers. MAX infinite The maximum number of elements per group. Specifying this results in adjacent groups containing equal values of the selected attributes. ENDS false An expression which evaluates to TRUE to indicate that the current element is to terminate the group. This is additional to any other reason to terminate the group. See also: Summary ### Copy This duplicates a data stream so that it can be used as input to two functions.  Port Direction Description DATA IN The data input stream. DATA1 DATA2 OUT Each data output is an exact copy of the input stream. #### Example:  Port Value DATA (in) [EOF] DATA 1 (out) [EOF] DATA 2 (out) [EOF] ### Buffer This function encourages pipelining by slightly reading ahead of its output. Normally, data processing occurs on demand - but there can be situations where there is no reason why a function could not start processing its next input before the demand has become apparent (e.g. because of a hold-up waiting for the completion of an external process).  Port Direction Description DATA IN The input data stream. DATA OUT The output data stream - which is an exact copy of the input #### Example:  Port Value DATA (in) [EOF] DATA(out) [EOF] ### Count This function counts elements and inserts an attribute into the element headers supplying their number in the stream. Since the increment can be obtained from the input stream it can also be used to accumulate a total. Note: this function is currently only defined for integer values and ascending counts.  Port Direction Description DATA IN The data stream to be processed. DATA OUT The resulting output data stream which is a copy of the input stream, but with an extra attribute added to the outermost element headers. The attribute name can be specified using the options and defaults to NUMBER.  Property Default Description TARGET 0 The target element, either the level number (defaults to 0 - the outermost level), or the element tag name. DEPTH 0 This sets the degree of nesting that is acceptable before the counter is reset. Use this to count inner-level elements over a wider extent than just the element in which they occur. By default the counting resets at the end of the enclosing element. Setting this parameter to 1, for example, would not reset the count at that point but wait for the next outer level. MIN 1 Specifies the lowest number to be used. This is to supply a starting value for when the number overflows the maximum. MAX large number Specifies the highest number to be used. The next element will then be numbered using the value of the MIN option. The default is very large. LAST last number Specifies the number to be given to the last element of the group. This option allows you to identify the final element of a group for subsequent special treatment. START 1 Specifies the starting number, i.e. the number that will be applied to the first element. NAME NUMBER Specifies the name of the attribute to be used for the number. STEP 1 Specifies the value by which the counter is incremented, or the name of the attribute to be used to obtain the increment value. RANGECOUNT Supplies an attribute name to be created with the range count number. This number starts at 0 and is incremented whenever the counter overflows. #### Example1: (with no options specified):  Port Value DATA (in) [EOF] DATA (out) [EOF] #### Example2: Options: TARGET="0" START="9" MIN="1" MAX="10" NAME="edition"  Port Value DATA (in) [EOF] DATA (out) [EOF] ### Eval This function allows string values to be converted to numeric form (e.g. for numeric-order sorting) and can sum a number of fields.  Port Direction Description DATA IN The input data stream. DATA OUT A copy of the input data stream with the numeric result added as an attribute of the target element.  Property Default Description TARGET 1 Identifies the element whose attributes are to be used for the numeric expression. It can either name an element tag or specify the target level as a number. NAME REQUIRED Specifies the name of the attribute that is to be created (or overwritten) by the result of the expression. SUM The NAME attribute A list of fields to be summed. Any attribute name prefixed with "-" will be negated before summing. A constant value can optionally be included in the sum list as the first item. SCALE 1 A scale factor to be applied to the result. You can specify a value to scale the result down by prefixing with '/' and note that if this value is specified as an integer then the result will be truncated to an integer result. This means that a scale factor of '/1' has the effect of converting the result to an integer. NAN task aborts on error Specifies the result for the situation where the expression value is not defined, e.g. because the source attribute value(s) could not be found or converted to numeric values. ### Sort This function sorts a list of elements in a data stream.  Port Direction Description DATA IN The input data stream. DATA OUT The output data stream.  Property Default Description TARGET 0 Identifies the element within which elements are to be sorted. It can either name an element tag or specify the target level as a number. KEY Element tags Selects the field attribute(s) to be used as the sort key. If a list of attribute names is required, use 'comma' as the separator (e.g. Key1,Key2,-Key3). Attributes are sorted in ascending order; use 'minus' as the prefix to sort in descending order. If none is specified then the element tags themselves are used as the sort key. DUPLICATE true Specifes the treatment for elements that have the same sort key. If FALSE then extra elements with the same key are discarded. If this option supplies a name then an attribute with the specified name is added to all elements with the value being the number of elements that had the same key. #### Example1: Options: KEY="FLD2"  Port Value DATA (in) [EOF] DATA (out) [EOF] #### Example2: Options: KEY="FLD3" DUPLICATE="false"  Port Value DATA (in) DATA (out) ### Separate This function creates a new data-stream containing the elements that are common to both input data-streams. A record is taken from each input stream and their values compared. If they are the same, the element is recorded (once) in the new data-stream (EQUAL) and removed from both output streams (DATA1, DATA2). Only the residue of the (unequal) records remain in their respective output data-streams.  Port Direction Description DATA1 IN Input data-stream 1. DATA2 IN Input data-stream 2. DATA1 OUT The output data-stream unique to input data stream 1. EQUAL OUT The output data-stream containing the elements that are the same in both input data streams. DATA2 OUT The output data-stream unique to input data-stream 2.  Property Default Description TARGET 0 Specifies the target level of the file. Note that this is one level above the record level. KEY Element tags Selects the field attribute(s) to be used as the comparison key. If a list of attribute names is required, use 'comma' as the separator (e.g. Key1,Key2,-Key3). Attributes are assumed to be sorted in ascending order; use 'minus' as the prefix for values sorted in descending order. If none is specified then the element tags themselves are used as the comparison key. ### Revise This function changes the names of attributes and tags. The description of how to change the names can either be explicitly supplied, or derived from the first targetted element (useful for CSV files which name their fields in the first record).  Port Direction Description DATA IN The data stream in which names will be changed. NAMES IN An optional data stream that supplies the details of the transformation required. When omitted, the first selected element supplies the replacement names. DATA OUT The resulting data stream.  Property Default Description TARGET 1 This can either be a level number (0 being the outermost level) or the name of the elements to be processed. #### Example: (with no options specified):  Port Value DATA (in) [EOF] DATA (out) [EOF] ### Format This converts attribute values to text and formats them, with the possibility of adding or removing leading and/or trailing spaces.  Port Direction Description DATA IN The input data stream. DATA OUT The output data stream.  Property Default Description TARGET 1 This can either be a level number (0 being the outermost level) or the name of the elements to be selected. FIELDS All attributes processed. This is a list of the attribute names to be processed. LEFT 0 The left-side size. If no pivot is specified then this effectively right-aligns the attribute value with the size specified. RIGHT 0 The right-side size. If no pivot is specified then this effectively left-aligns the attribute value with the size specified. PIVOT No pivot used. The pivot text to be used to find the centre of the attribute value, e.g. "." to format about a decimal point. ### Date This function will perform simple date calculations using dates in the ISO representation. Where FROM and USING both specify dates, the result is the difference, in the specified units (as a real numeric value). Where FROM is not specified it defaults to the current date and time, and so the result is the age of the date specified by USING in the specified units (default seconds). Where a date and a numeric value are specified, the result is a date made by subtracting the numeric value (scaled by the UNITS to seconds) from the date.  Port Direction Description DATA IN The input data stream. DATA OUT A copy of the input stream with the resulting values added.  Property Default Description TARGET 0 This can either be a level number (0 being the outermost level) or the name of the elements to be selected. USING REQUIRED The name of the attribute to be used in the calculation or a numeric value specifying an age. The attribute can select either an age value or a date. FROM The current time. One of: An ISO date. A numeric value. An attribute name. This attribute is used to supply the FROM value. Unspecified - defaults to the current time and date. CREATE REQUIRED The name to be used for the result attribute. UNITS 1 second. The units being used for numeric values (either as result or offset). Specify either the number of seconds (can be negative) or a letter "S" for seconds, "M" for minutes, "H" for hours, "D" for days. #### Example: Options: USING="WDATE" FROM="2001-01-01" CREATE="AGE" UNITS="D"  Port Value DATA (in) [EOF] DATA (out) [EOF] See also: Timer ### Select This compares element attributes, selecting those that pass the specified test. The rejects are available as an outer-level data stream.  Port Direction Description DATA IN The input data to be processed. DATA OUT A copy of the data stream excluding the rejected elements. Reject OUT The rejected elements (including their content). Note that by default only the rejected elements themselves are output.  Property Default Description TARGET 0 This can either be a level number (0 being the outermost level) or the name of the elements to be selected. TAG If specified, only elements with a name in the specified list will be selected. TEST REQUIRED A simple dyadic expression comparing an attribute with a value, or two attributes. Text values must be enclosed in quotes. If either operand is actually numeric then the comparison is made after converting both operands to a numeric value (if possible). DUPLICATE false Specify TRUE for the rejects stream to duplicate the outer element wrapping of the input stream. The default FALSE setting results in rejected elements being emitted without any wrapping. #### Example: Options: TARGET="1" TEST="FLD>3000"  Port Value DATA (in) [EOF] DATA (out) [EOF] Reject (out) [EOF] See also: Assert ### Assert This tests element attributes, selecting those that pass the specified test. If a test fails then the task is halted with an error.  Port Direction Description DATA IN The input data to be processed. DATA OUT A copy of the data stream.  Property Default Description TARGET 0 This can either be a level number (0 being the outermost level) or the name of the elements to be selected. TAG If specified, only elements with a name in the specified list will be selected. TEST REQUIRED A simple dyadic expression comparing an attribute with a value, or two attributes. Text values must be enclosed in quotes. If either operand is actually numeric then the comparison is made after converting both operands to a numeric value (if possible). If no test is specified then the error indication will be given for any data input. See also: Select ### Sync This function synchonises the flow of data of two streams. The second stream is held up until the required target element passes through the first.  Port Direction Description FIRST IN The main stream. SECOND IN The stream to be delayed. FIRST OUT The main stream output. SECOND OUT The delayed stream output.  Property Default Description TARGET outer When specified, this indicates which elements are to be cleared before the second stream passes an outer-level element. The default behaviour is for the second stream to be held up until the main stream is exhausted. Note that when the main stream becomes exhausted then all processing terminates. ### Apply This function selects elements from the main data stream and supplies them, or just their contents for separate processing. The results are then recombined with the main stream. Note that because of the way this function works it may not be possible to use it with any arbitary set of functions to process the inner stream. In the current implementation the inner stream is continuous, and is delimited by special processing instructions. If the inner stream is structurally altered so as to loose or duplicate these special markers then the APPLY function will not operate correctly and may not terminate or give a "Data structuring error". You must not make use of the markers as the details of the inner stream delimiting is subject to change.  Port Direction Description DATA IN The data stream to be processed. APPLY OUT The data stream of the selected elements, extracted from the main stream. DATA OUT The resulting output data stream. APPLY IN The return point for the separated elements (or content thereof). This stream will be merged back into the corresponding part of the main stream.  Property Default Description TARGET 1 This can either be a level number (0 being the outermost level) or the name of the elements to be selected. The targeted elements are then processed by dispatch through the APPLY output. TEST true A Boolean expression applied to the attributes of the selected elements to further refine the choice of which elements are to be seperated. CONTENT FALSE A Boolean value that selects whether the element start/end tags are to be processed. If TRUE then just the content is processed, FALSE will send the selected elements including their content through the processing stream. See also: Modify ### Modify This function primarily permits arbitary modifications of element headers by means of an ECMAScript program. There is also some provision for simple modification of the element content. The data content of the properties is compiled by the ECMAScript compiler and the result stored as global variables. This permits the definition of functions that will be used to do the processing. The current element is denoted by the global symbol "".

 Port Direction Description DATA IN The data stream to be modified. DATA OUT The modified data stream.

 Property Default Description TARGET 1 This can either be a level number (0 being the outermost level) or the name of the elements to be selected. The targeted elements are then processed by executing the ECMAScript statment(s) supplied by the ONTARGET option. ONTARGET REQUIRED ECMAScript statements to be applied to each targetted element. Note that the content part of the options is available to initialise global memory with variables and function definitions.

### Def

This function provides a way to associate names with values in the tasks local symbol table. The current main use for these symbol values is to parameterise task specifications as far as filestore directory names is concerned. The symbol names can be used at the start of a filename specification followed by double colon - and this notation is replaced by the associated value. Where no value is known the functional block that is accessing the symbol name will stop and wait until it is defined.
You can use this functional block quite unconnected to the rest of the task activity to create a particular named symbol at task startup.
Note that task local symbols cannot have there values changed once they are defined although no error will be raised if an attempt is made to redefine the local symbol with the same value.

 Port Direction Description DATA IN An optional input stream. If not used then the property values determine the symbol name and value.

 Property Default Description TARGET 0 This can either be a level number (0 being the outermost level) or the name of the elements to be selected. The targeted elements are then used to obtain the symbol values. Only specify a TARGET if the input stream is connected. NAME REQUIRED unless input stream is connected. Specifies the symbol name to be created. This property is required unless you are connecting an input stream - in which case ALL attribute associations will be added to the local symbol table. VALUE REQUIRED unless input stream is connected. This specifies the actual symbol value, where no input stream is connected. Otherwise it names the attribute to be used from the target element to obtain the symbol value.

See Appendix for list of global standard symbol values.

### HostInfo

This network service obtains information about a specified host from the domain name server. A host can be specified by name or IP address. The information returned depends on the domain services.

 Port Direction Description DATA IN The input data used to obtain the host name or IP address. DATA OUT A copy of the input data stream with the host information inserted as the initial content.

 Property Default Description TARGET 1 This can either be a level number (0 being the outermost level) or the name of the elements to be selected. The targeted elements are then used to obtain the host identification. HOST Specifies an actual host identification, or a default to use if USING is specified but not found. USING HOST Specifies the name of the attribute to read from the target element to obtain the host identification.

#### Example1:

(With no options specified)
 Port Value DATA (in) Example using host name Original content here Example using IP address [EOF] DATA (out) microsoft.com 207.46.130.14 207.46.130.149 207.46.130.45 207.46.131.137 207.46.131.30 www.microsoft.net Example using host name microsoft.com 207.46.131.137 207.46.131.30 207.46.130.14 207.46.130.149 207.46.130.45 www.microsoft.com Original content here www.microsoft.co.uk 212.158.20.155 119.119.119.46 109.105.99.114 111.115.111.102 116.46.99.111 microsoft.com 207.46.130.14 207.46.130.149 207.46.130.45 207.46.131.137 207.46.131.30 www.microsoft.net Example using IP address [EOF]

### TCP

This network function implements a simple TCP connection.
An attempt is made to make a connection to the specified port on the specified host. Note that the host can be specified by IP address or host name, and the port can be specified by number or service name.
If successful, the data content of the target element is transmitted to the specified port, then an end-of-data condition is signalled and data is received from the host and copied to the main output until the host signals end-of-data.
When the connection was unsuccessful, the target element (only - no outer levels) is copied to the reject output.

 Port Direction Description DATA IN The input data stream containing any data to be transmitted. DATA OUT The data received from the host when a successful connection was obtained. FAIL OUT A copy of the input element when no connection was obtained. An attribute named ERROR is added to the element header which supplies a nmemonic error code.

 Property Default Description TARGET 0 This can either be a level number (0 being the outermost level) or the name of the elements to be selected. The targeted elements are then used to obtain the host identification. HOST Specifies an actual host identification, or a default to use if USING is specified but not found. USING HOST Specifies the name of the attribute to read from the target element to obtain the host identification. PORT 80 (http) Specifies the name or number of the host port. If a name is specified then the name must be present in the Windows SERVICES file.

### TCPserver

This network function implements a simple TCP server. The server listens on the specified port - and performs a simple data transfer to clients.

 Port Direction Description DATA IN Input data to be sent to clients. DATA OUT The data received from clients.

 Property Default Description TARGET 0 This can either be a level number (0 being the outermost level) or the name of the elements to be selected. The data content will be transmitted to the next client. PORT REQUIRED Specifies the name or number of the port to service. If a name is specified then the name must be present in the Windows SERVICES file.

### Substitution

 Attribute Modification EQ A conditional section. The selected value is compared with the value of this attribute. If it is the same then the content of this element is copied to the result, otherwise it is omitted. NE A conditional section. The selected value is compared with the value of this attribute. If it is different then the content of this element is copied to the result, otherwise it is omitted. EMPTY The content is processed. If the result is entirely null the the attribute value is used instead. ATX The attribute supplies the starting character position (1 being the first). Characters before this are discarded. WIDTH The attribute specifies a maximum width for the value. It will be truncated if necessary. DEFAULT If the selected value is null then the attribute value is used instead. CASE The attribute value specifies what case-conversion should be done to the selected value. UC - convert to upper case. LC - convert to lower case. UL - convert first character to upper case and the rest to lower case.

### Conditional expressions

 Operator Test = Test for equality. <> Test for inequality. < Test for less-than. <= Test for less-than or equal. > Test for greater-than. >= Test for greater-than or equal. >=> Test for the right-hand string value being a substring of the left-hand. <=< Test for the left-hand string value being a substring of the right-hand.
The relational expressions can be combined by using one of these operators:
 Operator Result AND Expression is TRUE if both sub-expressions are TRUE OR Expression is TRUE if either sub-expression is TRUE

### Global symbols

 Symbol name Value EXE The directory from which nDP was run. NDP The nDP directory. NQROOT The root directory of the RedTitan nQ remote access server. This symbol is only created when remote access is enabled. NQDS The root directory for the RedTitan document server. This symbol is only created when remote access is enabled. TEMP A directory for temporary files.

Back to Contents