The runQueue lambda implements a peer to peer architecture which can be used to submit jobs from anywhere and which can be used to accept and run jobs from anywhere. At an extreme, to make a point, this would even include submitting a job to runQueue and accepting the same job or another job, via runQueue, from the same running context. Contexts which submit jobs to runQueue for execution are called suppliers, while contexts which accept jobs from runQueue for execution are called subscribers.
This architecture is currently implemented on a local area network of computers and may be implemented with slight variations in a cloud computing environment.
The runQueue Lambda, which is designed to be open source, provides added value through its job management API, through its console management API, via the thin job execution services it provides, and via the function hiding design of its API (ideally, if possible, the API design should not change regardless of a Network File Sharing implementation, a TCP/IP implementation, or a Cloud implementation). By far the most extensive value added will be provided by several function/class libraries providing remote execution services for specific applications. The function library will provide services for automatic data-matrix extraction, multiple regression distribution and re-accumulation, multiple regression reporting, trader lambda memoization (defaulting/learning/promotion) providing support of portfolio and factor research.
It is possible for the runQueue Lambda to function without requiring the user to study any of the available function/class libraries. All jobFiles contain actual source code for the distributed work task (unnamed lambda values to avoid unwanted side affects). These work tasks may be completely stand alone or they may extend existing function/class libraries. Thus runQueue will provide much of its added value in a non restrictive manner.
The submit function writes a set of job configuration files to the JobQueue. The resulting filenames are based on the jobConfig structure. The jobConfig structure must conform to the job config specifications. The jobData and jobLambda are both strings which will be compiled and evaluated by the subscriber. If present, the fourth parameter tells submit to return an error if there are any jobs in the JobQueue with the specified job name.
Type: Function
| Syntax | (runQueue.submit aJobConfig aJobData aJobLambda aExclusiveSW) |
| aJobConfig | A structure which contains the job configuration. |
| aJobData | A string which can be compiled and evaluated which produces the argument to be passed by the subscriber to the generated lambda. If not specified, aJobConfig.DataSourceFileName will be used. |
| aJobLambda | A string which can be compiled and evaluated which produces the lambda to be executed by the subscriber. If not specified, aJobConfig.LambdaSourceFileName will be used. |
| aExclusiveSW | Optional. When set to "exclusive", will cause an error if job/s with a similar name exists. |
The aJobConfig structure contains the following members:
| JobName | Job name. |
| MajorName | Optional. Job major name. |
| MinorName | Optional. Job minor name. |
| SeriesName | Optional. Job series name. |
| LambdaSourceFileName | Optional. Path to lambda source file. Used ONLY when aJobLambda was not specified. |
| DataSourceFileName | Optional. Path to data source file. Used ONLY when aJobData was not specified. |
| MinMemoryFree | Optional. The minimum amount of context memory needed to run the job. |
| RequiredLambdas | Optional. A list of lambdas required to run the job. Stored as a string and lambda names are separated by space. |
The run function is intended to be executed on each subscriber. It reads and executes jobs found in the JobQueue. Each job configuration contains the information necessary for the subscriber to execute the application in the subscriber context. Any generic library dependencies must be specified in the job configuration so the subscriber will be able to reasonably determine whether or not to accept the job.
Type: Function
| Syntax | (runQueue.run aRunOnce) |
| aRunOnce | Optional. When set to true, the run function will return after a single pass on the JobQueue. |
The getResult function reads the result of a specified job and returns a structure which contains the lambda return value and some additional job related information.
Type: Function
| Syntax | (runQueue.getResult aJobName aMajorName aMinorName aSeriesName) |
| aJobName | Job name. |
| aMajorName | Optional. Job major name. |
| aMinorName | Optional. Job minor name. |
| aSeriesName | Optional. Job series name. |
The getStatus function returns the status of a specified job.
Type: Function
| Syntax | (runQueue.getStatus aJobName aMajorName aMinorName aSeriesName) |
| aJobName | Job name. |
| aMajorName | Optional. Job major name. |
| aMinorName | Optional. Job minor name. |
| aSeriesName | Optional. Job series name. |
The getStatus returns one of the following states:
| ready | Job is ready to be picked up by the subscriber. |
| running | Job is running in the subscriber. |
| errored | Job errored out in the subscriber. |
| done | Job is complete. |
The remove function removes one or more jobs from the JobQueue. If any of the major, minor, or series names are missing, then all similar jobs will be removed.
Type: Function
| Syntax | (runQueue.remove aJobName aMajorName aMinorName aSeriesName) |
| aJobName | Job name. |
| aMajorName | Optional. Job major name. |
| aMinorName | Optional. Job minor name. |
| aSeriesName | Optional. Job series name. |