Class Stage

This is the Stage class. Derive your own class from it.
  stagename - full name for this stage (<jobname>-<stagename>)
  specs - ConfigObj specification for the supported keywords
  allow_extra_keywords - Whether to allow keywords that are not in the specification.
  cleanup - Whether to remove intermediate files
  inpipeline - Whether the state is running within a Python Pipeline.
               If the stage is manually created, do NOT set this flag.
               Python Pipeline will set it as needed.

Overrides: UserDict.UserDict.__init__

Validates the stored keywords. This is done by converting <self> to a ConfigObj instance, and calling validate() on it. The validated keywords are then updated back to <self>. This is done as part of Ev:87429

If a stage has a main product associated with it, the stage should overwrite this method with a method that returns the product string. For example, the LigPrepStage.mainProduct() will return "ligprep" Used by Pipeline.

Specify which product this stage is part of. Will determine which host the subjobs are run on.

A stage can accept one or more pipeio input objects. Use this method to specify the type of input object that is expected at each position.

position - an integer starting at 1. type - structures/grids/etc. required - whether this input always needs to be specified

A stage can return one or more pipeio objects. Use this method to specify the type of object that will be returned and whether or not it will always be produced by the stage.

position - an integer starting at 1. type - structures/grids/etc. always - whether this output is always produced

Returns the value for specified keyword, or default value.
Use as follows:
  precision_value = stageobj['PRECISION']

Get values for keywords this way from stage's operate() to determine
how to run. If the user did not specify a value for the keyword,
default value is returned.

Raises KeyError if a keyword is not supported by the stage.

Overrides: UserDict.UserDict.__getitem__

Specify a product that is required for this stage to run; optionally minimum version. Example: product="mmshare", min_release=16103 Overwrites any previous min_release versions for this product.

Raises RuntimeError if any of the required products are not installed or the version installed is less that minimum required version. It is possible to override this method. See ligprep.py for example.

Similar to requiredProduct() but can be used to specify required products at runtime. For example, ConvertStage doesn't know what products are required for conversion until runtime. Raises RuntimeError if product is not installed.

Returns the output IO object of the stage at specified position. Use this method after running the stage to get its output objects

This method is called by the pickle module to return the state to be serialized by dump().

Since Job objects and callable functions can not be pickled, it removes them from the a copy of the instance __dict__ attribute. _backend needs to be removed so that mmjobbe_terminate() would not be called more than once when the job exits.

This method dumps all the variables of the Stage to a restart file. Run it every time an important step is performed.

Raise exception if specified file does not exist. The message that is printed can be specified.

Prints specified objects to the stage log file. Obsolete. Use stage.info/debug/warning/error instead.

Prints specified objects to the stage log file. No EOF return

Print the value of each keyword for this stage to the stream specified as <fh>. Used by Pipeline

Return a dictionary of variable name of the inputs at each position. Key:position, value:name

Specify an input to use for this stage. position - input specified is for this position name - Variable name of this IO object obj - the IO object

This method is called by Pipeline.

Use in operate() to get the input object for specified position. Returns None if invalid position is specified.

Is called by Pipeline when starting the stage. Tell the stage what name to save each output under.

Tell this stage how to run the subjobs
None for njobs means determine automatically

subjob_hosts - list of hosts to run subjobs on
njobs - number of subjobs to generate
adjust - whether to adjust njobs such that job size is within limits
force - whether to continue with job if subjobs fail
max_retries - number of times to attempt to restart a subjob
              If not specified, use SCHRODINGER_MAX_RETRIES or 2.
cleanup - whether to delete intermediate files

Returns a list of hosts to run the subjobs on. localhost:1 may be in the list as well. Ideally, pass the output to JobDJ. Format: [ (host1,ncpus), (host2,ncpus) ] NOTE: whether or not to run with -LOCAL should be up to the stage. Pass this value to JobDJ.

Whether subjobs should be run with -LOCAL. Pass this value to JobDJ

Returns the requested target number of subjobs, and whether or not to adjust that number if it is unreasonable.

If -NJOBS was not specified, the # of CPUs or 10 is returned (whichever is smaller).

Used by Glide DockingStage and _adjustNJobs()

Returns the desired number of subjobs, and adjusts it for the the specified min & max job sizes if the user specified ADJUST option. If the number of desired jobs was specified by the user, the number of available cpus is used or 10, whichever is smaller. Specify the number of input ligands and the smallest and largest desired job sizes (Generally job lengths of 1 minute & 24 hours).

Returns a pre-set JobDJ instance for the stage to use. It already has it's hosts, local, max_retries, max_failures, default_max_retries, and verbosity set.

Return the number of max restarts to use. If -max_retries is specified, returns that value; otherwise if SCHRODINGER_MAX_RETRIES is defined, returns that value; otherwise returns default of 2. Pass this value to JobDJ.

Returns the total number of processors specified in the host string. For queued hosts with no CPU# specification, 10 is added.

Gets called periodically in order to update JobDJ's hosts. Will ask Pipeline for CPUS when needed, and will tell Pipeline when they are no longer needed.

Run the stage.

idle_function - function to call when idle

restart_file - file to periodically dump this instance to

verbosity - there are three verbosity levels: "quiet", "normal", and "verbose"
    "quiet" -   only warnings and errors are printed
    "normal" -  stage progress is printed - default
    "verbose" - additional debugging info is printed

logfh - where to send the loggin output

Adds the specified file to the stage's job control record. File must be specified as local (not absolute) path.

OVERWRITE: Perform an operation on the input Objects. use self.setOutput(position, obj) to set output objects

Generate a file name to be used by the stage.
Returns string: "<full-stagename>-<filenum><extension>"
            or: "<full-stagename>-<start>_<end><extension>"
            or: "<full-stagename><extension>"
            or: "<full-stagename>", etc.
Depending on given options.

Generate a file name to be used by the stage when writing
files for the output position <position>.
Returns string: "<full-varname>-<filenum><extension>"
            or: "<full-varname>-<start>_<end><extension>"
            or: "<full-varname><extension>"
            or: "<full-varname>", etc.
Depending on given options.

If the file does not exist, returs False; otherwise waits for the file to finish writing and returs Frue. Obsolete

Waits until the files are fully written. Default timeout 11 days. Obsolete.