schrodinger.utils.installation_check module¶
-
schrodinger.utils.installation_check.
main
()¶ The main “entry point” for the module, which gets run by the installation_check_run wrapper script.
-
schrodinger.utils.installation_check.
cd
(path)¶
-
schrodinger.utils.installation_check.
get_host_entries
(args)¶ Get all the host entries (jobcontrol.Host objects) from the hosts file. If no -test arguments were given, return them all. If -test arguments were given, return entries whose names are given by those arguments and which are present in the hosts file (warn on absent ones)
-
schrodinger.utils.installation_check.
run_test_jobs
(entries, launch_dir='.', timeout=None)¶ Run testapp jobs using a list of host entries (jobcontrol.Host objects). Return a list of all jobids, succeeded and failed, that were run
-
schrodinger.utils.installation_check.
run_postmortem
(jobids)¶ Run postmortem for a given set of jobids, log the results into the “postmortem.log”
-
schrodinger.utils.installation_check.
test_pyqt
()¶ Determine if we have correct libraries to load PyQt.
If test fails, raises RuntimeError with suggestion on how to fix.
-
schrodinger.utils.installation_check.
run_installation_check
(output_dir='.')¶ Runs diagnostic commands and writes output in the directory specified. Note: licadmin will write the license-info file in the current directory, not in output_dir. We may want to remove the output_dir argument and leave it to the caller to cd to the correct directory
-
schrodinger.utils.installation_check.
queue_inst_file
(queue_name, filename)¶
-
schrodinger.utils.installation_check.
queue_orig_dir
(queue_name)¶
-
schrodinger.utils.installation_check.
queue_orig_file
(queue_name, filename)¶
-
schrodinger.utils.installation_check.
queue_config_diffs
()¶ Return a collection of diffs of config and template queue files (original version from the installation vs what’s in the user’s queues dir)
-
schrodinger.utils.installation_check.
queue_config_file_pairs
()¶ Return a list of pairs of config/template queue files to be compared, and a list of warnings. A pair is one file from the user’s installation and the original version of the same file (from the ‘orig’ subdirectory). A warning is produced whenever the ‘orig’ subdirectory is not found, or when a user’s or original file that should be present, is not.
-
schrodinger.utils.installation_check.
queue_config_files_diff
(inst_filepath, orig_filepath)¶ Return a diff (list of strings) of two config/template queue files
Log the basic information.
-
schrodinger.utils.installation_check.
check_running_as_root
(logger)¶ Produce a warning if we are running as root.
-
schrodinger.utils.installation_check.
parse_args
(args=None)¶ Parse cmdline arguments.
-
class
schrodinger.utils.installation_check.
TestJobDJ
(entries, basedir=None, *args, **kwargs)¶ Bases:
schrodinger.job.queue.JobDJ
A flavor of JobDJ that takes a list of host entries (jobcontrol.Host objects) and lets the caller schedule exactly one job per entry.
Create an instance of this class, add the jobs, and call run().
-
__init__
(entries, basedir=None, *args, **kwargs)¶ Constructor.
Parameters: - hosts (A list of (<hostname>, <available cpus>) tuples, where
<hostname> is a string and <available cpus> is an integer.) – A host list specification. To convert a string representation
from -HOST to a list, use the jobcontrol.host_str_to_list()
function. If no hosts argument is specified, the hosts will be
read from the job control
SCHRODINGER_NODEFILE
environment variable. - local (bool) – If True, then all jobs will run with
-LOCAL
. Default is False. - max_retries (int) – Number of allowed retries per subjob. If this is set, it is
never overridden by the
SCHRODINGER_MAX_RETRIES
environment variable. If it is not set, the value in default_max_retries is used, andSCHRODINGER_MAX_RETRIES
is allowed to override. If you wish to disable restarting altogether, set this value to zero. - default_max_retries (int) – Number of allowed retries per subjob. This value can always be
overridden by the
SCHRODINGER_MAX_RETRIES
environment variable. Default is zero. - max_failures (int) – Total number of allowed subjob failures before
JobDJ
exits. If it is not defined, a default of zero will be used (exit on any failure after attempting to restart), but this can be overridden with theSCHRODINGER_MAX_FAILURES
environment variable. To allow an unlimited number of subjob failures, set max_failures to the module level NOLIMIT constant. - verbosity (str) – There are three allowed verbosity levels: “quiet” - only
warnings and errors are printed; “normal” -
JobDJ
progress is printed; and “verbose” - additional debugging info is printed. Default is “quiet”. - job_class (
BaseJob
subclass) – The class to use as the default job constructor when theaddJob
argument is not aBaseJob
instance. - update_delay (int) – The number of seconds to wait between job control database
reads for
JobControlJob
jobs. (This delay is for an individual job, not for any job database read.) Default is None, which causes the module level constant UPDATE_DELAY to be used.
- hosts (A list of (<hostname>, <available cpus>) tuples, where
<hostname> is a string and <available cpus> is an integer.) – A host list specification. To convert a string representation
from -HOST to a list, use the jobcontrol.host_str_to_list()
function. If no hosts argument is specified, the hosts will be
read from the job control
-
basedir
()¶
-
entryByName
(name)¶
-
completedJobids
()¶ The jobids of all jobs that have completed (with success or failure)
-
active_jobs
¶
-
addJob
(job, add_connected=True, **kwargs)¶ Add a job to run. If
job
is not aBaseJob
instance, aBaseJob
instance is constructed withjob
as the first argument. The defaultBaseJob
class for theJobDJ
instance can be specified in the constructor.Additional keyword arguments are passed on to the Job constructor.
All job prerequisites and dependencies need to be specified before adding a job to
JobDJ
.Parameters: add_connected (bool) – If True, for jobs with dependencies only one job per connected group should be added and all connected jobs will be discovered and added automatically. If False, it is the user’s responsibility to make sure that any prerequisites of a job are also added.
-
all_jobs
¶
-
disableSmartDistribution
()¶ Disable smart distribution of jobs.
Smart distribution allows subjobs to run on the machine that JobDJ is running on when JobDJ itself is running under a queuing system. This is usually desirable since the JobDJ process doesn’t generally consume significant computational resources and you don’t want to leave a queue slot mostly idle.
-
done_jobs
¶ Successfully completed jobs, sorted into the order they were marked as completed by JobDJ.
-
dump
(filename)¶ Pickle the
JobDJ
instance to the specified file name.
-
failed_jobs
¶
-
getActiveProcCounts
()¶ Return a dictionary containing the number of active jobs on each host.
-
hasStarted
()¶ Returns True if JobDJ has started already
-
isComplete
()¶ Returns True if JobDJ has completed, False otherwise.
-
killJobs
()¶ Kill all active jobs
-
markForRestart
(job, action)¶ Mark a job as dead, but make sure that it gets restarted.
Parameters: action – Describes the reason the job is being restarted.
-
printStatus
(job=None, action=None)¶ Prints the status of
JobDJ
and the action/status for the job.If no job is specified, prints the status header.
If no action is specified, the
status_string
attribute of the job is used.
-
run
(status_change_callback=None, periodic_callback=None, callback_interval=300, restart_failed=True)¶ Call this method to run all jobs that have been added. The method will return control when all jobs have completed.
Parameters: - status_change_callback – A command to call every time the status changes. An alternative approach is to use the updatedJobs generator, which makes it easy to determine which job changed its state.
- periodic_callback (callable) – A command to call periodically, regardless of whether job status has changed or not. The function will be called without any arguments.
- callback_interval (int) – The interval at which the periodic interval will be called. This time is only approximately enforced and will depend on the timing delay settings (e.g. MONITOR_DELAY).
- restart_failed (bool) – True (default) if previously failed jobs should be restarted, False if not.
-
setHostList
(host_list)¶ Tell JobDJ to use the specified host list.
Repeated hostnames will combine the available cpus and keep the list position of the first occurrence.
Active jobs are not affected by a change in the host list.
Parameters: host_list (A list of (<hostname>, <available cpus>) tuples, where <hostname> is a string and <available cpus> is an integer.) – A host list specification.
-
total_active
¶ The number of jobs currently running.
-
total_added
¶ The number of individual jobs that have been added to the JobDJ instance.
-
total_failed
¶ The number of jobs that have failed.
-
total_finished
¶ The number of jobs that have finished successfully.
-
updatedJobs
(periodic_callback=None, callback_interval=300, restart_failed=True)¶ A generator that starts job distribution and yields each job as its status changes. A status change occurs when a job is started, when it finishes, or fails. The
state
property of the yielded job can be examined to determine its current state.Use as:
for job in jobdj.updatedJobs(): if job.state == DONE and isinstance(job, JobControlJob): print "%s is done." % job.getJob().JobId
Parameters: restart_failed (bool) – True (default) if previously failed jobs should be restarted, False if not.
-
waiting_jobs
¶ Jobs waiting to be started.
-
-
class
schrodinger.utils.installation_check.
TestJob
(command_dir=None, duration=10, timeout=None)¶ Bases:
schrodinger.job.queue.JobControlJob
A flavor of JobControlJob to be used with TestJobDJ. Its command line will be determined when run() is called on it, by setup(); the job will get named after the host entry that it gets to run on.
-
__init__
(command_dir=None, duration=10, timeout=None)¶ Parameters: - command_dir (str) – The launch directory of the job
- duration (int) – The duration of the (testapp) job
- timeout (int) – Timeout (in seconds) after which the job will be killed. If None, the job is allowed to run indefinitely.
-
doCommand
(host, local)¶ Launch job on specified
host
using jobcontrol.launch_job(). The-LOCAL
flag is added to the job invocation command iflocal
is True.Parameters: - host (str) – Host on which the job will be executed.
- local (bool) – Should “-LOCAL” be appended to the command?
-
run
(host, local)¶ Run the job.
- The steps taken are as follows:
- Execute the preCommand method for things like changing the working directory.
- Call the doCommand to do the actual work of computation or job launching.
- Call the postCommand method to undo the changes from the preCommand that need to be undone.
-
formCommand
(entry)¶ Generate a command line appropriate for a given host entry
-
addFinalizer
(function, run_dir=None)¶ Add a function to be invoked when the job completes successfully.
See also the add_multi_job_finalizer function.
-
addGroupPrereq
(job)¶ Make all jobs connected to
job
prerequisites of all jobs connected to this Job.
-
addPrereq
(job)¶ Add a job that is an immediate prerequisite for this one.
-
cancelSubmitted
()¶ If the job is still in the ‘submitted’ state, cancel it, purge the jobrecord and set the job handle to None.
Return True if this was successful, False otherwise.
-
finalize
()¶ Clean up after a job successfully runs.
-
genAllJobs
(seen=None)¶ A generator that yields all jobs connected to this one.
-
genAllPrereqs
(seen=None)¶ A generator that yields all jobs that are prerequisites on this one.
-
getCommand
()¶ A hook method that can allow for the command to be generated at run time.
-
getCommandDir
()¶ Return the launch/command directory name. If None is returned, the job will be launched in the current directory.
-
getJob
()¶ Return the job record as a schrodinger.job.jobcontrol.Job instance.
Returns None if the job hasn’t been launched.
-
getJobDJ
()¶ Return the JobDJ instance that this job has been added to.
-
getPrereqs
()¶ Return a set of all immediate prerequisites for this job.
-
getStatusStrings
()¶ Return a tuple of status strings for printing by
JobDJ
.The strings returned are (status, jobid, name, host).
-
hasStarted
()¶ Returns True if this job has started (not waiting)
-
init_count
= 0¶
-
isComplete
()¶ Returns True if this job finished successfully
-
kill
()¶ Send kill request to jobcontrol managed job
-
maxFailuresReached
(msg)¶ Print an error summary, including the last 20 lines from each log file in the LogFiles list of the job record.
-
postCommand
()¶ A method to restore things to the pre-command state.
-
preCommand
()¶ A method to make pre-command changes, like cd’ing to the correct directory to run the command in.
-
retryFailure
(max_retries=0)¶ This method will be called when the job has failed, and JobDJ needs to know whether the job should be retried or not.
JobDJ’s value for the max_retries parameter is passed in, to be used when the job doesn’t have its own max_retries value.
Return True if this job should be retried, otherwise False.
-
runsLocally
()¶ Return True if the job runs on the
JobDJ
control host, False if not. Jobs that run locally don’t need hosts.There is no limit on the number of locally run jobs.
-
setup
()¶ A method to do initial setup; executed after
preCommand
, just beforedoCommand
.
-
state
¶ Return the current state of the job.
Note that this method can be overridden by subclasses that wish to provide for restartability at a higher level than unpickling
BaseJob
instances. For example, by examining some external condition (e.g. presence of output files) the state DONE could be returned immediately and the job would not run.
-
update
()¶ Checks for changes in job status, and updates the object appropriately (marks for restart, etc). Raises a RuntimeError if an unknown Job Status or ExitStatus is encountered.
-
usesJobServer
()¶ Detect, by looking at the jobId, whether this job uses a job server. Since the jobId is only set once, cache the answer (_uses_job_server) once it is established.
-
-
schrodinger.utils.installation_check.
find_user_libstdcpp
()¶ Returns path to system libstdc++.so.6 (or default libstdc++.so.6 before SCHRODINGER environment was found.
-
schrodinger.utils.installation_check.
get_libstdcpp_version
(library)¶ Returns GLIBCXX minor version from a given library.
Parameters: library (str) – pathname to a libstdc++.so.6
-
schrodinger.utils.installation_check.
get_bundled_libstdcpp
()¶
-
schrodinger.utils.installation_check.
is_bundled_cpp_too_old
()¶ Tests the version of libstdc++.so.6 that we ship with SCHRODINGER software versus the version that in system (or preceding LD_LIBRARY_PATH).
Returns True if the system libstdc++.so.6 should be used in preference to the bundled version.