public class CPlanner extends Executable
Modifier and Type | Field and Description |
---|---|
static java.lang.String |
CLEANUP_DIR
The basename of the directory that contains the submit files for the
cleanup DAG that for the concrete dag generated for the workflow.
|
static java.lang.String |
DEFAULT_MEGADAG_MODE
The default megadag mode that is used for generation of megadags in
deferred planning.
|
private static java.lang.String |
EMPTY_FINAL_WORKFLOW_MESSAGE
The final successful message that is to be logged.
|
static java.lang.String |
JAVA_COMMAND_LINE_PROPERTY_REGEX
The regex used to match against a java property that is set using
-Dpropertyname=value in the argument string
|
private PegasusBag |
mBag
The object containing the bag of pegasus objects
|
private java.text.NumberFormat |
mNumFormatter
The number formatter to format the run submit dir entries.
|
private PlannerMetrics |
mPMetrics
The PlannerMetrics object storing the metrics about this planning instance.
|
private PlannerOptions |
mPOptions
The object containing all the options passed to the Concrete Planner.
|
private boolean |
mSendMetrics
A boolean indicating whether metrics should be sent to metrics server or not
|
private java.lang.String |
mUser
The user name of the user running Pegasus.
|
static java.lang.String |
NOOP_PREFIX
The prefix for the NoOP jobs that are created.
|
static java.lang.String |
PEGASUS_MONITORD_LAUNCH_PROPERTY_KEY
The name of the property key that determines whether pegasus-run
should monitord or not.
|
private static java.lang.String |
SUCCESS_MESSAGE
The message to be logged in case of empty executable workflow.
|
mLogger, mLogMsg, mProps, mVersion
Constructor and Description |
---|
CPlanner()
Default constructor.
|
CPlanner(LogManager logger)
The overload constructor.
|
Modifier and Type | Method and Description |
---|---|
protected void |
construct(Job job,
java.lang.String key,
java.lang.String value)
Constructs a condor variable in the condor profile namespace
associated with the job.
|
protected Job |
createNoOPJob(java.lang.String name)
It creates a NoOP job that runs on the submit host.
|
protected boolean |
createSymbolicLink(java.lang.String source,
java.lang.String destination,
java.io.File directory,
boolean logErrorToDebug)
This method generates a symlink between two files
|
protected java.lang.String |
determineRelativeSubmitDirectory(ADag dag,
java.lang.String dir,
java.lang.String user,
java.lang.String vogroup,
boolean timestampBased)
Creates the submit directory for the workflow.
|
protected java.lang.String |
determineRelativeSubmitDirectory(java.lang.String label,
java.lang.String dir,
java.lang.String user,
java.lang.String vogroup,
boolean timestampBased)
Creates the submit directory for the workflow.
|
private java.lang.String |
determineRootWorkflowUUID(ADag dag,
PlannerOptions options,
PegasusProperties properties)
Determines the workflow uuid for a workflow
|
protected java.lang.String |
doBackupAndCreateSymbolicLinkForSubmitDirectory(java.lang.String baseDir,
java.lang.String relativeSubmitDir) |
void |
executeCommand()
Executes the command on the basis of the options specified.
|
java.util.Collection<java.io.File> |
executeCommand(PlannerOptions options)
Executes the command on the basis of the options specified.
|
private void |
generateStampedeEventsForAbstractWorkflow(ADag workflow,
PegasusBag bag)
Generates events for the abstract workflow.
|
gnu.getopt.LongOpt[] |
generateValidOptions()
Tt generates the LongOpt which contain the valid options that the command
will accept.
|
protected java.lang.String |
getDAGFilename(ADag dag,
PlannerOptions options)
Returns the basename of the dag file
|
java.lang.String |
getNOOPJobName(ADag dag)
Returns the name of the noop job.
|
private java.lang.String |
getPegasusRunInvocation()
Returns the pegasus-run invocation on the workflow planned.
|
protected java.lang.String |
getRandomDirectory(ADag dag)
Sets the basename of the random directory that is created on the remote
sites per workflow.
|
protected boolean |
handleRescueDAG(ADag dag,
PlannerOptions options)
Checks for rescue dags, and determines whether to plan or not.
|
protected boolean |
handleRescueDAG(java.lang.String dag,
java.lang.String dir,
int numOfRescues)
Checks for rescue dags, and determines whether to submit a rescue dag
or not.
|
void |
initialize(java.lang.String[] opts,
char confChar)
Initialize the executable object
|
void |
loadProperties()
Loads all the properties that are needed by this class.
|
private SiteStore |
loadSiteStore(java.util.Set<java.lang.String> sites)
Loads the sites from the site catalog into the site store
|
private void |
logMemoryUsage()
Logs memory usage of the JVM
|
private void |
logSuccessfulCompletion(boolean emptyWorkflow)
Logs the successful completion message.
|
static void |
main(java.lang.String[] args)
The main program for the CPlanner.
|
PlannerOptions |
parseCommandLineArguments(java.lang.String[] args)
Parses the command line arguments using GetOpt and returns a
PlannerOptions contains all the options passed by the
user at the command line. |
PlannerOptions |
parseCommandLineArguments(java.lang.String[] args,
boolean sanitizePath)
Parses the command line arguments using GetOpt and returns a
PlannerOptions contains all the options passed by the
user at the command line. |
private ADag |
parseDAX(java.lang.String dax)
Parses the DAX and returns the associated ADag object
|
void |
printLongVersion()
Prints the long description, displaying in detail what the various options
to the command stand for.
|
void |
printShortVersion()
Prints out a short description of what the command does.
|
protected static void |
sanityCheck(java.io.File dir)
Checks the destination location for existence, if it can
be created, if it is writable etc.
|
boolean |
submitWorkflow(java.lang.String invocation)
Submits the workflow for execution using pegasus-run, a wrapper around
pegasus-submit-dag.
|
convertException, convertException, getCommandLineOptions, getEnvValue, getGVDSVersion, initialize, log, lookupConfProperty, sanityCheckOnProperties, setupLogging
public static final java.lang.String DEFAULT_MEGADAG_MODE
public static final java.lang.String CLEANUP_DIR
public static final java.lang.String NOOP_PREFIX
public static final java.lang.String PEGASUS_MONITORD_LAUNCH_PROPERTY_KEY
public static final java.lang.String JAVA_COMMAND_LINE_PROPERTY_REGEX
private static final java.lang.String EMPTY_FINAL_WORKFLOW_MESSAGE
private static final java.lang.String SUCCESS_MESSAGE
private PlannerOptions mPOptions
private PegasusBag mBag
private PlannerMetrics mPMetrics
private java.text.NumberFormat mNumFormatter
private java.lang.String mUser
private boolean mSendMetrics
public CPlanner()
public CPlanner(LogManager logger)
logger
- the logger object to use. can be null.public void initialize(java.lang.String[] opts, char confChar)
Executable
initialize
in class Executable
opts
- the command line argument passed by the userconfChar
- the short option corresponding the conf property.public static void main(java.lang.String[] args)
args
- the main arguments passed to the planner.public void loadProperties()
loadProperties
in class Executable
public void executeCommand()
args
- the command line options.public java.util.Collection<java.io.File> executeCommand(PlannerOptions options)
options
- the command line options.File
objects for the files written
out.public java.lang.String getNOOPJobName(ADag dag)
dag
- the workflowprotected Job createNoOPJob(java.lang.String name)
name
- the name to be assigned to the noop jobprotected void construct(Job job, java.lang.String key, java.lang.String value)
job
- contains the job description.key
- the key of the profile.value
- the associated value.public PlannerOptions parseCommandLineArguments(java.lang.String[] args)
PlannerOptions
contains all the options passed by the
user at the command line.args
- the arguments passed by the user at command line.public PlannerOptions parseCommandLineArguments(java.lang.String[] args, boolean sanitizePath)
PlannerOptions
contains all the options passed by the
user at the command line.args
- the arguments passed by the user at command line.sanitizePath
- whether to sanitize path during construction of optionspublic boolean submitWorkflow(java.lang.String invocation)
invocation
- the pegasus run invocationprotected java.lang.String getRandomDirectory(ADag dag)
dag
- the DAG containing the abstract workflow.public gnu.getopt.LongOpt[] generateValidOptions()
generateValidOptions
in class Executable
LongOpt
objects , corresponding to the valid
optionspublic void printShortVersion()
printShortVersion
in class Executable
public void printLongVersion()
printLongVersion
in class Executable
private java.lang.String determineRootWorkflowUUID(ADag dag, PlannerOptions options, PegasusProperties properties)
dag
- the workflowoptions
- the options passed to the plannerproperties
- the properties passed to the plannerprotected java.lang.String determineRelativeSubmitDirectory(ADag dag, java.lang.String dir, java.lang.String user, java.lang.String vogroup, boolean timestampBased) throws java.io.IOException
dag
- the workflow being worked upon.dir
- the base directory specified by the user.user
- the username of the user.vogroup
- the vogroup to which the user belongs to.timestampBased
- boolean indicating whether to have a timestamp based dir or notjava.io.IOException
- in case of unable to create submit directory.protected java.lang.String determineRelativeSubmitDirectory(java.lang.String label, java.lang.String dir, java.lang.String user, java.lang.String vogroup, boolean timestampBased) throws java.io.IOException
label
- the label of the workflowdir
- the base directory specified by the user.user
- the username of the user.vogroup
- the vogroup to which the user belongs to.timestampBased
- boolean indicating whether to have a timestamp based dir or notjava.io.IOException
- in case of unable to create submit directory.protected static void sanityCheck(java.io.File dir) throws java.io.IOException
dir
- is the new base directory to optionally create.java.io.IOException
- in case of error while writing out files.protected java.lang.String getDAGFilename(ADag dag, PlannerOptions options)
dag
- the dag that was parsed.options
- the planner optionsprotected boolean handleRescueDAG(ADag dag, PlannerOptions options)
dag
- the dag that was parsed.options
- the planner optionsprotected boolean handleRescueDAG(java.lang.String dag, java.lang.String dir, int numOfRescues)
dag
- the dag file for the daxdir
- the submit directory.numOfRescues
- the number of rescues to handle.protected boolean createSymbolicLink(java.lang.String source, java.lang.String destination, java.io.File directory, boolean logErrorToDebug)
source
- the file that has to be symlinkeddestination
- the destination of the symlinkdirectory
- the directory in which to execute the commandlogErrorToDebug
- whether to log messeage to debug or notprivate void generateStampedeEventsForAbstractWorkflow(ADag workflow, PegasusBag bag)
workflow
- the parsed daxbag
- the initialized object bagprivate SiteStore loadSiteStore(java.util.Set<java.lang.String> sites)
sites
- private void logSuccessfulCompletion(boolean emptyWorkflow)
emptyWorkflow
- indicates whether the workflow created was empty or not.private java.lang.String getPegasusRunInvocation()
protected java.lang.String doBackupAndCreateSymbolicLinkForSubmitDirectory(java.lang.String baseDir, java.lang.String relativeSubmitDir) throws java.io.IOException
java.io.IOException
private void logMemoryUsage()
private ADag parseDAX(java.lang.String dax)
dax
- path to the DAX file.