Package edu.isi.pegasus.planner.common
Class PegasusProperties
- java.lang.Object
-
- edu.isi.pegasus.planner.common.PegasusProperties
-
- All Implemented Interfaces:
java.lang.Cloneable
public class PegasusProperties extends java.lang.Object implements java.lang.CloneableA Central Properties class that keeps track of all the properties used by Pegasus. All other classes access the methods in this class to get the value of the property. It access the CommonProperties class to read the property file.- Version:
- $Revision$
- Author:
- Karan Vahi, Gaurang Mehta
- See Also:
org.griphyn.common.util.CommonProperties
-
-
Nested Class Summary
Nested Classes Modifier and Type Class Description static classPegasusProperties.CLEANUP_SCOPEAn enum defining The scope for cleanup algorithm
-
Field Summary
Fields Modifier and Type Field Description static java.lang.StringALL_TRANSFER_PRIORITY_PROPERTY_KEYDefault properties that applies priorities to all kinds of transfer jobs.static java.lang.StringCONDOR_KICKSTARTstatic java.lang.StringDEFAULT_COMP_ERROR_PERCENTAGEstatic java.lang.StringDEFAULT_COMP_MULTIPLICATION_FACTORstatic java.lang.StringDEFAULT_COMP_VARIANCE_PERCENTAGEstatic java.lang.StringDEFAULT_CONDOR_BIN_DIRstatic java.lang.StringDEFAULT_CONDOR_CONFIG_DIRstatic java.lang.StringDEFAULT_DAGMAN_MAX_PRE_VALUEThe default value to be assigned for dagman.maxpre .static java.lang.StringDEFAULT_DATA_MULTIPLICATION_FACTORprivate static java.lang.StringDEFAULT_DAX_CALLBACKThe default DAXCallback that is loaded, if none is specified by the user.static java.lang.StringDEFAULT_EXEC_DIRstatic java.lang.StringDEFAULT_GRIDSTART_MODEstatic java.lang.StringDEFAULT_INVOKE_LENGTHstatic java.lang.StringDEFAULT_JOB_AGGREGATORstatic java.lang.StringDEFAULT_LOGGING_FILEstatic java.lang.StringDEFAULT_LRC_IGNORE_URLstatic java.lang.StringDEFAULT_POOL_MODEstatic java.lang.StringDEFAULT_RC_COLLECTIONstatic java.lang.StringDEFAULT_RLI_URLstatic java.lang.StringDEFAULT_RLS_EXIT_MODEstatic java.lang.StringDEFAULT_RLS_QUERY_ATTRIBstatic java.lang.StringDEFAULT_RLS_QUERY_MODEstatic java.lang.StringDEFAULT_RLS_TIMEOUTstatic java.lang.StringDEFAULT_SITE_SELECTORstatic java.lang.StringDEFAULT_SITE_SELECTOR_KEEPstatic java.lang.StringDEFAULT_SITE_SELECTOR_TIMEOUTstatic java.lang.StringDEFAULT_STAGING_DELIMITERstatic java.lang.StringDEFAULT_STORAGE_DIRstatic java.lang.StringDEFAULT_TC_MAPPER_MODEstatic java.lang.StringDEFAULT_TC_MODEstatic java.lang.StringDEFAULT_TRANSFER_PROCESSESstatic java.lang.StringDEFAULT_TRANSFER_STREAMSstatic java.lang.StringDEFAULT_TX_SELECTOR_MODEstatic java.lang.StringDISABLE_INVOKE_PROPERTYthe name of the property to disable invoke functionalityprivate java.lang.StringmDefaultPoolFileThe default path to the pool file.private java.lang.StringmDefaultTCThe default path to the transformation catalog.private java.lang.StringmDefaultTransferPriorityThe default transfer priority that needs to be applied to the transfer jobs.private java.util.SetmDeprecatedPropertiesThe set containing the deprecated properties specified by the user.private static java.util.Map<Profiles.NAMESPACES,java.lang.String>mNamepsaceToPropertiesPrefixprivate java.lang.StringmPegasusHomeThe value of the PEGASUS_HOME environment variable.private ProfilesmProfilesProfiles that are specified in the propertiesprivate CommonPropertiesmPropsThe object holding all the properties pertaining to the VDS system.private java.lang.StringmPropsInSubmitDirThe pointer to the properties file that is written out in the submit directory.static java.lang.StringPEGASUS_APP_METRICS_PREFIXstatic java.lang.StringPEGASUS_REPLICA_CATALOG_FILE_PROPERTYstatic java.lang.StringPEGASUS_REPLICA_CATALOG_PROPERTYstatic java.lang.StringPEGASUS_SITE_CATALOG_FILE_PROPERTYstatic java.lang.StringPEGASUS_SITE_CATALOG_PROPERTYstatic java.lang.StringPEGASUS_TRANSFER_WORKER_PACKAGE_PROPERTYstatic java.lang.StringPEGASUS_TRANSFORMATION_CATALOG_FILE_PROPERTYstatic java.lang.StringPEGASUS_TRANSFORMATION_CATALOG_PROPERTYstatic java.lang.StringPEGASUS_WORKER_NODE_EXECUTION_PROPERTYstatic java.lang.StringROOT_WORKFLOW_UUID_PROPERTY_KEYThe property key designated the root workflow uuid.static java.lang.StringSC_XML_FILEstatic java.lang.StringSC_XML3_FILEstatic java.lang.StringTC_DATA_FILEstatic java.lang.StringTC_TEXT_FILE
-
Constructor Summary
Constructors Modifier Constructor Description privatePegasusProperties(java.lang.String confProperties)The constructor that constructs the default paths to the various configuration files, and populates the singleton instance as required.
-
Method Summary
All Methods Static Methods Instance Methods Concrete Methods Modifier and Type Method Description booleanabortOnFirstJobFailure()Returns a boolean indicating whether seqexec trips on the first job failure.booleanaddDataDependencies()Returns a boolean indicating whether to automatically add edges as a result of underlying data dependecnies between jobs.booleanassignDefaultJobPriorities()Returns a boolean indicating whether we assign job priorities or not to the jobs Referred to by the "pegasus.job.priority.assign" property.booleanbypassFirstLevelStagingForInputs()Returns a boolean indicating whether to bypass first level staging of inputs.java.lang.Objectclone()Returns the clone of the object.booleancreateRegistrationJobs()Returns a boolean indicating whether we create registration jobs or not.booleandisableInvokeInGridStart()Returns a boolean indicating whether to disable use of invoke or not.booleandoStatWithKickstart()Return a boolean indicating whether to turn the stat option for kickstart on or not.booleanexecuteOnWorkerNode()Returns a boolean indicating whether to have jobs executing on worker node tmp or not.booleangenerateKickstartExtraOptions()Returns a boolean indicating whehter to pass extra options to kickstart or not.booleangenerateLOFFiles()Return a boolean indicating whether to generate the LOF files for the jobs or not.java.lang.StringgetAllIgnoredSites()Returns a comma separated list of sites, from which to ignore data transfers for all sites.java.lang.StringgetAllPreferredSites()Returns a comma separated list of sites, from which to prefer data transfers for all sites.java.lang.StringgetBaseSourceURLForSetupTransfers()Returns the base source URL where pointing to the directory where the worker package executables for pegasus releases are kept.java.io.FilegetBinDir()Accessor to the bin directory of the Pegasus installjava.lang.StringgetChmodDisabledSites()Returns the list of sites for which the chmod job creation has to be disabled for executable staging.java.lang.StringgetCleanupImplementation()Returns the name of the class that the user wants, to render the cleanup jobs.PegasusProperties.CLEANUP_SCOPEgetCleanupScope()Returns the scope for file cleanup.java.lang.StringgetCleanupStrategy()Returns the name of the Strategy class that the user wants, to insert the cleanup jobs in the graph.java.lang.StringgetClustererLabelKey()Returns the key that is to be used as a label key, for labelled clustering.java.lang.StringgetClusterSizeCleanupJobsPerLevel()Returns the fraction of cleanup jobs clustered into a single clustered cleanup job.java.lang.StringgetCollapseFactors()Returns a comma separated list for the node collapsing criteria for the execution pools.java.lang.StringgetCreateDirClass()Returns the name of the class that the user wants, to insert the create directory jobs in the graph in case of creating random directories.java.lang.StringgetCreateDirImplementation()Returns the name of the class that the user wants, to render the directory creation jobs.java.lang.StringgetCredName()Returns the credential name to be used for the stork transfer jobs.java.lang.StringgetDAXSchemaLocation()Returns the location of the schema for the DAX.java.lang.StringgetDAXSchemaLocation(java.lang.String defaultLocation)Returns the location of the schema for the DAX.java.lang.StringgetDefaultPathToCondorKickstart()Returns the default path to the condor kickstart.java.lang.StringgetDefaultPathToSC()Returns the default path to the site catalog file.java.lang.StringgetDefaultPathToTC()Returns the default path to the transformation catalog.private java.lang.StringgetDefaultThirdPartySites()Returns the default list of third party sites.private java.lang.StringgetDefaultTransferImplementation()Returns the default transfer implementation to be picked up for constructing transfer jobs.private java.lang.StringgetDefaultTransferPriority()Returns the default priority for the transfer jobs if specified in the properties file.java.lang.StringgetExecDirectory()Returns the execution directory suffix or absolute specified that is appended/replaced to the exec-mount-point specified in the pool catalog for the various pools.java.lang.StringgetGridFTPTimeout()Returns the timeout value in seconds after which to timeout in case of opening sockets to grid ftp server.java.lang.StringgetGridStart()Returns the GRIDSTART that is to be used to launch the jobs on the grid.longgetGridStartInvokeLength()Returns the trigger value for invoking an application through kickstart using kickstart.java.lang.StringgetHorizontalClusterPreference()Returns the users horizontal clustering preference.java.lang.StringgetHorizontalPartitionerBundleValue(java.lang.String name)Returns the bundle value for a particular transformation.java.lang.StringgetHorizontalPartitionerCollapseValue(java.lang.String name)Returns the collapse value for a particular transformation.java.lang.StringgetHttpLog4jURL()Returns the http url for log4j properties for windward project.static PegasusPropertiesgetInstance()Returns an instance to this properties object.static PegasusPropertiesgetInstance(java.lang.String confProperties)Returns an instance to this properties object.java.lang.StringgetJobAggregator()Returns what job aggregator is to be used to aggregate multiple compute jobs into a single condor job.java.lang.StringgetLog4JLogFile()Returns the location of the local log file where you want the messages to be logged.java.lang.StringgetLogFormatter()Returns the log formatter to use.java.lang.StringgetLoggingFile()Returns the file to which all the logging needs to be directed to.java.lang.StringgetLogManager()Returns the log manager to use.java.util.ListgetMatchingProperties(java.lang.String prefix, boolean system)Returns the properties matching a particular prefix as a list of sorted name value pairs, where name is the full name of the matching property (including the prefix) and value is it's value in the properties file.java.lang.StringgetMaximumCleanupJobsPerLevel()Returns the maximum number of clean up jobs created per level of the workflow in case of InPlace cleanup.java.lang.StringgetMetricsLogFile()Returns the path to the file that is used to be logging metrics Referred to by the "pegasus.log.metrics.file" property.java.lang.StringgetNumOfTransferProcesses()It returns the number of processes of g-u-c that the transfer script needs to spawn to do the transfers.java.lang.StringgetNumOfTransferStreams()It returns the number of streams that each transfer process uses to do the ftp transfer.java.lang.StringgetPartitionerDAXCallback()Returns the DAXCallback that is to be used while parsing the DAX.java.lang.StringgetPartitionerLabelKey()Returns the key that is to be used as a label key, for labelled partitioning.java.lang.StringgetPartitionParsingMode()Returns the mode for parsing the dax while writing out the partitioned daxes.java.lang.StringgetPDAXSchemaLocation(java.lang.String defaultLocation)Returns the location of the schema for the PDAX.java.lang.StringgetPoolFile()Returns the path to the pool file.java.lang.StringgetPoolMode()Returns the mode to be used for accessing the pool information.java.lang.StringgetPoolSchemaLocation()Returns the location of the schema for the DAX.java.lang.StringgetPoolSchemaLocation(java.lang.String defaultLocation)Returns the location of the schema for the site catalog file.NamespacegetProfiles(Profiles.NAMESPACES ns)Returns all the profiles relevant to a particular namespacejava.lang.StringgetPropertiesInSubmitDirectory()Returns the path to the property file that has been writting out in the submit directory.java.lang.StringgetProperty(java.lang.String key)It allows you to get any property from the property file without going through the corresponding accesor function in this class.private java.lang.StringgetProperty(java.lang.String newProperty, java.lang.String deprecatedProperty)This function is used to check whether a deprecated property is used or not.private java.lang.StringgetProperty(java.lang.String newProperty, java.lang.String deprecatedProperty, java.lang.String defaultValue)This function is used to check whether a deprecated property is used or not.java.lang.StringgetRefinementProvenanceStore()Returns the provenance store to use to log the refiner actions.java.lang.StringgetReplicaMode()Returns the replica mode.java.lang.StringgetReplicaSelector()Returns the name of the selector to be used for selection amongst the various replicas of a single lfn.java.lang.StringgetRLIURL()Returns the url to the RLI of the RLS.intgetRLSTimeout()It returns the timeout value in seconds after which to timeout in case of no activity from the RLS.java.lang.StringgetRootWorkflowUUID()Returns the root workflow UUID if defined in the properties, else null Referred to by the "pegasus.workflow.root.uuid" property.java.io.FilegetSchemaDir()Accessor to the schema directory of the Pegasus installjava.io.FilegetSharedDir()Accessor to the bin directory of the Pegasus installjava.lang.StringgetSiteSelectorKeep()Returns a value designating whether we need to keep the temporary files that are passed to the external site selectors.java.lang.StringgetSiteSelectorMode()Returns the class name of the site selector, that needs to be invoked to do the site selection.java.lang.StringgetSiteSelectorPath()Returns the path to the external site selector that needs to be called out to make the decision of site selection.intgetSiteSelectorTimeout()It returns the timeout value in seconds after which to timeout in case of no activity from the external site selector.java.lang.StringgetSLSTransferArguments()Returns the extra arguments with which the transfer executable used in PegasusLite needs to be invoked.java.lang.StringgetSLSTransferImplementation()Returns the sls transfer implementation that is to be used for constructing the transfer jobs.java.lang.StringgetStagingDelimiter()Returns the delimiter to be used for constructing the staged executable name, during transfer of executables to remote sites.java.lang.StringgetStorageDirectory()Returns the storage directory suffix or absolute specified that is appended/replaced to the storage-mount-point specified in the pool catalog for the various pools.java.lang.StringgetSubmitLogsDirectory()Returns the the path to the logs directory on the submit host.java.lang.StringgetSubmitMode()Returns which submit mode to be used to submit the jobs on to the grid.java.io.FilegetSysConfDir()Accessor to $PEGASUS_HOME/etc.java.lang.StringgetTCMapperMode()Returns the mode for loading the transformation mapper that sits in front of the transformation catalog.java.lang.StringgetTCMode()Returns the mode to be used for accessing the Transformation Catalog.java.lang.StringgetTCPath()Returns the location of the transformation catalog.java.lang.StringgetThirdPartySites(java.lang.String property)Returns the comma separated list of third party sites, specified in the properties.java.lang.StringgetThirdPartySitesRemote(java.lang.String property)Returns the comma separated list of third party sites for which the third party transfers are executed on the remote sites.java.lang.StringgetTransferArguments()Returns the arguments with which the transfer executable needs to be invoked.java.lang.StringgetTransferImplementation()Returns the transfer implementation that is to be used for constructing the transfer jobs.java.lang.StringgetTransferImplementation(java.lang.String property)Returns the transfer implementation.java.lang.StringgetTransferInterPriority()Returns the priority to be set for the interpool transfer job.private java.lang.StringgetTransferPriority(java.lang.String property)Returns the transfer priority.java.lang.StringgetTransferRefiner()Returns the transfer refiner that is to be used for adding in the transfer jobs in the workflow Referred to by the "pegasus.transfer.refiner" property.java.lang.StringgetTransferStageInPriority()Returns the priority to be set for the stage in transfer job.java.lang.StringgetTransferStageOutPriority()Returns the priority to be set for the stage out transfer job.java.lang.StringgetTXSelectorMode()Returns the mode for loading the transformation selector that selects amongst the various candidate transformation catalog entry objects.booleangetUseOfSymbolicLinks()It returns whether the use of symbolic links in case where the source and destination files happen to be on the same file system.CommonPropertiesgetVDSProperties()Returns the CommonProperties that this object encapsulates.java.lang.StringgetWingsPropertiesFile()Returns the path to the wings properties file.java.lang.StringgetWingsRequestID()Returns the request id.private voidinitializePropertyFile(java.lang.String confProperties)Gets the handle to the properties file.booleanlabelBasedSubmitDirectoryForSubWorkflows()Returns a boolean indicating whether the submit directory for the sub workflows should include the label of the sub workflow or not.private voidlogDeprecatedWarning(java.lang.String deprecatedProperty, java.lang.String newProperty)Logs a warning about the deprecated property.booleanlogJobAggregatorProgress()Returns whether the seqexec job aggregator should log progress to a log or not.booleanlogJobAggregatorProgressToGlobal()Returns whether the seqexec job aggregator should write to a global log or not.booleanlogMemoryUsage()Returns a boolean indicating whether to log JVM memory usage or not.java.util.PropertiesmatchingSubset(java.lang.String prefix, boolean keepPrefix)Extracts a specific property key subset from the known properties.java.util.Map<Profiles.NAMESPACES,java.lang.String>namespaceToPropertiesPrefix()static PegasusPropertiesnonSingletonInstance()To get a reference to the the object.protected static PegasusPropertiesnonSingletonInstance(java.lang.String confProperties)To get a reference to the the object.booleanpreserveParserLineBreaks()Returns a boolean indicating whether to preserver line breaks.booleanquoteTransferURL()Returns whether to introduce quotes around url's before handing to g-u-c and condor.ProfilesretrieveProfilesFromProperties()Retrieves profiles from the propertiesprotected ProfilesretrieveProfilesFromProperties(CommonProperties properties)Retrieves profiles from the propertiesprivate voidsanitizePathForProperty(java.util.Properties properties, java.lang.String key)Santizes the value in the properties .protected static voidsanityCheck(java.io.File dir)Checks the destination location for existence, if it can be created, if it is writable etc.booleansetPostSCRIPTDebugON()Returns a boolean indicating whether to turn debug on or not for exitcode.java.lang.ObjectsetProperty(java.lang.String key, java.lang.String value)Accessor: Overwrite any properties from within the program.booleansetXBitWithKickstart()Returns a boolean indicating whether kickstart should set x bit on staged executables before launching them.booleanstageSLSFilesViaFirstLevelStaging()Returns a boolean indicating whether to stage sls files via Pegasus First Level Staging or let Condor do it.booleansymlinkCommonLog()Completely disable placing a symlink for Condor common log (indiscriminately).booleantransferWorkerPackage()It specifies if the worker package needs to be staged to the remote site or not.booleantreatCacheAsRC()Returns a boolean indicating whether to treat the entries in the cache files as a replica catalog or not.booleanuseCondorQuotingForArguments()Returns a boolean indicating whether we want to Condor Quote the arguments of the job or not.booleanuseDeepStorageDirectoryStructure()Returns a boolean indicating whether to have a deep storage directory structure or not while staging out data to the output site.booleanuseExtendedTimeStamp()It specifies whether to use the extended timestamp format for generation of timestamps that are used to create the random directory name, and for the classads generation.booleanuseForceInTransfer()It specifies whether the underlying transfer mechanism being used should use the force option if available to transfer the files.booleanuseInvokeInGridStart()Returns a boolean indicating whether to use invoke in kickstart always or not.booleanuseTimestampForDirectoryStructure()Returns a boolean indicating whether to use timestamp for directory name creation or not.booleanwriteOutMetrics()Returns a boolean indicating whether to write out the planner metrics or not.java.lang.StringwriteOutProperties(java.lang.String directory)Writes out the properties to a temporary file in the directory passed.java.lang.StringwriteOutProperties(java.lang.String directory, boolean sanitizePath)Writes out the properties to a temporary file in the directory passed.java.lang.StringwriteOutProperties(java.lang.String directory, boolean sanitizePath, boolean setInternalVariable)Writes out the properties to a temporary file in the directory passed.
-
-
-
Field Detail
-
DISABLE_INVOKE_PROPERTY
public static final java.lang.String DISABLE_INVOKE_PROPERTY
the name of the property to disable invoke functionality- See Also:
- Constant Field Values
-
PEGASUS_WORKER_NODE_EXECUTION_PROPERTY
public static final java.lang.String PEGASUS_WORKER_NODE_EXECUTION_PROPERTY
- See Also:
- Constant Field Values
-
PEGASUS_TRANSFER_WORKER_PACKAGE_PROPERTY
public static final java.lang.String PEGASUS_TRANSFER_WORKER_PACKAGE_PROPERTY
- See Also:
- Constant Field Values
-
PEGASUS_TRANSFORMATION_CATALOG_PROPERTY
public static final java.lang.String PEGASUS_TRANSFORMATION_CATALOG_PROPERTY
- See Also:
- Constant Field Values
-
PEGASUS_TRANSFORMATION_CATALOG_FILE_PROPERTY
public static final java.lang.String PEGASUS_TRANSFORMATION_CATALOG_FILE_PROPERTY
- See Also:
- Constant Field Values
-
PEGASUS_REPLICA_CATALOG_PROPERTY
public static final java.lang.String PEGASUS_REPLICA_CATALOG_PROPERTY
- See Also:
- Constant Field Values
-
PEGASUS_REPLICA_CATALOG_FILE_PROPERTY
public static final java.lang.String PEGASUS_REPLICA_CATALOG_FILE_PROPERTY
- See Also:
- Constant Field Values
-
PEGASUS_SITE_CATALOG_PROPERTY
public static final java.lang.String PEGASUS_SITE_CATALOG_PROPERTY
- See Also:
- Constant Field Values
-
PEGASUS_SITE_CATALOG_FILE_PROPERTY
public static final java.lang.String PEGASUS_SITE_CATALOG_FILE_PROPERTY
- See Also:
- Constant Field Values
-
PEGASUS_APP_METRICS_PREFIX
public static final java.lang.String PEGASUS_APP_METRICS_PREFIX
- See Also:
- Constant Field Values
-
DEFAULT_RC_COLLECTION
public static final java.lang.String DEFAULT_RC_COLLECTION
- See Also:
- Constant Field Values
-
DEFAULT_RLI_URL
public static final java.lang.String DEFAULT_RLI_URL
-
DEFAULT_RLS_QUERY_MODE
public static final java.lang.String DEFAULT_RLS_QUERY_MODE
- See Also:
- Constant Field Values
-
DEFAULT_RLS_EXIT_MODE
public static final java.lang.String DEFAULT_RLS_EXIT_MODE
- See Also:
- Constant Field Values
-
DEFAULT_RLS_QUERY_ATTRIB
public static final java.lang.String DEFAULT_RLS_QUERY_ATTRIB
- See Also:
- Constant Field Values
-
DEFAULT_LRC_IGNORE_URL
public static final java.lang.String DEFAULT_LRC_IGNORE_URL
-
DEFAULT_RLS_TIMEOUT
public static final java.lang.String DEFAULT_RLS_TIMEOUT
- See Also:
- Constant Field Values
-
DEFAULT_EXEC_DIR
public static final java.lang.String DEFAULT_EXEC_DIR
- See Also:
- Constant Field Values
-
DEFAULT_STORAGE_DIR
public static final java.lang.String DEFAULT_STORAGE_DIR
- See Also:
- Constant Field Values
-
DEFAULT_TC_MODE
public static final java.lang.String DEFAULT_TC_MODE
- See Also:
- Constant Field Values
-
TC_TEXT_FILE
public static final java.lang.String TC_TEXT_FILE
- See Also:
- Constant Field Values
-
TC_DATA_FILE
public static final java.lang.String TC_DATA_FILE
- See Also:
- Constant Field Values
-
DEFAULT_POOL_MODE
public static final java.lang.String DEFAULT_POOL_MODE
- See Also:
- Constant Field Values
-
DEFAULT_CONDOR_BIN_DIR
public static final java.lang.String DEFAULT_CONDOR_BIN_DIR
- See Also:
- Constant Field Values
-
DEFAULT_CONDOR_CONFIG_DIR
public static final java.lang.String DEFAULT_CONDOR_CONFIG_DIR
- See Also:
- Constant Field Values
-
SC_XML_FILE
public static final java.lang.String SC_XML_FILE
- See Also:
- Constant Field Values
-
SC_XML3_FILE
public static final java.lang.String SC_XML3_FILE
- See Also:
- Constant Field Values
-
CONDOR_KICKSTART
public static final java.lang.String CONDOR_KICKSTART
- See Also:
- Constant Field Values
-
DEFAULT_STAGING_DELIMITER
public static final java.lang.String DEFAULT_STAGING_DELIMITER
- See Also:
- Constant Field Values
-
DEFAULT_TRANSFER_PROCESSES
public static final java.lang.String DEFAULT_TRANSFER_PROCESSES
- See Also:
- Constant Field Values
-
DEFAULT_TRANSFER_STREAMS
public static final java.lang.String DEFAULT_TRANSFER_STREAMS
- See Also:
- Constant Field Values
-
DEFAULT_GRIDSTART_MODE
public static final java.lang.String DEFAULT_GRIDSTART_MODE
- See Also:
- Constant Field Values
-
DEFAULT_INVOKE_LENGTH
public static final java.lang.String DEFAULT_INVOKE_LENGTH
- See Also:
- Constant Field Values
-
DEFAULT_SITE_SELECTOR
public static final java.lang.String DEFAULT_SITE_SELECTOR
- See Also:
- Constant Field Values
-
DEFAULT_SITE_SELECTOR_TIMEOUT
public static final java.lang.String DEFAULT_SITE_SELECTOR_TIMEOUT
- See Also:
- Constant Field Values
-
DEFAULT_SITE_SELECTOR_KEEP
public static final java.lang.String DEFAULT_SITE_SELECTOR_KEEP
- See Also:
- Constant Field Values
-
DEFAULT_DATA_MULTIPLICATION_FACTOR
public static final java.lang.String DEFAULT_DATA_MULTIPLICATION_FACTOR
- See Also:
- Constant Field Values
-
DEFAULT_COMP_MULTIPLICATION_FACTOR
public static final java.lang.String DEFAULT_COMP_MULTIPLICATION_FACTOR
- See Also:
- Constant Field Values
-
DEFAULT_COMP_ERROR_PERCENTAGE
public static final java.lang.String DEFAULT_COMP_ERROR_PERCENTAGE
- See Also:
- Constant Field Values
-
DEFAULT_COMP_VARIANCE_PERCENTAGE
public static final java.lang.String DEFAULT_COMP_VARIANCE_PERCENTAGE
- See Also:
- Constant Field Values
-
DEFAULT_JOB_AGGREGATOR
public static final java.lang.String DEFAULT_JOB_AGGREGATOR
- See Also:
- Constant Field Values
-
DEFAULT_TC_MAPPER_MODE
public static final java.lang.String DEFAULT_TC_MAPPER_MODE
- See Also:
- Constant Field Values
-
DEFAULT_TX_SELECTOR_MODE
public static final java.lang.String DEFAULT_TX_SELECTOR_MODE
- See Also:
- Constant Field Values
-
DEFAULT_LOGGING_FILE
public static final java.lang.String DEFAULT_LOGGING_FILE
- See Also:
- Constant Field Values
-
ALL_TRANSFER_PRIORITY_PROPERTY_KEY
public static final java.lang.String ALL_TRANSFER_PRIORITY_PROPERTY_KEY
Default properties that applies priorities to all kinds of transfer jobs.- See Also:
- Constant Field Values
-
ROOT_WORKFLOW_UUID_PROPERTY_KEY
public static final java.lang.String ROOT_WORKFLOW_UUID_PROPERTY_KEY
The property key designated the root workflow uuid.- See Also:
- Constant Field Values
-
DEFAULT_DAGMAN_MAX_PRE_VALUE
public static final java.lang.String DEFAULT_DAGMAN_MAX_PRE_VALUE
The default value to be assigned for dagman.maxpre .- See Also:
- Constant Field Values
-
DEFAULT_DAX_CALLBACK
private static final java.lang.String DEFAULT_DAX_CALLBACK
The default DAXCallback that is loaded, if none is specified by the user.- See Also:
- Constant Field Values
-
mPegasusHome
private java.lang.String mPegasusHome
The value of the PEGASUS_HOME environment variable.
-
mProps
private CommonProperties mProps
The object holding all the properties pertaining to the VDS system.
-
mDefaultTC
private java.lang.String mDefaultTC
The default path to the transformation catalog.
-
mDefaultPoolFile
private java.lang.String mDefaultPoolFile
The default path to the pool file.
-
mDefaultTransferPriority
private java.lang.String mDefaultTransferPriority
The default transfer priority that needs to be applied to the transfer jobs.
-
mDeprecatedProperties
private java.util.Set mDeprecatedProperties
The set containing the deprecated properties specified by the user.
-
mPropsInSubmitDir
private java.lang.String mPropsInSubmitDir
The pointer to the properties file that is written out in the submit directory.
-
mProfiles
private Profiles mProfiles
Profiles that are specified in the properties
-
mNamepsaceToPropertiesPrefix
private static java.util.Map<Profiles.NAMESPACES,java.lang.String> mNamepsaceToPropertiesPrefix
-
-
Constructor Detail
-
PegasusProperties
private PegasusProperties(java.lang.String confProperties)
The constructor that constructs the default paths to the various configuration files, and populates the singleton instance as required. If the properties file passed is null, then the singleton instance is invoked, else the non singleton instance is invoked.- Parameters:
confProperties- the path to conf properties, that supersede the loading of properties from $PEGASUS_HOME/.pegasusrc
-
-
Method Detail
-
namespaceToPropertiesPrefix
public java.util.Map<Profiles.NAMESPACES,java.lang.String> namespaceToPropertiesPrefix()
-
getInstance
public static PegasusProperties getInstance()
Returns an instance to this properties object.- Returns:
- a handle to the Properties class.
-
getInstance
public static PegasusProperties getInstance(java.lang.String confProperties)
Returns an instance to this properties object.- Parameters:
confProperties- the path to conf properties, that supersede the loading of properties from $PEGASUS_HOME/.pegasusrc- Returns:
- a handle to the Properties class.
-
nonSingletonInstance
protected static PegasusProperties nonSingletonInstance(java.lang.String confProperties)
To get a reference to the the object. The properties file that is loaded is from the path specified in the argument. This is *not implemented* as singleton. However the invocation of this does modify the internally held singleton object.- Parameters:
confProperties- the path to conf properties, that supersede the loading of properties from $PEGASUS_HOME/.pegasusrc- Returns:
- a handle to the Properties class.
-
nonSingletonInstance
public static PegasusProperties nonSingletonInstance()
To get a reference to the the object. The properties file that is loaded is from the path specified in the argument. This is *not implemented* as singleton. However the invocation of this does modify the internally held singleton object.- Returns:
- a handle to the Properties class.
-
retrieveProfilesFromProperties
public Profiles retrieveProfilesFromProperties()
Retrieves profiles from the properties- Parameters:
properties- the common properties so far- Returns:
- profiles object.
-
retrieveProfilesFromProperties
protected Profiles retrieveProfilesFromProperties(CommonProperties properties)
Retrieves profiles from the properties- Parameters:
properties- the common properties so far- Returns:
- profiles object.
-
clone
public java.lang.Object clone()
Returns the clone of the object.- Overrides:
clonein classjava.lang.Object- Returns:
- the clone
-
getBinDir
public java.io.File getBinDir()
Accessor to the bin directory of the Pegasus install- Returns:
- the "etc" directory of the VDS runtime system.
-
getSchemaDir
public java.io.File getSchemaDir()
Accessor to the schema directory of the Pegasus install- Returns:
- the "etc" directory of the VDS runtime system.
-
getSharedDir
public java.io.File getSharedDir()
Accessor to the bin directory of the Pegasus install- Returns:
- the "etc" directory of the VDS runtime system.
-
getProfiles
public Namespace getProfiles(Profiles.NAMESPACES ns)
Returns all the profiles relevant to a particular namespace- Parameters:
ns- the namespace corresponding to which you need the profiles
-
getDefaultPathToTC
public java.lang.String getDefaultPathToTC()
Returns the default path to the transformation catalog. Currently the default path defaults to $PEGASUS_HOME/etc/tc.text if transformation type is Text else $PEGASUS_HOME/etc/tc.data- Returns:
- the default path to transformation catalog file
-
getDefaultPathToSC
public java.lang.String getDefaultPathToSC()
Returns the default path to the site catalog file. The default path is constructed on the basis of the mode set by the user.- Returns:
- $PEGASUS_HOME/etc/sites.xml3 if the pool mode is XML3, else $PEGASUS_HOME/etc/sites.xml
- See Also:
getPoolMode()
-
getDefaultPathToCondorKickstart
public java.lang.String getDefaultPathToCondorKickstart()
Returns the default path to the condor kickstart. Currently the path defaults to $PEGASUS_HOME/bin/kickstart-condor.- Returns:
- default path to kickstart condor.
-
initializePropertyFile
private void initializePropertyFile(java.lang.String confProperties)
Gets the handle to the properties file. The singleton instance is invoked if the properties file is null (partly due to the way CommonProperties is implemented ), else the non singleton is invoked.- Parameters:
confProperties- the path to conf properties, that supersede the loading of properties from $PEGASUS_HOME/.pegasusrc
-
getProperty
public java.lang.String getProperty(java.lang.String key)
It allows you to get any property from the property file without going through the corresponding accesor function in this class. For coding and clarity purposes, the function should be used judiciously, and the accessor function should be used as far as possible.- Parameters:
key- the property whose value is desired.- Returns:
- String
-
getVDSProperties
public CommonProperties getVDSProperties()
Returns the CommonProperties that this object encapsulates. Use only when absolutely necessary. Use accessor methods whereever possible.- Returns:
- CommonProperties
-
setProperty
public java.lang.Object setProperty(java.lang.String key, java.lang.String value)Accessor: Overwrite any properties from within the program.- Parameters:
key- is the key to look upvalue- is the new property value to place in the system.- Returns:
- the old value, or null if it didn't exist before.
-
matchingSubset
public java.util.Properties matchingSubset(java.lang.String prefix, boolean keepPrefix)Extracts a specific property key subset from the known properties. The prefix may be removed from the keys in the resulting dictionary, or it may be kept. In the latter case, exact matches on the prefix will also be copied into the resulting dictionary.- Parameters:
prefix- is the key prefix to filter the properties by.keepPrefix- if true, the key prefix is kept in the resulting dictionary. As side-effect, a key that matches the prefix exactly will also be copied. If false, the resulting dictionary's keys are shortened by the prefix. An exact prefix match will not be copied, as it would result in an empty string key.- Returns:
- a property dictionary matching the filter key. May be an empty dictionary, if no prefix matches were found.
- See Also:
is used to assemble matches
-
getMatchingProperties
public java.util.List getMatchingProperties(java.lang.String prefix, boolean system)Returns the properties matching a particular prefix as a list of sorted name value pairs, where name is the full name of the matching property (including the prefix) and value is it's value in the properties file.- Parameters:
prefix- the prefix for the property names.system- boolean indicating whether to match only System properties or all including the ones in the property file.- Returns:
- list of
NameValueobjects corresponding to the matched properties sorted by keys. null if no matching property is found.
-
getSysConfDir
public java.io.File getSysConfDir()
Accessor to $PEGASUS_HOME/etc. The files in this directory have a low change frequency, are effectively read-only, they reside on a per-machine basis, and they are valid usually for a single user.- Returns:
- the "etc" directory of the VDS runtime system.
-
getDAXSchemaLocation
public java.lang.String getDAXSchemaLocation()
Returns the location of the schema for the DAX. Referred to by the "pegasus.schema.dax" property.- Returns:
- location to the DAX schema.
-
getDAXSchemaLocation
public java.lang.String getDAXSchemaLocation(java.lang.String defaultLocation)
Returns the location of the schema for the DAX. Referred to by the "pegasus.schema.dax" property.- Parameters:
defaultLocation- the default location to the schema.- Returns:
- location to the DAX schema specified in the properties file, else the default location if no value specified.
-
getPDAXSchemaLocation
public java.lang.String getPDAXSchemaLocation(java.lang.String defaultLocation)
Returns the location of the schema for the PDAX. Referred to by the "pegasus.schema.pdax" property- Parameters:
defaultLocation- the default location to the schema.- Returns:
- location to the PDAX schema specified in the properties file, else the default location if no value specified.
-
getCreateDirClass
public java.lang.String getCreateDirClass()
Returns the name of the class that the user wants, to insert the create directory jobs in the graph in case of creating random directories. Referred to by the "pegasus.dir.create.strategy" property.- Returns:
- the create dir classname if specified in the properties file, else Minimal.
-
getCreateDirImplementation
public java.lang.String getCreateDirImplementation()
Returns the name of the class that the user wants, to render the directory creation jobs. It dictates what mechanism is used to create the directory for a workflow. Referred to by the "pegasus.dir.create.impl" property.- Returns:
- the create dir classname if specified in the properties file, else DefaultImplementation.
-
useExtendedTimeStamp
public boolean useExtendedTimeStamp()
It specifies whether to use the extended timestamp format for generation of timestamps that are used to create the random directory name, and for the classads generation. Referred to by the "pegasus.dir.timestamp.extended" property.- Returns:
- the value specified in the properties file if valid boolean, else false.
-
useTimestampForDirectoryStructure
public boolean useTimestampForDirectoryStructure()
Returns a boolean indicating whether to use timestamp for directory name creation or not. Referred to by "pegasus.dir.useTimestamp" property.- Returns:
- the boolean value specified in the properties files, else false.
-
getExecDirectory
public java.lang.String getExecDirectory()
Returns the execution directory suffix or absolute specified that is appended/replaced to the exec-mount-point specified in the pool catalog for the various pools. Referred to by the "pegasus.dir.exec" property- Returns:
- the value specified in the properties file, else the default suffix.
- See Also:
DEFAULT_EXEC_DIR
-
getSubmitLogsDirectory
public java.lang.String getSubmitLogsDirectory()
Returns the the path to the logs directory on the submit host. This is the directory where the condor logs for the workflows are created. The logs directory should be on the local filesystem else condor may complain Referred to by the "pegasus.dir.submit.logs" property- Returns:
- the value in the properties file, else null
-
labelBasedSubmitDirectoryForSubWorkflows
public boolean labelBasedSubmitDirectoryForSubWorkflows()
Returns a boolean indicating whether the submit directory for the sub workflows should include the label of the sub workflow or not. Referred to by the "pegasus.dir.submit.subwf.labelbased" property- Returns:
- the value in the properties file, else false
-
getStorageDirectory
public java.lang.String getStorageDirectory()
Returns the storage directory suffix or absolute specified that is appended/replaced to the storage-mount-point specified in the pool catalog for the various pools. Referred to by the "pegasus.dir.storage" property.- Returns:
- the value specified in the properties file, else the default suffix.
- See Also:
DEFAULT_STORAGE_DIR
-
useDeepStorageDirectoryStructure
public boolean useDeepStorageDirectoryStructure()
Returns a boolean indicating whether to have a deep storage directory structure or not while staging out data to the output site. Referred to by the "pegasus.dir.storage.deep" property.- Returns:
- the boolean value specified in the properties files, else false.
-
getCleanupStrategy
public java.lang.String getCleanupStrategy()
Returns the name of the Strategy class that the user wants, to insert the cleanup jobs in the graph. Referred to by the "pegasus.file.cleanup.strategy" property.- Returns:
- the create dir classname if specified in the properties file, else InPlace.
-
getCleanupImplementation
public java.lang.String getCleanupImplementation()
Returns the name of the class that the user wants, to render the cleanup jobs. It dictates what mechanism is used to remove the files on a remote system. Referred to by the "pegasus.file.cleanup.impl" property.- Returns:
- the cleanup implementation classname if specified in the properties file, else Cleanup.
-
getMaximumCleanupJobsPerLevel
public java.lang.String getMaximumCleanupJobsPerLevel()
Returns the maximum number of clean up jobs created per level of the workflow in case of InPlace cleanup. Referred to by the "pegasus.file.cleanup.clusters.num" property- Returns:
- the value in the property file , else null
-
getClusterSizeCleanupJobsPerLevel
public java.lang.String getClusterSizeCleanupJobsPerLevel()
Returns the fraction of cleanup jobs clustered into a single clustered cleanup job. Referred to by the "pegasus.file.cleanup.clusters.size" property- Returns:
- the value in the property file , else null
-
getCleanupScope
public PegasusProperties.CLEANUP_SCOPE getCleanupScope()
Returns the scope for file cleanup. It is used to trigger cleanup in case of deferred planning. The vaild property values accepted are - fullahead - deferred Referred to by the property "pegasus.file.cleanup.scope"- Returns:
- the value in property file if specified, else fullahead
-
getTCMode
public java.lang.String getTCMode()
Returns the mode to be used for accessing the Transformation Catalog. Referred to by the "pegasus.catalog.transformation" property.- Returns:
- the value specified in properties file, else DEFAULT_TC_MODE.
- See Also:
DEFAULT_TC_MODE
-
getTCPath
public java.lang.String getTCPath()
Returns the location of the transformation catalog. Referred to by "pegasus.catalog.transformation.file" property.- Returns:
- the value specified in the properties file, else default path specified by mDefaultTC.
- See Also:
mDefaultTC
-
getTCMapperMode
public java.lang.String getTCMapperMode()
Returns the mode for loading the transformation mapper that sits in front of the transformation catalog. Referred to by the "pegasus.catalog.transformation.mapper" property.- Returns:
- the value specified in the properties file, else default tc mapper mode.
- See Also:
DEFAULT_TC_MAPPER_MODE
-
getReplicaMode
public java.lang.String getReplicaMode()
Returns the replica mode. It identifies the ReplicaMechanism being used by Pegasus to determine logical file locations. Referred to by the "pegasus.catalog.replica" property.- Returns:
- the replica mode, that is used to load the appropriate implementing class if property is specified, else null
-
getRLIURL
public java.lang.String getRLIURL()
Returns the url to the RLI of the RLS. Referred to by the "pegasus.rls.url" property.- Returns:
- the value specified in properties file, else DEFAULT_RLI_URL.
- See Also:
DEFAULT_RLI_URL
-
getRLSTimeout
public int getRLSTimeout()
It returns the timeout value in seconds after which to timeout in case of no activity from the RLS. Referred to by the "pegasus.rc.rls.timeout" property.- Returns:
- the timeout value if specified else, DEFAULT_RLS_TIMEOUT.
- See Also:
DEFAULT_RLS_TIMEOUT
-
getPoolMode
public java.lang.String getPoolMode()
Returns the mode to be used for accessing the pool information. Referred to by the "pegasus.catalog.site" property.- Returns:
- the pool mode, that is used to load the appropriate implementing class if the property is specified, else default pool mode specified by DEFAULT_POOL_MODE
- See Also:
DEFAULT_POOL_MODE
-
getPoolFile
public java.lang.String getPoolFile()
Returns the path to the pool file. Referred to by the "pegasus.catalog.site.file" property.- Returns:
- the path to the pool file specified in the properties file, else the default path specified by mDefaultPoolFile.
- See Also:
mDefaultPoolFile
-
getPoolSchemaLocation
public java.lang.String getPoolSchemaLocation()
Returns the location of the schema for the DAX. Referred to by the "pegasus.schema.sc" property.- Returns:
- the location of pool schema if specified in properties file, else null.
-
getPoolSchemaLocation
public java.lang.String getPoolSchemaLocation(java.lang.String defaultLocation)
Returns the location of the schema for the site catalog file. Referred to by the "pegasus.schema.sc" property- Parameters:
defaultLocation- the default location where the schema should be if no other location is specified.- Returns:
- the location specified by the property, else defaultLocation.
-
getRefinementProvenanceStore
public java.lang.String getRefinementProvenanceStore()
Returns the provenance store to use to log the refiner actions. Referred to by the "pegasus.catalog.provenance.refinement" property.- Returns:
- the value set in the properties, else null if not set.
-
getTransferImplementation
public java.lang.String getTransferImplementation()
Returns the transfer implementation that is to be used for constructing the transfer jobs. Referred to by the "pegasus.transfer.*.impl" property.- Returns:
- the transfer implementation
-
getSLSTransferImplementation
public java.lang.String getSLSTransferImplementation()
Returns the sls transfer implementation that is to be used for constructing the transfer jobs. Referred to by the "pegasus.transfer.lite.*.impl" property.- Returns:
- the transfer implementation
-
getTransferImplementation
public java.lang.String getTransferImplementation(java.lang.String property)
Returns the transfer implementation.- Parameters:
property- property name.- Returns:
- the transfer implementation, else the one specified by "pegasus.transfer.*.impl",
-
stageSLSFilesViaFirstLevelStaging
public boolean stageSLSFilesViaFirstLevelStaging()
Returns a boolean indicating whether to stage sls files via Pegasus First Level Staging or let Condor do it. Referred to by the property "pegasus.transfer.stage.lite.file"- Returns:
- boolean value mentioned in the properties or else the default value which is true.
-
getDefaultThirdPartySites
private java.lang.String getDefaultThirdPartySites()
Returns the default list of third party sites. Referred to by the "pegasus.transfer.*.thirdparty.sites" property.- Returns:
- the value specified in the properties file, else null.
-
getDefaultTransferImplementation
private java.lang.String getDefaultTransferImplementation()
Returns the default transfer implementation to be picked up for constructing transfer jobs. Referred to by the "pegasus.transfer.*.impl" property.- Returns:
- the value specified in the properties file, else null.
-
bypassFirstLevelStagingForInputs
public boolean bypassFirstLevelStagingForInputs()
Returns a boolean indicating whether to bypass first level staging of inputs. Useful in case of PegasusLite setup Referred to by the "pegasus.transfer.bypass.input.staging" property.- Returns:
- boolean value specified , else false
-
getDefaultTransferPriority
private java.lang.String getDefaultTransferPriority()
Returns the default priority for the transfer jobs if specified in the properties file.- Returns:
- the value specified in the properties file, else null if non integer value or no value specified.
-
getBaseSourceURLForSetupTransfers
public java.lang.String getBaseSourceURLForSetupTransfers()
Returns the base source URL where pointing to the directory where the worker package executables for pegasus releases are kept. Referred to by the "pegasus.transfer.setup.source.base.url- Returns:
- the value in the property file, else null
-
getTransferRefiner
public java.lang.String getTransferRefiner()
Returns the transfer refiner that is to be used for adding in the transfer jobs in the workflow Referred to by the "pegasus.transfer.refiner" property.- Returns:
- the transfer refiner, else null
-
quoteTransferURL
public boolean quoteTransferURL()
Returns whether to introduce quotes around url's before handing to g-u-c and condor. Referred to by "pegasus.transfer.single.quote" property.- Returns:
- boolean value specified in the properties file, else true in case of non boolean value being specified or property not being set.
-
getNumOfTransferProcesses
public java.lang.String getNumOfTransferProcesses()
It returns the number of processes of g-u-c that the transfer script needs to spawn to do the transfers. This is applicable only in the case where the transfer executable has the capability of spawning processes. It should not be confused with the number of streams that each process opens. By default it is set to 4. In case a non integer value is specified in the properties file it returns the default value. Referred to by "pegasus.transfer.throttle.processes" property.- Returns:
- the number of processes specified in properties file, else DEFAULT_TRANSFER_PROCESSES
- See Also:
DEFAULT_TRANSFER_PROCESSES
-
getNumOfTransferStreams
public java.lang.String getNumOfTransferStreams()
It returns the number of streams that each transfer process uses to do the ftp transfer. By default it is set to 1.In case a non integer value is specified in the properties file it returns the default value. Referred to by "pegasus.transfer.throttle.streams" property.- Returns:
- the number of streams specified in the properties file, else DEFAULT_TRANSFER_STREAMS.
- See Also:
DEFAULT_TRANSFER_STREAMS
-
useForceInTransfer
public boolean useForceInTransfer()
It specifies whether the underlying transfer mechanism being used should use the force option if available to transfer the files. Referred to by "pegasus.transfer.force" property.- Returns:
- boolean value specified in the properties file,else false in case of non boolean value being specified or property not being set.
-
getUseOfSymbolicLinks
public boolean getUseOfSymbolicLinks()
It returns whether the use of symbolic links in case where the source and destination files happen to be on the same file system. Referred to by "pegasus.transfer.links" property.- Returns:
- boolean value specified in the properties file, else false in case of non boolean value being specified or property not being set.
-
getThirdPartySites
public java.lang.String getThirdPartySites(java.lang.String property)
Returns the comma separated list of third party sites, specified in the properties.- Parameters:
property- property name.- Returns:
- the comma separated list of sites.
-
getThirdPartySitesRemote
public java.lang.String getThirdPartySitesRemote(java.lang.String property)
Returns the comma separated list of third party sites for which the third party transfers are executed on the remote sites.- Parameters:
property- property name.- Returns:
- the comma separated list of sites.
-
getStagingDelimiter
public java.lang.String getStagingDelimiter()
Returns the delimiter to be used for constructing the staged executable name, during transfer of executables to remote sites. Referred to by the "pegasus.transfer.staging.delimiter" property.- Returns:
- the value specified in the properties file, else DEFAULT_STAGING_DELIMITER
- See Also:
DEFAULT_STAGING_DELIMITER
-
getChmodDisabledSites
public java.lang.String getChmodDisabledSites()
Returns the list of sites for which the chmod job creation has to be disabled for executable staging. Referred to by the "pegasus.transfer.disable.chmod" property.- Returns:
- a comma separated list of site names.
-
transferWorkerPackage
public boolean transferWorkerPackage()
It specifies if the worker package needs to be staged to the remote site or not. Referred to by "pegasus.transfer.worker.package" property.- Returns:
- boolean value specified in the properties file,else false in case of non boolean value being specified or property not being set.
-
getTransferArguments
public java.lang.String getTransferArguments()
Returns the arguments with which the transfer executable needs to be invoked. Referred to by "pegasus.transfer.arguments" property.- Returns:
- the arguments specified in the properties file, else null if property is not specified.
-
getSLSTransferArguments
public java.lang.String getSLSTransferArguments()
Returns the extra arguments with which the transfer executable used in PegasusLite needs to be invoked. Referred to by "pegasus.transfer.lite.arguments" property.- Returns:
- the arguments specified in the properties file, else null if property is not specified.
-
getTransferStageInPriority
public java.lang.String getTransferStageInPriority()
Returns the priority to be set for the stage in transfer job. Referred to by "pegasus.transfer.stagein.priority" property if set, else by "pegasus.transfer.*.priority" property.- Returns:
- the priority as String if a valid integer specified in the properties, else null.
-
getTransferStageOutPriority
public java.lang.String getTransferStageOutPriority()
Returns the priority to be set for the stage out transfer job. Referred to by "pegasus.transfer.stageout.priority" property if set, else by "pegasus.transfer.*.priority" property.- Returns:
- the priority as String if a valid integer specified in the properties, else null.
-
getTransferInterPriority
public java.lang.String getTransferInterPriority()
Returns the priority to be set for the interpool transfer job. Referred to by "pegasus.transfer.inter.priority" property if set, else by "pegasus.transfer.*.priority" property.- Returns:
- the priority as String if a valid integer specified in the properties, else null.
-
getTransferPriority
private java.lang.String getTransferPriority(java.lang.String property)
Returns the transfer priority.- Parameters:
property- property name.- Returns:
- the priority as String if a valid integer specified in the properties as value to property, else null.
-
getTXSelectorMode
public java.lang.String getTXSelectorMode()
Returns the mode for loading the transformation selector that selects amongst the various candidate transformation catalog entry objects. Referred to by the "pegasus.selector.transformation" property.- Returns:
- the value specified in the properties file, else default transformation selector.
- See Also:
DEFAULT_TC_MAPPER_MODE
-
getReplicaSelector
public java.lang.String getReplicaSelector()
Returns the name of the selector to be used for selection amongst the various replicas of a single lfn. Referred to by the "pegasus.selector.replica" property.- Returns:
- the name of the selector if the property is specified, else null
-
getAllPreferredSites
public java.lang.String getAllPreferredSites()
Returns a comma separated list of sites, from which to prefer data transfers for all sites. Referred to by the "pegasus.selector.replica.*.prefer.stagein.sites" property.- Returns:
- comma separated list of sites.
-
getAllIgnoredSites
public java.lang.String getAllIgnoredSites()
Returns a comma separated list of sites, from which to ignore data transfers for all sites. Replaces the old pegasus.rc.restricted.sites property. Referred to by the "pegasus.selector.ignore.*.prefer.stagein.sites" property.- Returns:
- comma separated list of sites.
-
getSiteSelectorMode
public java.lang.String getSiteSelectorMode()
Returns the class name of the site selector, that needs to be invoked to do the site selection. Referred to by the "pegasus.selector.site" property.- Returns:
- the classname corresponding to the site selector that needs to be invoked if specified in the properties file, else the default selector specified by DEFAULT_SITE_SELECTOR.
- See Also:
DEFAULT_SITE_SELECTOR
-
getSiteSelectorPath
public java.lang.String getSiteSelectorPath()
Returns the path to the external site selector that needs to be called out to make the decision of site selection. Referred to by the "pegasus.selector.site.path" property.- Returns:
- the path to the external site selector if specified in the properties file, else null.
-
getSiteSelectorTimeout
public int getSiteSelectorTimeout()
It returns the timeout value in seconds after which to timeout in case of no activity from the external site selector. Referred to by the "pegasus.selector.site.timeout" property.- Returns:
- the timeout value if specified else, DEFAULT_SITE_SELECTOR_TIMEOUT.
- See Also:
DEFAULT_SITE_SELECTOR_TIMEOUT
-
getSiteSelectorKeep
public java.lang.String getSiteSelectorKeep()
Returns a value designating whether we need to keep the temporary files that are passed to the external site selectors. The check for the valid tristate value should be done at the calling function end. This just passes on the value user specified in the properties file. Referred to by the "pegasus.selector.site.keep.tmp" property.- Returns:
- the value of the property is specified, else DEFAULT_SITE_SELECTOR_KEEP
- See Also:
DEFAULT_SITE_SELECTOR_KEEP
-
getGridStart
public java.lang.String getGridStart()
Returns the GRIDSTART that is to be used to launch the jobs on the grid. Referred to by the "pegasus.gridstart" property.- Returns:
- the value specified in the property file, else DEFAULT_GRIDSTART_MODE
- See Also:
DEFAULT_GRIDSTART_MODE
-
setXBitWithKickstart
public boolean setXBitWithKickstart()
Returns a boolean indicating whether kickstart should set x bit on staged executables before launching them. Referred to by the "pegasus.gridstart.kickstart.set.xbit" property.- Returns:
- the value specified in the property file, else false
-
doStatWithKickstart
public boolean doStatWithKickstart()
Return a boolean indicating whether to turn the stat option for kickstart on or not. By default it is turned on. Referred to by the "pegasus.gridstart.kickstart.stat" property.- Returns:
- the boolean value specified in the property file, else false if not specified or non boolean specified.
-
generateLOFFiles
public boolean generateLOFFiles()
Return a boolean indicating whether to generate the LOF files for the jobs or not. This is used to generate LOF files, but not trigger the stat option Referred to by the "pegasus.gridstart.kickstart.generate.loft" property.- Returns:
- the boolean value specified in the property file, else false if not specified or non boolean specified.
-
useInvokeInGridStart
public boolean useInvokeInGridStart()
Returns a boolean indicating whether to use invoke in kickstart always or not. Referred to by the "pegasus.gridstart.invoke.always" property.- Returns:
- the boolean value specified in the property file, else false if not specified or non boolean specified.
-
disableInvokeInGridStart
public boolean disableInvokeInGridStart()
Returns a boolean indicating whether to disable use of invoke or not. Referred to by the "pegasus.gridstart.invoke.disable" property.- Returns:
- the boolean value specified in the property file, else false if not specified or non boolean specified.
-
getGridStartInvokeLength
public long getGridStartInvokeLength()
Returns the trigger value for invoking an application through kickstart using kickstart. If the arguments value being constructed in the condor submit file is more than this value, then invoke is used to pass the arguments to the remote end. Helps in bypassing the Condor 4K limit. Referred to by "pegasus.gridstart.invoke.length" property.- Returns:
- the long value specified in the properties files, else DEFAULT_INVOKE_LENGTH
- See Also:
DEFAULT_INVOKE_LENGTH
-
generateKickstartExtraOptions
public boolean generateKickstartExtraOptions()
Returns a boolean indicating whehter to pass extra options to kickstart or not. The extra options have appeared only in VDS version 1.4.2 (like -L and -T). Referred to by "pegasus.gridstart.label" property.- Returns:
- the boolean value specified in the property file, else true if not specified or non boolean specified.
-
setPostSCRIPTDebugON
public boolean setPostSCRIPTDebugON()
Returns a boolean indicating whether to turn debug on or not for exitcode. By default false is returned. Referred to by the "pegasus.exitcode.debug" property.- Returns:
- boolean value.
-
symlinkCommonLog
public boolean symlinkCommonLog()
Completely disable placing a symlink for Condor common log (indiscriminately). Starting 4.2.1 this defaults to "false" . Referred to by the "pegasus.condor.logs.symlink" property.- Returns:
- value specified by the property. Defaults to false.
-
useCondorQuotingForArguments
public boolean useCondorQuotingForArguments()
Returns a boolean indicating whether we want to Condor Quote the arguments of the job or not. Referred to by the "pegasus.condor.arguments.quote" property.- Returns:
- boolean
-
getCredName
public java.lang.String getCredName()
Returns the credential name to be used for the stork transfer jobs. Referred to by the "pegasus.transfer.stork.cred" property.- Returns:
- the credential name if specified by the property, else null.
-
getLogManager
public java.lang.String getLogManager()
Returns the log manager to use. Referred to by the "pegasus.log.manager" property.- Returns:
- the value in the properties file, else Default
-
getLogFormatter
public java.lang.String getLogFormatter()
Returns the log formatter to use. Referred to by the "pegasus.log.formatter" property.- Returns:
- the value in the properties file, else Simple
-
getHttpLog4jURL
public java.lang.String getHttpLog4jURL()
Returns the http url for log4j properties for windward project. Referred to by the "log4j.configuration" property.- Returns:
- the value in the properties file, else null
-
getLoggingFile
public java.lang.String getLoggingFile()
Returns the file to which all the logging needs to be directed to. Referred to by the "pegasus.log.*" property.- Returns:
- the value of the property that is specified, else null
-
getLog4JLogFile
public java.lang.String getLog4JLogFile()
Returns the location of the local log file where you want the messages to be logged. Not used for the moment. Referred to by the "pegasus.log4j.log" property.- Returns:
- the value specified in the property file,else null.
-
writeOutMetrics
public boolean writeOutMetrics()
Returns a boolean indicating whether to write out the planner metrics or not. Referred to by the "pegasus.log.metrics" property.- Returns:
- boolean in the properties, else true
-
getMetricsLogFile
public java.lang.String getMetricsLogFile()
Returns the path to the file that is used to be logging metrics Referred to by the "pegasus.log.metrics.file" property.- Returns:
- path to the metrics file if specified, else rundir/pegasus.metrics
-
logMemoryUsage
public boolean logMemoryUsage()
Returns a boolean indicating whether to log JVM memory usage or not. Referred to by the "pegasus.log.memory.usage" property.- Returns:
- boolean value specified in properties else false.
-
assignDefaultJobPriorities
public boolean assignDefaultJobPriorities()
Returns a boolean indicating whether we assign job priorities or not to the jobs Referred to by the "pegasus.job.priority.assign" property.- Returns:
- boolean value specified in properties else true.
-
createRegistrationJobs
public boolean createRegistrationJobs()
Returns a boolean indicating whether we create registration jobs or not. Referred to by the "pegasus.register" property.- Returns:
- boolean value specified in properties else true.
-
executeOnWorkerNode
public boolean executeOnWorkerNode()
Returns a boolean indicating whether to have jobs executing on worker node tmp or not. Referred to by the "pegasus.execute.*.filesystem.local" property.- Returns:
- boolean value in the properties file, else false if not specified or an invalid value specified.
-
treatCacheAsRC
public boolean treatCacheAsRC()
Returns a boolean indicating whether to treat the entries in the cache files as a replica catalog or not.- Returns:
- boolean
-
preserveParserLineBreaks
public boolean preserveParserLineBreaks()
Returns a boolean indicating whether to preserver line breaks. Referred to by the "pegasus.parser.dax.preserve.linebreaks" property.- Returns:
- boolean value in the properties file, else false if not specified or an invalid value specified.
-
addDataDependencies
public boolean addDataDependencies()
Returns a boolean indicating whether to automatically add edges as a result of underlying data dependecnies between jobs. Referred to by the "pegasus.parser.dax.data.dependencies" property.- Returns:
- boolean value in the properties file, else true if not specified or an invalid value specified.
-
getWingsPropertiesFile
public java.lang.String getWingsPropertiesFile()
Returns the path to the wings properties file. Referred to by the "pegasus.wings.properties" property.- Returns:
- value in the properties file, else null.
-
getWingsRequestID
public java.lang.String getWingsRequestID()
Returns the request id. Referred to by the "pegasus.wings.request-id" property.- Returns:
- value in the properties file, else null.
-
getGridFTPTimeout
public java.lang.String getGridFTPTimeout()
Returns the timeout value in seconds after which to timeout in case of opening sockets to grid ftp server. Referred to by the "pegasus.auth.gridftp.timeout" property.- Returns:
- the timeout value if specified else, null.
- See Also:
DEFAULT_SITE_SELECTOR_TIMEOUT
-
getSubmitMode
public java.lang.String getSubmitMode()
Returns which submit mode to be used to submit the jobs on to the grid. Referred to by the "pegasus.code.generator" property.- Returns:
- the submit mode specified in the property file, else the default i.e condor.
-
getPartitionParsingMode
public java.lang.String getPartitionParsingMode()
Returns the mode for parsing the dax while writing out the partitioned daxes. Referred to by the "pegasus.partition.parser.load" property.- Returns:
- the value specified in the properties file, else the default value i.e single.
-
getCollapseFactors
public java.lang.String getCollapseFactors()
Returns a comma separated list for the node collapsing criteria for the execution pools. This determines how many jobs one fat node gobbles up. Referred to by the "pegasus.cluster.nodes" property.- Returns:
- the value specified in the properties file, else null.
-
getHorizontalClusterPreference
public java.lang.String getHorizontalClusterPreference()
Returns the users horizontal clustering preference. This property determines how to cluster horizontal jobs. If this property is set with a value value of runtime, the jobs will be grouped into into clusters according to their runtimes as specified byjob.runtimeproperty. For all other cases the default horizontal clustering approach will be used.- Returns:
- the value specified in the properties file, else null.
-
getJobAggregator
public java.lang.String getJobAggregator()
Returns what job aggregator is to be used to aggregate multiple compute jobs into a single condor job. Referred to by the "pegasus.cluster.job.aggregator" property.- Returns:
- the value specified in the properties file, else DEFAULT_JOB_AGGREGATOR
- See Also:
DEFAULT_JOB_AGGREGATOR
-
logJobAggregatorProgress
public boolean logJobAggregatorProgress()
Returns whether the seqexec job aggregator should log progress to a log or not. Referred to by the "pegasus.clusterer.job.aggregator.seqexec.log" property.- Returns:
- the value specified in the properties file, else false
-
logJobAggregatorProgressToGlobal
public boolean logJobAggregatorProgressToGlobal()
Returns whether the seqexec job aggregator should write to a global log or not. This comes into play only if "pegasus.clusterer.job.aggregator.seqexec.log" is set to true. Referred to by the "pegasus.clusterer.job.aggregator.seqexec.log.global" property.- Returns:
- the value specified in the properties file, else true
-
abortOnFirstJobFailure
public boolean abortOnFirstJobFailure()
Returns a boolean indicating whether seqexec trips on the first job failure. Referred to by the "pegasus.cluster.job.aggregator.seqexec.firstjobfail" property.- Returns:
- the value specified in the properties file, else true
-
getRootWorkflowUUID
public java.lang.String getRootWorkflowUUID()
Returns the root workflow UUID if defined in the properties, else null Referred to by the "pegasus.workflow.root.uuid" property.- Returns:
- the value in the properties file else, null
-
getPartitionerDAXCallback
public java.lang.String getPartitionerDAXCallback()
Returns the DAXCallback that is to be used while parsing the DAX. Referred to by the "pegasus.partitioner.parser.dax.callback" property.- Returns:
- the value specified in the properties file, else DEFAULT_DAX_CALLBACK
- See Also:
DEFAULT_DAX_CALLBACK
-
getPartitionerLabelKey
public java.lang.String getPartitionerLabelKey()
Returns the key that is to be used as a label key, for labelled partitioning. Referred to by the "pegasus.partitioner.label.key" property.- Returns:
- the value specified in the properties file.
-
getHorizontalPartitionerBundleValue
public java.lang.String getHorizontalPartitionerBundleValue(java.lang.String name)
Returns the bundle value for a particular transformation. Referred to by the "pegasus.partitioner.horziontal.bundle.[txname]" property, where [txname] is replaced by the name passed an input to this function.- Parameters:
name- the logical name of the transformation.- Returns:
- the path to the postscript if specified in properties file, else null.
-
getHorizontalPartitionerCollapseValue
public java.lang.String getHorizontalPartitionerCollapseValue(java.lang.String name)
Returns the collapse value for a particular transformation. Referred to by the "pegasus.partitioner.horziontal.collapse.[txname]" property, where [txname] is replaced by the name passed an input to this function.- Parameters:
name- the logical name of the transformation.- Returns:
- the path to the postscript if specified in properties file, else null.
-
getClustererLabelKey
public java.lang.String getClustererLabelKey()
Returns the key that is to be used as a label key, for labelled clustering. Referred to by the "pegasus.clusterer.label.key" property.- Returns:
- the value specified in the properties file.
-
getPropertiesInSubmitDirectory
public java.lang.String getPropertiesInSubmitDirectory()
Returns the path to the property file that has been writting out in the submit directory.- Returns:
- path to the property file
- Throws:
java.lang.RuntimeException- in case of file not being generated.
-
writeOutProperties
public java.lang.String writeOutProperties(java.lang.String directory) throws java.io.IOExceptionWrites out the properties to a temporary file in the directory passed.- Parameters:
directory- the directory in which the properties file needs to be written to.- Returns:
- the absolute path to the properties file written in the directory.
- Throws:
java.io.IOException- in case of error while writing out file.
-
writeOutProperties
public java.lang.String writeOutProperties(java.lang.String directory, boolean sanitizePath) throws java.io.IOExceptionWrites out the properties to a temporary file in the directory passed.- Parameters:
directory- the directory in which the properties file needs to be written to.sanitizePath- boolean indicating whether to sanitize paths for certain properties or not.- Returns:
- the absolute path to the properties file written in the directory.
- Throws:
java.io.IOException- in case of error while writing out file.
-
writeOutProperties
public java.lang.String writeOutProperties(java.lang.String directory, boolean sanitizePath, boolean setInternalVariable) throws java.io.IOExceptionWrites out the properties to a temporary file in the directory passed.- Parameters:
directory- the directory in which the properties file needs to be written to.sanitizePath- boolean indicating whether to sanitize paths for certain properties or not.setInternalVariable- whether to set the internal variable that stores the path to the properties file.- Returns:
- the absolute path to the properties file written in the directory.
- Throws:
java.io.IOException- in case of error while writing out file.
-
sanitizePathForProperty
private void sanitizePathForProperty(java.util.Properties properties, java.lang.String key)Santizes the value in the properties . Ensures that the path is absolute.- Parameters:
properties- the propertieskey- the key whose value needs to be sanitized
-
sanityCheck
protected static void sanityCheck(java.io.File dir) throws java.io.IOExceptionChecks the destination location for existence, if it can be created, if it is writable etc.- Parameters:
dir- is the new base directory to optionally create.- Throws:
java.io.IOException- in case of error while writing out files.
-
getProperty
private java.lang.String getProperty(java.lang.String newProperty, java.lang.String deprecatedProperty)This function is used to check whether a deprecated property is used or not. If a deprecated property is used,it logs a warning message specifying the new property. If both properties are not set by the user, the function returns the default property. If no default property then null.- Parameters:
newProperty- the new property that should be used.deprecatedProperty- the deprecated property that needs to be replaced.- Returns:
- the appropriate value.
-
getProperty
private java.lang.String getProperty(java.lang.String newProperty, java.lang.String deprecatedProperty, java.lang.String defaultValue)This function is used to check whether a deprecated property is used or not. If a deprecated property is used,it logs a warning message specifying the new property. If both properties are not set by the user, the function returns the default property. If no default property then null.- Parameters:
newProperty- the new property that should be used.deprecatedProperty- the deprecated property that needs to be replaced.defaultValue- the default value that should be returned.- Returns:
- the appropriate value.
-
logDeprecatedWarning
private void logDeprecatedWarning(java.lang.String deprecatedProperty, java.lang.String newProperty)Logs a warning about the deprecated property. Logs a warning only if it has not been displayed before.- Parameters:
deprecatedProperty- the deprecated property that needs to be replaced.newProperty- the new property that should be used.
-
-