Package edu.isi.pegasus.planner.transfer.sls

Class Condor

java.lang.Object

edu.isi.pegasus.planner.transfer.sls.Condor

All Implemented Interfaces:

SLS

public class Condor
extends java.lang.Object
implements SLS

This uses the Condor File Transfer mechanism for the second level staging. It will work only if the Pegasus Style profile ( pegasus::style ) has a value of condor.

Version:

$Revision$

Author:

Karan Vahi

Field Summary

Fields

Modifier and Type	Field	Description
static java.lang.String	DESCRIPTION	A short description of the transfer implementation.
private boolean	mBypassStagingForInputs	A boolean indicating whether to bypass first level staging for inputs
protected LogManager	mLogger	The handle to the logging manager.
private PlannerCache	mPlannerCache	A SimpleFile Replica Catalog, that tracks all the files that are being materialized as part of workflow execution.
protected PlannerOptions	mPOptions	The handle to the planner options.
protected PegasusProperties	mProps	The handle to the properties.
protected SiteStore	mSiteStore	The handle to the site catalog.

Fields inherited from interface edu.isi.pegasus.planner.transfer.SLS

VERSION

Constructor Summary

Constructors

Constructor	Description
Condor()	The default constructor.

Method Summary

Modifier and Type	Method	Description
java.util.Collection<FileTransfer>	determineSLSInputTransfers(Job job, java.lang.String fileName, FileServer stagingSiteServer, java.lang.String stagingSiteDirectory, java.lang.String workerNodeDirectory)	Generates a second level staging file of the input files to the worker node directory.
java.util.Collection<FileTransfer>	determineSLSOutputTransfers(Job job, java.lang.String fileName, FileServer stagingSiteServer, java.lang.String stagingSiteDirectory, java.lang.String workerNodeDirectory)	Generates a second level staging file of the input files to the worker node directory.
boolean	doesCondorModifications()	Returns a boolean whether the SLS implementation does a condor based modification or not.
java.lang.String	getSLSInputLFN(Job job)	Returns the LFN of sls input file.
java.lang.String	getSLSOutputLFN(Job job)	Returns the LFN of sls output file.
void	initialize(PegasusBag bag)	Initializes the SLS implementation.
java.lang.String	invocationString(Job job, java.io.File slsFile)	Constructs a command line invocation for a job, with a given sls file.
boolean	modifyJobForFirstLevelStaging(Job job, java.lang.String submitDir, java.lang.String slsInputLFN, java.lang.String slsOutputLFN)	Modifies a job for the first level staging to headnode.This is to add any files that needs to be staged to the head node for a job specific to the SLS implementation.
boolean	modifyJobForWorkerNodeExecution(Job job, java.lang.String stagingSiteURLPrefix, java.lang.String stagingSiteDirectory, java.lang.String workerNodeDirectory)	Modifies a compute job for second level staging.
boolean	needsSLSInputTransfers(Job job)	Returns a boolean indicating whether it will an input file for a job to do the transfers.
boolean	needsSLSOutputTransfers(Job job)	Returns a boolean indicating whether it will an output file for a job to do the transfers.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

DESCRIPTION

public static final java.lang.String DESCRIPTION

A short description of the transfer implementation.

See Also:

Constant Field Values

mSiteStore

protected SiteStore mSiteStore

The handle to the site catalog.

mProps

protected PegasusProperties mProps

The handle to the properties.

mPOptions

protected PlannerOptions mPOptions

The handle to the planner options.

mLogger

protected LogManager mLogger

The handle to the logging manager.

mBypassStagingForInputs

private boolean mBypassStagingForInputs

A boolean indicating whether to bypass first level staging for inputs

mPlannerCache

private PlannerCache mPlannerCache

A SimpleFile Replica Catalog, that tracks all the files that are being materialized as part of workflow execution.

Constructor Detail

Condor

public Condor()

The default constructor.

Method Detail

initialize

public void initialize(PegasusBag bag)

Initializes the SLS implementation.

Specified by:

initialize in interface SLS

Parameters:

bag - the bag of objects. Contains access to catalogs etc.

doesCondorModifications

public boolean doesCondorModifications()

Returns a boolean whether the SLS implementation does a condor based modification or not. By condor based modification we mean whether it uses condor specific classads to achieve the second level staging or not.

Specified by:

doesCondorModifications in interface SLS

Returns:

false

invocationString

public java.lang.String invocationString(Job job,
                                         java.io.File slsFile)

Constructs a command line invocation for a job, with a given sls file. The SLS maybe null. In the case where SLS impl does not read from a file, it is advised to create a file in generateSLSXXX methods, and then read the file in this function and put it on the command line.

Specified by:

invocationString in interface SLS

Parameters:

job - the job that is being sls enabled

slsFile - the slsFile can be null

Returns:

invocation string

needsSLSInputTransfers

public boolean needsSLSInputTransfers(Job job)

Returns a boolean indicating whether it will an input file for a job to do the transfers. Transfer reads from stdin the file transfers that it needs to do. Always returns true, as we need to transfer the proxy always.

Specified by:

needsSLSInputTransfers in interface SLS

Parameters:

job - the job being detected.

Returns:

false

needsSLSOutputTransfers

public boolean needsSLSOutputTransfers(Job job)

Returns a boolean indicating whether it will an output file for a job to do the transfers. Transfer reads from stdin the file transfers that it needs to do.

Specified by:

needsSLSOutputTransfers in interface SLS

Parameters:

job - the job being detected.

Returns:

false

getSLSInputLFN

public java.lang.String getSLSInputLFN(Job job)

Returns the LFN of sls input file.

Specified by:

getSLSInputLFN in interface SLS

Parameters:

job - Job

Returns:

the name of the sls input file.

getSLSOutputLFN

public java.lang.String getSLSOutputLFN(Job job)

Returns the LFN of sls output file.

Specified by:

getSLSOutputLFN in interface SLS

Parameters:

job - Job

Returns:

the name of the sls input file.

determineSLSInputTransfers

public java.util.Collection<FileTransfer> determineSLSInputTransfers(Job job,
                                                                     java.lang.String fileName,
                                                                     FileServer stagingSiteServer,
                                                                     java.lang.String stagingSiteDirectory,
                                                                     java.lang.String workerNodeDirectory)

Generates a second level staging file of the input files to the worker node directory.

Specified by:

determineSLSInputTransfers in interface SLS

Parameters:

job - job for which the file is being created

fileName - name of the file that needs to be written out.

stagingSiteServer - the file server on the staging site to be used for retrieval of files i.e the get operation

stagingSiteDirectory - directory on the head node of the compute site.

workerNodeDirectory - worker node directory

Returns:

a Collection of FileTransfer objects listing the transfers that need to be done.

See Also:

needsSLSInputTransfers( Job)

determineSLSOutputTransfers

public java.util.Collection<FileTransfer> determineSLSOutputTransfers(Job job,
                                                                      java.lang.String fileName,
                                                                      FileServer stagingSiteServer,
                                                                      java.lang.String stagingSiteDirectory,
                                                                      java.lang.String workerNodeDirectory)

Generates a second level staging file of the input files to the worker node directory.

Specified by:

determineSLSOutputTransfers in interface SLS

Parameters:

job - the job for which the file is being created

fileName - the name of the file that needs to be written out.

stagingSiteServer - the file server on the staging site to be used for retrieval of files i.e the put operation

stagingSiteDirectory - the directory on the head node of the compute site.

workerNodeDirectory - the worker node directory

Returns:

a Collection of FileTransfer objects listing the transfers that need to be done.

See Also:

needsSLSOutputTransfers( Job)

modifyJobForFirstLevelStaging

public boolean modifyJobForFirstLevelStaging(Job job,
                                             java.lang.String submitDir,
                                             java.lang.String slsInputLFN,
                                             java.lang.String slsOutputLFN)

Modifies a job for the first level staging to headnode.This is to add any files that needs to be staged to the head node for a job specific to the SLS implementation. If any file needs to be added, a FileTransfer object should be created and added as an input or an output file.

Parameters:

job - the job

submitDir - the submit directory

slsInputLFN - the sls input file if required, that is used for staging in from the head node to worker node directory.

slsOutputLFN - the sls output file if required, that is used for staging in from the head node to worker node directory.

Returns:

boolean

modifyJobForWorkerNodeExecution

public boolean modifyJobForWorkerNodeExecution(Job job,
                                               java.lang.String stagingSiteURLPrefix,
                                               java.lang.String stagingSiteDirectory,
                                               java.lang.String workerNodeDirectory)

Modifies a compute job for second level staging. Adds the appropriate condor classads. It assumes that all the files are being moved to and from the submit directory directly. Ignores any headnode parameters passed.

Specified by:

modifyJobForWorkerNodeExecution in interface SLS

Parameters:

job - the job to be modified.

stagingSiteURLPrefix - the url prefix for the server on the staging site

stagingSitedirectory - the directory on the staging site, where the nput data is read from and the output data written out.

workerNodeDirectory - the directory in the worker node tmp

Returns:

boolean indicating whether job was successfully modified or not.