Package edu.isi.pegasus.planner.cluster.aggregator

Class Abstract

  • All Implemented Interfaces:
    JobAggregator
    Direct Known Subclasses:
    MPIExec, SeqExec
    public abstract class Abstract
extends java.lang.Object
implements JobAggregator
    An abstract implementation of the JobAggregator interface, which the other implementations can choose to extend.
    Version:
    $Revision$
    Author:
    Karan Vahi vahi@isi.edu

    • Field Detail

      • CLUSTERED_JOB_PREFIX

        public static final java.lang.String CLUSTERED_JOB_PREFIX
        The prefix that is assigned to the jobname of the collapsed jobs to get the jobname for the fat job.
        See Also:
        Constant Field Values

      • TRANSFORMATION_NAMESPACE

        public static final java.lang.String TRANSFORMATION_NAMESPACE
        The transformation namespace for the cluster jobs.
        See Also:
        Constant Field Values

      • TRANSFORMATION_VERSION

        public static final java.lang.String TRANSFORMATION_VERSION
        The version number for the derivations for cluster jobs

      • DERIVATION_NAMESPACE

        public static final java.lang.String DERIVATION_NAMESPACE
        The derivation namespace for the cluster jobs.
        See Also:
        Constant Field Values

      • DERIVATION_VERSION

        public static final java.lang.String DERIVATION_VERSION
        The version number for the derivations for cluster jobs.
        See Also:
        Constant Field Values

      • MONITORD_COMMENT_MARKER

        public static final java.lang.String MONITORD_COMMENT_MARKER
        The marker to designate a line in the input file reserved for monitord purposes.
        See Also:
        Constant Field Values

      • mDirectory

        protected java.lang.String mDirectory
        The directory, where the stdin file of the fat jobs are created. It should be the submit file directory that the user mentions at runtime.

      • mProps

        protected PegasusProperties mProps
        The object holding all the properties pertaining to Pegasus.

      • mLogger

        protected LogManager mLogger
        The handle to the LogManager that logs all the messages.

      • mSiteStore

        protected SiteStore mSiteStore
        Handle to the site catalog store

      • mClusteredADag

        protected ADag mClusteredADag
        The handle to the ADag object that contains the workflow being clustered.

      • mGridStartFactory

        protected GridStartFactory mGridStartFactory
        The handle to the GridStart Factory.

      • mBag

        protected PegasusBag mBag
        Bag of initialization objects.

    • Constructor Detail

      • Abstract

        public Abstract()
        The default constructor.

    • Method Detail

      • getCompleteTranformationName

        public static java.lang.String getCompleteTranformationName​(java.lang.String name)
        A convenience method to return the complete transformation name being used to construct jobs in this class.
        Parameters:
        name - the name of the transformation
        Returns:
        the complete transformation name

      • initialize

        public void initialize​(ADag dag,
                       PegasusBag bag)
        Initializes the JobAggregator impelementation
        Specified by:
        initialize in interface JobAggregator
        Parameters:
        dag - the workflow that is being clustered.
        bag - the bag of objects that is useful for initialization.

      • aggregatedJobArguments

        public abstract java.lang.String aggregatedJobArguments​(AggregatedJob job)
        Returns the arguments with which the AggregatedJob needs to be invoked with.
        Parameters:
        job - the AggregatedJob for which the arguments have to be constructed.
        Returns:
        argument string

      • constructAbstractAggregatedJob

        public AggregatedJob constructAbstractAggregatedJob​(java.util.List jobs,
                                                    java.lang.String name,
                                                    java.lang.String id)
        Constructs a new aggregated job that contains all the jobs passed to it. The new aggregated job, appears as a single job in the workflow and replaces the jobs it contains in the workflow.
        Specified by:
        constructAbstractAggregatedJob in interface JobAggregator
        Parameters:
        jobs - the list of Job objects that need to be collapsed. All the jobs being collapsed should be scheduled at the same pool, to maintain correct semantics.
        name - the logical name of the jobs in the list passed to this function.
        id - the id that is given to the new job.
        Returns:
        the Job object corresponding to the aggregated job containing the jobs passed as List in the input, null if the list of jobs is empty

      • constructAbstractAggregatedJob

        public AggregatedJob constructAbstractAggregatedJob​(java.util.List jobs,
                                                    java.lang.String name,
                                                    java.lang.String id,
                                                    java.lang.String mergeLFN)
        Constructs an abstract aggregated job that has a handle to the appropriate JobAggregator that will be used to aggregate the jobs.
        Parameters:
        jobs - the list of SubInfo objects that need to be collapsed. All the jobs being collapsed should be scheduled at the same pool, to maintain correct semantics.
        name - the logical name of the jobs in the list passed to this function.
        id - the id that is given to the new job.
        mergeLFN - the logical name for the aggregated job that has to be constructed.
        Returns:
        the SubInfo object corresponding to the aggregated job containing the jobs passed as List in the input, null if the list of jobs is empty

      • makeAbstractAggregatedJobConcrete

        public void makeAbstractAggregatedJobConcrete​(AggregatedJob job)
        Enables the abstract clustered job for execution and converts it to it's executable form
        Specified by:
        makeAbstractAggregatedJobConcrete in interface JobAggregator
        Parameters:
        job - the abstract clustered job

      • getCommentString

        protected java.lang.String getCommentString​(Job job,
                                            int taskid)
        Generates the comment string for the job . It generates a comment of the format # task_id transformation derivation.
        Parameters:
        job - the job for which
        taskid - the task id to put in.
        Returns:
        the comment invocation

      • getTCEntry

        protected TransformationCatalogEntry getTCEntry​(Job job)
        Helper method to get an entry from the transformation catalog for an installed executable. It does the traversal from the list of entries to return a single TransformationCatalogEntry object, and dies with an appropriate error message if the object is not found. The pool and the name are retrieved from job object.
        Parameters:
        job - the job whose corresponding TransformationCatalogEntry you want.
        Returns:
        the TransformationCatalogEntry corresponding to the entry in the TC.

      • defaultTCEntry

        private TransformationCatalogEntry defaultTCEntry​(java.lang.String name,
                                                  java.lang.String executableBasename,
                                                  java.lang.String site)
        Returns a default TC entry to be used in case entry is not found in the transformation catalog.
        Parameters:
        name - the logical name for the clustering transformation.
        executableBasename - the basename for the executable in the bin directory of a Pegasus installation
        site - the site for which the default entry is required.
        Returns:
        the default entry.

      • entryNotInTC

        protected boolean entryNotInTC​(java.lang.String namespace,
                               java.lang.String name,
                               java.lang.String version,
                               java.lang.String executableBasename,
                               java.lang.String site)
        Determines whether there is NOT an entry in the transformation catalog for a particular transformation on a particular site.
        Parameters:
        namespace - the logical namespace of the transformation.
        name - the logical name of the transformation.
        version - the version of the transformation.
        executableBasename - basename of the executable that does the clustering.
        site - the site at which existence check is required.
        Returns:
        boolean true if an entry does not exists, false otherwise.

      • setDirectory

        protected void setDirectory​(java.lang.String directory)
        Sets the directory where the stdin files are to be generated.
        Parameters:
        directory - the path to the directory to which it needs to be set.

      • writeOutInputFileForJobAggregator

        protected java.io.File writeOutInputFileForJobAggregator​(AggregatedJob job)
        Writes out the input file for the aggregated job
        Parameters:
        job - the aggregated job
        Returns:
        path to the input file