Package edu.isi.pegasus.planner.code.generator

Class Stampede

  • All Implemented Interfaces:
    CodeGenerator
    public class Stampede
extends java.lang.Object
implements CodeGenerator
    A Stampede Events Code Generator that generates events in netlogger format for the exectuable workflow. This generators generates the events about 
       the tasks int he abstract workflow
   the edges in the abstract workflow
   jobs in the executable workflow
   the edges in the executable workflow
   relationship about how the tasks in the abstract workflow map to jobs in the
   executable workflow.
    Version:
    $Revision$
    Author:
    Karan Vahi

    • Constructor Summary

      Constructors 
      Constructor Description
      Stampede()  

    • Method Summary

      All Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      java.lang.String booleanToInt​(boolean value)
      Returns boolean as an integer
      java.util.Collection<java.io.File> generateCode​(ADag dag)
      Generates the code for the executable workflow in terms of a braindump file that contains workflow metadata useful for monitoring daemons etc.
      void generateCode​(ADag dag, Job job)
      Method not implemented.
      protected void generateEventsForDAXTask​(java.io.PrintWriter writer, ADag workflow, Job job)
      Generates stampede events corresponding to jobs/tasks in the DAX
      protected void generateEventsForExecutableJob​(java.io.PrintWriter writer, ADag dag, Job job)
      Generates stampede events corresponding to an executable job
      protected void generateTaskMapEvents​(java.io.PrintWriter writer, ADag dag, Job job)
      Generates the task.map events that link the jobs in the DAX with the jobs in the executable workflow
      private int getTaskCount​(Job job)
      Returns the task count for a job.
      void initialize​(PegasusBag bag)
      Initializes the Code Generator implementation.
      void reset()
      Resets the Code Generator implementation.
      boolean startMonitoring()
      Starts monitoring of the workflow by invoking a workflow monitor daemon.

      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

    • Field Detail

      • NETLOGGER_BP_FILE_SUFFIX

        public static final java.lang.String NETLOGGER_BP_FILE_SUFFIX
        The suffix to use while constructing the name of the metrics file
        See Also:
        Constant Field Values

      • NETLOGGER_LOG_FORMATTER_IMPLEMENTOR

        public static final java.lang.String NETLOGGER_LOG_FORMATTER_IMPLEMENTOR
        See Also:
        Constant Field Values

      • WORKFLOW_ID_KEY

        public static final java.lang.String WORKFLOW_ID_KEY
        The attribute key for workflow id.
        See Also:
        Constant Field Values

      • TASK_EVENT_NAME

        public static final java.lang.String TASK_EVENT_NAME
        The event name for task info
        See Also:
        Constant Field Values

      • TASK_ID_KEY

        public static final java.lang.String TASK_ID_KEY
        The attribute key for task id
        See Also:
        Constant Field Values

      • TYPE_KEY

        public static final java.lang.String TYPE_KEY
        The attribute key for task type
        See Also:
        Constant Field Values

      • TYPE_DESCRIPTION_KEY

        public static final java.lang.String TYPE_DESCRIPTION_KEY
        The attribute key for type description
        See Also:
        Constant Field Values

      • TASK_TRANSFORMATION_KEY

        public static final java.lang.String TASK_TRANSFORMATION_KEY
        The attribute key for transformation
        See Also:
        Constant Field Values

      • ARGUMENTS_KEY

        public static final java.lang.String ARGUMENTS_KEY
        The attribute key for task arguments.
        See Also:
        Constant Field Values

      • TASK_EDGE_EVENT_NAME

        public static final java.lang.String TASK_EDGE_EVENT_NAME
        The event name for task edge
        See Also:
        Constant Field Values

      • PARENT_TASK_ID_KEY

        public static final java.lang.String PARENT_TASK_ID_KEY
        The atrribute key for parent task id.
        See Also:
        Constant Field Values

      • CHILD_TASK_ID_KEY

        public static final java.lang.String CHILD_TASK_ID_KEY
        The atrribute key for child task id.
        See Also:
        Constant Field Values

      • JOB_EVENT_NAME

        public static final java.lang.String JOB_EVENT_NAME
        The event name for a job
        See Also:
        Constant Field Values

      • JOB_ID_KEY

        public static final java.lang.String JOB_ID_KEY
        The attribute key for job id
        See Also:
        Constant Field Values

      • JOB_SUBMIT_FILE_KEY

        public static final java.lang.String JOB_SUBMIT_FILE_KEY
        Teh attribute key for the submit file
        See Also:
        Constant Field Values

      • JOB_CLUSTERED_KEY

        public static final java.lang.String JOB_CLUSTERED_KEY
        The attribute key for whether a job is clustered or not
        See Also:
        Constant Field Values

      • JOB_MAX_RETRIES_KEY

        public static final java.lang.String JOB_MAX_RETRIES_KEY
        The attribute key for how many times a job is retried
        See Also:
        Constant Field Values

      • JOB_TASK_COUNT_KEY

        public static final java.lang.String JOB_TASK_COUNT_KEY
        The attribute key for the number of tasks in the job
        See Also:
        Constant Field Values

      • JOB_EXECUTABLE_KEY

        public static final java.lang.String JOB_EXECUTABLE_KEY
        The attribute key for the executable
        See Also:
        Constant Field Values

      • JOB_EDGE_EVENT_NAME

        public static final java.lang.String JOB_EDGE_EVENT_NAME
        The event name for job edge
        See Also:
        Constant Field Values

      • PARENT_JOB_ID_KEY

        public static final java.lang.String PARENT_JOB_ID_KEY
        The atrribute key for parent job id.
        See Also:
        Constant Field Values

      • CHILD_JOB_ID_KEY

        public static final java.lang.String CHILD_JOB_ID_KEY
        The atrribute key for child job id.
        See Also:
        Constant Field Values

      • TASK_MAP_EVENT_NAME

        public static final java.lang.String TASK_MAP_EVENT_NAME
        The event name for task map event
        See Also:
        Constant Field Values

      • mLogFormatter

        private LogFormatter mLogFormatter
        The handle to the netlogger log formatter.

      • mBag

        protected PegasusBag mBag
        The bag of initialization objects.

      • mSubmitFileDir

        protected java.lang.String mSubmitFileDir
        The directory where all the submit files are to be generated.

      • mProps

        protected PegasusProperties mProps
        The object holding all the properties pertaining to Pegasus.

      • mPOptions

        protected PlannerOptions mPOptions
        The object containing the command line options specified to the planner at runtime.

      • mLogger

        protected LogManager mLogger
        The handle to the logging object.

    • Constructor Detail

      • Stampede

        public Stampede()

    • Method Detail

      • generateCode

        public java.util.Collection<java.io.File> generateCode​(ADag dag)
                                                throws CodeGeneratorException
        Generates the code for the executable workflow in terms of a braindump file that contains workflow metadata useful for monitoring daemons etc.
        Specified by:
        generateCode in interface CodeGenerator
        Parameters:
        dag - the concrete workflow.
        Returns:
        the Collection of File objects for the files written out.
        Throws:
        CodeGeneratorException - in case of any error occuring code generation.

      • generateEventsForDAXTask

        protected void generateEventsForDAXTask​(java.io.PrintWriter writer,
                                        ADag workflow,
                                        Job job)
                                 throws CodeGeneratorException
        Generates stampede events corresponding to jobs/tasks in the DAX
        Parameters:
        writer - the writer stream to write the events too
        workflow - the workflow.
        job - the job for which to generate the events.
        Throws:
        CodeGeneratorException

      • generateEventsForExecutableJob

        protected void generateEventsForExecutableJob​(java.io.PrintWriter writer,
                                              ADag dag,
                                              Job job)
                                       throws CodeGeneratorException
        Generates stampede events corresponding to an executable job
        Parameters:
        writer - the writer stream to write the events too
        workflow - the workflow.
        job - the job for which to generate the events.
        Throws:
        CodeGeneratorException

      • generateTaskMapEvents

        protected void generateTaskMapEvents​(java.io.PrintWriter writer,
                                     ADag dag,
                                     Job job)
        Generates the task.map events that link the jobs in the DAX with the jobs in the executable workflow
        Parameters:
        writer - the writer stream to write the events too
        workflow - the workflow.
        job - the job for which to generate the events.

      • getTaskCount

        private int getTaskCount​(Job job)
        Returns the task count for a job. The task count is the number of tasks/jobs in the DAX that map to this job. jobs inserted by Pegasus, which do not have a mapped task from the DAX, will have its task_count set to 0.
        Parameters:
        job - the executable job.
        Returns:
        task count

      • booleanToInt

        public java.lang.String booleanToInt​(boolean value)
        Returns boolean as an integer
        Parameters:
        value - the boolean value
        Returns:
        0 for false and 1 for true

      • startMonitoring

        public boolean startMonitoring()
        Description copied from interface: CodeGenerator
        Starts monitoring of the workflow by invoking a workflow monitor daemon. The monitoring should start only after the output files have been generated. FIXME: It should actually happen after the workflow has been submitted. Eventually should be a separate monitor interface, and submit writers should be loaded by an AbstractFactory.
        Specified by:
        startMonitoring in interface CodeGenerator
        Returns:
        boolean indicating whether could successfully start the monitor daemon or not.