Package edu.isi.pegasus.planner.classes
Class ADag
- java.lang.Object
-
- edu.isi.pegasus.planner.classes.Data
-
- edu.isi.pegasus.planner.classes.ADag
-
- All Implemented Interfaces:
Graph,GraphNodeContent,java.lang.Cloneable
public class ADag extends Data implements Graph
This class object contains the info about a Dag. DagInfo object contains the information to create the .dax file. vJobSubInfos is a Vector containing Job objects of jobs making the Dag. Each subinfo object contains information needed to generate a submit file for that job.
-
-
Field Summary
Fields Modifier and Type Field Description private DagInfomDAGInfoThe DagInfo object which contains the information got from parsing the dax file.private GraphmGraphImplementorHandle to the Graph implementor.protected NotificationsmNotificationsAll the notifications associated with the jobprotected ReplicaStoremReplicaStoreHandle to the replica store that stores the replica catalog user specifies in the DAXprivate java.lang.StringmRequestIDThe optional request ID associated with the DAX.protected java.lang.StringmRootWorkflowUUIDThe Root Workflow UUID.private java.lang.StringmSubmitDirectoryThe root of the submit directory hierarchy for the DAG.protected TransformationStoremTransformationStoreHandle to the transformation store that stores the transformation catalog user specifies in the DAXprotected booleanmWorkflowRefinementStartedBoolean indicating whether the refinement process on the workflow has started or not.protected java.lang.StringmWorkflowUUIDThe UUID associated with the workflow.
-
Constructor Summary
Constructors Constructor Description ADag()Initialises the class member variables.
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description voidadd(Job job)This adds a new job to the ADAG object.voidaddEdge(GraphNode parent, GraphNode child)Adds an edge between two already existing nodes in the graph.voidaddEdge(java.lang.String parent, java.lang.String child)Adds an edge between two already existing nodes in the graph.voidaddEdges(java.lang.String child, java.util.List<java.lang.String> parents)A convenience method that allows for bulk addition of edges between already existing nodes in the graph.voidaddNewRelation(java.lang.String parent, java.lang.String child)Adds a new PCRelation pair to the Vector ofPCRelationpairs.voidaddNode(GraphNode node)Adds a node to the Graph.voidaddNotification(Invoke invoke)Adds a Invoke object corresponding to a notification.voidaddNotifications(Notifications invokes)Adds all the notifications passed to the underlying container.voidaddRoot(GraphNode root)Adds a single root node to the Graph.java.util.Iterator<GraphNode>bottomUpIterator()Returns an iterator that traverses the graph bottom up from the leaves.voidcheckForCorruption()Checks the underlying graph structure for any corruption.java.lang.Objectclone()Returns a new copy of the Object.private java.lang.Stringcomplain(java.lang.String desc, GraphNode node, GraphNode linkedNode)Convenience method to complain for a linked node from a node that does not exist in the DAGprivate voidedgeToDOT(java.io.Writer stream, java.lang.String indent, java.lang.String parent, java.lang.String child)Returns the DOT description of the object.voidgenerateFlowID()Generates the flow id for this current run.voidgenerateFlowName()Generates the name of the flow.protected java.lang.StringgenerateWorkflowUUID()Generates the UUID for the workflowjava.lang.StringgetAbstractWorkflowName()Returns the workflow idjava.lang.StringgetBaseSubmitDirectory()Returns the root of submit directory hierarchy for the workflow.java.lang.StringgetCount()Returns the count of the workflow, that was specified in the DAX.DagInfogetDAGInfo()Returns the DAGInfo that stores the metadata about the DAXjava.lang.StringgetDAXVersion()Returns the dax versionjava.lang.StringgetExecutableWorkflowName()Returns the workflow idjava.lang.StringgetFlowID()Returns the flow ID for the workflow.java.lang.StringgetFlowName()Returns the flow namejava.lang.StringgetFlowTimestamp()Returns the flow timestamp for the workflow.java.lang.StringgetIndex()Returns the index of the workflow, that was specified in the DAX.java.lang.StringgetLabel()Returns the label of the workflow, that was specified in the DAX.java.util.List<GraphNode>getLeaves()Returns the leaf nodes of the Graph.java.lang.StringgetMTime()Returns the last modified time for the file containing the workflow description.GraphNodegetNode(java.lang.String identifier)Returns the node matching the id passed.intgetNoOfJobs()Returns the number of jobs in the dag on the basis of number of elements in thedagJobsVector.NotificationsgetNotifications()Returns all the notifications associated with the job.java.util.Collection<Invoke>getNotifications(Invoke.WHEN when)Returns a collection of all the notifications that need to be done for a particular conditionjava.lang.StringgetReleaseVersion()Return the release versionReplicaStoregetReplicaStore()Returns the Replica Storejava.lang.StringgetRequestID()Returns the request id.java.util.List<GraphNode>getRoots()Returns the root nodes of the Graph.java.lang.StringgetRootWorkflowUUID()Returns the UUID for the Root workflowTransformationStoregetTransformationStore()Returns the Transformation StoreWorkflowMetricsgetWorkflowMetrics()Returns the metrics about the workflow.java.lang.StringgetWorkflowUUID()Returns the UUID for the workflowbooleanhasWorkflowRefinementStarted()Returns a boolean indicating whether the workflow refinement has started or notbooleanisEmpty()Returns a boolean if there are no nodes in the graph.java.util.Iterator<GraphNode>iterator()Returns an iterator that traverses through the graph using a graph traversal algorithm.java.util.Iterator<GraphNode>jobIterator()Returns an iterator for traversing through the jobs in the workflow.java.util.Iterator<GraphNode>nodeIterator()Returns an iterator for the nodes in the Graph.booleanremove(Job job)Removes a particular job from the workflow.booleanremove(java.lang.String identifier)Removes a node from the Graph.voidresetEdges()Resets all the dependencies in the Graph, while preserving the nodes.voidresetStores()Resets the replica and transformation stores;voidsetBaseSubmitDirectory(java.lang.String dir)Sets the submit directory for the workflow.voidsetCount(java.lang.String count)Set the count of the workflow, that was specified in the DAX.voidsetDAXMTime(java.io.File f)Sets the mtime (last modified time) for the DAX.voidsetDAXVersion(java.lang.String version)Sets the dax versionvoidsetFlowTimestamp(java.lang.String timestamp)Sets the flow timestamp for the workflow.voidsetIndex(java.lang.String index)Set the index of the workflow, that was specified in the DAX.voidsetLabel(java.lang.String label)Sets the label for the workflow.voidsetReleaseVersion()Grabs the release version from VDS.Properties file.voidsetReplicaStore(ReplicaStore store)Sets the Replica StorevoidsetRequestID(java.lang.String id)Sets the request id.voidsetRootWorkflowUUID(java.lang.String uuid)Sets the root UUID for the workflowvoidsetTransformationStore(TransformationStore store)Sets the Transformation StorevoidsetWorkflowRefinementStarted(boolean state)Sets whether the workflow refinement has started or notvoidsetWorkflowUUID(java.lang.String uuid)Sets the UUID for the workflowintsize()Returns the number of nodes in the graph.java.lang.StringtoDOT()Returns the DOT description of the object.voidtoDOT(java.io.Writer stream, java.lang.String indent)Returns the DOT description of the object.java.util.Iterator<GraphNode>topologicalSortIterator()Returns an iterator for the graph that traverses in topological sort order.java.lang.StringtoString()Returns the String description of the dag associated with this object.voidwriteDOTHeader(java.io.Writer stream, java.lang.String indent)Writes out the static DOT Header.-
Methods inherited from class edu.isi.pegasus.planner.classes.Data
setToString, vectorToString
-
-
-
-
Field Detail
-
mDAGInfo
private DagInfo mDAGInfo
The DagInfo object which contains the information got from parsing the dax file.
-
mSubmitDirectory
private java.lang.String mSubmitDirectory
The root of the submit directory hierarchy for the DAG. This is the directory where generally the DAG related files like the log files, .dag and dagman output files reside.
-
mRequestID
private java.lang.String mRequestID
The optional request ID associated with the DAX.
-
mReplicaStore
protected ReplicaStore mReplicaStore
Handle to the replica store that stores the replica catalog user specifies in the DAX
-
mTransformationStore
protected TransformationStore mTransformationStore
Handle to the transformation store that stores the transformation catalog user specifies in the DAX
-
mRootWorkflowUUID
protected java.lang.String mRootWorkflowUUID
The Root Workflow UUID.
-
mWorkflowUUID
protected java.lang.String mWorkflowUUID
The UUID associated with the workflow.
-
mWorkflowRefinementStarted
protected boolean mWorkflowRefinementStarted
Boolean indicating whether the refinement process on the workflow has started or not.
-
mNotifications
protected Notifications mNotifications
All the notifications associated with the job
-
mGraphImplementor
private Graph mGraphImplementor
Handle to the Graph implementor.
-
-
Method Detail
-
addNotification
public void addNotification(Invoke invoke)
Adds a Invoke object corresponding to a notification.- Parameters:
invoke- the invoke object containing the notification
-
addNotifications
public void addNotifications(Notifications invokes)
Adds all the notifications passed to the underlying container.- Parameters:
invokes- the notifications to be added
-
getNotifications
public java.util.Collection<Invoke> getNotifications(Invoke.WHEN when)
Returns a collection of all the notifications that need to be done for a particular condition- Parameters:
when- the condition- Returns:
-
getNotifications
public Notifications getNotifications()
Returns all the notifications associated with the job.- Returns:
- the notifications
-
resetStores
public void resetStores()
Resets the replica and transformation stores;
-
clone
public java.lang.Object clone()
Returns a new copy of the Object.- Overrides:
clonein classjava.lang.Object- Returns:
- the clone of the object.
-
getRootWorkflowUUID
public java.lang.String getRootWorkflowUUID()
Returns the UUID for the Root workflow- Returns:
- the UUID of the workflow
-
setRootWorkflowUUID
public void setRootWorkflowUUID(java.lang.String uuid)
Sets the root UUID for the workflow- Parameters:
uuid- the UUID of the workflow
-
getWorkflowUUID
public java.lang.String getWorkflowUUID()
Returns the UUID for the workflow- Returns:
- the UUID of the workflow
-
setWorkflowUUID
public void setWorkflowUUID(java.lang.String uuid)
Sets the UUID for the workflow- Parameters:
uuid- the UUID of the workflow
-
generateWorkflowUUID
protected java.lang.String generateWorkflowUUID()
Generates the UUID for the workflow- Returns:
- the UUID of the workflow
-
hasWorkflowRefinementStarted
public boolean hasWorkflowRefinementStarted()
Returns a boolean indicating whether the workflow refinement has started or not- Returns:
- boolean
-
setWorkflowRefinementStarted
public void setWorkflowRefinementStarted(boolean state)
Sets whether the workflow refinement has started or not- Parameters:
state- the boolean value
-
toString
public java.lang.String toString()
Returns the String description of the dag associated with this object.
-
add
public void add(Job job)
This adds a new job to the ADAG object. It ends up adding both the job name and the job description to the internal structure.- Parameters:
job- the new job that is to be added to the ADag.
-
remove
public boolean remove(Job job)
Removes a particular job from the workflow. It however does not delete the relations the edges that refer to the job.- Parameters:
job- theJobobject containing the job description.- Returns:
- boolean indicating whether the removal was successful or not.
-
getNoOfJobs
public int getNoOfJobs()
Returns the number of jobs in the dag on the basis of number of elements in thedagJobsVector.- Returns:
- the number of jobs.
-
setRequestID
public void setRequestID(java.lang.String id)
Sets the request id.- Parameters:
id- the request id.
-
getRequestID
public java.lang.String getRequestID()
Returns the request id.- Returns:
- the request id.
-
getAbstractWorkflowName
public java.lang.String getAbstractWorkflowName()
Returns the workflow id- Returns:
- the abstract workflow id
-
getExecutableWorkflowName
public java.lang.String getExecutableWorkflowName()
Returns the workflow id- Returns:
- the executable workflow id
-
addNewRelation
public void addNewRelation(java.lang.String parent, java.lang.String child)Adds a new PCRelation pair to the Vector ofPCRelationpairs. For the new relation the isDeleted parameter is set to false.- Parameters:
parent- The parent in the relation pairchild- The child in the relation pair
-
setBaseSubmitDirectory
public void setBaseSubmitDirectory(java.lang.String dir)
Sets the submit directory for the workflow.- Parameters:
dir- the submit directory.
-
getLabel
public java.lang.String getLabel()
Returns the label of the workflow, that was specified in the DAX.- Returns:
- the label of the workflow.
-
setLabel
public void setLabel(java.lang.String label)
Sets the label for the workflow.- Parameters:
label- the label to be assigned to the workflow
-
getIndex
public java.lang.String getIndex()
Returns the index of the workflow, that was specified in the DAX.- Returns:
- the index of the workflow.
-
setIndex
public void setIndex(java.lang.String index)
Set the index of the workflow, that was specified in the DAX.- Parameters:
index- the count
-
setCount
public void setCount(java.lang.String count)
Set the count of the workflow, that was specified in the DAX.- Parameters:
count- the count
-
getCount
public java.lang.String getCount()
Returns the count of the workflow, that was specified in the DAX.- Returns:
- the count
-
getDAXVersion
public java.lang.String getDAXVersion()
Returns the dax version- Returns:
- teh dax version.
-
getMTime
public java.lang.String getMTime()
Returns the last modified time for the file containing the workflow description.- Returns:
- the MTime
-
getBaseSubmitDirectory
public java.lang.String getBaseSubmitDirectory()
Returns the root of submit directory hierarchy for the workflow.- Returns:
- the directory.
-
checkForCorruption
public void checkForCorruption()
Checks the underlying graph structure for any corruption. Corruption can be where a parent or a child of a node refers to an object, that is not in underlying graph node list.- Throws:
java.lang.RuntimeException- in case of corruption.
-
complain
private java.lang.String complain(java.lang.String desc, GraphNode node, GraphNode linkedNode)Convenience method to complain for a linked node from a node that does not exist in the DAG- Parameters:
desc-node-linkedNode-
-
setReplicaStore
public void setReplicaStore(ReplicaStore store)
Sets the Replica Store- Parameters:
store- the Replica Store
-
getReplicaStore
public ReplicaStore getReplicaStore()
Returns the Replica Store- Returns:
- the Replica Store
-
setTransformationStore
public void setTransformationStore(TransformationStore store)
Sets the Transformation Store- Parameters:
store- the Transformation Store
-
getTransformationStore
public TransformationStore getTransformationStore()
Returns the Transformation Store- Returns:
- the Replica Store
-
getDAGInfo
public DagInfo getDAGInfo()
Returns the DAGInfo that stores the metadata about the DAX- Returns:
-
generateFlowID
public void generateFlowID()
Generates the flow id for this current run. It is made of the name of the dag and a timestamp. This is a simple concat of the mFlowTimestamp and the flowName. For it work correctly the function needs to be called after the flow name and timestamp have been generated.
-
getFlowID
public java.lang.String getFlowID()
Returns the flow ID for the workflow.- Returns:
-
generateFlowName
public void generateFlowName()
Generates the name of the flow. It is same as the mNameOfADag if specified in dax generated by Chimera.
-
getFlowName
public java.lang.String getFlowName()
Returns the flow name
-
setDAXVersion
public void setDAXVersion(java.lang.String version)
Sets the dax version- Parameters:
version- the version of the DAX
-
setDAXMTime
public void setDAXMTime(java.io.File f)
Sets the mtime (last modified time) for the DAX. It is the time, when the DAX file was last modified. If the DAX file does not exist or an IO error occurs, the MTime is set to OL i.e . The DAX mTime is always generated in an extended format. Generating not in extended format, leads to the XML parser tripping while parsing the invocation record generated by Kickstart.- Parameters:
f- the file descriptor to the DAX|PDAX file.
-
getReleaseVersion
public java.lang.String getReleaseVersion()
Return the release version
-
setReleaseVersion
public void setReleaseVersion()
Grabs the release version from VDS.Properties file.
-
getFlowTimestamp
public java.lang.String getFlowTimestamp()
Returns the flow timestamp for the workflow.- Returns:
- the flowtimestamp
-
setFlowTimestamp
public void setFlowTimestamp(java.lang.String timestamp)
Sets the flow timestamp for the workflow.- Parameters:
timestamp- the flowtimestamp
-
jobIterator
public java.util.Iterator<GraphNode> jobIterator()
Returns an iterator for traversing through the jobs in the workflow.- Returns:
- a bative java failsafe iterator to the underlying collection.
-
getWorkflowMetrics
public WorkflowMetrics getWorkflowMetrics()
Returns the metrics about the workflow.- Returns:
- the WorkflowMetrics
-
toDOT
public java.lang.String toDOT() throws java.io.IOExceptionReturns the DOT description of the object. This is used for visualizing the workflow.- Returns:
- String containing the Partition object in XML.
- Throws:
java.io.IOException- if something fishy happens to the stream.
-
toDOT
public void toDOT(java.io.Writer stream, java.lang.String indent) throws java.io.IOExceptionReturns the DOT description of the object. This is used for visualizing the workflow.- Parameters:
stream- is a stream opened and ready for writing. This can also be a StringWriter for efficient output.indent- is aStringof spaces used for pretty printing. The initial amount of spaces should be an empty string. The parameter is used internally for the recursive traversal.- Throws:
java.io.IOException- if something fishy happens to the stream.
-
edgeToDOT
private void edgeToDOT(java.io.Writer stream, java.lang.String indent, java.lang.String parent, java.lang.String child) throws java.io.IOExceptionReturns the DOT description of the object. This is used for visualizing the workflow.- Parameters:
stream- is a stream opened and ready for writing. This can also be a StringWriter for efficient output.indent- is aStringof spaces used for pretty printing. The initial amount of spaces should be an empty string. The parameter is used internally for the recursive traversal.parent- the parentchild- the child- Throws:
java.io.IOException- if something fishy happens to the stream.
-
writeDOTHeader
public void writeDOTHeader(java.io.Writer stream, java.lang.String indent) throws java.io.IOExceptionWrites out the static DOT Header.- Parameters:
stream- is a stream opened and ready for writing. This can also be a StringWriter for efficient output.indent- is aStringof spaces used for pretty printing. The initial amount of spaces should be an empty string. The parameter is used internally for the recursive traversal.- Throws:
java.io.IOException- if something fishy happens to the stream.
-
addNode
public void addNode(GraphNode node)
Adds a node to the Graph. It overwrites an already existing node with the same ID.
-
addEdge
public void addEdge(java.lang.String parent, java.lang.String child)Adds an edge between two already existing nodes in the graph.
-
addEdge
public void addEdge(GraphNode parent, GraphNode child)
Adds an edge between two already existing nodes in the graph.
-
addEdges
public void addEdges(java.lang.String child, java.util.List<java.lang.String> parents)A convenience method that allows for bulk addition of edges between already existing nodes in the graph.
-
getNode
public GraphNode getNode(java.lang.String identifier)
Returns the node matching the id passed.
-
addRoot
public void addRoot(GraphNode root)
Adds a single root node to the Graph. All the exisitng roots of the Graph become children of the root.
-
remove
public boolean remove(java.lang.String identifier)
Removes a node from the Graph.
-
resetEdges
public void resetEdges()
Resets all the dependencies in the Graph, while preserving the nodes. The resulting Graph is a graph of independent nodes.- Specified by:
resetEdgesin interfaceGraph
-
nodeIterator
public java.util.Iterator<GraphNode> nodeIterator()
Returns an iterator for the nodes in the Graph. These iterators are fail safe.- Specified by:
nodeIteratorin interfaceGraph- Returns:
- Iterator
-
iterator
public java.util.Iterator<GraphNode> iterator()
Returns an iterator that traverses through the graph using a graph traversal algorithm.
-
bottomUpIterator
public java.util.Iterator<GraphNode> bottomUpIterator()
Returns an iterator that traverses the graph bottom up from the leaves. At any one time, only one iterator can iterate through the graph.- Specified by:
bottomUpIteratorin interfaceGraph- Returns:
- Iterator through the nodes of the graph.
-
topologicalSortIterator
public java.util.Iterator<GraphNode> topologicalSortIterator()
Returns an iterator for the graph that traverses in topological sort order.- Specified by:
topologicalSortIteratorin interfaceGraph- Returns:
- Iterator through the nodes of the graph.
-
size
public int size()
Returns the number of nodes in the graph.
-
getRoots
public java.util.List<GraphNode> getRoots()
Returns the root nodes of the Graph.
-
getLeaves
public java.util.List<GraphNode> getLeaves()
Returns the leaf nodes of the Graph.
-
-