Class File
- java.lang.Object
-
- edu.isi.pegasus.planner.catalog.transformation.impl.Abstract
-
- edu.isi.pegasus.planner.catalog.transformation.impl.File
-
- All Implemented Interfaces:
Catalog,TransformationCatalog
public class File extends Abstract implements TransformationCatalog
This is the new file based TC implementation storing the contents of the file in memory. For the old tc file implementation see OldTC.java- Version:
- $Revision$
- Author:
- Gaurang Mehta, Karan Vahi
-
-
Field Summary
Fields Modifier and Type Field Description private booleanmFlushOnCloseBoolean indicating whether to flush the contents back to the file on close.protected LogManagermLoggerThe LogManager object which is used to log all the messages.private booleanmodifyFileURLBoolean indicating whether to modify the file URL or notprivate PegasusPropertiesmPropsThe handle to the properties object.private java.lang.StringmTCFileThe path to the file based TC.private static FilemTCFileHandleThe singleton handler to the contents of the transformation catalog.private java.util.MapmTreeMapThe Tree Map which stores the contents of the file.protected java.util.ListmvExecPoolsThe List containing the user specified list of pools on which he wants the dag to run.-
Fields inherited from interface edu.isi.pegasus.planner.catalog.Catalog
DB_ALL_PREFIX
-
Fields inherited from interface edu.isi.pegasus.planner.catalog.TransformationCatalog
MODIFY_FOR_FILE_URLS_KEY, VERSION
-
-
Constructor Summary
Constructors Constructor Description File()
-
Method Summary
All Methods Static Methods Instance Methods Concrete Methods Deprecated Methods Modifier and Type Method Description intaddLFNProfile(java.lang.String namespace, java.lang.String name, java.lang.String version, java.util.List profiles)Add additional profile to a logical transformation .intaddPFNProfile(java.lang.String pfn, TCType type, java.lang.String resourcename, java.util.List profiles)Add additional profile to a physical transformation.protected booleanaddTCEntry(java.lang.String namespace, java.lang.String name, java.lang.String version, java.lang.String physicalname, TCType type, java.lang.String resourceid, java.util.List pfnprofiles, java.util.List lfnprofiles, SysInfo system, Notifications invokes, boolean write)Add an single entry into the transformation catalog.intclear()Deletes the entire transformation catalog.voidclose()Explicitely free resources before the garbage collection hits.booleanconnect(java.util.Properties props)Establishes a link between the implementation and the thing the implementation is build upon.intdeleteLFNProfiles(java.lang.String namespace, java.lang.String name, java.lang.String version, java.util.List profiles)Delete a list of profiles or all the profiles associated with a logical transformation.intdeletePFNProfiles(java.lang.String physicalname, TCType type, java.lang.String resourceid, java.util.List profiles)Delete a list of profiles or all the profiles associated with a pfn on a resource and of a type.voidFile()The default constructor.java.util.List<TransformationCatalogEntry>getContents()List all the contents of the TCjava.lang.StringgetDescription()Returns a textual description of the transformation mode.static TransformationCataloggetInstance()Deprecated.java.util.List<java.lang.String[]>getTCLogicalNames(java.lang.String resourceid, TCType type)Get the list of LogicalNames available on a particular resource.voidinitialize(PegasusBag bag)Initialize the implementation, and return an instance of the implementation.intinsert(TransformationCatalogEntry entry)Add single TCEntry to the Catalog.intinsert(TransformationCatalogEntry entry, boolean write)Add single TCEntry object temporarily to the in memory Catalog.intinsert(java.lang.String namespace, java.lang.String name, java.lang.String version, java.lang.String physicalname, TCType type, java.lang.String resourceid, java.util.List pfnprofiles, java.util.List lfnprofiles, SysInfo system)Add an single entry into the transformation catalog.intinsert(java.util.List<TransformationCatalogEntry> entries)Add multiple TCEntries to the Catalog.booleanisClosed()Predicate to check, if the connection with the catalog's implementation is still active.protected voidlogMessage(java.lang.String msg)Logs the message to a logging stream.java.util.List<TransformationCatalogEntry>lookup(java.lang.String namespace, java.lang.String name, java.lang.String version, java.lang.String resourceid, TCType type)Returns TC entries for a particular logical transformation and/or on a particular resource and/or of a particular type.java.util.List<TransformationCatalogEntry>lookup(java.lang.String namespace, java.lang.String name, java.lang.String version, java.util.List resourceids, TCType type)Returns TC entries for a particular logical transformation and/or on a number of resources and/or of a particular type.java.util.List<Profile>lookupLFNProfiles(java.lang.String namespace, java.lang.String name, java.lang.String version)Get the list of Profiles associated with a particular logical transformation.java.util.List<TransformationCatalogEntry>lookupNoProfiles(java.lang.String namespace, java.lang.String name, java.lang.String version, java.lang.String resourceid, TCType type)Get the list of PhysicalNames for a particular transformation on a site/sites for a particular type/types;java.util.List<Profile>lookupPFNProfiles(java.lang.String pfn, java.lang.String resourceid, TCType type)Get the list of Profiles associated with a particular physical transformation.java.util.List<java.lang.String>lookupSites(java.lang.String namespace, java.lang.String name, java.lang.String version, TCType type)Get the list of Resource ID's where a particular transformation may reside.private booleanpopulateTC()Populates the internal copy of the transformation catalog from the file containing the transformation catalog in the 6 column format.private booleanpopulateTC(java.io.InputStream reader)Populates the internal copy of the transformation catalog from a byte stream (input stream).private booleanpopulateTC(java.io.Reader reader)Adds multiple entries into the TC.intremoveByLFN(java.lang.String namespace, java.lang.String name, java.lang.String version, java.lang.String resourceid, TCType type)Delete all entries in the transformation catalog for a give logical tranformation and/or on a resource and/or of a particular typeintremoveByPFN(java.lang.String physicalname, java.lang.String namespace, java.lang.String name, java.lang.String version, java.lang.String resourceid, TCType type)Delete all entries in the transformation catalog for pair of logical and physical transformation.intremoveBySiteID(java.lang.String resourceid)Delete all entries on a particular resource from the transformation catalog.intremoveBySysInfo(SysInfo sysinfo)Deletes entries from the catalog which have a particular system information.intremoveByType(TCType type, java.lang.String resourceid)Delete a particular type of transformation, and/or on a particular resourceprivate voidwriteTC()-
Methods inherited from class edu.isi.pegasus.planner.catalog.transformation.impl.Abstract
modifyForFileURLS, modifyForFileURLS
-
-
-
-
Field Detail
-
mTCFileHandle
private static File mTCFileHandle
The singleton handler to the contents of the transformation catalog.
-
mLogger
protected LogManager mLogger
The LogManager object which is used to log all the messages. It's values are set in the CPlanner (the main toolkit) class.
-
mvExecPools
protected java.util.List mvExecPools
The List containing the user specified list of pools on which he wants the dag to run.
-
mTreeMap
private java.util.Map mTreeMap
The Tree Map which stores the contents of the file. The key is the transformationname.
-
mTCFile
private java.lang.String mTCFile
The path to the file based TC.
-
mProps
private PegasusProperties mProps
The handle to the properties object.
-
mFlushOnClose
private boolean mFlushOnClose
Boolean indicating whether to flush the contents back to the file on close.
-
modifyFileURL
private boolean modifyFileURL
Boolean indicating whether to modify the file URL or not
-
-
Method Detail
-
getInstance
public static TransformationCatalog getInstance()
Deprecated.Returns an instance of the File TC.- Returns:
- TransformationCatalog
-
File
public void File()
The default constructor.
-
initialize
public void initialize(PegasusBag bag)
Initialize the implementation, and return an instance of the implementation.- Specified by:
initializein interfaceTransformationCatalog- Parameters:
bag- the bag of Pegasus initialization objects.
-
getDescription
public java.lang.String getDescription()
Returns a textual description of the transformation mode.- Specified by:
getDescriptionin interfaceTransformationCatalog- Returns:
- String containing the description.
-
lookup
public java.util.List<TransformationCatalogEntry> lookup(java.lang.String namespace, java.lang.String name, java.lang.String version, java.util.List resourceids, TCType type) throws java.lang.Exception
Returns TC entries for a particular logical transformation and/or on a number of resources and/or of a particular type.- Specified by:
lookupin interfaceTransformationCatalog- Parameters:
namespace- String The namespace of the logical transformation.name- String the name of the logical transformation.version- String The version of the logical transformation.resourceids- List The List resourceid where the transformation is located. If NULL it returns all resources.type- TCType The type of the transformation to search for. If NULL it returns all types.- Returns:
- List Returns a list of TransformationCatalogEntry objects containing the corresponding entries from the TC. Returns null if no entry found.
- Throws:
java.lang.Exception- See Also:
TCType,edu.isi.pegasus.planner.catalog.TransformationCatalogEntry
-
lookup
public java.util.List<TransformationCatalogEntry> lookup(java.lang.String namespace, java.lang.String name, java.lang.String version, java.lang.String resourceid, TCType type) throws java.lang.Exception
Returns TC entries for a particular logical transformation and/or on a particular resource and/or of a particular type.- Specified by:
lookupin interfaceTransformationCatalog- Parameters:
namespace- String The namespace of the logical transformation.name- String the name of the logical transformation.version- String The version of the logical transformation.resourceid- String The resourceid where the transformation is located. If NULL it returns all resources.type- TCType The type of the transformation to search for. If NULL it returns all types.- Returns:
- List Returns a list of TransformationCatalogEntry objects containing the corresponding entries from the TC. Returns null if no entry found.
- Throws:
java.lang.Exception- See Also:
TCType,edu.isi.pegasus.planner.catalog.TransformationCatalogEntry
-
lookupSites
public java.util.List<java.lang.String> lookupSites(java.lang.String namespace, java.lang.String name, java.lang.String version, TCType type) throws java.lang.ExceptionGet the list of Resource ID's where a particular transformation may reside.- Specified by:
lookupSitesin interfaceTransformationCatalog- Parameters:
namespace- String The namespace of the transformation to search for.name- String The name of the transformation to search for.version- String The version of the transformation to search for.type- TCType The type of the transformation to search for.
(Enumerated type includes SOURCE, STATIC-BINARY, DYNAMIC-BINARY, PACMAN, INSTALLED, SCRIPT)
If NULL it returns all types.- Returns:
- List Returns a list of Resource Id's as strings. Returns NULL if no results found.
- Throws:
java.lang.Exception- NotImplementedException if not implemented- See Also:
TCType
-
lookupNoProfiles
public java.util.List<TransformationCatalogEntry> lookupNoProfiles(java.lang.String namespace, java.lang.String name, java.lang.String version, java.lang.String resourceid, TCType type) throws java.lang.Exception
Get the list of PhysicalNames for a particular transformation on a site/sites for a particular type/types;- Specified by:
lookupNoProfilesin interfaceTransformationCatalog- Parameters:
namespace- String The namespace of the transformation to search for.name- String The name of the transformation to search for.version- String The version of the transformation to search for.resourceid- String The id of the resource on which you want to search.
If NULL then returns entries on all resourcestype- TCType The type of the transformation to search for.
(Enumerated type includes source, binary, dynamic-binary, pacman, installed)
If NULL then returns entries of all types.- Returns:
- List Returns a List of
objects with the profiles not populated. - Throws:
java.lang.Exception- NotImplementedException if not implemented.- See Also:
TCType
-
getTCLogicalNames
public java.util.List<java.lang.String[]> getTCLogicalNames(java.lang.String resourceid, TCType type) throws java.lang.ExceptionGet the list of LogicalNames available on a particular resource.- Specified by:
getTCLogicalNamesin interfaceTransformationCatalog- Parameters:
resourceid- String The id of the resource on which you want to searchtype- TCType The type of the transformation to search for.
(Enumerated type includes source, binary, dynamic-binary, pacman, installed)
If NULL then return logical name for all types.- Returns:
- List Returns a list of String Arrays. Each array contains the resourceid, logical transformation in the format namespace::name:version and type. Returns NULL if no results found.
- Throws:
java.lang.Exception- NotImplementedException if not implemented.
-
lookupLFNProfiles
public java.util.List<Profile> lookupLFNProfiles(java.lang.String namespace, java.lang.String name, java.lang.String version) throws java.lang.Exception
Get the list of Profiles associated with a particular logical transformation.- Specified by:
lookupLFNProfilesin interfaceTransformationCatalog- Parameters:
namespace- String The namespace of the transformation to search for.name- String The name of the transformation to search for.version- String The version of the transformation to search for.- Returns:
- List Returns a list of Profile Objects containing profiles assocaited with the transformation. Returns NULL if no profiles found.
- Throws:
java.lang.Exception- NotImplementedException if not implemented.- See Also:
org.griphyn.cPlanner.classes.Profile
-
lookupPFNProfiles
public java.util.List<Profile> lookupPFNProfiles(java.lang.String pfn, java.lang.String resourceid, TCType type) throws java.lang.Exception
Get the list of Profiles associated with a particular physical transformation.- Specified by:
lookupPFNProfilesin interfaceTransformationCatalog- Parameters:
pfn- The physical file name to search the transformation by.resourceid- String The id of the resource on which you want to search.type- TCType The type of the transformation to search for.
(Enumerated type includes source, binary, dynamic-binary, pacman, installed)- Returns:
- List Returns a list of Profile Objects containing profiles assocaited with the transformation. Returns NULL if no profiless found.
- Throws:
java.lang.Exception- NotImplementedException if not implemented.- See Also:
org.griphyn.cPlanner.classes.Profile
-
getContents
public java.util.List<TransformationCatalogEntry> getContents() throws java.lang.Exception
List all the contents of the TC- Specified by:
getContentsin interfaceTransformationCatalog- Returns:
- List Returns a List of TransformationCatalogEntry objects.
- Throws:
java.lang.Exception
-
insert
public int insert(java.util.List<TransformationCatalogEntry> entries) throws java.lang.Exception
Add multiple TCEntries to the Catalog.- Specified by:
insertin interfaceTransformationCatalog- Parameters:
tcentry- List Takes a list of TransformationCatalogEntry objects as input- Returns:
- number of insertions On failure,throw an exception, don't use zero.
- Throws:
java.lang.Exception- See Also:
edu.isi.pegasus.planner.catalog.TransformationCatalogEntry
-
insert
public int insert(TransformationCatalogEntry entry) throws java.lang.Exception
Add single TCEntry to the Catalog.- Specified by:
insertin interfaceTransformationCatalog- Parameters:
tcentry- Takes a single TransformationCatalogEntry object as input- Returns:
- number of insertions, should always be 1. On failure, throw an exception, don't use zero.
- Throws:
java.lang.Exception- See Also:
edu.isi.pegasus.planner.catalog.TransformationCatalogEntry
-
insert
public int insert(TransformationCatalogEntry entry, boolean write) throws java.lang.Exception
Add single TCEntry object temporarily to the in memory Catalog. This is a hack to get around for adding soft state entries to the TC- Specified by:
insertin interfaceTransformationCatalog- Parameters:
tcentry- Takes a single TransformationCatalogEntry object as inputwrite- boolean enable write commits to backed catalog or not.- Returns:
- number of insertions, should always be 1. On failure, throw an exception, don't use zero.
- Throws:
java.lang.Exception- See Also:
edu.isi.pegasus.planner.catalog.TransformationCatalogEntry
-
insert
public int insert(java.lang.String namespace, java.lang.String name, java.lang.String version, java.lang.String physicalname, TCType type, java.lang.String resourceid, java.util.List pfnprofiles, java.util.List lfnprofiles, SysInfo system) throws java.lang.ExceptionAdd an single entry into the transformation catalog.- Specified by:
insertin interfaceTransformationCatalog- Parameters:
namespace- String The namespace of the transformation to be added (Can be null)name- String The name of the transformation to be added.version- String The version of the transformation to be added. (Can be null)physicalname- String The physical name/location of the transformation to be added.type- TCType The type of the physical transformation.resourceid- String The resource location id where the transformation is located.lfnprofiles- List The List of Profile objects associated with a Logical Transformation. (can be null)pfnprofiles- List The List of Profile objects associated with a Physical Transformation. (can be null)sysinfo- SysInfo The System information associated with a physical transformation.- Returns:
- number of insertions, should always be 1. On failure, throw an exception, don't use zero.
- Throws:
java.lang.Exception- See Also:
edu.isi.pegasus.planner.catalog.TransformationCatalogEntry,SysInfo,org.griphyn.cPlanner.classes.Profile
-
addTCEntry
protected boolean addTCEntry(java.lang.String namespace, java.lang.String name, java.lang.String version, java.lang.String physicalname, TCType type, java.lang.String resourceid, java.util.List pfnprofiles, java.util.List lfnprofiles, SysInfo system, Notifications invokes, boolean write) throws java.lang.ExceptionAdd an single entry into the transformation catalog.- Parameters:
namespace- the namespace of the transformation to be added (Can be null)name- the name of the transformation to be added.version- the version of the transformation to be added. (Can be null)physicalname- the physical name/location of the transformation to be added.type- the type of the physical transformation.resourceid- the resource location id where the transformation is located.lfnprofiles- the List ofProfileobjects associated with a Logical Transformation. (can be null)pfnprofiles- the list ofProfileobjects associated with a Physical Transformation. (can be null)system- the System information associated with a physical transformation.invokes- the Notifications associated with the transformation.write- boolean to commit changes to backend catalog- Returns:
- boolean true if succesfully added, returns false if error and throws exception.
- Throws:
java.lang.Exception- See Also:
org.griphyn.common.catalog.TransformationCatalogEntry,SysInfo,org.griphyn.cPlanner.classes.Profile
-
addLFNProfile
public int addLFNProfile(java.lang.String namespace, java.lang.String name, java.lang.String version, java.util.List profiles) throws java.lang.ExceptionAdd additional profile to a logical transformation .- Specified by:
addLFNProfilein interfaceTransformationCatalog- Parameters:
namespace- String The namespace of the transformation to be added. (can be null)name- String The name of the transformation to be added.version- String The version of the transformation to be added. (can be null)profiles- List The List of Profile objects that are to be added to the transformation.- Returns:
- number of insertions. On failure, throw an exception, don't use zero.
- Throws:
java.lang.Exception- See Also:
org.griphyn.cPlanner.classes.Profile
-
addPFNProfile
public int addPFNProfile(java.lang.String pfn, TCType type, java.lang.String resourcename, java.util.List profiles) throws java.lang.ExceptionAdd additional profile to a physical transformation.- Specified by:
addPFNProfilein interfaceTransformationCatalog- Parameters:
pfn- String The physical name of the transformationtype- TCType The type of transformation that the profile is associated with.resourcename- String The resource on which the physical transformation existsprofiles- The List of Profile objects that are to be added to the transformation.- Returns:
- number of insertions. On failure, throw an exception, don't use zero.
- Throws:
java.lang.Exception- See Also:
org.griphyn.cPlanner.classes.Profile
-
removeByLFN
public int removeByLFN(java.lang.String namespace, java.lang.String name, java.lang.String version, java.lang.String resourceid, TCType type) throws java.lang.ExceptionDelete all entries in the transformation catalog for a give logical tranformation and/or on a resource and/or of a particular type- Specified by:
removeByLFNin interfaceTransformationCatalog- Parameters:
namespace- String The namespace of the transformation to be deleted. (can be null)name- String The name of the transformation to be deleted.version- String The version of the transformation to be deleted. ( can be null)resourceid- String The resource id for which the transformation is to be deleted. If NULL then transformation on all resource are deletedtype- TCType The type of the transformation. If NULL then all types are deleted for the transformation.- Returns:
- the number of removed entries.
- Throws:
java.lang.Exception- See Also:
TCType
-
removeByPFN
public int removeByPFN(java.lang.String physicalname, java.lang.String namespace, java.lang.String name, java.lang.String version, java.lang.String resourceid, TCType type) throws java.lang.ExceptionDelete all entries in the transformation catalog for pair of logical and physical transformation.- Specified by:
removeByPFNin interfaceTransformationCatalog- Parameters:
physicalname- String The physical name of the transformationnamespace- String The namespace assocaited in the logical name of the transformation.name- String The name of the logical transformation.version- String The version number of the logical transformation.resourceid- String The resource on which the transformation is to be deleted. If NULL then it searches all the resource id.type- TCType The type of transformation. If NULL then it search and deletes entries for all types.- Returns:
- the number of removed entries.
- Throws:
java.lang.Exception- See Also:
TCType
-
removeByType
public int removeByType(TCType type, java.lang.String resourceid) throws java.lang.Exception
Delete a particular type of transformation, and/or on a particular resource- Specified by:
removeByTypein interfaceTransformationCatalog- Parameters:
type- TCType The type of the transformationresourceid- String The resource on which the transformation exists. If NULL then that type of transformation is deleted from all the resources.- Returns:
- the number of removed entries.
- Throws:
java.lang.Exception- See Also:
TCType
-
removeBySysInfo
public int removeBySysInfo(SysInfo sysinfo) throws java.lang.Exception
Deletes entries from the catalog which have a particular system information.- Specified by:
removeBySysInfoin interfaceTransformationCatalog- Parameters:
sysinfo- SysInfo The System Information by which you want to delete- Returns:
- the number of removed entries.
- Throws:
java.lang.Exception- See Also:
SysInfo
-
removeBySiteID
public int removeBySiteID(java.lang.String resourceid) throws java.lang.ExceptionDelete all entries on a particular resource from the transformation catalog.- Specified by:
removeBySiteIDin interfaceTransformationCatalog- Parameters:
resourceid- String The resource which you want to remove.- Returns:
- the number of removed entries.
- Throws:
java.lang.Exception
-
clear
public int clear() throws java.lang.ExceptionDeletes the entire transformation catalog. CLEAN............. USE WITH CAUTION.- Specified by:
clearin interfaceTransformationCatalog- Returns:
- the number of removed entries.
- Throws:
java.lang.Exception
-
deletePFNProfiles
public int deletePFNProfiles(java.lang.String physicalname, TCType type, java.lang.String resourceid, java.util.List profiles) throws java.lang.ExceptionDelete a list of profiles or all the profiles associated with a pfn on a resource and of a type.- Specified by:
deletePFNProfilesin interfaceTransformationCatalog- Parameters:
physicalname- String The physical name of the transformation.type- TCType The type of the transformation.resourceid- String The resource of the transformation.profiles- List The list of profiles to be deleted. If NULL then all profiles for that pfn+resource+type are deleted.- Returns:
- the number of removed entries.
- Throws:
java.lang.Exception- See Also:
org.griphyn.cPlanner.classes.Profile
-
deleteLFNProfiles
public int deleteLFNProfiles(java.lang.String namespace, java.lang.String name, java.lang.String version, java.util.List profiles) throws java.lang.ExceptionDelete a list of profiles or all the profiles associated with a logical transformation.- Specified by:
deleteLFNProfilesin interfaceTransformationCatalog- Parameters:
namespace- String The namespace of the logical transformation.name- String The name of the logical transformation.version- String The version of the logical transformation.profiles- List The List of profiles to be deleted. If NULL then all profiles for the logical transformation are deleted.- Returns:
- the number of removed entries.
- Throws:
java.lang.Exception- See Also:
org.griphyn.cPlanner.classes.Profile
-
connect
public boolean connect(java.util.Properties props)
Description copied from interface:CatalogEstablishes a link between the implementation and the thing the implementation is build upon.FIXME: The cause for failure is lost without exceptions.
-
isClosed
public boolean isClosed()
Description copied from interface:CatalogPredicate to check, if the connection with the catalog's implementation is still active. This helps determining, if it makes sense to callclose().- Specified by:
isClosedin interfaceCatalog- Returns:
- true, if the implementation is disassociated, false otherwise.
- See Also:
Catalog.close()
-
close
public void close()
Description copied from interface:CatalogExplicitely free resources before the garbage collection hits.
-
writeTC
private void writeTC()
-
populateTC
private boolean populateTC(java.io.InputStream reader)
Populates the internal copy of the transformation catalog from a byte stream (input stream). Used in webservices, when clients upload their files. It uses the default character encoding.- Parameters:
reader- theInputStreancontaining the bytes to be read.- Returns:
- boolean
-
populateTC
private boolean populateTC()
Populates the internal copy of the transformation catalog from the file containing the transformation catalog in the 6 column format.- Returns:
- boolean
-
populateTC
private boolean populateTC(java.io.Reader reader)
Adds multiple entries into the TC. Calls the above api multiple times.- Parameters:
reader- the input stream from where to read the contents of the transformation catalog.- Returns:
- boolean
-
logMessage
protected void logMessage(java.lang.String msg)
Logs the message to a logging stream. Currently does not log to any stream.- Parameters:
msg- the message to be logged.
-
-