ndcctools.work_queue.PythonTask Class Reference

Python PythonTask object. More...

Inheritance diagram for ndcctools.work_queue.PythonTask:

Public Member Functions
	__init__ (self, func, *args, **kwargs)
	Creates a new python task.
	output (self)
	returns the result of a python task as a python variable
Public Member Functions inherited from ndcctools.work_queue.Task
	clone (self)
	Return a copy of this task.
	specify_command (self, command)
	Set the command to be executed by the task.
	specify_coprocess (self, coprocess)
	Set the coprocess at the worker that should execute the task's command.
	specify_algorithm (self, algorithm)
	Set the worker selection algorithm for task.
	specify_tag (self, tag)
	Attach a user defined logical name to the task.
	specify_category (self, name)
	Label the task with the given category.
	specify_feature (self, name)
	Label the task with the given user-defined feature.
	specify_file (self, local_name, remote_name=None, type=None, flags=None, cache=None, failure_only=None)
	Add a file to the task.
	specify_url (self, url, remote_name, type=None, flags=None, cache=None, failure_only=None)
	Add a url to the task which will be provided as an input file.
	specify_file_command (self, cmd, remote_name, type=None, flags=None, cache=None, failure_only=None)
	Add an input file produced by a Unix shell command.
	specify_file_piece (self, local_name, remote_name=None, start_byte=0, end_byte=0, type=None, flags=None, cache=None, failure_only=None)
	Add a file piece to the task.
	specify_input_file (self, local_name, remote_name=None, flags=None, cache=None)
	Add a input file to the task.
	specify_output_file (self, local_name, remote_name=None, flags=None, cache=None, failure_only=None)
	Add a output file to the task.
	specify_directory (self, local_name, remote_name=None, type=None, flags=None, recursive=False, cache=None, failure_only=None)
	Add a directory to the task.
	specify_buffer (self, buffer, remote_name, flags=None, cache=None)
	Add an input bufer to the task.
	specify_snapshot_file (self, filename)
	When monitoring, indicates a json-encoded file that instructs the monitor to take a snapshot of the task resources.
	specify_max_retries (self, max_retries)
	Indicate the number of times the task should be retried.
	specify_cores (self, cores)
	Indicate the number of cores required by this task.
	specify_memory (self, memory)
	Indicate the memory (in MB) required by this task.
	specify_disk (self, disk)
	Indicate the disk space (in MB) required by this task.
	specify_gpus (self, gpus)
	Indicate the number of GPUs required by this task.
	specify_priority (self, priority)
	Indicate the the priority of this task (larger means better priority, default is 0).
	specify_environment_variable (self, name, value=None)
	Set this environment variable before running the task.
	specify_monitor_output (self, directory)
	Set a name for the resource summary output directory from the monitor.
	tag (self)
	Get the user-defined logical name for the task.
	category (self)
	Get the category name for the task.
	command (self)
	Get the shell command executed by the task.
	priority (self)
	Get the priority of the task.
	algorithm (self)
	Get the algorithm for choosing worker to run the task.
	std_output (self)
	Get the standard output of the task.
	id (self)
	Get the task id number.
	return_status (self)
	Get the exit code of the command executed by the task.
	result (self)
	Get the result of the task as an integer code, such as successful, missing file, etc.
	result_str (self)
	Return a string that explains the result of a task.
	total_submissions (self)
	Get the number of times the task has been resubmitted internally.
	exhausted_attempts (self)
	Get the number of times the task has been failed given resource exhaustion.
	host (self)
	Get the address and port of the host on which the task ran.
	hostname (self)
	Get the name of the host on which the task ran.
	submit_time (self)
	Get the time at which this task was submitted.
	finish_time (self)
	Get the time at which this task was finished.
	total_cmd_exhausted_execute_time (self)
	Get the total time the task executed and failed given resource exhaustion.
	app_delay (self)
	Get the time spent in upper-level application (outside of work_queue_wait).
	send_input_start (self)
	Get the time at which the task started to transfer input files.
	send_input_finish (self)
	Get the time at which the task finished transferring input files.
	execute_cmd_start (self)
	The time at which the task began.
	execute_cmd_finish (self)
	Get the time at which the task finished (discovered by the manager).
	receive_output_start (self)
	Get the time at which the task started to transfer output files.
	receive_output_finish (self)
	Get the time at which the task finished transferring output files.
	total_bytes_received (self)
	Get the number of bytes received since task started receiving input data.
	total_bytes_sent (self)
	Get the number of bytes sent since task started sending input data.
	total_bytes_transferred (self)
	Get the number of bytes transferred since task started transferring input data.
	total_transfer_time (self)
	Get the time comsumed in microseconds for transferring total_bytes_transferred.
	cmd_execution_time (self)
	Time spent in microseconds for executing the command until completion on a single worker.
	total_cmd_execution_time (self)
	Accumulated time spent in microseconds for executing the command on any worker, regardless of whether the task finished (i.e., this includes time running on workers that disconnected).
	resources_measured (self)
	Get the resources measured for the task execution if resource monitoring is enabled.
	limits_exceeded (self)
	Get the resources the task exceeded.
	resources_requested (self)
	Get the resources the task requested to run.
	resources_allocated (self)
	Get the resources allocated to the task in its latest attempt.

Protected Member Functions
	_create_wrapper (self)
	creates the wrapper script which will execute the function.

Detailed Description

Python PythonTask object.

this class is used to create a python task

Constructor & Destructor Documentation

__init__()

ndcctools.work_queue.PythonTask.__init__	(		self,
			func,
		*	args,
		**	kwargs
	)

Creates a new python task.

Parameters

self	Reference to the current python task object
func	python function to be executed by task
args	arguments used in function to be executed by task
kwargs	keyword arguments used in function to be executed by task

Reimplemented from ndcctools.work_queue.Task.

Member Function Documentation

output()

ndcctools.work_queue.PythonTask.output

(

self

)

returns the result of a python task as a python variable

Parameters

self	reference to the current python task object

Reimplemented from ndcctools.work_queue.Task.

_create_wrapper()

ndcctools.work_queue.PythonTask._create_wrapper (   self )

protected

creates the wrapper script which will execute the function.

pickles output.

---

The documentation for this class was generated from the following file:

• work_queue.py