XrdPfc::IOFile Class Reference

Downloads original file into a single file on local disk. Handles read requests as they come along. More...

#include <XrdPfcIOFile.hh>

Inheritance diagram for XrdPfc::IOFile:

Collaboration diagram for XrdPfc::IOFile:

Public Member Functions
	IOFile (XrdOucCacheIO *io, Cache &cache)
	~IOFile ()
void	DetachFinalize () override
	Abstract virtual method of XrdPfc::IO Called to destruct the IO object after it is no longer used.
long long	FSize () override
int	Fstat (struct stat &sbuff) override
bool	HasFile () const
	Check if File was opened successfully.
bool	ioActive () override
	Abstract virtual method of XrdPfc::IO Called to check if destruction needs to be done in a separate task.
virtual int	pgRead (char *buff, long long offs, int rdlen, std::vector< uint32_t > &csvec, uint64_t opts=0, int *csfix=0)
void	pgRead (XrdOucCacheIOCB &iocb, char *buff, long long off, int size, std::vector< uint32_t > &csvec, uint64_t opts=0, int *csfix=0) override
int	Read (char *buff, long long off, int size) override
	Pass Read request to the corresponding File object.
void	Read (XrdOucCacheIOCB &iocb, char *buff, long long off, int size) override
int	ReadV (const XrdOucIOVec *readV, int n) override
	Pass ReadV request to the corresponding File object.
void	ReadV (XrdOucCacheIOCB &iocb, const XrdOucIOVec *readV, int n) override
void	Update (XrdOucCacheIO &iocp) override
Public Member Functions inherited from XrdPfc::IO
	IO (XrdOucCacheIO *io, Cache &cache)
virtual XrdOucCacheIO *	Base ()
	Original data source.
bool	Detach (XrdOucCacheIOCD &iocdP) final
XrdOucCacheIO *	GetInput ()
const char *	GetLocation ()
XrdSysTrace *	GetTrace ()
const char *	Path () override
	Original data source URL.
int	Sync () override
virtual void	Sync (XrdOucCacheIOCB &iocb)
int	Trunc (long long Offset) override
virtual void	Trunc (XrdOucCacheIOCB &iocb, long long offs)
int	Write (char *Buffer, long long Offset, int Length) override
virtual void	Write (XrdOucCacheIOCB &iocb, char *buff, long long offs, int wlen)
Public Member Functions inherited from XrdOucCacheIO
	XrdOucCacheIO ()
	Construct and Destructor.
virtual const char *	Location (bool refresh=false)
virtual int	pgWrite (char *buff, long long offs, int wrlen, std::vector< uint32_t > &csvec, uint64_t opts=0, int *csfix=0)
virtual void	pgWrite (XrdOucCacheIOCB &iocb, char *buff, long long offs, int wrlen, std::vector< uint32_t > &csvec, uint64_t opts=0, int *csfix=0)
virtual void	Preread (aprParms &Parms)
virtual void	Preread (long long offs, int rlen, int opts=0)
virtual int	WriteV (const XrdOucIOVec *writV, int wnum)
virtual void	WriteV (XrdOucCacheIOCB &iocb, const XrdOucIOVec *writV, int wnum)

Additional Inherited Members
Static Public Attributes inherited from XrdOucCacheIO
static const uint64_t	forceCS = 0x0000000000000001ULL
static const int	SingleUse = 0x0001
	Mark pages for single use.
Protected Member Functions inherited from XrdPfc::IO
std::string	GetFilename ()
const char *	GetPath ()
unsigned short	ObtainReadSid ()
const char *	RefreshLocation ()
bool	register_block_error (int res)
bool	register_incomplete_read ()
Protected Member Functions inherited from XrdOucCacheIO
virtual	~XrdOucCacheIO ()
Protected Attributes inherited from XrdPfc::IO
RAtomic_int	m_active_read_reqs
	number of active read requests
Cache &	m_cache
	reference to Cache object
std::map< int, int >	m_error_counts
int	m_incomplete_count {0}
const char *	m_traceID

Detailed Description

Downloads original file into a single file on local disk. Handles read requests as they come along.

Definition at line 38 of file XrdPfcIOFile.hh.

Constructor & Destructor Documentation

IOFile()

IOFile::IOFile	(	XrdOucCacheIO *	io,
		Cache &	cache )

Definition at line 36 of file XrdPfcIOFile.cc.

                                               :
   IO(io, cache),
   m_file(0)
{
   m_file = Cache::GetInstance().GetFile(GetFilename(), this);
}

References XrdPfc::IO::IO(), XrdOucCacheIO::XrdOucCacheIO(), XrdPfc::Cache::GetFile(), XrdPfc::IO::GetFilename(), and XrdPfc::Cache::GetInstance().

Referenced by pgRead(), Read(), and ReadV().

Here is the call graph for this function:

Here is the caller graph for this function:

~IOFile()

IOFile::~IOFile

(

)

Definition at line 44 of file XrdPfcIOFile.cc.

{
   // called from Detach() if no sync is needed or
   // from Cache's sync thread
   TRACEIO(Debug, "~IOFile() " << this);
}

References Debug, and TRACEIO.

Member Function Documentation

DetachFinalize()

void IOFile::DetachFinalize ( )

overridevirtual

Abstract virtual method of XrdPfc::IO Called to destruct the IO object after it is no longer used.

Implements XrdPfc::IO.

Definition at line 116 of file XrdPfcIOFile.cc.

{
   // Effectively a destructor.
 
   TRACE(Debug, "DetachFinalize() " << this);
 
   m_file->RequestSyncOfDetachStats();
   Cache::GetInstance().ReleaseFile(m_file, this);
 
   if (( ! m_error_counts.empty() || m_incomplete_count > 0) && XRD_TRACE What >= TRACE_Error) {
      char info[1024];
      int pos = 0, cap = 1024;
      bool truncated = false;
      for (auto [err, cnt] : m_error_counts) {
         int len = snprintf(&info[pos], cap, " ( %d : %d)", err, cnt);
         if (len >= cap) {
            truncated = true;
            break;
         }
         pos += len;
         cap -= len;
      }
      TRACE(Error, "DetachFinalize() " << this << " n_incomplete_reads=" << m_incomplete_count
            << ", block (error : count) report:" << info << (truncated ? " - truncated" : ""));
   }
 
   delete this;
}

References Debug, Error, XrdPfc::Cache::GetInstance(), XrdPfc::IO::m_error_counts, XrdPfc::IO::m_incomplete_count, XrdPfc::Cache::ReleaseFile(), TRACE, TRACE_Error, and XRD_TRACE.

Here is the call graph for this function:

FSize()

long long IOFile::FSize ( )

overridevirtual

Obtain size of the file.

Returns

Size of the file in bytes.

Implements XrdOucCacheIO.

Definition at line 66 of file XrdPfcIOFile.cc.

{
   return m_file->GetFileSize();
}

Fstat()

int IOFile::Fstat ( struct stat & sbuff )

overridevirtual

Perform an fstat() operation (defaults to passthrough).

Parameters

sbuff

reference to the stat buffer to be filled in. Only fields st_size, st_blocks, st_mtime (st_atime and st_ctime may be set to st_mtime), st_ino, and st_mode need to be set. All other fields are preset and should not be changed.

Returns

<0 - fstat failed, value is -errno. =0 - fstat succeeded, sbuff holds stat information. >0 - fstat could not be done, forward operation to next level.

Reimplemented from XrdOucCacheIO.

Definition at line 52 of file XrdPfcIOFile.cc.

{
   // This is only called during Cache::Attach / Cache::GetFile() for file creation.
   // Should really be separate name but one needs to change virtual interface
   // so initialStat() becomes virtual in the base-class.
   // Also, IOFileBlock should be ditched.
   if ( ! m_file) {
      return initialStat(sbuff);
   }
 
   return m_file->Fstat(sbuff);
}

References stat.

HasFile()

bool XrdPfc::IOFile::HasFile ( ) const

inline

Check if File was opened successfully.

Definition at line 48 of file XrdPfcIOFile.hh.

{ return m_file != 0; }

Referenced by XrdPfc::Cache::Attach().

Here is the caller graph for this function:

ioActive()

bool IOFile::ioActive ( )

overridevirtual

Abstract virtual method of XrdPfc::IO Called to check if destruction needs to be done in a separate task.

Implements XrdPfc::IO.

Definition at line 109 of file XrdPfcIOFile.cc.

{
   RefreshLocation();
   return m_file->ioActive(this);
}

References XrdPfc::IO::RefreshLocation().

Here is the call graph for this function:

pgRead() [1/2]

int XrdOucCacheIO::pgRead ( char * buff, long long offs, int rdlen, std::vector< uint32_t > & csvec, uint64_t opts = 0, int * csfix = 0 )

virtual

Reimplemented from XrdOucCacheIO.

Definition at line 190 of file XrdOucCache.cc.

{
   (void)csfix;
   int bytes;
 
// Read the data into the buffer
//
   bytes = Read(buff, offs, rdlen);
 
// Calculate checksums if so wanted
//
   if (bytes > 0 && (opts & forceCS))
       XrdOucPgrwUtils::csCalc((const char *)buff, (ssize_t)offs,
                               (size_t)bytes, csvec);
 
// All done
//
   return bytes;
}

pgRead() [2/2]

void IOFile::pgRead ( XrdOucCacheIOCB & iocb, char * buff, long long offs, int rdlen, std::vector< uint32_t > & csvec, uint64_t opts = 0, int * csfix = 0 )

overridevirtual

Read file pages and checksums using asynchronous I/O (default sync).

Parameters

iocb	reference to the callback object that receives the result. All results are returned via this object's Done() method. If the caller holds any locks they must be recursive locks as the callback may occur on the calling thread.
buff	pointer to buffer where the bytes are to be placed.
offs	The offset where the read is to start.
rdlen	The number of bytes to read.
csvec	A vector which will be filled with the corresponding CRC32C checksum for each page or page segment.
opts	Processing options: forceCS - always return checksums even when not available.
csfix	When not nil, returns the number of corrected checksum errs.

Reimplemented from XrdOucCacheIO.

Definition at line 198 of file XrdPfcIOFile.cc.

{
   struct ZHandler : ReadReqRH
   {  using ReadReqRH::ReadReqRH;
      IOFile *m_io = nullptr;
      std::function<void (int)> m_lambda {0};
 
      void Done(int result) override {
         if (m_lambda) m_lambda(result);
         m_io->ReadEnd(result, this);
      }
   };
 
   ++m_active_read_reqs;
 
   auto *rh = new ZHandler(ObtainReadSid(), &iocb);
   rh->m_io = this;
 
   TRACEIO(Dump, "pgRead() async " << this << " sid: " << Xrd::hex1 << rh->m_seq_id << " off: " << off << " size: " << size);
 
   if (opts & XrdOucCacheIO::forceCS)
      rh->m_lambda = [=, &csvec](int result) {
         if (result > 0)
            XrdOucPgrwUtils::csCalc((const char *)buff, (ssize_t)off, (size_t)result, csvec);
      };
 
   int retval = ReadBegin(buff, off, size, rh);
   if (retval != kReadPending)
   {
      rh->Done(retval);
   }
}

References IOFile(), XrdPfc::ReadReqRH::ReadReqRH(), XrdOucPgrwUtils::csCalc(), XrdOucCacheIO::forceCS, Xrd::hex1, XrdPfc::kReadPending, XrdPfc::IO::m_active_read_reqs, XrdPfc::IO::ObtainReadSid(), opts, and TRACEIO.

Here is the call graph for this function:

Read() [1/2]

int IOFile::Read ( char * buff, long long off, int size )

overridevirtual

Pass Read request to the corresponding File object.

Implements XrdOucCacheIO.

Definition at line 151 of file XrdPfcIOFile.cc.

{
   ++m_active_read_reqs;
 
   auto *rh = new ReadReqRHCond(ObtainReadSid(), nullptr);
 
   TRACEIO(Dump, "Read() sync " << this << " sid: " << Xrd::hex1 << rh->m_seq_id << " off: " << off << " size: " << size);
 
   rh->m_cond.Lock();
   int retval = ReadBegin(buff, off, size, rh);
   if (retval == kReadPending)
   {
      rh->m_cond.Wait();
      retval = rh->m_retval;
   }
   rh->m_cond.UnLock();
 
   return ReadEnd(retval, rh);
}

References Xrd::hex1, XrdPfc::kReadPending, XrdPfc::IO::m_active_read_reqs, XrdPfc::IO::ObtainReadSid(), and TRACEIO.

Here is the call graph for this function:

Read() [2/2]

void IOFile::Read ( XrdOucCacheIOCB & iocb, char * buff, long long offs, int rlen )

overridevirtual

Perform an asynchronous read (defaults to synchronous).

Parameters

iocb	reference to the callback object that receives the result. All results are returned via this object's Done() method. If the caller holds any locks they must be recursive locks as the callback may occur on the calling thread.
buff	pointer to the buffer to receive the results. The buffer must remain valid until the callback is invoked.
offs	the offset into the file.
rlen	the number of bytes to read.

Reimplemented from XrdOucCacheIO.

Definition at line 172 of file XrdPfcIOFile.cc.

{
   struct ZHandler : ReadReqRH
   {  using ReadReqRH::ReadReqRH;
      IOFile *m_io = nullptr;
 
      void Done(int result) override {
         m_io->ReadEnd(result, this);
      }
   };
 
   ++m_active_read_reqs;
 
   auto *rh = new ZHandler(ObtainReadSid(), &iocb);
   rh->m_io = this;
 
   TRACEIO(Dump, "Read() async " << this << " sid: " << Xrd::hex1 << rh->m_seq_id << " off: " << off << " size: " << size);
 
   int retval = ReadBegin(buff, off, size, rh);
   if (retval != kReadPending)
   {
      rh->Done(retval);
   }
}

References IOFile(), XrdPfc::ReadReqRH::ReadReqRH(), Xrd::hex1, XrdPfc::kReadPending, XrdPfc::IO::m_active_read_reqs, XrdPfc::IO::ObtainReadSid(), and TRACEIO.

Here is the call graph for this function:

ReadV() [1/2]

int IOFile::ReadV ( const XrdOucIOVec * readV, int n )

overridevirtual

Pass ReadV request to the corresponding File object.

Reimplemented from XrdOucCacheIO.

Definition at line 277 of file XrdPfcIOFile.cc.

{
   ++m_active_read_reqs;
 
   auto *rh = new ReadReqRHCond(ObtainReadSid(), nullptr);
 
   TRACEIO(Dump, "ReadV() sync " << this << " sid: " << Xrd::hex1 << rh->m_seq_id << " n_chunks: " <<  n);
 
   rh->m_cond.Lock();
   int retval = ReadVBegin(readV, n, rh);
   if (retval == kReadPending)
   {
      rh->m_cond.Wait();
      retval = rh->m_retval;
   }
   rh->m_cond.UnLock();
   return ReadVEnd(retval, rh);
}

References Xrd::hex1, XrdPfc::kReadPending, XrdPfc::IO::m_active_read_reqs, XrdPfc::ReadReqRH::m_seq_id, XrdPfc::IO::ObtainReadSid(), and TRACEIO.

Here is the call graph for this function:

ReadV() [2/2]

void IOFile::ReadV ( XrdOucCacheIOCB & iocb, const XrdOucIOVec * readV, int rnum )

overridevirtual

Perform an asynchronous vector read (defaults to synchronous).

Parameters

iocb	reference to the callback object that receives the result. All results are returned via this object's Done() method. If the caller holds any locks they must be recursive locks as the callback may occur on the calling thread.
readV	pointer to a vector of read requests.
rnum	the number of elements in the vector.

Reimplemented from XrdOucCacheIO.

Definition at line 297 of file XrdPfcIOFile.cc.

{
   struct ZHandler : ReadReqRH
   {  using ReadReqRH::ReadReqRH;
      IOFile *m_io = nullptr;
 
      void Done(int result) override { m_io-> ReadVEnd(result, this); }
   };
 
   ++m_active_read_reqs;
 
   auto *rh = new ZHandler(ObtainReadSid(), &iocb);
   rh->m_io = this;
 
   TRACEIO(Dump, "ReadV() async " << this << " sid: " << Xrd::hex1 << rh->m_seq_id << " n_chunks: " <<  n);
 
   int retval = ReadVBegin(readV, n, rh);
   if (retval != kReadPending)
   {
      rh->Done(retval);
   }
}

References IOFile(), XrdPfc::ReadReqRH::ReadReqRH(), XrdOucCacheIOCB::Done(), Xrd::hex1, XrdPfc::kReadPending, XrdPfc::IO::m_active_read_reqs, XrdPfc::ReadReqRH::m_seq_id, XrdPfc::IO::ObtainReadSid(), and TRACEIO.

Here is the call graph for this function:

Update()

void IOFile::Update ( XrdOucCacheIO & iocp )

overridevirtual

Update the originally passed XrdOucCacheIO object with the object passed. All future uses underlying XrdOucCacheIO object must now use this object. Update() is called when Prepare() indicated that the file should not be physically opened and a file method was invoked in the XrdOucCacheIO passed to Attach(). When this occurs, the file is actually opened and Update() called to replace the original XrdOucCacheIO object with one that uses the newly opened file.

Parameters

iocp	reference to the new XrdOucCacheIO object.

Reimplemented from XrdPfc::IO.

Definition at line 102 of file XrdPfcIOFile.cc.

{
   IO::Update(iocp);
   m_file->ioUpdated(this);
}

References XrdOucCacheIO::XrdOucCacheIO(), and XrdPfc::IO::Update().

Here is the call graph for this function:

---

The documentation for this class was generated from the following files:

• XrdPfcIOFile.hh
• XrdPfcIOFile.cc