gevent.fileobject – Wrappers to make file-like objects cooperative

class FileObject

The main entry point to the file-like gevent-compatible behaviour. It will be defined to be the best available implementation.

There are two main implementations of FileObject. On all systems, there is FileObjectThread which uses the built-in native threadpool to avoid blocking the entire interpreter. On UNIX systems (those that support the fcntl module), there is also FileObjectPosix which uses native non-blocking semantics.

A third class, FileObjectBlock, is simply a wrapper that executes everything synchronously (and so is not gevent-compatible). It is provided for testing and debugging purposes.

Configuration

You may change the default value for FileObject using the GEVENT_FILE environment variable. Set it to posix, thread, or block to choose from FileObjectPosix, FileObjectThread and FileObjectBlock, respectively. You may also set it to the fully qualified class name of another object that implements the file interface to use one of your own objects.

Note

The environment variable must be set at the time this module is first imported.

Classes

class FileObjectPosix(fobj, mode='rb', bufsize=-1, close=True)

Bases: object

A file-like object that operates on non-blocking files but provides a synchronous, cooperative interface.

Caution

This object is most effective wrapping files that can be used appropriately with select.select() such as sockets and pipes.

In general, on most platforms, operations on regular files (e.g., open('/etc/hosts')) are considered non-blocking already, even though they can take some time to complete as data is copied to the kernel and flushed to disk (this time is relatively bounded compared to sockets or pipes, though). A read() or write() call on such a file will still effectively block for some small period of time. Therefore, wrapping this class around a regular file is unlikely to make IO gevent-friendly: reading or writing large amounts of data could still block the event loop.

If you’ll be working with regular files and doing IO in large chunks, you may consider using FileObjectThread or tp_read() and tp_write() to bypass this concern.

Note

Random read/write (e.g., mode='rwb') is not supported. For that, use io.BufferedRWPair around two instance of this class.

Tip

Although this object provides a fileno() method and so can itself be passed to fcntl.fcntl(), setting the os.O_NONBLOCK flag will have no effect; however, removing that flag will cause this object to no longer be cooperative.

Changed in version 1.1: Now uses the io package internally. Under Python 2, previously used the undocumented class socket._fileobject. This provides better file-like semantics (and portability to Python 3).

Parameters:

• fobj – Either an integer fileno, or an object supporting the usual socket.fileno() method. The file will be put in non-blocking mode using gevent.os.make_nonblocking().
• mode (str) – The manner of access to the file, one of “rb”, “rU” or “wb” (where the “b” or “U” can be omitted). If “U” is part of the mode, IO will be done on text, otherwise bytes.
• bufsize (int) – If given, the size of the buffer to use. The default value means to use a platform-specific default, and a value of 0 is translated to a value of 1. Other values are interpreted as for the io package. Buffering is ignored in text mode.

closed

True if the file is closed

readable()

New in version 1.1b2.

writable()

New in version 1.1b2.

class FileObjectThread(fobj, *args, **kwargs)

Bases: object

A file-like object wrapping another file-like object, performing all blocking operations on that object in a background thread.

Parameters:

• fobj – The underlying file-like object to wrap, or an integer fileno that will be pass to os.fdopen() along with everything in args.
• lock (bool) – If True (the default) then all operations will be performed one-by-one. Note that this does not guarantee that, if using this file object from multiple threads/greenlets, operations will be performed in any particular order, only that no two operations will be attempted at the same time. You can also pass your own gevent.lock.Semaphore to synchronize file operations with an external resource.
• close (bool) – If True (the default) then when this object is closed, the underlying object is closed as well.

close()

Changed in version 1.1b1: The file object is closed using the threadpool. Note that whether or not this action is synchronous or asynchronous is not documented.

FileObject

alias of FileObjectPosix

Next page: gevent.subprocess – Cooperative subprocess module