<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html>
<head>
<title>[impl-test] </title>
<meta content="text/html;charset=ISO-8859-1" name="Content-Type"/>
<link href="style.css" media="screen" rel="stylesheet" type="text/css"/></head>
<body>
<div><a href="http://codespeak.net"><img alt="py lib" height="114" id="pyimg" src="http://codespeak.net/img/pylib.png" width="154"/></a></div>
<div id="metaspace">
<div class="project_title">[impl-test] </div>
<div id="menubar"><a class="menu" href="index.html">index</a> <a class="menu" href="../../apigen/api/index.html">api</a> <a class="menu" href="../../apigen/source/index.html">source</a> <a class="menu" href="contact.html">contact</a> <a class="menu" href="download.html">download</a></div></div>
<div id="contentspace">
<div id="docinfoline">
<div style="float: right; font-style: italic;"> </div></div>
<div class="document" id="implementation-and-customization-of-py-test">
<h1 class="title">Implementation and Customization of <tt class="docutils literal"><span class="pre">py.test</span></tt></h1>
<div class="contents topic">
<p class="topic-title first"><a id="contents" name="contents">Contents</a></p>
<ul class="auto-toc simple">
<li><a class="reference" href="#collecting-and-running-tests-implementation-remarks" id="id1" name="id1">1 Collecting and running tests / implementation remarks</a><ul class="auto-toc">
<li><a class="reference" href="#collectors-and-the-test-collection-process" id="id2" name="id2">1.1 Collectors and the test collection process</a></li>
<li><a class="reference" href="#test-items-are-collectors-as-well" id="id3" name="id3">1.2 test items are collectors as well</a></li>
<li><a class="reference" href="#constructing-the-package-name-for-modules" id="id4" name="id4">1.3 constructing the package name for modules</a></li>
<li><a class="reference" href="#module-collector" id="id5" name="id5">1.4 Module Collector</a></li>
</ul>
</li>
<li><a class="reference" href="#customizing-the-testing-process" id="id6" name="id6">2 Customizing the testing process</a><ul class="auto-toc">
<li><a class="reference" href="#writing-conftest-py-files" id="id7" name="id7">2.1 writing conftest.py files</a><ul class="auto-toc">
<li><a class="reference" href="#adding-custom-options" id="id8" name="id8">2.1.1 adding custom options</a></li>
</ul>
</li>
<li><a class="reference" href="#customizing-the-collecting-and-running-process" id="id9" name="id9">2.2 customizing the collecting and running process</a><ul class="auto-toc">
<li><a class="reference" href="#example-perform-additional-rest-checks" id="id10" name="id10">2.2.1 example: perform additional ReST checks</a></li>
</ul>
</li>
<li><a class="reference" href="#customizing-the-collection-process-in-a-module" id="id11" name="id11">2.3 Customizing the collection process in a module</a></li>
<li><a class="reference" href="#customizing-execution-of-functions" id="id12" name="id12">2.4 Customizing execution of Functions</a></li>
</ul>
</li>
</ul>
</div>
<div class="section">
<h1><a class="toc-backref" href="#id1" id="collecting-and-running-tests-implementation-remarks" name="collecting-and-running-tests-implementation-remarks"><span id="basicpicture"></span>1 Collecting and running tests / implementation remarks</a></h1>
<p>In order to customize <tt class="docutils literal"><span class="pre">py.test</span></tt> it's good to understand
its basic architure (WARNING: these are not guaranteed
yet to stay the way they are now!):</p>
<pre class="literal-block">
___________________
| |
| Collector |
|___________________|
/ \
| Item.run()
| ^
receive test Items /
| /execute test Item
| /
___________________/
| |
| Session |
|___________________|
.............................
. conftest.py configuration .
. cmdline options .
.............................
</pre>
<p>The <em>Session</em> basically receives test <em>Items</em> from a <em>Collector</em>,
and executes them via the <tt class="docutils literal"><span class="pre">Item.run()</span></tt> method. It monitors
the outcome of the test and reports about failures and successes.</p>
<div class="section">
<h2><a class="toc-backref" href="#id2" id="collectors-and-the-test-collection-process" name="collectors-and-the-test-collection-process"><span id="collection-process"></span>1.1 Collectors and the test collection process</a></h2>
<p>The collecting process is iterative, i.e. the session
traverses and generates a <em>collector tree</em>. Here is an example of such
a tree, generated with the command <tt class="docutils literal"><span class="pre">py.test</span> <span class="pre">--collectonly</span> <span class="pre">py/xmlobj</span></tt>:</p>
<pre class="literal-block">
<Directory 'xmlobj'>
<Directory 'testing'>
<Module 'test_html.py' (py.__.xmlobj.testing.test_html)>
<Function 'test_html_name_stickyness'>
<Function 'test_stylenames'>
<Function 'test_class_None'>
<Function 'test_alternating_style'>
<Module 'test_xml.py' (py.__.xmlobj.testing.test_xml)>
<Function 'test_tag_with_text'>
<Function 'test_class_identity'>
<Function 'test_tag_with_text_and_attributes'>
<Function 'test_tag_with_subclassed_attr_simple'>
<Function 'test_tag_nested'>
<Function 'test_tag_xmlname'>
</pre>
<p>By default all directories not starting with a dot are traversed,
looking for <tt class="docutils literal"><span class="pre">test_*.py</span></tt> and <tt class="docutils literal"><span class="pre">*_test.py</span></tt> files. Those files
are imported under their <a class="reference" href="#package-name">package name</a>.</p>
</div>
<div class="section">
<h2><a class="toc-backref" href="#id3" id="test-items-are-collectors-as-well" name="test-items-are-collectors-as-well"><span id="collector-api"></span>1.2 test items are collectors as well</a></h2>
<p>To make the reporting life simple for the session object
items offer a <tt class="docutils literal"><span class="pre">run()</span></tt> method as well. In fact the session
distinguishes "collectors" from "items" solely by interpreting
their return value. If it is a list, then we recurse into
it, otherwise we consider the "test" as passed.</p>
</div>
<div class="section">
<h2><a class="toc-backref" href="#id4" id="constructing-the-package-name-for-modules" name="constructing-the-package-name-for-modules"><span id="package-name"></span>1.3 constructing the package name for modules</a></h2>
<p>Test modules are imported under their fully qualified
name. Given a module <tt class="docutils literal"><span class="pre">path</span></tt> the fully qualified package
name is constructed as follows:</p>
<ul class="simple">
<li>determine the last "upward" directory from <tt class="docutils literal"><span class="pre">path</span></tt> that
contains an <tt class="docutils literal"><span class="pre">__init__.py</span></tt> file. Going upwards
means repeatedly calling the <tt class="docutils literal"><span class="pre">dirpath()</span></tt> method
on a path object (which returns the parent directory
as a path object).</li>
<li>insert this base directory into the sys.path list
as its first element</li>
<li>import the root package</li>
<li>determine the fully qualified name for the module located
at <tt class="docutils literal"><span class="pre">path</span></tt> ...<ul>
<li>if the imported root package has a __package__ object
then call <tt class="docutils literal"><span class="pre">__package__.getimportname(path)</span></tt></li>
<li>otherwise use the relative path of the module path to
the base dir and turn slashes into dots and strike
the trailing <tt class="docutils literal"><span class="pre">.py</span></tt>.</li>
</ul>
</li>
</ul>
<p>The Module collector will eventually trigger
<tt class="docutils literal"><span class="pre">__import__(mod_fqdnname,</span> <span class="pre">...)</span></tt> to finally get to
the live module object.</p>
<p>Side note: this whole logic is performed by local path
object's <tt class="docutils literal"><span class="pre">pyimport()</span></tt> method.</p>
</div>
<div class="section">
<h2><a class="toc-backref" href="#id5" id="module-collector" name="module-collector">1.4 Module Collector</a></h2>
<p>The default Module collector looks for test functions
and test classes and methods. Test functions and methods
are prefixed <tt class="docutils literal"><span class="pre">test</span></tt> by default. Test classes must
start with a capitalized <tt class="docutils literal"><span class="pre">Test</span></tt> prefix.</p>
</div>
</div>
<div class="section">
<h1><a class="toc-backref" href="#id6" id="customizing-the-testing-process" name="customizing-the-testing-process">2 Customizing the testing process</a></h1>
<div class="section">
<h2><a class="toc-backref" href="#id7" id="writing-conftest-py-files" name="writing-conftest-py-files">2.1 writing conftest.py files</a></h2>
<p>You may put conftest.py files containing project-specific
configuration in your project's root directory, it's usually
best to put it just into the same directory level as your
topmost <tt class="docutils literal"><span class="pre">__init__.py</span></tt>. In fact, <tt class="docutils literal"><span class="pre">py.test</span></tt> performs
an "upwards" search starting from the directory that you specify
to be tested and will lookup configuration values right-to-left.
You may have options that reside e.g. in your home directory
but note that project specific settings will be considered
first. There is a flag that helps you debugging your
conftest.py configurations:</p>
<pre class="literal-block">
py.test --traceconfig
</pre>
<div class="section">
<h3><a class="toc-backref" href="#id8" id="adding-custom-options" name="adding-custom-options">2.1.1 adding custom options</a></h3>
<p>To register a project-specific command line option
you may have the following code within a <tt class="docutils literal"><span class="pre">conftest.py</span></tt> file:</p>
<pre class="literal-block">
import py
Option = py.test.config.Option
option = py.test.config.addoptions("pypy options",
Option('-V', '--view', action="store_true", dest="view", default=False,
help="view translation tests' flow graphs with Pygame"),
)
</pre>
<p>and you can then access <tt class="docutils literal"><span class="pre">option.view</span></tt> like this:</p>
<pre class="literal-block">
if option.view:
print "view this!"
</pre>
<p>The option will be available if you type <tt class="docutils literal"><span class="pre">py.test</span> <span class="pre">-h</span></tt>
Note that you may only register upper case short
options. <tt class="docutils literal"><span class="pre">py.test</span></tt> reserves all lower
case short options for its own cross-project usage.</p>
</div>
</div>
<div class="section">
<h2><a class="toc-backref" href="#id9" id="customizing-the-collecting-and-running-process" name="customizing-the-collecting-and-running-process">2.2 customizing the collecting and running process</a></h2>
<p>To introduce different test items you can create
one or more <tt class="docutils literal"><span class="pre">conftest.py</span></tt> files in your project.
When the collection process traverses directories
and modules the default collectors will produce
custom Collectors and Items if they are found
in a local <tt class="docutils literal"><span class="pre">conftest.py</span></tt> file.</p>
<div class="section">
<h3><a class="toc-backref" href="#id10" id="example-perform-additional-rest-checks" name="example-perform-additional-rest-checks">2.2.1 example: perform additional ReST checks</a></h3>
<p>With your custom collectors or items you can completely
derive from the standard way of collecting and running
tests in a localized manner. Let's look at an example.
If you invoke <tt class="docutils literal"><span class="pre">py.test</span> <span class="pre">--collectonly</span> <span class="pre">py/documentation</span></tt>
then you get:</p>
<pre class="literal-block">
<DocDirectory 'documentation'>
<DocDirectory 'example'>
<DocDirectory 'pytest'>
<Module 'test_setup_flow_example.py' (test_setup_flow_example)>
<Class 'TestStateFullThing'>
<Instance '()'>
<Function 'test_42'>
<Function 'test_23'>
<ReSTChecker 'TODO.txt'>
<ReSTSyntaxTest 'TODO.txt'>
<LinkCheckerMaker 'checklinks'>
<ReSTChecker 'api.txt'>
<ReSTSyntaxTest 'api.txt'>
<LinkCheckerMaker 'checklinks'>
<CheckLink 'getting-started.html'>
...
</pre>
<p>In <tt class="docutils literal"><span class="pre">py/documentation/conftest.py</span></tt> you find the following
customization:</p>
<pre class="literal-block">
class DocDirectory(py.test.collect.Directory):
def run(self):
results = super(DocDirectory, self).run()
for x in self.fspath.listdir('*.txt', sort=True):
results.append(x.basename)
return results
def join(self, name):
if not name.endswith('.txt'):
return super(DocDirectory, self).join(name)
p = self.fspath.join(name)
if p.check(file=1):
return ReSTChecker(p, parent=self)
Directory = DocDirectory
</pre>
<p>The existence of the 'Directory' name in the
<tt class="docutils literal"><span class="pre">pypy/documentation/conftest.py</span></tt> module makes the collection
process defer to our custom "DocDirectory" collector. We extend
the set of collected test items by <tt class="docutils literal"><span class="pre">ReSTChecker</span></tt> instances
which themselves create <tt class="docutils literal"><span class="pre">ReSTSyntaxTest</span></tt> and <tt class="docutils literal"><span class="pre">LinkCheckerMaker</span></tt>
items. All of this instances (need to) follow the <a class="reference" href="#collector-api">collector API</a>.</p>
</div>
</div>
<div class="section">
<h2><a class="toc-backref" href="#id11" id="customizing-the-collection-process-in-a-module" name="customizing-the-collection-process-in-a-module">2.3 Customizing the collection process in a module</a></h2>
<blockquote>
REPEATED WARNING: details of the collection and running process are
still subject to refactorings and thus details will change.
If you are customizing py.test at "Item" level then you
definitely want to be subscribed to the <a class="reference" href="http://codespeak.net/mailman/listinfo/py-dev">py-dev mailing list</a>
to follow ongoing development.</blockquote>
<p>If you have a module where you want to take responsibility for
collecting your own test Items and possibly even for executing
a test then you can provide <a class="reference" href="test.html#generative-tests">generative tests</a> that yield
callables and possibly arguments as a tuple. This should
serve some immediate purposes like paramtrized tests.</p>
<p>The other extension possibility goes deeper into the machinery
and allows you to specify a custom test <tt class="docutils literal"><span class="pre">Item</span></tt> class which
is responsible for setting up and executing an underlying
test. [XXX not working: You can integrate your custom <tt class="docutils literal"><span class="pre">py.test.collect.Item</span></tt> subclass
by binding an <tt class="docutils literal"><span class="pre">Item</span></tt> name to a test class.] Or you can
extend the collection process for a whole directory tree
by putting Items in a <tt class="docutils literal"><span class="pre">conftest.py</span></tt> configuration file.
The collection process constantly looks at according names
in the <em>chain of conftest.py</em> modules to determine collectors
and items at <tt class="docutils literal"><span class="pre">Directory</span></tt>, <tt class="docutils literal"><span class="pre">Module</span></tt>, <tt class="docutils literal"><span class="pre">Class</span></tt>, <tt class="docutils literal"><span class="pre">Function</span></tt>
or <tt class="docutils literal"><span class="pre">Generator</span></tt> level. Note that, right now, except for <tt class="docutils literal"><span class="pre">Function</span></tt>
items all classes are pure collectors, i.e. will return a list
of names (possibly empty).</p>
<p>XXX implement doctests as alternatives to <tt class="docutils literal"><span class="pre">Function</span></tt> items.</p>
</div>
<div class="section">
<h2><a class="toc-backref" href="#id12" id="customizing-execution-of-functions" name="customizing-execution-of-functions">2.4 Customizing execution of Functions</a></h2>
<ul class="simple">
<li>Function test items allow total control of executing their
contained test method. <tt class="docutils literal"><span class="pre">function.run()</span></tt> will get called by the
session in order to actually run a test. The method is responsible
for performing proper setup/teardown ("Test Fixtures") for a
Function test.</li>
<li><tt class="docutils literal"><span class="pre">Function.execute(target,</span> <span class="pre">*args)</span></tt> methods are invoked by
the default <tt class="docutils literal"><span class="pre">Function.run()</span></tt> to actually execute a python
function with the given (usually empty set of) arguments.</li>
</ul>
</div>
</div>
</div>
</div></body></html> 