<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> <html> <head> <title>[pytest-overview] </title> <meta content="text/html;charset=ISO-8859-1" name="Content-Type"/> <link href="../style.css" media="screen" rel="stylesheet" type="text/css"/></head> <body> <div id="logo"><a href="http://codespeak.net"><img alt="py lib" height="114" id="pyimg" src="http://codespeak.net/img/pylib.png" width="154"/></a></div> <div id="metaspace"> <div class="project_title">[pytest-overview] </div> <div id="menubar"><a class="menu" href="../index.html">index</a> <a class="menu" href="../../../apigen/api/index.html">api</a> <a class="menu" href="../../../apigen/source/index.html">source</a> <a class="menu" href="../contact.html">contact</a> <a class="menu" href="../download.html">download</a></div></div> <div id="contentspace"> <div id="docinfoline"> <div style="float: right; font-style: italic;"> </div></div> <div class="document" id="py-test-flexible-and-powerful-automated-testing"> <h1 class="title">py.test - flexible and powerful automated testing</h1> <table class="docinfo" frame="void" rules="none"> <col class="docinfo-name" /> <col class="docinfo-content" /> <tbody valign="top"> <tr><th class="docinfo-name">Authors:</th> <td>Holger Krekel <br />merlinux GmbH</td></tr> <tr><th class="docinfo-name">Date:</th> <td>13th March 2006</td></tr> </tbody> </table> <!-- Definitions of interpreted text roles (classes) for S5/HTML data. --> <!-- This data file has been placed in the public domain. --> <!-- Colours ======= --> <!-- Text Sizes ========== --> <!-- Display in Slides (Presentation Mode) Only ========================================== --> <!-- Display in Outline Mode Only ============================ --> <!-- Display in Print Only ===================== --> <!-- Incremental Display =================== --> <div class="section"> <h1><a id="intro-benefits-of-automated-testing" name="intro-benefits-of-automated-testing">Intro: Benefits of Automated Testing</a></h1> <ul class="simple"> <li>prove that code changes actually fix a certain issue</li> <li>minimizing Time to Feedback for developers</li> <li>reducing overall Time to Market</li> <li>document usage of plugins</li> <li>tests as a means of communication</li> <li>easing entry for newcomers</li> </ul> </div> <div class="section"> <h1><a id="py-test-purposes-goals" name="py-test-purposes-goals">py.test Purposes & Goals</a></h1> <ul class="simple"> <li>automated cross-project open source testing tool</li> <li>flexible per-project customization</li> <li>reusing test methods/reporting across projects</li> <li>various iterative test collection methods</li> <li>support for distributed testing</li> <li>py lib is a development support library</li> </ul> </div> <div class="section"> <h1><a id="what-is-python" name="what-is-python">What is Python?</a></h1> <ul class="simple"> <li>easy-to-learn flexible OO high level language</li> <li>glue-language for connecting C++, Java and scripting</li> <li>used e.g. by Google for deployment/testing/implementation</li> <li>used by BIND (mainstream DNS internet server) for testing</li> <li>Jython provides Python for JVM</li> <li>IronPython provides Python for .NET</li> <li>CPython is mainstream C-based platform</li> <li>PyPy - Python in Python implementation</li> </ul> </div> <div class="section"> <h1><a id="main-drivers-of-py-test-development" name="main-drivers-of-py-test-development">Main drivers of py.test development</a></h1> <ul class="simple"> <li>PyPy project testing needs (part of EU project)</li> <li>needs by individual (freely contributing) projects</li> <li>at least 20 projects using py.test and py lib</li> <li>higher level innovation plans by merlinux & experts</li> <li>commercial needs</li> <li>almost three years of (non-fulltime) development</li> </ul> </div> <div class="section"> <h1><a id="authors-copyrights" name="authors-copyrights">Authors & copyrights</a></h1> <ul class="simple"> <li>initial: Holger Krekel, Armin Rigo</li> <li>major contributors: Jan Balster, Brian Dorsey, Grig Gheorghiu</li> <li>many others with small patches</li> <li>MIT license</li> </ul> </div> <div class="section"> <h1><a id="who-is-merlinux" name="who-is-merlinux">who is merlinux?</a></h1> <ul class="simple"> <li>small company founded in 2004 by Holger Krekel and Laura Creighton</li> <li>purpose: research and development / open source technologies</li> <li>7 employees (no win32 experts!), 6 freelancers</li> <li>three larger projects:<ul> <li>PyPy - next generation Python implementation</li> <li>mailwitness - digital invoicing/signatures</li> <li>provider of development servers</li> </ul> </li> <li>technologies: virtualization, deployment and testing</li> </ul> </div> <div class="section"> <h1><a id="main-features-of-py-test" name="main-features-of-py-test">Main Features of py.test</a></h1> <ul class="simple"> <li>simplest possible <tt class="docutils literal"><span class="pre">assert</span></tt> approach</li> <li>clean setup/teardown semantics</li> <li>stdout/stderr capturing per test</li> <li>per-project/directory cmdline options (many predefined)</li> <li>test selection support</li> <li>customizable auto-collection of tests</li> <li><a class="reference" href="../test.html#features">more features</a> ...</li> </ul> </div> <div class="section"> <h1><a id="main-user-level-entry-points" name="main-user-level-entry-points">Main User-Level entry points</a></h1> <ul class="simple"> <li><tt class="docutils literal"><span class="pre">py.test.raises(Exc,</span> <span class="pre">func,</span> <span class="pre">*args,</span> <span class="pre">**kwargs)</span></tt></li> <li><tt class="docutils literal"><span class="pre">py.test.fail(msg)</span></tt> -> fail a test</li> <li><tt class="docutils literal"><span class="pre">py.test.skip(msg)</span></tt> -> skip a test</li> <li><tt class="docutils literal"><span class="pre">py.test.ensuretemp(prefix)</span></tt> -> per-test session temporary directory</li> <li><tt class="docutils literal"><span class="pre">conftest.py</span></tt> can modify almost arbitrary testing aspects (but it's a bit involved)</li> </ul> </div> <div class="section"> <h1><a id="some-py-lib-components" name="some-py-lib-components">some py lib components</a></h1> <ul class="simple"> <li><tt class="docutils literal"><span class="pre">py.execnet</span></tt> provides ad-hoc means to distribute programs</li> <li><tt class="docutils literal"><span class="pre">py.path</span></tt> objects abstract local and svn files</li> <li><tt class="docutils literal"><span class="pre">py.log</span></tt> offers (preliminary) logging support</li> <li><tt class="docutils literal"><span class="pre">py.xml.html</span></tt> for programmatic html generation</li> <li>lazy import ...``import py`` is enough</li> </ul> </div> <div class="section"> <h1><a id="py-test-implementation" name="py-test-implementation">py.test Implementation</a></h1> <ul class="simple"> <li><a class="reference" href="../test.html">basic picture</a></li> <li>Session objects (Terminal and Tcl-GUI)</li> <li>reporting hooks are on session objects</li> <li>Collector hierarchy yield iteratively tests</li> <li>uses py lib extensively (py.path/py.execnet)</li> <li>"conftest.py" per-directory configuration mechanism</li> </ul> </div> <div class="section"> <h1><a id="session-objects" name="session-objects">Session objects</a></h1> <ul class="simple"> <li>responsible for driving the testing process</li> <li>make use of iterative Collector hierarchies</li> <li>responsible for reporting (XXX)</li> <li>can be split to a Frontend and BackendSession for distributed testing (GUI frontend uses it)</li> </ul> </div> <div class="section"> <h1><a id="collector-objects" name="collector-objects">Collector objects</a></h1> <ul class="simple"> <li>Collectors / Test Items form a tree</li> <li>the tree is build iteratively (driven from Sessions)</li> <li>collector tree can be viewed with <tt class="docutils literal"><span class="pre">--collectonly</span></tt></li> <li><tt class="docutils literal"><span class="pre">run()</span></tt> returns list of (test) names or runs the test</li> <li><tt class="docutils literal"><span class="pre">join(name)</span></tt> returns a sub collector/item</li> <li>various helper methods to e.g. determine file/location</li> </ul> </div> <div class="section"> <h1><a id="extensions-rest-documentation-checking" name="extensions-rest-documentation-checking">Extensions: ReST documentation checking</a></h1> <ul class="simple"> <li><a class="reference" href="../conftest.py">py/documentation/conftest.py</a> provides test items for checking documentation and link integrity</li> <li>uses its own collector/testitem hierarchy</li> <li>invokes <tt class="docutils literal"><span class="pre">docutils</span></tt> processing, reports errors</li> </ul> </div> <div class="section"> <h1><a id="extensions-distributed-testing" name="extensions-distributed-testing">Extensions: Distributed Testing</a></h1> <ul class="simple"> <li>using py.execnet to dispatch on different python versions</li> <li>using py.execnet to dispatch tests on other hosts/platforms</li> <li>currently: Popen, SSH and Socket gateways</li> <li>missing support pushing tests to "the other side"</li> <li>missing for deployment on multiple machines</li> <li>but it's already possible ...</li> </ul> </div> <div class="section"> <h1><a id="example-using-pywinauto-from-linux" name="example-using-pywinauto-from-linux">Example using pywinauto from linux</a></h1> <ul class="simple"> <li>start socketserver.py on windows</li> <li>connect a SocketGateway e.g. from linux</li> <li>send tests, execute and report tracebacks through the gateway</li> <li>remotely use pywinauto to automate testing of GUI work flow</li> <li>interactive example ...</li> </ul> </div> <div class="section"> <h1><a id="status-of-py-lib" name="status-of-py-lib">Status of py lib</a></h1> <ul class="simple"> <li>mostly developed on linux/OSX</li> <li>basically all tests pass on win32 as well</li> <li>but missing some win32 convenience</li> <li>some support for generation of html/ReST/PDFs reports</li> <li>py.execnet works rather reliably (pending deeper win32 testing)</li> <li>flexible configuration but sometimes non-obvious/documented (requires understanding of internals)</li> </ul> </div> <div class="section"> <h1><a id="next-steps-py-lib-py-test" name="next-steps-py-lib-py-test">Next Steps py lib / py.test</a></h1> <ul class="simple"> <li>refined py.execnet distribution of programs</li> <li>more configurable and customizable reporting</li> <li>implement support for testing distribution</li> <li>explore refined win32 support</li> <li>automated collection of unittest.py based tests</li> <li>make spawning processes/gateways more robust</li> <li>doctest support</li> <li>unify logging approaches (py.log.*)</li> <li>...</li> </ul> </div> </div> <div class="footer"> <hr class="footer" /> Holger Krekel (merlinux) • • 13th March 2006 </div> </div></body></html>