twill’s Python API

twill is essentially a web browsing and testing tool based on the httpx and lxml packages. All twill commands are implemented in the commands.py file, and pyparsing does the work of parsing the input and converting it into Python commands (see parse.py). Interactive shell work and readline support is implemented via the cmd module (from the standard Python library).

Using twill from Python

There are two fairly simple ways to use twill from Python. (They are compatible with each other, so you don’t need to choose between them; just use whichever is appropriate.)

The first is to simply import all of the commands in commands.py and use them directly from Python. For example,

from twill.commands import *
go("http://www.python.org/")
showforms()

This has the advantage of being very simple, as well as being tied directly to the documented set of commands in the commands reference.

However, the functions in commands.py are too simple for some situations. In particular, most of them do not have any return values, so in order to e.g. get the HTML for a particular page, you will need to ask the actual “Web browser” object that twill uses.

To make use of to the Web browser directly, import the browser object:

from twill import browser

browser.go("https://www.python.org/")
assert 'Python' in browser.html
browser.showforms()

This is the second way to use twill from Python, and it is much more flexible. All of the functions in commands.py use this same browser object, so you can mix and match:

from twill import browser

from twill.commands import *
go("https://www.python.org/")
assert 'Python' in browser.html
find("Documentation")
browser.showforms()

The basic difference is that functions available through the browser object are intended for use from Python, while commands.py functions define the twill language. In fact, all of the functions in commands.py are small snippets of code wrapped around the browser object.

Most importantly, the browser object also provides several properties that can be used to introspect the current state of the browser and evaluated programmatically in Python, such as url, code, html, title, links, forms, cookies, response_headers or history.

For more information on the functions exposed by the browser object, see the code of the TwillBrowser class in twill.browser.

Extending twill

Right now twill is very easy to extend: just build a Python module that exports the functions you want to call, place it in the PYTHONPATH, and run extend_with <modulename>. See extensions/mailman_sf.py for an extension that helps deal with mailman lists on SourceForge; this extension is used by examples/discard-sf-mailman-msgs.

Notes:

• If your extension raises SystemExit, twill will stop processing the script. This is a useful way to build in conditionals, e.g. see the discard-sf-mailman-msgs example script.

Passing variables from Python into twill

Suppose you write an extension function, and you want to return some values from that extension function for later use in filling out forms. How do you do this?

The simplest way is to simply insert them into the global or local dictionary, like so:

from twill.namespaces import get_twill_glocals
global_dict, local_dict = get_twill_glocals()

global_dict['varname'] = value

You can then use the usual variable access methods to substitute for varname, e.g.

formvalue 1 field $varname

Using twill in other Python programs

All of the commands available in twill are implemented as top-level functions in the twill.commands module. For example, to use twill functionality from another Python program, you can do:

from twill.commands import go, showforms, formclear, fv, submit

go('http://localhost:8080/')
go('./widgets')
showforms()

formclear('1')
fv('1', 'name', 'test')
fv('1', 'password', 'testpass')
fv('1', 'confirm', 'yes')
showforms()

submit('0')

Note that all arguments need to be strings, at least for the moment.

You can capture command output by passing any write-enabled file handle to twill.set_output, e.g.

twill.set_output(StringIO())

will send all non-error output into a StringIO() object.