twill Extension Modules ¶

Several different extension modules are distributed with twill, under ‘twill.extensions’.

check_links – a simple link checker ¶

A simple link checker is included with twill. To use it, do

go <page>

extend_with check_links
check_links

‘check_links’ will visit each link on the current page and verify that the HTTP status code is 200 (success). A list of failing links will be printed out at the end of the function.

The ‘check_links’ function takes an optional argument, a regex to filter for links that should be checked. By default the filter matches everything; if you only wanted to check links on e.g. idyll.org, you could do

go http://idyll.org/
check_links .*\.idyll\.org

A few notes:

‘follow’ is used to visit each link, so the referrer URL should be set correctly.

HTTP basic authentication and cookies work as you’d expect, so you can set up twill with all of the permissions necessary to visit restricted links & only then run check_links.

match_parse – extension for regular expressions with multiple matches ¶

The ‘match_parse’ extension module contains a number of functions for dealing with multiple matches, etc. Here’s some example code:

extend_with match_parse
go http://www.idyll.org/

split "org"
echo __matchlist__

findall "t."
echo __matchlist__

split "org"
popmatch 0
getmatch test 'm[0].split()'
showvar test

split "org"
setmatch "m.split()[0]"

popmatch 0
echo __matchlist__
echo __match__

require – assert that specific conditions hold after each page is loaded ¶

The ‘require’ extension module contains four functions that let you assert specific conditions. For example,

extend_with require
require success

will assert that after each page load, the HTTP code is 200 (“success”). ‘require’ does nothing on a ‘back’ call (which doesn’t actually reload a page).

Currently there are only two assertions available, ‘success’ and ‘links_ok’. ‘links_ok’ automatically runs ‘check_links’ (see above) after each page is loaded. ‘links_ok’ will not check links twice unless you call ‘flush_visited’ (see below).

The other functions in this module are:

skip_require – don’t check the requirements for the next action.

no_require – turn off requirements checking & reset conditions.

flush_visited – flush the list of visited links.

mailman_sf – discard spam messages from your SourceForge mailing lists ¶

The ‘mailman_sf’ extension module contains two functions, ‘discard_all_messages’ and ‘exit_if_empty’. Here’s some example code:

# load in the mailman_sf extensions module
extend_with mailman_sf

# unfortunately we have to hard-code in the mailing list name for
# the moment. not sure how to do substitutions here.
go https://lists.sourceforge.net/lists/admindb/pywx-announce

# fill out the page with the list password.
getpassword "Enter list password: "
formvalue 1 adminpw __password__
submit 0
code 200

# if there aren't any messages on the page, exit.
exit_if_empty

# if not empty, discard all messages & submit
discard_all_messages
submit 0

argparse – pass arguments into twill scripts via sys.argv ¶

The ‘argparse’ extension module contains one function, ‘get_args’. ‘get_args’ loads all of the post-scriptfile command-line arguments into variables, e.g.

$ twill script1 script2 script3 -- val1 val2 val3
...
>> extend_with argparse
>> get_args
>> echo "args are ${arg1} ${args} ${arg3}"
args are val1 val2 val3

dns_check – make assertions about domain name service records ¶

dns_check implements functions to test domain name service (DNS) resolution. You’ll need dnspython to use it.

Functions:

dns_resolves – assert that a host resolves to a specific IP address.

dns_a – assert that a host directly resolves to a specific IP address

dns_cname – assert that a host is an alias for another hostname.

dnx_mx – assert that a given host is a mail exchanger for the given name.

dns_ns – assert that a given hostname is a name server for the given name.

csv_iterate – iterate over lists of values ¶

csv_iterate provides a single function, ‘csv_iterate’, that lets you run a script once for each row in a file of comma-separated values. For example, if you had a file ‘test.csv’ containing

value1, value2
value3, value4

and a script ‘test’ containing

echo "first value is ${col1}, second is ${col2}"

then

>> csv_iterate test.csv test

would produce

first value is value1, second is value2
first value is value3, second is value4

This is designed for building test suites containing many different combinations of values for specific forms.

dirstack – manipulate the current working directory (cwd)¶

dirstack provides two functions, ‘chdir’ and ‘popd’, which change the directory and recover the original directory, respectively. The cwd is kept in the global variable ‘__dir__’.

For example

>> chdir /tmp
>> echo __dir__
>> popd
>> echo __dir__

will change to the ‘/tmp’ directory, print out ‘/tmp’, and then change back to the original directory.

formfill – convenience functions for filling out forms ¶

formfill provides three functions, ‘fv_match’, ‘fv_multi’, and ‘fv_multi_sub’.

‘fv_match’ allows you to set many form fields all at once, based on regex matching to the form field name, e.g.

fv_match <form> test-(\d+) 'hello world!'

will set all fields with the name ‘test-#’ (# is any number) to ‘hello world!’

‘fv_multi’ allows you to set many form fields at once, using a more compact notation:

fv_multi <form> field1=value1 field2=value2 ...

‘fv_multi’ uses the same field-matching convention that ‘formvalue’ uses: it simply iterates over all arguments, splits at the first ‘=’, and runs ‘formvalue’ directly.

‘fv_multi_sub’ does the same thing as ‘fv_multi’ and then executes ‘submit’.

(Thanks to Ben Bangert for the idea!)