web Package

web Package

The routines, modules, etc. that are involved in the presentation and handling of content over HTTP.

These are the parts that makes it TiddlyWeb, not Tiddly something else.

challenge Module

WSGI App for running the base challenge system, which lists and links to the available challengers. If there is only one challenger, redirect to it.

tiddlyweb.web.challenge.base(environ, start_response)

The basic listing page that shows all available challenger systems. If there is only one challenger, we redirect to that instead of listing.

tiddlyweb.web.challenge.challenge_get(environ, start_response)

Dispatch a GET request to the chosen challenger.

tiddlyweb.web.challenge.challenge_post(environ, start_response)

Dispatch a POST request to the chosen challenger.

extractor Module

Extract of user credentials from incoming web requests. UserExtract passes to a stack of extractors. If an extractor returns something other than None, we have found valid data with which to set tiddlyweb.usersign.

class tiddlyweb.web.extractor.UserExtract(application)

Bases: object

WSGI Middleware to set the tiddlyweb.usersign, if it can be found in the request.

listentities Module

Common code used for listing bags and recipes in HTTP responses.

tiddlyweb.web.listentities.list_entities(environ, start_response, method_name, store_list=None, serializer_list=None)

Get an optionally filtered list of all the bags or recipes the current tiddlyweb.usersign can read.

negotiate Module

WSGI Middleware to do a limited version of content negotiation and put the type in tiddlyweb.type. On GET and HEAD requests the Accept header is examined. On POST and PUT, Content-Type. If extensions are provided on a URI used in a GET request if the extension matches something in extension_types in config, the type indicated by the extension wins over the Accept header.

class tiddlyweb.web.negotiate.Negotiate(application)

Bases: object

Perform a form of content negotiation to provide information to the WSGI environment that will later be used to choose serializers.

tiddlyweb.web.negotiate.figure_type(environ)

Determine either the Content-Type (for POST and PUT) or Accept header (for GET) and put that information in tiddlyweb.type in the WSGI environment.

query Module

WSGI Middleware that extracts CGI parameters from the QUERY_STRING and puts them in tiddlyweb.query in the environ in the same structure that cgi.py uses (dictionary of lists). If the current request is a POST of HTML form data, parse that too.

class tiddlyweb.web.query.Query(application)

Bases: object

Extract CGI parameter data from QUERY_STRING and POSTed form data.

extract_query(environ)

Read the QUERY_STRING and body (if a POSTed form) to extract query parameters. Put the results in tiddlyweb.query in environ. The query names and values are decoded from UTF-8 to unicode.

If there are file uploads in posted form data, the files are not put into tiddlyweb.query. Instead the file handles are appended to tiddlyweb.input_files.

sendentity Module

Send a bag or recipe out over HTTP, first serializing to the correct type.

This consolidates common code for bags and recipes.

tiddlyweb.web.sendentity.send_entity(environ, start_response, entity)

Send a bag or recipe out over HTTP, first serializing to the correct type. If an incoming Etag validates, raise a 304 response.

sendtiddlers Module

Routines related to sending a list of tiddlers out to the web, including optionally filtering those tiddlers and validating cache-oriented request headers.

tiddlyweb.web.sendtiddlers.send_tiddlers(environ, start_response, tiddlers=None)

Output the tiddlers contained in the provided Tiddlers collection in a Negotiated representation.

serve Module

Functions and Classes for running a TiddlyWeb server, including optionally a built in web server.

class tiddlyweb.web.serve.Configurator(application, config)

Bases: object

WSGI middleware to set tiddlyweb.config in environ for every request from config.

class tiddlyweb.web.serve.RequestStarter(application)

Bases: object

WSGI middleware that logs basic request information and cleans PATH_INFO in the environment.

PATH_INFO cleaning is done to ensure that there is a well known encoding of special characters and to support / in entity names (see clean_path_info()).

clean_path_info(environ)

Clean PATH_INFO in the environment.

This is necessary because WSGI servers tend to decode the URI before putting it in PATH_INFO. This means that uri encoded data, such as the %2F encoding of / will be decoded before we get to route dispatch handling, by which time the / is treated as a separator. People say that the right thing to do here is not use %2F. This is hogwash. The right thing to do is not decode PATH_INFO. In this solution if REQUEST_URI is present we use a portion of it to set PATH_INFO.

tiddlyweb.web.serve.load_app(app_prefix=None, dirname=None)

Create our application from a series of layers. The innermost layer is a Selector application based on urls_map defined in config. This is surrounded by wrappers, which either set something in the environment, modify the request, or transform the response. The wrappers are WSGI middleware defined by server_request_filters and server_response_filters in tiddlyweb.config.

tiddlyweb.web.serve.start_server(config)

Start a simple webserver, from wsgiref, to run our app.

util Module

General utility routines shared by various web related modules.

tiddlyweb.web.util.bag_etag(environ, bag)

Construct an etag for a bag.

tiddlyweb.web.util.bag_url(environ, bag, full=True)

Construct a URL for a bag.

tiddlyweb.web.util.check_bag_constraint(environ, bag, constraint)

Check to see if the provided bag allows the current tiddlyweb.usersign to perform the action described by constraint. Lets NoBagError raise if the bag does not exist.

This is a web util because user and store come from the WSGI environ.

tiddlyweb.web.util.check_incoming_etag(environ, etag_string, cache_control='no-cache', last_modified=None, vary='Accept')

Raise 304 if the provided etag_string is the same as that found in the If-None-Match header of the incoming request.

Return incoming_etag to indicate if an etag was there but did not match.

tiddlyweb.web.util.check_last_modified(environ, last_modified_string, etag='', cache_control='no-cache', vary='Accept')

Raise 304 if an If-Modified-Since header matches last_modified_string.

tiddlyweb.web.util.content_length_and_type(environ)

For PUT or POST request there must be Content-Length and Content-Type headers. Raise 400 if not present in the request.

tiddlyweb.web.util.datetime_from_http_date(http_datestring)

Turn an HTTP formatted date into a datetime object. Return None if the date string is invalid.

tiddlyweb.web.util.encode_name(name)

Encode a unicode value as utf-8 and then URL encode that string. Use for entity titles in URLs.

tiddlyweb.web.util.entity_etag(environ, entity)

Construct an Etag from the digest of the JSON reprepresentation of an entity.

The JSON representation provides a reasonably repeatable and unique string of data.

tiddlyweb.web.util.escape_attribute_value(text)

Escape common HTML character entities, including double quotes in attribute values

This assumes values are enclosed in double quotes (key=”value”).

tiddlyweb.web.util.get_route_value(environ, name)

Retrieve and decode name from data provided in WSGI route.

If name is not present in the route, allow KeyError to raise.

tiddlyweb.web.util.get_serialize_type(environ, collection=False, accept_type=False)

Look in the environ to determine which serializer should be used for this request.

If collection is True, then the presence of an extension on the URI which does not match any serializer should lead to a 415.

tiddlyweb.web.util.handle_extension(environ, resource_name)

Look for an extension (as defined in config) on the provided resource_name and trim it off to give the “real” resource name.

tiddlyweb.web.util.html_encode(text)

Encode &, < and > entities in text that will be used in or as HTML.

tiddlyweb.web.util.html_frame(environ, title='')

Return the header and footer from the current HTML serialization.

tiddlyweb.web.util.http_date_from_timestamp(timestamp)

Turn a modifier or created tiddler timestamp into a properly formatted HTTP date. If the timestamp is invalid use the current time as the timestamp.

Create a cookie string, optionally with a MAC, path and expires value. If expires is provided, its value should be in seconds.

tiddlyweb.web.util.read_request_body(environ, length)

Read the wsgi.input handle to get the request body.

Length is a required parameter because it is tested for existence earlier in the process.

tiddlyweb.web.util.recipe_etag(environ, recipe)

Construct an etag for a recipe.

tiddlyweb.web.util.recipe_url(environ, recipe, full=True)

Construct a URL for a recipe.

tiddlyweb.web.util.server_base_url(environ)

Using information in tiddlyweb.config, construct the base URL of the server, without the trailing /.

tiddlyweb.web.util.server_host_url(environ)

Generate the scheme and host portion of our server url.

tiddlyweb.web.util.tiddler_etag(environ, tiddler)

Construct an etag for a tiddler from the tiddler’s attributes, but not its text.

tiddlyweb.web.util.tiddler_url(environ, tiddler, container='bags', full=True)

Construct a URL for a tiddler.

validator Module

A collection of routines for validating, santizing and otherwise messing with content coming in from the web to be tiddlers, bags or recipes.

The validators can be extended by adding functions to the BAG_VALIDATORS, RECIPE_VALIDATORS and TIDDLER_VALIDATORS. The functions take an entity object, and an optional WSGI environ dict.

exception tiddlyweb.web.validator.InvalidBagError

Bases: exceptions.Exception

The provided bag has not passed a validation routine and has been rejected. The caller should stop processing and return an error to calling code or user-agent.

exception tiddlyweb.web.validator.InvalidRecipeError

Bases: exceptions.Exception

The provided recipe has not passed a validation routine and has been rejected. The caller should stop processing and return an error to calling code or user-agent.

exception tiddlyweb.web.validator.InvalidTiddlerError

Bases: exceptions.Exception

The provided tiddler has not passed a validation routine and has been rejected. The caller should stop processing and return an error to calling code or user-agent.

tiddlyweb.web.validator.sanitize_desc(entity, environ)

Strip any dangerous HTML which may be present in a bag or recipe description.

tiddlyweb.web.validator.sanitize_html_fragment(fragment)

Santize an HTML fragment, returning a copy of the fragment that has been cleaned up.

tiddlyweb.web.validator.validate_bag(bag, environ=None)

Pass the bag to each of the functions in BAG_VALIDATORS, in order, either changing the content of the bags’s attributes, or if some aspect of the bag can not be accepted raising InvalidBagError.

BAG_VALIDATORS may be extended by plugins.

validate_bag is called whenever a bag is PUT via HTTP.

tiddlyweb.web.validator.validate_recipe(recipe, environ=None)

Pass the recipe to each of the functions in RECIPE_VALIDATORS, in order, either changing the content of the recipes’s attributes, or if some aspect of the recipe can not be accepted raising InvalidRecipeError.

RECIPE_VALIDATORS may be extended by plugins.

validate_recipe is called whenever a recipe is PUT via HTTP.

tiddlyweb.web.validator.validate_tiddler(tiddler, environ=None)

Pass the tiddler to each of the functions in TIDDLER_VALIDATORS, in order, either changing the content of the tiddler’s attributes, or if some aspect of the tiddler can not be accepted raising InvalidTiddlerError.

TIDDLER_VALIDATORS is an empty list which may be extended by plugins.

validate_tiddler is called from web handlers, when the accept constraint on the policy of the bag containing the tiddler does not pass.

wsgi Module

WSGI Middleware apps that haven’t gotten around to being extracted to their own modules.

class tiddlyweb.web.wsgi.EncodeUTF8(application)

Bases: object

WSGI Middleware to ensure that the unicode content sent out the pipe is encoded to UTF-8. Within the application string-based content is unicode (i.e. not encoded).

class tiddlyweb.web.wsgi.Header(application)

Bases: object

If REQUEST_METHOD is HEAD, change it internally to GET and consume the generated output so the response has no body.

class tiddlyweb.web.wsgi.PermissionsExceptor(application)

Bases: object

Trap permissions exceptions and turn them into HTTP exceptions so the errors are propagated to clients.

class tiddlyweb.web.wsgi.SimpleLog(application)

Bases: object

WSGI Middleware to write a very simple log to stdout.

Borrowed from Paste Translogger

format = '%(REMOTE_ADDR)s - %(REMOTE_USER)s [%(time)s] "%(REQUEST_METHOD)s %(REQUEST_URI)s %(HTTP_VERSION)s" %(status)s %(bytes)s "%(HTTP_REFERER)s" "%(HTTP_USER_AGENT)s"'
write_log(environ, req_uri, status, size)

Write the log info out in a formatted form to logging.info.

This is rather more complex than desirable because there is a mix of str and unicode in the gathered data and it needs to be made acceptable for output.

class tiddlyweb.web.wsgi.StoreSet(application)

Bases: object

WSGI Middleware that sets our choice of Store in the environ. That is, initialize the store for each request.

class tiddlyweb.web.wsgi.TransformProtect(application)

Bases: object

WSGI Middleware to add a Cache-Control: no-transform` header so that mobile companies that transcode content over their 3G (etc) networks don’t, as it will break various JavaScript things, including TiddlyWiki.