filters Package

filters Package

Overarching handler for TiddlyWeb filters.

Filters provide an extensible syntax for limiting any Collection by attributes on the entities in the collection. Though primarily for Tiddlers, Bags and Recipes can be filtered as well.

The basic filters provide for selecting and sorting on attributes of the entities and for limiting (the number of) entities. These basic types of filter can be extended with plugins, and the ways attributes are processed can also be extended.

Filters are parsed from a string that is formatted as a CGI query string with parameters and arguments. The parameter is a filter type. Each filter is processed in sequence: the first processing all the entities handed to it, the next taking only those that result from the first.

Filters can be extended by adding more parsers to FILTER_PARSERS. Parsers for existing filter types may be extended as well (see the documentation for each type).

The call signature for a filter is:

filter(entities, indexable=indexable, environ=environ)

The attribute and value for which a filter filters is established in the parsing stage and are set as upvalues of the filter closure that gets created.

indexable and environ are optional parameters that in special cases allow a select style filter to be optimized with the use of an index. In the current implementation this is only done when:

• the select filter is the first filter in a stack of filters passed to recursive_filter()
• the entities to be filtered are tiddlers in the context of a bag (this helps to constrain the index)

When both of the above are true the system looks for a module named by tiddlyweb.config['indexer'], imports it, looks for a function called indexy_query, and passes environ and information about the bag and the attribute being selected.

What index_query does to satisfy the query is up to the module. It should return a list of tiddlers that have been loaded from the tiddlyweb.store.Store.

If for some reason index_query does not wish to perform the query (e.g. the index cannot satisfy the query) it may raise FilterIndexRefused and the normal filtering process will be performed.

Note that testing should be done to determine if using an index is of any benefit. In some stores (for example caching stores) traversing the tiddlers is faster than using an index.

exception tiddlyweb.filters.FilterError

Bases: exceptions.Exception

An exception to throw when an attempt is made to filter on an unavailable attribute.

exception tiddlyweb.filters.FilterIndexRefused

Bases: tiddlyweb.filters.FilterError

A filter index has refused to satisfy a filter with its index.

tiddlyweb.filters.parse_for_filters(query_string, environ=None)

Take a string that looks like a CGI query string and parse it for filters. Return a tuple of a list of filter functions and a string of whatever was in the query string that did not result in a filter.

tiddlyweb.filters.recursive_filter(filters, entities, indexable=False)

Recursively process the list of filters found by parse_for_filters() against the given list of entities.

Each next filter processes only those entities that were results of the previous filter.

Misnamed, early versions were more truly recursive.

limit Module

A filter type to limit a group of entities using a syntax similar to SQL Limit:

limit=<index>,<count>
limit=<count>

tiddlyweb.filters.limit.limit(entities, count=0, index=0)

Make a slice of a list of entities based on a count and index.

tiddlyweb.filters.limit.limit_parse(count='0')

Parse the argument of a limit filter for a count and index argument, return a function which does the limiting.

Exceptions while parsing are passed up the stack.

select Module

A filter type for selecting only some entities, usually tiddlers, from a collection of entities, usually by an attribute of the tiddlers.

The syntax is:

select=attribute:value    # attribute is value
select=attribute:!value   # attribute is not value
select=attribute:>value   # attribute is greater than value
select=attribute:<value   # attribute is less than value

ATTRIBUTE_SELECTOR is checked for a function which returns True or False for whether the provided value matches for the entity being tested. The default case is lower case string equality. Other functions may be provided by plugins. Attributes may be virtual, i.e. not real attributes on entity. For example we can check for the presence of a tag in a tiddlers tags attribute with:

select=tag:tagvalue

An attribute function takes an entity, an attribute name and a value. It may then do anything it wants with it, and must return True or False.

• ! negates a selection, getting all those entities that don’t match.
• > gets those entities that sort greater than the value.
• < gets those entities that sort less than the value.

When doing sorting ATTRIBUTE_SORT_KEY is consulted to canonicalize the value. See tiddlyweb.filters.sort.

tiddlyweb.filters.select.bag_in_recipe(entity, attribute, value)

Return True if the named bag is in the recipe.

tiddlyweb.filters.select.default_func(entity, attribute, value)

Look in the entity for an attribute with the provided value. First real object attributes are checked, then, if available, extended fields. If neither of these are present, return False.

tiddlyweb.filters.select.field_in_fields(entity, attribute, value)

Return True if the entity has the named field.

tiddlyweb.filters.select.select_by_attribute(attribute, value, entities, negate=False, indexable=None, environ=None)

Select entities where value of attribute matches the provide value.

If negate is True, get those that don’t match.

tiddlyweb.filters.select.select_parse(command)

Parse a select filter string into attributes and arguments and return a function (for later use) which will do the selecting.

tiddlyweb.filters.select.select_relative_attribute(attribute, value, entities, greater=False, lesser=False, environ=None)

Select entities that sort greater or less than the provided value for the provided attribute.

tiddlyweb.filters.select.tag_in_tags(entity, attribute, value)

Return True if the provided entity has a tag of value in its tag list.

tiddlyweb.filters.select.text_in_text(entity, attribute, value)

Return True if the provided entity has the string provided in value within its text attribute.

sort Module

A filter type to sort a collection of entities by some attribute. The syntax is:

sort=attribute   # sort ascending
sort=-attribute  # sort descending

Atribute is either a real entity attribute or a key in ATTRIBUTE_SORT_KEY that has as its value a function used to generate a key to pass to the sort. ATTRIBUTE_SORT_KEY can be extended by plugins.

tiddlyweb.filters.sort.as_int(attribute)

Treat attribute as int if it looks like one.

tiddlyweb.filters.sort.date_to_canonical(datestring)

Take a (TiddlyWiki-style) string of 14 or less digits and turn it into 14 digits for the sake of comparing entity dates.

tiddlyweb.filters.sort.sort_by_attribute(attribute, entities, reverse=False, environ=None)

Sort a group of entities by some attribute. Inspect ATTRIBUTE_SORT_KEY to see if there is a special function by which we should generate the value for this attribute.

tiddlyweb.filters.sort.sort_parse(attribute)

Create a function which will sort a collection of entities.