core.db.query

Attributes

TOrderHook

TFilterHook

Classes

Query

Base Class for querying the datastore. Its API is similar to the google.cloud.datastore.query API,

Functions

_entryMatchesQuery(entry, singleFilter)

Utility function which checks if the given entity could have been returned by a query filtering by the

Module Contents

core.db.query.TOrderHook
core.db.query.TFilterHook
core.db.query._entryMatchesQuery(entry, singleFilter)

Utility function which checks if the given entity could have been returned by a query filtering by the properties in singleFilter. This can be used if a list of entities have been retrieved (e.g. by a 3rd party full text search engine) and these have now to be checked against the filter returned by their modules viur.core.prototypes.list.listFilter() method. :param entry: The entity which will be tested :param singleFilter: A dictionary containing all the filters from the query :return: True if the entity could have been returned by such an query, False otherwise

Parameters:
Return type:

bool

class core.db.query.Query(kind, srcSkelClass=None, *args, **kwargs)

Bases: object

Base Class for querying the datastore. Its API is similar to the google.cloud.datastore.query API, but it provides the necessary hooks for relational or random queries, the fulltext search as well as support for IN filters.

Constructs a new Query. :param kind: The kind to run this query on. This may be later overridden to run on a different kind (like

viur-relations), but it’s guaranteed to return only entities of that kind.

Parameters:

  • srcSkelClass (Union[viur.core.skeleton.SkeletonInstance, None]) – If set, enables data-model depended queries (like relational queries) as well as the :meth:fetch method

  • kind (str)

kind
srcSkel = None
queries: None | core.db.types.QueryDefinition | List[core.db.types.QueryDefinition]
_filterHook: TFilterHook | None = None
_orderHook: TOrderHook | None = None
_customMultiQueryMerge: None | Callable[[Query, List[List[core.db.types.Entity]], int], List[core.db.types.Entity]] = None
_calculateInternalMultiQueryLimit: None | Callable[[Query, int], int] = None
customQueryInfo
origKind
_lastEntry = None
_fulltextQueryString: None | str = None
lastCursor = None
setFilterHook(hook)

Installs hook as a callback function for new filters.

hook will be called each time a new filter constrain is added to the query. This allows e.g. the relationalBone to rewrite constrains added after the initial processing of the query has been done (e.g. by listFilter() methods).

Parameters:

hook (TFilterHook) – The function to register as callback. A value of None removes the currently active hook.

Returns:

The previously registered hook (if any), or None.

Return type:

TFilterHook | None

setOrderHook(hook)

Installs hook as a callback function for new orderings.

hook will be called each time a db.Query.order() is called on this query.

Parameters:

hook (TOrderHook) – The function to register as callback. A value of None removes the currently active hook.

Returns:

The previously registered hook (if any), or None.

Return type:

TOrderHook | None

mergeExternalFilter(filters)

Safely merges filters according to the data model.

Its only valid to call this function if the query has been created using core.skeleton.Skeleton.all().

It’s safe to pass filters received from an external source (a user); unknown/invalid filters will be ignored, so the query-object is kept in a valid state even when processing malformed data.

If complex queries are needed (e.g. filter by relations), this function shall also be used.

See also filter() for simple filters.

Parameters:

filters (dict) – A dictionary of attributes and filter pairs.

Returns:

Returns the query itself for chaining.

Return type:

Self

filter(prop, value)

Adds a new constraint to this query.

See also mergeExternalFilter() for a safer filter implementation.

Parameters:

  • prop (str) – Name of the property + operation we’ll filter by

  • value (core.db.types.DATASTORE_BASE_TYPES | list[core.db.types.DATASTORE_BASE_TYPES]) – The value of that filter.

Returns:

Returns the query itself for chaining.

Return type:

Self

order(*orderings)

Specify a query sorting.

Resulting entities will be sorted by the first property argument, then by the second, and so on.

The following example

query = Query("Person")
query.order(("bday" db.SortOrder.Ascending), ("age", db.SortOrder.Descending))

sorts every Person in order of their birthday, starting with January 1. People with the same birthday are sorted by age, oldest to youngest.

order() may be called multiple times. Each call resets the sort order from scratch.

If an inequality filter exists in this Query it must be the first property passed to order(). t.Any number of sort orders may be used after the inequality filter property. Without inequality filters, any number of filters with different orders may be specified.

Entities with multiple values for an order property are sorted by their lowest value.

Note that a sort order implies an existence filter! In other words, Entities without the sort order property are filtered out, and not included in the query results.

If the sort order property has different types in different entities - e.g. if bob[“id”] is an int and fred[“id”] is a string - the entities will be grouped first by the property type, then sorted within type. No attempt is made to compare property values across types.

Parameters:

orderings (Tuple[str, core.db.types.SortOrder]) – The properties to sort by, in sort order. Each argument must be a (name, direction) 2-tuple.

Returns:

Returns the query itself for chaining.

Return type:

Self

setCursor(startCursor, endCursor=None)

Sets the start and optionally end cursor for this query.

The result set will only include results between these cursors. The cursor is generated by an earlier query with exactly the same configuration.

It’s safe to use client-supplied cursors, a cursor can’t be abused to access entities which don’t match the current filters.

Parameters:

  • startCursor (str) – The start cursor for this query.

  • endCursor (Optional[str]) – The end cursor for this query.

Returns:

Returns the query itself for chaining.

Return type:

Self

limit(limit)

Sets the query limit to limit entities in the result.

Parameters:

limit (int) – The maximum number of entities per batch.

Returns:

Returns the query itself for chaining.

Return type:

Self

distinctOn(keyList)

Ensure only entities with distinct values on the fields listed are returned. This will implicitly override your SortOrder as all fields listed in keyList have to be sorted first.

Parameters:

keyList (List[str])

Return type:

Self

getCursor()

Get a valid cursor from the last run of this query.

The source of this cursor varies depending on what the last call was: - run(): A cursor that points immediately behind the

last result pulled off the returned iterator.

  • get(): A cursor that points immediately behind the

    last result in the returned list.

Returns:

A cursor that can be used in subsequent query requests or None if that query does not support cursors or there are no more elements to fetch

Return type:

Optional[str]

get_orders()

Get the orders from this query.

Returns:

The orders form this query as a list if there is no orders set it returns None

Return type:

List[Tuple[str, core.db.types.SortOrder]] | None

getKind()
Returns:

the current kind of this query. This may not be the kind this query has been constructed with as relational bones may rewrite this.

Return type:

str

_run_single_filter_query(query, limit)

Internal helper function that runs a single query definition on the datastore and returns a list of entities found. :param query: The querydefinition (filters, orders, distinct etc.) to run against the datastore :param limit: How many results should at most be returned :return: The first limit entities that matches this query

Parameters:
Return type:

List[core.db.types.Entity]

_merge_multi_query_results(input_result)

Merge the lists of entries into a single list; removing duplicates and restoring sort-order :param input_result: Nested Lists of Entries returned by each individual query run :return: Sorted & deduplicated list of entries

Parameters:

input_result (List[List[core.db.types.Entity]])

Return type:

List[core.db.types.Entity]

_resort_result(entities, filters, orders)

Internal helper that takes a (deduplicated) list of entities that has been fetched from different internal queries (the datastore does not support IN filters itself, so we have to query each item in that array separately) and resorts the list so it matches the query again.

Parameters:

  • entities (List[core.db.types.Entity]) – t.List of entities to resort

  • filters (Dict[str, core.db.types.DATASTORE_BASE_TYPES]) – The filter used in the query (used to determine implicit sort order by an inequality filter)

  • orders (List[Tuple[str, core.db.types.SortOrder]]) – The sort-orders to apply

Returns:

The sorted list

Return type:

List[core.db.types.Entity]

_fixKind(resultList)

Jump to parentKind if necessary (used in relations)

Parameters:

resultList (List[core.db.types.Entity])

Return type:

List[core.db.types.Entity]

run(limit=-1)

Run this query.

It is more efficient to use limit if the number of results is known.

If queried data is wanted as instances of Skeletons, fetch() should be used.

Parameters:

limit (int) – Limits the query to the defined maximum entities.

Returns:

The list of found entities

Raises:

BadFilterError if a filter string is invalid

Raises:

BadValueError if a filter value is invalid.

Raises:

BadQueryError if an IN filter in combination with a sort order on another property is provided

Return type:

List[core.db.types.Entity]

count(up_to=2**63 - 1)

The count operation cost one entity read for up to 1,000 index entries matched (https://cloud.google.com/datastore/docs/aggregation-queries#pricing) :param up_to can be sigend int 64 bit (max positive 2^31-1)

Returns:

Count entries for this query.

Parameters:

up_to (int)

Return type:

int

fetch(limit=-1)

Run this query and fetch results as core.skeleton.SkelList.

This function is similar to run(), but returns a core.skeleton.SkelList instance instead of Entities.

Warning:

The query must be limited!

Parameters:

limit (int)

Return type:

viur.core.skeleton.SkelList

If queried data is wanted as instances of Entity, run() should be used.

Parameters:

limit (int) – Limits the query to the defined maximum entities.

Raises:

BadFilterError if a filter string is invalid

Raises:

BadValueError if a filter value is invalid.

Raises:

BadQueryError if an IN filter in combination with a sort order on another property is provided

Return type:

viur.core.skeleton.SkelList

iter()

Run this query and return an iterator for the results.

The advantage of this function is, that it allows for iterating over a large result-set, as it hasn’t have to be pulled in advance from the datastore.

This function intentionally ignores a limit set by limit().

Warning:

If iterating over a large result set, make sure the query supports cursors. Otherwise, it might not return all results as the AppEngine doesn’t maintain the view for a query for more than ~30 seconds.

Return type:

Iterator[core.db.types.Entity]

getEntry()

Returns only the first entity of the current query.

Returns:

The first entity on success, or None if the result-set is empty.

Return type:

Union[None, core.db.types.Entity]

getSkel()

Returns a matching core.db.skeleton.Skeleton instance for the current query.

It’s only possible to use this function if this query has been created using core.skeleton.Skeleton.all().

Returns:

The Skeleton or None if the result-set is empty.

Return type:

Optional[viur.core.skeleton.SkeletonInstance]

clone()

Returns a deep copy of the current query.

Returns:

The cloned query.

Return type:

Self

__repr__()
Return type:

str