core.modules.file¶

Attributes¶

`VALID_FILENAME_REGEX`
`GOOGLE_STORAGE_CLIENT`
`PRIVATE_BUCKET_NAME`
`PUBLIC_BUCKET_NAME`
`PUBLIC_DLKEY_SUFFIX`
`_private_bucket`
`_public_bucket`

Classes¶

`FilePath`
`DownloadUrlBone`	This bone is used to inject a freshly signed download url into a FileSkel.
`FileLeafSkel`	Default file leaf skeleton.
`FileNodeSkel`	Default file node skeleton.
`File`	Tree module prototype.

Functions¶

`importBlobFromViur2`(dlKey, fileName)
`thumbnailer`(fileSkel, existingFiles, params)
`cloudfunction_thumbnailer`(fileSkel, existingFiles, params)	External Thumbnailer for images.
`startCheckForUnreferencedBlobs`()	Start searching for blob locks that have been recently freed
`doCheckForUnreferencedBlobs`([cursor])
`startCleanupDeletedFiles`()	Increase deletion counter on each blob currently not referenced and delete
`doCleanupDeletedFiles`([cursor])
`start_delete_pending_files`()	Start deletion of pending FileSkels that are older than 7 days.
`__getattr__`(attr)

Module Contents¶

core.modules.file.VALID_FILENAME_REGEX¶

core.modules.file.GOOGLE_STORAGE_CLIENT¶

core.modules.file.PRIVATE_BUCKET_NAME = 'Uninferable.appspot.com'¶

core.modules.file.PUBLIC_BUCKET_NAME = 'public-dot-Uninferable'¶

core.modules.file.PUBLIC_DLKEY_SUFFIX = '_pub'¶

core.modules.file._private_bucket¶

core.modules.file._public_bucket = None¶

class core.modules.file.FilePath¶

Bases: tuple

dlkey¶

is_derived¶

filename¶

core.modules.file.importBlobFromViur2(dlKey, fileName)¶

core.modules.file.thumbnailer(fileSkel, existingFiles, params)¶

core.modules.file.cloudfunction_thumbnailer(fileSkel, existingFiles, params)¶

External Thumbnailer for images.

The corresponding cloudfunction can be found here . https://github.com/viur-framework/viur-cloudfunctions/tree/main/thumbnailer

You can use it like so: main.py:

from viur.core.modules.file import cloudfunction_thumbnailer

conf.file_thumbnailer_url = "https://xxxxx.cloudfunctions.net/imagerenderer"
conf.file_derivations = {"thumbnail": cloudfunction_thumbnailer}

conf.derives_pdf = {
    "thumbnail": [{"width": 1920,"sites":"1,2"}]
}

skeletons/xxx.py: .. code-block:: python

test = FileBone(derive=conf.derives_pdf)

class core.modules.file.DownloadUrlBone(*, compute=None, defaultValue=None, descr=None, getEmptyValueFunc=None, indexed=True, isEmptyFunc=None, languages=None, multiple=False, params=None, readOnly=None, required=False, searchable=False, type_suffix='', unique=None, vfunc=None, visible=True, clone_behavior=None)¶

Bases: viur.core.bones.BaseBone

This bone is used to inject a freshly signed download url into a FileSkel.

Initializes a new Bone.

Parameters:

compute (Compute)
defaultValue (Any)
descr (Optional[str | viur.core.i18n.translate])
getEmptyValueFunc (callable)
indexed (bool)
isEmptyFunc (callable)
languages (None | list[str])
multiple (bool | MultipleConstraints)
params (dict)
readOnly (bool)
required (bool | list[str] | tuple[str])
searchable (bool)
type_suffix (str)
unique (None | UniqueValue)
vfunc (callable)
visible (bool)
clone_behavior (CloneBehavior | CloneStrategy | None)

unserialize(skel, name)¶

Deserialize bone data from the datastore and populate the bone with the deserialized values.

This function is the inverse of the serialize function. It converts data from the datastore into a format that can be used by the bones in the skeleton.

Parameters:

skel – A SkeletonInstance object containing the values to be deserialized.
name – The property name of the bone in its Skeleton (not the description).

Returns:

True if deserialization is successful, False otherwise.

class core.modules.file.FileLeafSkel(*args, **kwargs)¶

Bases: viur.core.prototypes.tree.TreeSkel

Default file leaf skeleton.

kindName = 'file'¶: Specifies the entity kind name this Skeleton is associated with. Will be determined automatically when not explicitly set.

name¶

alt¶

size¶

dlkey¶

mimetype¶

weak¶

pending¶

width¶

height¶

downloadUrl¶

derived¶

pendingparententry¶

crc32c_checksum¶

md5_checksum¶

public¶

serving_url¶

classmethod _inject_serving_url(skel)¶

Inject the serving url for public image files into a FileSkel

Parameters:: skel (viur.core.skeleton.SkeletonInstance)
Return type:: None

preProcessBlobLocks(locks)¶: Ensure that our dlkey is locked even if we don’t have a filebone here

classmethod refresh(skel)¶

Refresh the bones current content.

This function causes a refresh of all relational bones and their associated information.

classmethod write(skel, **kwargs)¶

Write current Skeleton to the datastore.

Stores the current data of this instance into the database. If an key value is set to the object, this entity will ne updated; Otherwise a new entity will be created.

To read a Skeleton object from the data store, see read().

Parameters:

key – Allows to specify a key that is set to the skeleton and used for writing.
update_relations – If False, this entity won’t be marked dirty; This avoids from being fetched by the background task updating relations.

Returns:

The Skeleton.

class core.modules.file.FileNodeSkel(*args, **kwargs)¶

Bases: viur.core.prototypes.tree.TreeSkel

Default file node skeleton.

kindName = 'file_rootNode'¶: Specifies the entity kind name this Skeleton is associated with. Will be determined automatically when not explicitly set.

name¶

rootNode¶

public¶

viurCurrentSeoKeys = None¶

class core.modules.file.File(moduleName, modulePath, *args, **kwargs)¶

Bases: viur.core.prototypes.tree.Tree

Tree module prototype.

It is used for hierarchical structures, either as a tree with nodes and leafs, or as a hierarchy with nodes only.

PENDING_POSTFIX = ' (pending)'¶

DOWNLOAD_URL_PREFIX = '/file/download/'¶

INTERNAL_SERVING_URL_PREFIX = '/file/serve/'¶

MAX_FILENAME_LEN = 256¶

IMAGE_META_MAX_SIZE: viur.core.decorators.t.Final[int] = 10485760¶: Maximum size of image files that should be analysed in set_image_meta(). Default: 10 MiB

leafSkelCls¶

nodeSkelCls¶

handler = 'tree.simple.file'¶: This is the module’s handler, respectively its type. Use the @property-decorator in specific Modules to construct the handler’s value dynamically. A module without a handler setting cannot be described, so cannot be handled by admin-tools.

adminInfo¶

This is a dict holding the information necessary for the Vi/Admin to handle this module.

name: str
Human-readable module name that will be shown in the admin tool.

handler: str (list, tree or singleton):
Allows to override the handler provided by the module. Set this only when really necessary, otherwise it can be left out and is automatically injected by the Module’s prototype.

icon: str
(Optional) Either the Shoelace icon library name or a path relative to the project’s deploy folder (e.g. /static/icons/viur.svg) for the icon used in the admin tool for this module.

columns: List[str]
(Optional) List of columns (bone names) that are displayed by default. Used only by the List handler.

filter: Dict[str, str]
(Optional) Dictionary of additional parameters that will be send along when fetching entities from the server. Can be used to filter the entities being displayed on the client-side.

display: str (“default”, “hidden” or “group”)
(Optional) “hidden” will hide the module in the admin tool’s main bar. (itwill not be accessible directly, however it’s registered with the frontend so it can be used in a relational bone). “group” will show this module in the main bar, but it will not be clickable. Clicking it will just try to expand it (assuming there are additional views defined).

preview: Union[str, Dict[str, str]]
(Optional) A url that will be opened in a new tab and is expected to display the entity selected in the table. Can be “/{{module}}/view/{{key}}”, with {{module}} and {{key}} getting replaced as needed. If more than one preview-url is needed, supply a dictionary where the key is the URL and the value the description shown to the user.

views: List[Dict[str, t.Any]]
(Optional) List of nested adminInfo like dictionaries. Used to define additional views on the module. Useful f.e. for an order module, where you want separate list of “payed orders”, “unpayed orders”, “orders waiting for shipment”, etc. If such views are defined, the top-level entry in the menu bar will expand if clicked, revealing these additional filters.

actions: List[str]
(Optional) List of actions supported by this modules. Actions can be defined by the frontend (like “add”, “edit”, “delete” or “preview”); it can be an action defined by a plugin loaded by the frontend; or it can be a so called “server side action” (see “customActions” below)

customActions: Dict[str, dict]
(Optional) A mapping of names of server-defined actions that can be used in the actions list above to their definition dictionary. See …. for more details.

disabledActions: List[str, dict]
(Optional) A list of disabled actions. The frontend will inject default actions like add or edit even if they’re not listed in actions. Listing them here will prevent that. It’s up to the frontend to decide if that action won’t be visible at all or it’s button just being disabled.

sortIndex: int
(Optional) Defines the order in which the modules will appear in the main bar in ascrending order.

indexedBones: List[str]
(Optional) List of bones, for which an (composite?) index exists in this view. This allows the fronted to signal the user that a given list can be sorted or filtered by this bone. If no additional filters are enforced by the listFilter and filter is not set, this should be all bones which are marked as indexed.

changeInvalidates: List[str]
(Optional) A list of module-names which depend on the entities handled from this module. This allows the frontend to invalidate any caches in these depended modules if the data in this module changes. Example: This module may be a list-module handling the file_rootNode entities for the file module, so a edit/add/deletion action on this module should be reflected in the rootNode-selector in the file-module itself. In this case, this property should be set to ["file"].

moduleGroup: str
(Optional) If set, should be a key of a moduleGroup defined in …. .

editViews: Dict[str, t.Any]
(Optional) If set, will embed another list-widget in the edit forms for a given entity. See …. for more details.

If this is a function, it must take no parameters and return the dictionary as shown above. This can be used to customize the appearance of the Vi/Admin to individual users.

roles¶

Allows to specify role settings for a module.

Defaults to no role definition, which ignores the module entirely in the role-system. In this case, access rights can still be set individually on the user’s access bone.

A “*” wildcard can either be used as key or as value to allow for “all roles”, or “all rights”.

# Example
roles = {
    "*": "view",                # Any role may only "view"
    "editor": ("add", "edit"),  # Role "editor" may "add" or "edit", but not "delete"
    "admin": "*",               # Role "admin" can do everything
}

default_order = 'name'¶

Allows to specify a default order for this module, which is applied when no other order is specified.

Setting a default_order might result in the requirement of additional indexes, which are being raised and must be specified.

static get_bucket(dlkey)¶

Retrieves a Google Cloud Storage bucket for the given dlkey.

Parameters:: dlkey (str)
Return type:: google.cloud.storage.bucket.Bucket

classmethod is_valid_filename(filename)¶

Verifies a valid filename.

The filename should be valid on Linux, Mac OS and Windows. It should not be longer than MAX_FILENAME_LEN chars.

Rule set: https://stackoverflow.com/a/31976060/3749896 Regex test: https://regex101.com/r/iBYpoC/1

Parameters:: filename (str)
Return type:: bool

static hmac_sign(data)¶

Parameters:: data (viur.core.decorators.t.Any)
Return type:: str

classmethod hmac_verify(data, signature)¶

Parameters:

data (viur.core.decorators.t.Any)
signature (str)

Return type:

bool

classmethod create_internal_serving_url(serving_url, size=0, filename='', options='', download=False)¶

Helper function to generate an internal serving url (endpoint: /file/serve) from a Google serving url.

This is needed to hide requests to Google as they are internally be routed, and can be the result of a legal requirement like GDPR.

Parameters:

serving_url (str) – Is the original serving URL as generated from FileLeafSkel._inject_serving_url()
size (int) – Optional size setting
filename (str) – Optonal filename setting
options (str) – Additional options parameter-pass through to /file/serve
download (bool) – Download parameter-pass through to /file/serve

Return type:

str

classmethod create_download_url(dlkey, filename, derived=False, expires=datetime.timedelta(hours=1), download_filename=None)¶

Utility function that creates a signed download-url for the given folder/filename combination

Parameters:

folder – The GCS-Folder (= the download-key) for that file
filename (str) – The name of the file. Either the original filename or the name of a derived file.
derived (bool) – True, if it points to a derived file, False if it points to the original uploaded file
expires (viur.core.decorators.t.Optional[datetime.timedelta | int]) – None if the file is supposed to be public (which causes it to be cached on the google ede caches), otherwise a datetime.timedelta of how long that link should be valid
download_filename (viur.core.decorators.t.Optional[str]) – If set, browser is enforced to download this blob with the given alternate filename
dlkey (str)

Returns:

The signed download-url relative to the current domain (eg /download/…)

Return type:

str

classmethod parse_download_url(url)¶

Parses a file download URL in the format /file/download/xxxx?sig=yyyy into its FilePath.

If the URL cannot be parsed, the function returns None.

Parameters:: url – The file download URL to be parsed.
Returns:: A FilePath on success, None otherwise.
Return type:: viur.core.decorators.t.Optional[FilePath]

classmethod create_src_set(file, expires=datetime.timedelta(hours=1), width=None, height=None, language=None)¶

Generates a string suitable for use as the srcset tag in html. This functionality provides the browser with a list of images in different sizes and allows it to choose the smallest file that will fill it’s viewport without upscaling.

Parameters:

file (viur.core.decorators.t.Union[viur.core.skeleton.SkeletonInstance, dict, str]) – The file skeleton (or if multiple=True a single value from it) to generate the srcset.
expires (viur.core.decorators.t.Optional[datetime.timedelta | int]) – None if the file is supposed to be public (which causes it to be cached on the google edecaches), otherwise it’s lifetime in seconds
width (viur.core.decorators.t.Optional[int]) – A list of widths that should be included in the srcset. If a given width is not available, it will be skipped.
height (viur.core.decorators.t.Optional[int]) – A list of heights that should be included in the srcset. If a given height is not available, it will be skipped.
language (viur.core.decorators.t.Optional[str]) – Language overwrite if file has multiple languages, and we want to explicitly specify one

Returns:

The srctag generated or an empty string if a invalid file object was supplied

Return type:

str

write(filename, content, mimetype='text/plain', *, width=None, height=None, public=False, rootnode=None, folder=())¶

Write a file from any bytes-like object into the file module.

If folder and rootnode are both set, the file is added to the repository in that folder. If only folder is set, the file is added to the default repository in that folder. If only rootnode is set, the file is added to that repository in the root folder.

If both are not set, the file is added without a path or repository as a weak file. It will not be visible in admin in this case.

Parameters:

filename (str) – Filename to be written.
content (viur.core.decorators.t.Any) – The file content to be written, as bytes-like object.
mimetype (str) – The file’s mimetype.
width (int) – Optional width information for the file.
height (int) – Optional height information for the file.
public (bool) – True if the file should be publicly accessible.
rootnode (viur.core.decorators.t.Optional[viur.core.db.Key]) – Optional root-node of the repository to add the file to
folder (viur.core.decorators.t.Iterable[str] | str) – Optional folder the file should be written into.

Returns:

Returns the key of the file object written. This can be associated e.g. with a FileBone.

Return type:

viur.core.db.Key

read(key=None, path=None)¶

Read a file from the Cloud Storage.

If a key and a path are provided, the key is preferred. This means that the entry in the db is searched first and if this is not found, the path is used.

Parameters:

key (viur.core.db.Key | int | str | None) – Key of the LeafSkel that contains the “dlkey” and the “name”.
path (str | None) – The path of the file in the Cloud Storage Bucket.

Returns:

Returns the file as a io.BytesIO buffer and the content-type

Return type:

tuple[io.BytesIO, str]

deleteRecursive(parentKey)¶

Recursively processes a delete request.

This will delete all entries which are children of nodeKey, except key nodeKey.

Parameters:: parentKey – URL-safe key of the node which children should be deleted.

getUploadURL(fileName, mimeType, size=None, node=None, authData=None, authSig=None, public=False)¶

Parameters:

fileName (str)
mimeType (str)
size (viur.core.decorators.t.Optional[int])
node (viur.core.decorators.t.Optional[str | viur.core.db.Key])
authData (viur.core.decorators.t.Optional[str])
authSig (viur.core.decorators.t.Optional[str])
public (bool)

download(blobKey, fileName='', download=False, sig='', *args, **kwargs)¶

Download a file. :param blobKey: The unique blob key of the file. :param fileName: Optional filename to provide in the header. :param download: Set header to attachment retrival, set explictly to “1” if download is wanted.

Parameters:

blobKey (str)
fileName (str)
download (bool)
sig (str)

SERVE_VALID_OPTIONS¶: Valid modification option shorts for the serve-function. This is passed-through to the Google UserContent API, and hast to be supported there.

SERVE_VALID_FORMATS¶: Valid file-formats to the serve-function. This is passed-through to the Google UserContent API, and hast to be supported there.

serve(host, key, size=None, filename=None, options='', download=False)¶

Requests an image using the serving url to bypass direct Google requests.

Parameters:

host (str) – the google host prefix i.e. lh3
key (str) – the serving url key
size (viur.core.decorators.t.Optional[int]) – the target image size
filename (viur.core.decorators.t.Optional[str]) – a random string with an extention, valid extentions are (defined in File.SERVE_VALID_FORMATS).
options (str) –
- seperated options (defined in File.SERVE_VALID_OPTIONS).
c - crop p - face crop fv - vertrical flip fh - horizontal flip rXXX - rotate 90, 180, 270 nu - no upscale
download (bool) – Serves the content as download (Content-Disposition) or not.

Returns:

Returns the requested content on success, raises a proper HTTP exception otherwise.

add(skelType, node=None, *args, **kwargs)¶

Add a new entry with the given parent node, and render the entry, eventually with error notes on incorrect data. Data is taken by any other arguments in kwargs.

The function performs several access control checks on the requested entity before it is added.