Model

Thinc uses just one model class for almost all layer types. Instead of creating subclasses, you’ll usually instantiate the Model class directly, passing in a forward function that actually performs the computation. You can find examples of this in the library itself, in thinc.layers (also see the layers documentation).

ModelClass for implementing Thinc models and layers.
UtilitiesHelper functions for implementing models.
ShimInterface for external models. Users can create subclasses of Shim to wrap external libraries.

Model class

Class for implementing Thinc models and layers.

Typing

Model can be used as a generic type with two parameters: the expected input and expected output. For instance, Model[List[Floats2d], Floats2d] denotes a model that takes a list of two-dimensional arrays of floats as inputs and outputs a two-dimensional array of floats. A mismatch will cause a type error. For more details, see the docs on type checking.

from typing import List
from thinc.api import Model
from thinc.types import Floats2d

def my_function(model: Model[List[Floats2d], Floats2d]):
    ...

Attributes

NameTypeDescription
namestrThe name of the layer type.
opsOpsPerform array operations.
idintID number for the model instance.

Properties

NameTypeDescription
layersList[Model]A list of child layers of the model. You can use the list directly, including modifying it in-place: the standard way to add a layer to a model is simply model.layers.append(layer). However, you cannot reassign the model.layers attribute to a new variable: model.layers = [] will fail.
shimsList[Shim]A list of child shims added to the model.
attrsDict[str, Any]The model attributes. You can use the dict directly and assign to it – but you cannot reassign model.attrs to a new variable: model.attrs = {} will fail.
param_namesTuple[str,]Get the names of registered parameter (including unset).
grad_namesTuple[str,]Get the names of parameters with registered gradients (including unset).
dim_namesTuple[str,]Get the names of registered dimensions (including unset).
ref_namesTuple[str,]Get the names of registered node references (including unset).

Model.__init__ method

Initialize a new model.

Examplemodel = Model(
    "linear",
    linear_forward,
    init=linear_init,
    dims={"nO": nO, "nI": nI},
    params={"W": None, "b": None},
)
ArgumentTypeDescription
namestrThe name of the layer type.
forwardCallableFunction to compute the forward result and the backpropagation callback.
keyword-only
initCallableFunction to define the initialization logic.
dimsDict[str, Optional[int]]Dictionary describing the model’s dimensions. Map unknown dimensions to None.
paramsDict[str, Optional[FloatsXd]]Dictionary with the model’s parameters. Set currently unavailable parameters to None.
refsDict[str, Optional[Model]]Dictionary mapping specific nodes (sublayers) of the network to a name.
attrsDict[str, Any]Dictionary of non-parameter attributes.
layersList[Model]List of child layers.
shimsList[Shim]List of interfaces for external models.
opsOptional[Union[NumpyOps, AppleOps, CupyOps, MPSOps]]An Ops instance, which provides mathematical and memory operations.

Model.define_operators classmethodcontextmanager

Bind arbitrary binary functions to Python operators, for use in any Model instance. The method can (and should) be used as a contextmanager, so that the overloading is limited to the immediate block. This allows concise and expressive model definition. The following operators are supported: +, -, *, @, /, //, %, **, <<, >>, &, ^ and |.

Examplefrom thinc.api import Model, Relu, Softmax, chain

with Model.define_operators({">>": chain}):
    model = Relu(512) >> Relu(512) >> Softmax()
ArgumentTypeDescription
operatorsDict[str, Callable]Functions mapped to operators.

Model.initialize method

Finish initialization of the model, optionally providing a batch of example input and output data to perform shape inference. Until Model.initialize is called, the model may not be in a ready state to operate on data: parameters or dimensions may be unset. The Model.initialize method will usually delegate to the init function given to Model.__init__.

If sample data is provided, it will be validated against the type annotations of the forward function, if available. A DataValidationError is raised in case of a mismatch, e.g. if the forward pass expects a two-dimensional array but the model is initialized with a three-dimensional array.

Examplefrom thinc.api import Linear, zero_init
import numpy

X = numpy.zeros((128, 16), dtype="f")
Y = numpy.zeros((128, 10), dtype="f")
model = Linear(init_W=zero_init)
model.initialize(X=X, Y=Y)
ArgumentTypeDescription
XOptional[Any]An example batch of input data.
YOptional[Any]An example batch of output data.
RETURNSModelThe model instance.

Model.__call__ method

Call the model’s forward function, returning the output and a callback to compute the gradients via backpropagation.

Examplefrom thinc.api import Linear
import numpy

X = numpy.zeros((128, 10), dtype="f")
model = Linear(10)
model.initialize(X=X)
Y, backprop = model(X, is_train=True)
ArgumentTypeDescription
XAnyA batch of input data.
is_trainboolA boolean indicating whether the model is running in a training (as opposed to prediction) context.
RETURNSTuple[Any, Callable]A batch of output data and the backprop callback.

Model.begin_update method

Call the model’s forward function with is_train=True, and return the output and the backpropagation callback, which is a function that takes a gradient of outputs and returns the corresponding gradient of inputs. The backpropagation callback may also increment the gradients of the model parameters, via calls to Model.inc_grad.

Examplefrom thinc.api import Linear
import numpy

X = numpy.zeros((128, 10), dtype="f")
model = Linear(10)
model.initialize(X=X)
Y, backprop = model.begin_update(X)
ArgumentTypeDescription
XAnyA batch of input data.
RETURNSTuple[Any, Callable]A batch of output data and the backprop callback.

Model.predict method

Call the model’s forward function with is_train=False, and return only the output, instead of the (output, callback) tuple.

Examplefrom thinc.api import Linear
import numpy

X = numpy.zeros((128, 10), dtype="f")
model = Linear(10)
model.initialize(X=X)
Y = model.predict(X)
ArgumentTypeDescription
XAnyA batch of input data.
RETURNSAnyA batch of output data.

Model.finish_update method

Update parameters using the current parameter gradients. The Optimizer instance contains the functionality to perform the stochastic gradient descent.

Examplefrom thinc.api import Adam, Linear

optimizer = Adam()
model = Linear(10)
model.finish_update(optimizer)
ArgumentTypeDescription
optimizerOptimizerThe optimizer, which is called with each parameter and gradient of the model.

Model.use_params contextmanager

Contextmanager to temporarily set the model’s parameters to specified values.

ArgumentTypeDescription
paramsDict[int, FloatsXd]A dictionary keyed by model IDs, whose values are arrays of weight values.

Model.walk method

Iterate out layers of the model, in breadth-first order.

Examplefrom thinc.api import Relu

model = Relu(512, normalize=True)
for node in model.walk():
    print(node.name)

The walk method supports three iteration orders through the order argument:

  • "bfs": breadth-first. Iteration order of the example above: 1 - 2 - 4 - 3 - 5
  • "dfs_pre": depth-first preorder, outputs a node before its children. Iteration order of the example above: 1 - 2 - 3 - 4 - 5
  • "dfs_post": depth-first postorder, outputs children before a node itself. Iteration order of the example above: 3 - 2 - 5 - 4 - 1
ArgumentTypeDescription
orderstrNode iteration order. "bfs" (breadth-first), "dfs_pre" (depth-first preorder), "dfs_post" (depth-first postorder) Default: "bfs".
RETURNSIterable[Model]The layers of the model.

Model.remove_node method

Remove a node from all layers lists, and then update references. References that no longer point to a node within the tree will be set to None. For instance, if a node has its grandchild as a reference and the child is removed, the grandchild reference will be left dangling, so will be set to None.

ArgumentTypeDescription
nodeModelThe node to remove.

Model.has_dim method

Check whether the model has a dimension of a given name. If the dimension is registered but the value is unset, returns None.

Examplefrom thinc.api import Linear
import numpy

model = Linear(10)
assert model.has_dim("nI") is None
model.initialize(X=numpy.zeros((128, 16), dtype="f"))
assert model.has_dim("nI") is True
ArgumentTypeDescription
namestrThe name of the dimension, e.g. "nO".
RETURNSOptional[bool]A ternary value (True, False, or None).

Model.get_dim method

Retrieve the value of a dimension of the given name. Raises a KeyError if the dimension is either unregistered or the value is currently unset.

Examplefrom thinc.api import Linear
import numpy

model = Linear(10)
model.initialize(X=numpy.zeros((128, 16), dtype="f"))
assert model.get_dim("nI") == 16
ArgumentTypeDescription
namestrThe name of the dimension, e.g. "nO".
RETURNSintThe size of the dimension.

Model.maybe_get_dim method

Retrieve the value of a dimension of the given name, or None if the dimension is either unregistered or the value is currently unset.

ArgumentTypeDescription
namestrThe name of the dimension, e.g. "nO".
RETURNSOptional[int]The size of the dimension, or None.

Model.set_dim method

Set a value for a dimension. This raises a ValueError if the dimension was previously defined with a different value, unless force is set to True.

Examplefrom thinc.api import Linear

model = Linear(10)
model.set_dim("nI", 16)
assert model.get_dim("nI") == 16
ArgumentTypeDescription
namestrThe name of the dimension to set.
valueintThe new value for the dimension.
forceboolWhen set to True, allow changing the value of the dimension if it was set before. Default: False.

Model.has_param method

Check whether the model has a weights parameter of the given name. Returns None if the parameter is registered but currently unset.

Examplefrom thinc.api import Linear, zero_init
import numpy

model = Linear(10, init_W=zero_init)
assert model.has_param("W") is None
model.initialize(X=numpy.zeros((128, 16), dtype="f"))
assert model.has_param("W") is True
ArgumentTypeDescription
namestrThe name of the parameter.
RETURNSOptional[bool]A ternary value (True, False, or None).

Model.get_param method

Retrieve a weights parameter by name. Raises a KeyError if the parameter is unregistered or its value is undefined.

Examplefrom thinc.api import Linear, zero_init
import numpy

model = Linear(10, init_W=zero_init)
assert model.has_param("W") is None
model.initialize(X=numpy.zeros((128, 16), dtype="f"))
W = model.get_param("W")
assert W.shape == (10, 16)
ArgumentTypeDescription
namestrThe name of the parameter to get.
RETURNSFloatsXdThe current parameter.

Model.maybe_get_param method

Retrieve a weights parameter by name. Returns None if the parameter is unregistered or its value is undefined.

ArgumentTypeDescription
namestrThe name of the parameter to get.
RETURNSOptional[FloatsXd]The current parameter, or None.

Model.set_param method

Set a weights parameter’s value.

Examplefrom thinc.api import Linear, zero_init
import numpy

model = Linear(10, init_W=zero_init)
assert model.has_param("W") is None
model.set_param("W", numpy.zeros((10, 16), dtype="f"))
assert model.has_param("W") is True
ArgumentTypeDescription
namestrThe name of the parameter to set a value for.
valueOptional[FloatsXd]The new value of the parameter.

Model.has_ref method

Check whether the model has a reference of a given name. If the reference is registered but the value is unset, returns None.

ArgumentTypeDescription
namestrThe name of the reference.
RETURNSOptional[bool]A ternary value (True, False, or None).

Model.get_ref method

Retrieve the value of a reference of the given name. Raises a KeyError if unset.

ArgumentTypeDescription
namestrThe name of the reference.
RETURNSModelThe reference.

Model.maybe_get_ref method

Retrieve the value of a reference of the given name, or None if unset.

ArgumentTypeDescription
namestrThe name of the reference.
RETURNSOptional[Model]The reference, or None.

Model.set_ref method

Set a value for a reference.

ArgumentTypeDescription
namestrThe name of the reference.
valueOptional[Model]The new value for the attribute.

Model.has_grad method

Check whether the model has a non-zero gradient for the given parameter. If the gradient is allocated but is zeroed, returns None.

ArgumentTypeDescription
namestrThe parameter to check the gradient for.
RETURNSOptional[bool]A ternary value (True, False, or None).

Model.get_grad method

Get the gradient for a parameter, if one is available. If the parameter is undefined or no gradient has been allocated, raises a KeyError.

ArgumentTypeDescription
namestrThe name of the parameter to get the gradient for.
RETURNSFloatsXdThe current gradient of the parameter.

Model.maybe_get_grad method

Get the gradient for a parameter, if one is available. If the parameter is undefined or no gradient has been allocated, returns None.

ArgumentTypeDescription
namestrThe name of the parameter to get the gradient for.
RETURNSOptional[FloatsXd]The current gradient of the parameter, or None.

Model.set_grad method

Set a parameter gradient to a new value.

ArgumentTypeDescription
namestrThe name of the parameter to assign the gradient for.
valueFloatsXdThe new gradient.

Model.inc_grad method

Increment the gradient of a parameter by value.

ArgumentTypeDescription
namestrThe name of the parameter.
valueFloatsXdThe value to add to its gradient.

Model.get_gradients method

Get non-zero gradients of the model’s parameters, as a dictionary keyed by the parameter ID. The values are (weights, gradients) tuples.

ArgumentTypeDescription
RETURNSDict[Tuple[int, str], Tuple[FloatsXd, FloatsXd]]The gradients keyed by parameter ID.

Model.copy method

Create a copy of the model, its attributes, and its parameters. Any child layers will also be deep-copied. The copy will receive a distinct model.id value.

Examplefrom thinc.api import Linear

model = Linear()
model_copy = model.copy()
ArgumentTypeDescription
RETURNSModelA new copy of the model.

Model.to_gpu method

Transfer the model to a given GPU device.

Exampledevice = model.to_gpu(0)
ArgumentTypeDescription
gpu_idintDevice index to select.
RETURNScupy.cuda.DeviceThe device.

Model.to_cpu method

Copy the model to CPU.

Examplemodel.to_cpu()

Model.to_dict method

Serialize the model to a Python dictionary. Model.to_bytes delegates to this method to create the dict, which it then dumps with MessagePack. Serialization should round-trip identically, i.e. the same dict should result from loading and serializing a model.

Examplemodel_data = model.to_dict()
ArgumentTypeDescription
RETURNSdictThe serialized model.

Model.from_dict method

Load the model from a Python dictionary.

Examplemodel_data = model.to_dict()
model = Model("model_name", forward).from_dict(model_data)
ArgumentTypeDescription
msgdictThe data to load.
RETURNSModelThe loaded model.

Model.to_bytes method

Serialize the model to a bytes representation.

Examplebytes_data = model.to_bytes()
ArgumentTypeDescription
RETURNSbytesThe serialized model.

Model.from_bytes method

Deserialize the model from a bytes representation.

Examplebytes_data = model.to_bytes()
model = Model("model_name", forward).from_bytes(bytes_data)
ArgumentTypeDescription
bytes_databytesThe bytestring to load.
RETURNSModelThe loaded model.

Model.to_disk method

Serialize the model to disk. Most models will serialize to a single file, which should just be the bytes contents of Model.to_bytes.

Examplemodel.to_disk("/path/to/model")
ArgumentTypeDescription
 pathUnion[Path, str]File or directory to save the model to.

Model.from_disk method

Deserialize the model from disk. Most models will serialize to a single file, which should just be the bytes contents of Model.to_bytes.

Examplemodel = Model().from_disk("/path/to/model")
ArgumentTypeDescription
 pathUnion[Path, str]Directory to load the model from.
RETURNSModelThe loaded model.

Model.can_from_bytes method

Check whether bytes data is compatible with the model for deserialization.

ArgumentTypeDescription
 bytes_databytesThe bytestring to check.
 strictboolWhether to require attributes to match.
RETURNSboolWhether the data is compatible.

Model.can_from_bytes method

Check whether a path is compatible with the model for deserialization.

ArgumentTypeDescription
 pathUnion[Path, str]The path to check.
 strictboolWhether to require attributes to match.
RETURNSModelWhether the path is compatible.

Model.can_from_dict method

Check whether a dictionary is compatible with the model for deserialization.

ArgumentTypeDescription
msgdictThe data to check.
strictboolWhether to require attributes to match.
RETURNSModelWhether the data is compatible.

Utilities

serialize_attr functionsingle-dispatch

Single-dispatch generic function that serializes a model attribute in Model.attrs to bytes and can be customized to support other objects and data types. By default, the function uses MessagePack to serialize the attribute value to bytes. To register a serialization function for a custom type, you can use the @serialize_attr.register decorator and call it with the custom type. If an attribute of that type exists on a model, the registered function will be used to serialize it.

Examplefrom thinc.api import serialize_attr

@serialize_attr.register(MyCustomClass)
def serialize_my_custom_class(_, value: MyCustomClass, name: str, model) -> bytes:
    # value is an instance of MyCustomClass that needs to be serialized. You
    # can perform any custom serialization here and return bytes
    return value.custom_serialization_method()
ArgumentTypeDescription
_AnyAn instance of the value to serialize. Its type will be used to determine which registered serialization function to apply.
valueAnyThe value to serialize.
namestrThe attribute name.
modelModelThe model that’s being serialized, e.g. to retrieve other information.
RETURNSbytesThe serialized attribute.

deserialize_attr functionsingle-dispatch

Single-dispatch generic function that deserializes a model attribute in Model.attrs from bytes and can be customized to support other objects and data types. By default, the function uses MessagePack to load the attribute value from bytes. To register a deserialization function for a custom type, you can use the @deserialize_attr.register decorator and call it with the custom type. If an attribute of that type exists on a model, the registered function will be used to deserialize it.

Examplefrom thinc.api import deserialize_attr

@deserialize_attr.register(MyCustomClass)
def deserialize_my_custom_class(_, value: bytes, name: str, model) -> MyCustomClass:
    # value is a bytestring that needs to be deserialized and transformed into
    # MyCustomClass. You can perform any custom deserialization here and return
    # an instance of MyCustomClass.
    return MyCustomClass().custom_load_method(value)
ArgumentTypeDescription
_AnyAn instance of the value to deserialize (the default value of the attribute). Its type will be used to determine which registered deserialization function to apply.
valuebytesThe bytestring to load.
namestrThe attribute name.
modelModelThe model that’s being deserialized, e.g. to perform other side-effects.
RETURNSAnyThe loaded attribute.

Shim class

Define a basic interface for external models. Users can create subclasses of Shim to wrap external libraries. The Thinc Model class treats Shim objects as a sort of special type of sublayer: it knows they’re not actual Thinc Model instances, but it also knows to talk to the shim instances when doing things like using transferring between devices, loading in parameters, optimization. It also knows Shim objects need to be serialized and deserialized with to/from bytes/disk, rather than expecting that they’ll be msgpack-serializable. A Shim can implement the following methods:

MethodDescription
 __init__Initialize the model.
 __call__Call the model and return the output and a callback to compute the gradients via backpropagation.
 predictCall the model and return only the output, instead of the (output, callback) tuple.
 begin_updateRun the model over a batch of data, returning the output and a callback to complete the backward pass.
 finish_updateUpdate parameters with current gradients.
 use_paramsContext manager to temporarily set the model’s parameters to specified values.
 to_gpuTransfer the model to a given GPU device.
 to_cpuCopy the model to CPU.
 to_bytesSerialize the model to bytes.
 from_bytesLoad the model from bytes.
 to_diskSerialize the model to disk. Defaults to writing the bytes representation to a file.
 from_diskLoad the model from disk. Defaults to loading the byte representation from a file.

Available shims

PyTorchShimInterface between a PyTorch model and a Thinc Model. For more details and examples, see the PyTorchWrapper layer and docs on integrating other frameworks.
TorchScriptShimInterface between a TorchScript model and a Thinc Model. For more details and examples, see the TorchScriptWrapper layer and docs on integrating other frameworks.
TensorFlowShimInterface between a TensorFlow model and a Thinc Model. For more details, see the TensorFlowWrapper layer and docs on integrating other frameworks
MXNetShimInterface between a MXNet model and a Thinc Model. For more details, see the MXNetWrapper layer and docs on integrating other frameworks