Loss Calculators

All loss calculators follow the same API: they’re classes that are initialized with optional settings and have a get_grad method returning the gradient of the loss with respect to the model outputs and a get_loss method returning the scalar loss.

Loss base class

Loss.__init__ method

Initialize the loss calculator.

ArgumentTypeDescription
**kwargsAnyOptional calculator-specific settings. Can also be provided via the config.

Loss.__call__ method

Calculate the gradient and the scalar loss. Returns a tuple of the results of Loss.get_grad and Loss.get_loss.

ArgumentTypeDescription
guessesAnyThe model outputs.
truthsAnyThe training labels.
RETURNSTuple[Any, Any]The gradient and scalar loss.

Loss.get_grad method

Calculate the gradient of the loss with respect with the model outputs.

ArgumentTypeDescription
guessesAnyThe model outputs.
truthsAnyThe training labels.
RETURNSAnyThe gradient.

Loss.get_loss method

Calculate the scalar loss. Typically returns a float.

ArgumentTypeDescription
guessesAnyThe model outputs.
truthsAnyThe training labels.
RETURNSAnyThe scalar loss.

Loss Calculators

CategoricalCrossentropy class

from thinc.api import CategoricalCrossentropy
loss_calc = CategoricalCrossentropy()
config.cfg[loss]
@losses = "CategoricalCrossentropy.v1"
normalize = true
ArgumentType Description
keyword-only
normalizeboolNormalize and divide by number of examples given.

SequenceCategoricalCrossentropy class

from thinc.api import SequenceCategoricalCrossentropy
loss_calc = SequenceCategoricalCrossentropy()
config.cfg[loss]
@losses = "SequenceCategoricalCrossentropy.v1"
normalize = true
ArgumentType Description
keyword-only
normalizeboolNormalize and divide by number of examples given.

L2Distance class

from thinc.api import L2Distance
loss_calc = L2Distance(margin=0.2)
config.cfg[loss]
@losses = "L2Distance.v1"
normalize = true
ArgumentType Description
keyword-only
normalizeboolNormalize and divide by number of examples given.

CosineDistance function

from thinc.api import CosineDistance
loss_calc = CosineDistance(ignore_zeros=False)
config.cfg[loss]
@losses = "CosineDistance.v1"
normalize = true
ignore_zeros = false
ArgumentType Description
keyword-only
normalizeboolNormalize and divide by number of examples given.
ignore_zerosboolDon’t count zero vectors.

Usage via config and function registry

Defining the loss calculators in the config will return the initialized object. Within your script, you can then call it or its methods and pass in the data.

config.cfg[loss]
@losses = "L2Distance.v1"
normalize = true
Usagefrom thinc.api import registry, Config

config = Config().from_disk("./config.cfg")
C = registry.make_from_config(config)
loss_calc = C["loss"]
loss = loss_calc.get_grad(guesses, truths)