Shortcuts

modules.losses

Losses module contains implementations for various losses used generally in vision and language space. One can register custom losses to be detected by MMF using the following example.

from mmf.common.registry import registry
from torch import nn


@registry.register_loss("custom")
class CustomLoss(nn.Module):
    ...

Then in your model’s config you can specify losses attribute to use this loss in the following way:

model_config:
    some_model:
        losses:
            - type: custom
            - params: {}
class mmf.modules.losses.AttentionSupervisionLoss[source]

Loss for attention supervision. Used in case you want to make attentions similar to some particular values.

forward(sample_list, model_output)[source]

Calculates and returns the multi loss.

Parameters:
  • sample_list (SampleList) – SampleList containing targets attribute.
  • model_output (Dict) – Model output containing scores attribute.
Returns:

Float value for loss.

Return type:

torch.FloatTensor

class mmf.modules.losses.BCEAndKLLoss(weight_softmax)[source]

binary_cross_entropy_with_logits and kl divergence loss. Calculates both losses and returns a dict with string keys. Similar to bce_kl_combined, but returns both losses.

forward(sample_list, model_output)[source]

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class mmf.modules.losses.BinaryCrossEntropyLoss[source]
forward(sample_list, model_output)[source]

Calculates and returns the binary cross entropy.

Parameters:
  • sample_list (SampleList) – SampleList containing targets attribute.
  • model_output (Dict) – Model output containing scores attribute.
Returns:

Float value for loss.

Return type:

torch.FloatTensor

class mmf.modules.losses.CaptionCrossEntropyLoss[source]
forward(sample_list, model_output)[source]

Calculates and returns the cross entropy loss for captions.

Parameters:
  • sample_list (SampleList) – SampleList containing targets attribute.
  • model_output (Dict) – Model output containing scores attribute.
Returns:

Float value for loss.

Return type:

torch.FloatTensor

class mmf.modules.losses.CombinedLoss(weight_softmax)[source]
forward(sample_list, model_output)[source]

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class mmf.modules.losses.ContrastiveLoss[source]

This is a generic contrastive loss typically used for pretraining. No modality assumptions are made here.

forward(sample_list: Dict[str, torch.Tensor], model_output: Dict[str, torch.Tensor])[source]

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class mmf.modules.losses.CosineEmbeddingLoss[source]

Cosine embedding loss

forward(sample_list, model_output)[source]

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class mmf.modules.losses.CrossEntropyLoss(**params)[source]
forward(sample_list, model_output)[source]

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class mmf.modules.losses.InBatchHinge(margin: float = 0.0, hard: bool = False)[source]

Based on the code from https://github.com/fartashf/vsepp/blob/master/model.py

forward(sample_list: Dict[str, torch.Tensor], model_output: Dict[str, torch.Tensor])[source]

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class mmf.modules.losses.LabelSmoothingCrossEntropyLoss(label_smoothing=0.1, reduction='mean', ignore_index=-100)[source]

Cross-entropy loss with label smoothing. If label_smoothing = 0, then it’s canonical cross entropy. The smoothed one-hot encoding is 1 - label_smoothing for true label and label_smoothing / (num_classes - 1) for the rest.

Reference: https://stackoverflow.com/questions/55681502/label-smoothing-in-pytorch

forward(sample_list, model_output)[source]

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class mmf.modules.losses.LogitBinaryCrossEntropy[source]

Returns Binary Cross Entropy for logits.

Attention

Key: logit_bce

forward(sample_list, model_output)[source]

Calculates and returns the binary cross entropy for logits

Parameters:
  • sample_list (SampleList) – SampleList containing targets attribute.
  • model_output (Dict) – Model output containing scores attribute.
Returns:

Float value for loss.

Return type:

torch.FloatTensor

class mmf.modules.losses.LossConfig(type: str = '???', params: Dict[str, Any] = '???')[source]
class mmf.modules.losses.Losses(loss_list: List[Union[str, mmf.modules.losses.LossConfig]])[source]

Losses acts as an abstraction for instantiating and calculating losses. BaseModel instantiates this class based on the losses attribute in the model’s configuration model_config. loss_list needs to be a list for each separate loss containing type and params attributes.

Parameters:loss_list (ListConfig) – Description of parameter loss_list.

Example:

# losses:
# - type: logit_bce
# Can also contain `params` to specify that particular loss's init params
# - type: combined
config = [{"type": "logit_bce"}, {"type": "combined"}]
losses = Losses(config)

Note

Since, Losses is instantiated in the BaseModel, normal end user mostly doesn’t need to use this class.

losses

List containing instantiations of each loss passed in config

forward(sample_list: Dict[str, torch.Tensor], model_output: Dict[str, torch.Tensor])[source]

Takes in the original SampleList returned from DataLoader and model_output returned from the model and returned a Dict containing loss for each of the losses in losses.

Parameters:
  • sample_list (SampleList) – SampleList given be the dataloader.
  • model_output (Dict) – Dict returned from model as output.
Returns:

Dictionary containing loss value for each of the loss.

Return type:

Dict

class mmf.modules.losses.M4CDecodingBCEWithMaskLoss[source]
forward(sample_list, model_output)[source]

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class mmf.modules.losses.MMFLoss(params=None)[source]

Internal MMF helper and wrapper class for all Loss classes. It makes sure that the value returned from a Loss class is a dict and contain proper dataset type in keys, so that it is easy to figure out which one is the val loss and which one is train loss.

For example: it will return {"val/vqa2/logit_bce": 27.4}, in case logit_bce is used and SampleList is from val set of dataset vqa2.

Parameters:params (type) – Description of parameter params.

Note

Since, MMFLoss is used by the Losses class, end user doesn’t need to worry about it.

forward(sample_list: Dict[str, torch.Tensor], model_output: Dict[str, torch.Tensor])[source]

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class mmf.modules.losses.MSELoss[source]

Mean Squared Error loss

forward(sample_list, model_output)[source]

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class mmf.modules.losses.MultiLoss(params)[source]

A loss for combining multiple losses with weights.

Parameters:params (List(Dict)) – A list containing parameters for each different loss and their weights.

Example:

# MultiLoss works with config like below where each loss's params and
# weights are defined
losses:
- type: multi
  params:
  - type: logit_bce
    weight: 0.3
    params: {}
  - type: attention_supervision
    weight: 0.7
    params: {}
forward(sample_list, model_output, *args, **kwargs)[source]

Calculates and returns the multi loss.

Parameters:
  • sample_list (SampleList) – SampleList containing attentions attribute.
  • model_output (Dict) – Model output containing attention_supervision attribute.
Returns:

Float value for loss.

Return type:

torch.FloatTensor

class mmf.modules.losses.NLLLoss[source]

Negative log likelikehood loss.

forward(sample_list, model_output)[source]

Calculates and returns the negative log likelihood.

Parameters:
  • sample_list (SampleList) – SampleList containing targets attribute.
  • model_output (Dict) – Model output containing scores attribute.
Returns:

Float value for loss.

Return type:

torch.FloatTensor

class mmf.modules.losses.SoftLabelCrossEntropyLoss(ignore_index=-100, reduction='mean', normalize_targets=True)[source]
compute_loss(targets, scores)[source]

for N examples and C classes - scores: N x C these are raw outputs (without softmax/sigmoid) - targets: N x C or N corresponding targets

Target elements set to ignore_index contribute 0 loss.

Samples where all entries are ignore_index do not contribute to the loss reduction.

forward(sample_list, model_output)[source]

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class mmf.modules.losses.SoftmaxKlDivLoss[source]
forward(sample_list, model_output)[source]

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class mmf.modules.losses.TripleLogitBinaryCrossEntropy[source]

This is used for Three-branch fusion only. We predict scores and compute cross entropy loss for each of branches.

forward(sample_list, model_output)[source]

Calculates and returns the binary cross entropy for logits :param sample_list: SampleList containing targets attribute. :type sample_list: SampleList :param model_output: Model output containing scores attribute. :type model_output: Dict

Returns:Float value for loss.
Return type:torch.FloatTensor
class mmf.modules.losses.WeightedSoftmaxLoss[source]
forward(sample_list, model_output)[source]

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class mmf.modules.losses.WrongLoss[source]
forward(sample_list, model_output)[source]

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

Read the Docs v: latest
Versions
latest
stable
website
configuration_docs
Downloads
pdf
html
On Read the Docs
Project Home
Builds

Free document hosting provided by Read the Docs.