metarl.torch.embeddings package

PyTorch embedding modules for meta-learning algorithms.

class MLPEncoder(input_dim, output_dim, hidden_sizes, hidden_nonlinearity=<sphinx.ext.autodoc.importer._MockObject object>, hidden_w_init=<sphinx.ext.autodoc.importer._MockObject object>, hidden_b_init=<sphinx.ext.autodoc.importer._MockObject object>, output_nonlinearity=None, output_w_init=<sphinx.ext.autodoc.importer._MockObject object>, output_b_init=<sphinx.ext.autodoc.importer._MockObject object>, layer_normalization=False)[source]

Bases: metarl.torch.modules.mlp_module.MLPModule, metarl.np.embeddings.encoder.Encoder

This MLP network encodes context of RL tasks.

Context is stored in the terms of observation, action, and reward, and this network uses an MLP module for encoding it.

Parameters:
  • input_dim (int) – Dimension of the network input.
  • output_dim (int) – Dimension of the network output.
  • hidden_sizes (list[int]) – Output dimension of dense layer(s). For example, (32, 32) means this MLP consists of two hidden layers, each with 32 hidden units.
  • hidden_nonlinearity (callable or torch.nn.Module) – Activation function for intermediate dense layer(s). It should return a torch.Tensor.Set it to None to maintain a linear activation.
  • hidden_w_init (callable) – Initializer function for the weight of intermediate dense layer(s). The function should return a torch.Tensor.
  • hidden_b_init (callable) – Initializer function for the bias of intermediate dense layer(s). The function should return a torch.Tensor.
  • output_nonlinearity (callable or torch.nn.Module) – Activation function for output dense layer. It should return a torch.Tensor. Set it to None to maintain a linear activation.
  • output_w_init (callable) – Initializer function for the weight of output dense layer(s). The function should return a torch.Tensor.
  • output_b_init (callable) – Initializer function for the bias of output dense layer(s). The function should return a torch.Tensor.
  • layer_normalization (bool) – Bool for using layer normalization or not.
input_dim

Dimension of the encoder input.

Type:int
output_dim

Dimension of the encoder output (embedding).

Type:int
reset(do_resets=None)[source]

Reset the encoder.

This is effective only to recurrent encoder. do_resets is effective only to vectoried encoder.

For a vectorized encoder, do_resets is an array of boolean indicating which internal states to be reset. The length of do_resets should be equal to the length of inputs.

Parameters:do_resets (numpy.ndarray) – Bool array indicating which states to be reset.
spec

Input and output space.

Type:metarl.InOutSpec