harl.models.base package

Submodules

harl.models.base.act module

class harl.models.base.act.ACTLayer(action_space, inputs_dim, initialization_method, gain, args=None)

Bases: Module

MLP Module to compute actions.

evaluate_actions(x, action, available_actions=None, active_masks=None)

Compute action log probability, distribution entropy, and action distribution.

Parameters:

• x – (torch.Tensor) input to network.

• action – (torch.Tensor) actions whose entropy and log probability to evaluate.

• available_actions – (torch.Tensor) denotes which actions are available to agent (if None, all actions available)

• active_masks – (torch.Tensor) denotes whether an agent is active or dead.

Returns:

(torch.Tensor) log probabilities of the input actions. dist_entropy: (torch.Tensor) action distribution entropy for the given inputs. action_distribution: (torch.distributions) action distribution.

Return type:

action_log_probs

forward(x, available_actions=None, deterministic=False)

Compute actions and action logprobs from given input.

Parameters:

• x – (torch.Tensor) input to network.

• available_actions – (torch.Tensor) denotes which actions are available to agent (if None, all actions available)

• deterministic – (bool) whether to sample from action distribution or return the mode.

Returns:

(torch.Tensor) actions to take. action_log_probs: (torch.Tensor) log probabilities of taken actions.

Return type:

actions

get_logits(x, available_actions=None)

Get action logits from inputs.

Parameters:

• x – (torch.Tensor) input to network.

• available_actions – (torch.Tensor) denotes which actions are available to agent (if None, all actions available)

Returns:

(torch.Tensor) logits of actions for the given inputs.

Return type:

action_logits

harl.models.base.cnn module

class harl.models.base.cnn.CNNBase(args, obs_shape)

Bases: Module

A CNN base module for actor and critic.

forward(x)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class harl.models.base.cnn.CNNLayer(obs_shape, hidden_sizes, initialization_method, activation_func, kernel_size=3, stride=1)

Bases: Module

forward(x)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

harl.models.base.distributions module

Modify standard PyTorch distributions so they to make compatible with this codebase.

class harl.models.base.distributions.Categorical(num_inputs, num_outputs, initialization_method='orthogonal_', gain=0.01)

Bases: Module

A linear layer followed by a Categorical distribution.

forward(x, available_actions=None)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class harl.models.base.distributions.DiagGaussian(num_inputs, num_outputs, initialization_method='orthogonal_', gain=0.01, args=None)

Bases: Module

A linear layer followed by a Diagonal Gaussian distribution.

forward(x, available_actions=None)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class harl.models.base.distributions.FixedCategorical(probs=None, logits=None, validate_args=None)

Bases: Categorical

Modify standard PyTorch Categorical.

log_probs(actions)

mode()

Returns the mode of the distribution.

sample()

Generates a sample_shape shaped sample or sample_shape shaped batch of samples if the distribution parameters are batched.

class harl.models.base.distributions.FixedNormal(loc, scale, validate_args=None)

Bases: Normal

Modify standard PyTorch Normal.

entropy()

Method to compute the entropy using Bregman divergence of the log normalizer.

log_probs(actions)

mode()

Returns the mode of the distribution.

harl.models.base.flatten module

class harl.models.base.flatten.Flatten(*args, **kwargs)

Bases: Module

forward(x)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

harl.models.base.mlp module

class harl.models.base.mlp.MLPBase(args, obs_shape)

Bases: Module

A MLP base module.

forward(x)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class harl.models.base.mlp.MLPLayer(input_dim, hidden_sizes, initialization_method, activation_func)

Bases: Module

forward(x)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

harl.models.base.plain_cnn module

class harl.models.base.plain_cnn.PlainCNN(obs_shape, hidden_size, activation_func, kernel_size=3, stride=1)

Bases: Module

Plain CNN

forward(x)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

harl.models.base.plain_mlp module

class harl.models.base.plain_mlp.PlainMLP(sizes, activation_func, final_activation_func='identity')

Bases: Module

Plain MLP

forward(x)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

harl.models.base.rnn module

class harl.models.base.rnn.RNNLayer(inputs_dim, outputs_dim, recurrent_n, initialization_method)

Bases: Module

forward(x, hxs, masks)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

Module contents