metarl.experiment.task_sampler module

Efficient and general interfaces for sampling tasks for Meta-RL.

class ConstructEnvsSampler(env_constructors)

Bases: metarl.experiment.task_sampler.TaskSampler

TaskSampler where each task has its own constructor.

Generally, this is used when the different tasks are completely different environments.

Parameters:env_constructors (list[Callable[gym.Env]]) – Callables that produce environments (for example, environment types).

n_tasks

the number of tasks.

Type:int

sample(n_tasks, with_replacement=False)

Sample a list of environment updates.

Parameters:

• n_tasks (int) – Number of updates to sample.
• with_replacement (bool) – Whether tasks can repeat when sampled. Note that if more tasks are sampled than exist, then tasks may repeat, but only after every environment has been included at least once in this batch. Ignored for continuous task spaces.

Returns:

Batch of sampled environment updates, which, when

invoked on environments, will configure them with new tasks. See EnvUpdate for more information.

Return type:

list[EnvUpdate]

class EnvPoolSampler(envs)

Bases: metarl.experiment.task_sampler.TaskSampler

TaskSampler that samples from a finite pool of environments.

This can be used with any environments, but is generally best when using in-process samplers with environments that are expensive to construct.

Parameters:envs (list[gym.Env]) – List of environments to use as a pool.

grow_pool(new_size)

Increase the size of the pool by copying random tasks in it.

Note that this only copies the tasks already in the pool, and cannot create new original tasks in any way.

Parameters:new_size (int) – Size the pool should be after growning.

n_tasks

the number of tasks.

Type:int

sample(n_tasks, with_replacement=False)

Sample a list of environment updates.

Parameters:

• n_tasks (int) – Number of updates to sample.
• with_replacement (bool) – Whether tasks can repeat when sampled. Since this cannot be easily implemented for an object pool, setting this to True results in ValueError.

Raises:

ValueError – If the number of requested tasks is larger than the pool, or with_replacement is set.

Returns:

Batch of sampled environment updates, which, when

invoked on environments, will configure them with new tasks. See EnvUpdate for more information.

Return type:

list[EnvUpdate]

class SetTaskSampler(env_constructor)

Bases: metarl.experiment.task_sampler.TaskSampler

TaskSampler where the environment can sample “task objects”.

This is used for environments that implement sample_tasks and set_task. For example, HalfCheetahVelEnv, as implemented in MetaRL.

Parameters:env_constructor (Callable[gym.Env]) – Callable that produces an environment (for example, an environment type).

n_tasks

The number of tasks if known and finite.

Type:int or None

sample(n_tasks, with_replacement=False)

Sample a list of environment updates.

Parameters:

• n_tasks (int) – Number of updates to sample.
• with_replacement (bool) – Whether tasks can repeat when sampled. Note that if more tasks are sampled than exist, then tasks may repeat, but only after every environment has been included at least once in this batch. Ignored for continuous task spaces.

Returns:

Batch of sampled environment updates, which, when

invoked on environments, will configure them with new tasks. See EnvUpdate for more information.

Return type:

list[EnvUpdate]

class TaskSampler

Bases: abc.ABC

Class for sampling batches of tasks, represented as `~EnvUpdate`s.

n_tasks

Number of tasks, if known and finite.

Type:int or None

n_tasks

The number of tasks if known and finite.

Type:int or None

sample(n_tasks, with_replacement=False)

Sample a list of environment updates.

Parameters:

• n_tasks (int) – Number of updates to sample.
• with_replacement (bool) – Whether tasks can repeat when sampled. Note that if more tasks are sampled than exist, then tasks may repeat, but only after every environment has been included at least once in this batch. Ignored for continuous task spaces.

Returns:

Batch of sampled environment updates, which, when

invoked on environments, will configure them with new tasks. See EnvUpdate for more information.

Return type:

list[EnvUpdate]