Update parallel RL

[lihuoran]

Description

Update parallel RL. Now, MARO RL has full ability to run rollout & training under parallel mode.

• Refine EnvSampler
  • Do not explicitly pass info_list. Instead, store info_list in the EnvSampler.
  • Add merge_metrics() to support metrics aggregation in parallel rollout.
  • Add EnvSamplerInterface as the base interface for EnvSampler and BatchEnvSampler.
  • Complete other previously missing features.
• Trainer
  • Complete functions for parallel training (@remote functions).
• MISCs
  • Add parallel RL configs for GYM tasks.
  • Fix several minor bugs.

Linked issue(s)/Pull request(s)

• issue_number

Type of Change

• Non-breaking bug fix
• Breaking bug fix
• New feature
• Test
• Doc update
• Docker update

Related Component

• Simulation toolkit
• RL toolkit
• Distributed toolkit

Has Been Tested

• OS:
  • Windows
  • Mac OS
  • Linux
• Python version:
  • 3.7
  • 3.8
  • 3.9
• Key information snapshot(s):

Needs Follow Up Actions

• New release package
• New docker image

Checklist

• Add/update the related comments
• Add/update the related tests
• Add/update the related documentations
• Update the dependent downstream modules usage