World Models

Meta

  • Author：David Ha
  • Conference：NeurIPS 2018
  • Link：World Models

Intro

  1. 理解大脑内部预测模型的一种方式认为：它或许不只是泛泛地预测未来，而是在给定当前运动动作的条件下预测未来的感官数据
  2. RNN-based world models + controllers

Method

  1. framework:
    1

    • VAE (V) Model
      1

    • MDN-RNN (M) Model
      (RNN with a Mixture Density Network output layer)
      1

    • Controller (C) Model
      1

    • All in All
      1

Experiments

  • CarRacing-v0: World Model for Feature Extraction
  • VizDoom Take Cover: Learning Inside of a Dream

DreamerV3

Meta

  • Author：Google DeepMind
  • Conference：Nature
  • Link：World Models

Method

  • Framework
    1

  • World model learning

    • Recurrent State-Space Model
      1
    • loss
      1

  • Critic learning
    1
    1

  • Actor learning
    1
    1

  • Robust predictions

    • symlog 变换 + symlog 平方损失
    • symexp 双热编码损失（twohot loss）