Actor-Critic in practice

Architecture design

2 networks design

+: simple & stable

- : no shared features between actor and critic

shares network design

+: shared features

-: need more time for hyper-parameter tuning

In practice

Online version perhaps works best with a batch(e.g., parallel)

parallel actor-critic