PG in practice

Policy gradient with automatic differentiation: ignored.

1. Gradient has high variance and will be noisy. Because this isn't the same as supervised learning.

2. Consider using much larger batches.

3. Tweaking learning rates $\alpha$ is very hard, and adaptive step size rules like ADAM can be OK-ish.