2 Matching Annotations
  1. Mar 2023
    1. Pre-LN Transformerlayer

      Q: Why does the layer norm come before the MHA / before the FFN in Pre-LN?

    1. Installing Spinning Up

      Troubleshooting in March 2023 on an M2 Mac:

      1. I could only get tensorflow 2 running on my M2 Mac, not tensorflow 1 like the setup.py file requires. To do so, I changed the dep from tensorflow to tensorflow-macos and removed the version constraint.

      2. I also removed the version constraint for torch.

      3. I then also had to change how tensorflow is imported e.g. in run.py:

      import tensorflow.compat.v1 as tf tf.disable_v2_behavior()

      By making these changes, I was able to successfully run the installtest.