Soak a little penetrating oil into it and let it sit a while- You should be able to punch it out.
Clean the keyway on both items really well prior to assembly.