We present new traversable wormhole and non-singular black hole solutions in pure, scale-free $R^2$ gravity. These exotic solutions require no null energy condition violating or "exotic" matter and are supported only by the vacuum of the theory. It is well known that $f(R)$ theories of gravity may be recast as dual theories in the Einstein frame. The solutions we present are found when the conformal transformation required to move to the dual frame is singular. For quadratic $R^2$ gravity, the required conformal factor is identically zero for spacetimes with $R=0$. Solutions in this case are argued to arise in the strong coupling limit of General Relativity.