Could you just lay the fill dirt and sod over the gravel?