If you devise some kind of latch system, you can mount each side as solid as you'd like, and then not have to worry about stripping anything out since the only movement should be the latch itself. Either something similar to a simple gate/door latch, or something more complex like a ring around one pipe and a latch on the other, like the way you close a cookie jar.