Your solution seems to be a decent one. Let's say, for instance, client A publishes to client B by using topic customer/group/B/switch, then client B will reply to customer/group/A/switch.
Alternatively, client A could also publish to customer/group/B/switch and specify the topic where it expects any replies in its payload, like this:
{
"data":"your message",
"reply-to":"customer/group/A/segretreply-fromB-1345313"
}
That was just an example of JSON payload. You even can use timestamps, MACs or other unique sources to build the reply-to topic. Client A can subscribe to different reply-to topics for maintaining separate channels for every different client.
However, MQTT is not a one-to-one communication and each client can, potentially, subscribe to # and receive all messages. So, you'll need some sort of authorization in your broker to take a call on whether a client gets to publish/subscribe to a topic or not.
Give the following article a good read: MQTT Security Fundamentals: Authorization. You'll, then, be able to set up topics and authorization rules according to your requirements.