That's a good question. I haven't tested the scenario, but if a user responds to a thread, the default behavior is for them to follow that discussion to be kept in the loop. If they unfollow it, and reply again...then I could see how the platform would auto-follow them again onto that discussion.
From a solution perspective, you could create an Add-On and lace the discussion with meta-data and listen for reply events on that discussion. When found, you could trigger the auto un-follow logic (although, it would have to be done as the user) ... but it might be a bit heavy handed. The best way to do this would be to have the users use their stream follow configuration.
If the system keeps getting in the way, I would recommend filing an idea on the feature in the Jive Platform space and try to get something put into core.
Hope this helps.