Such a thing will need a little coding but not a hack as such. It could be achieved using the built in WordPress system of filters and that code can be placed in a child theme to protect it from updates. Probably the easiest way is to just remove the 'Reply' buttons when appropriate. Of course it could only be made to work for registered users so I assume you would not be allowing unregistered access.