There is no built in mechanism at the moment although I did open a ticket recently to discuss this (for forums but that could include groups).
I have not tried it but I do believe if you removed all of the user-group/permission assignments for each forum in the group then it should stop showing up – except for Admins of course. As I say – I have never actually tried that and it is a little long-winded.

Oke i will ceap it simpel and just delete the groups-forums i dont nead at the moment.

Havent seen your ticket but you got my vote to intergrade this in future updates.

on the list…