Unfortunately there is currently no way to convert a Public Group into a Private Group.
Your best bet would be to create a new Private Group and invite all the people you'd like to chat with from your Public Group.
The same applies to switching a Private Group to a Public Group. Your best bet is to create a Public Group and invite your Private Group members.