If you want to unshare your calendar with a specific person or group of people to protect your privacy, Google Calendar makes ... contact's email and choose "See only free/busy (hide details)".