Make each user able to restrict their own documents by having a list of people who are allowed to view / edit each document.