we had similar situation, and we did it through organization name. As in our case, all site A users will belong to Org A and site B users will belong to Org B.So now Org A people won't see Org B people.
In the people-finder.get.html file , we got the current users's organization name and checked that against in the people-finder.js file (included another if loop in function PeopleFinder_onReady() after Remove the current user from the list comment). You have to avoid this check for admin.
Thanks,
Yaso