Create a new organization and invite only those people who you want the binder to see. Then create the binder in this organization.

OR

Go to Setup -> Manage Organization Roles and Privileges -> Create New Role and create a new role and include only those people who you do not want to see the binder. Now create the binder in the issue tracker and click on the permission icon for that project. Click on the Edit button for that role and uncheck all the access. Save it and verify if it is showing the correct access.