Access Right Settings

Need some help with setting access rights as it seems its not working as intended.

How I would like to setup access

/ - root

/gallery1/Pictures Public - access by admin
/gallery1/Pictures Private - access by visitor group

I gave visitor group access to /gallery1/Pictures Private but I dont see any albums, if I give visitor group access to root then all other galleries inherit access rights and they can see all.

What is the right way to configure album.

Basically I categorize my pictures by year/month/event and inside event like TripToXXX I i have 2 folders Private and Public pictures.

p.s. With the removal of inherit rights would be one way of fixing it.