Actually, either this did not work or I've found a bug.
Test scenario:
As admin, I created a site called Marketing, and two test users (test1, test2.) I also created a group called Marketing, to which I added the two test users.
I added the Marketing group to the site, and granted the group Site Manager permission.
Neither of the test users was able to view the More -> Customize Site page - it loads the page, but everything between the Customize Site banner and the footer (where the components - Wiki, Documents, Links, etc. would normally be) is completely blank.
I then used my admin account to invite user test1 to the site via the normal method (again, granting account test1 Site Manager access.) Once I did the manual invite process, user test1 was able to access the customize site page.
So is this a typical difference of users getting permission from invites vs. users getting permission from groups, is this a bug, or have I just done something horribly wrong?