Alfresco Community does not support MSSQL as the database server. Unless you find some community-driven project that adds the necessary database mapping configuration, you will not be able to set it up on MSSQL sucessfully.
Some time ago I created a download for earlier versions of Alfresco which was taken by another member and turned into an Alfresco add+ons project http://addons.alfresco.com/tags/mssql
I have not yet contributed back to this project for the latest version but I tend to create updates for each new community version as one of my clients uses both MSSQL 2005 and MSSQL 2008R2
Anyway, checkout the add+ons and see if this helps you.
I WILL get around to updating this add+ons project with my latest changes but its just finding the time I have trouble with.