SSO / SiteMinder setup - what are we missing?

[Kevin_Perron]

We have performed the following steps to setup SSO in version 13.0.1.120

• 1) Ran "Hyland Single Sign On.msi"

• Ran "C:\Program Files (x86)\Hyland\Single Sign On\SingleSignOnConfig.exe" (Run as Administrator) on web server

Selected "c:\inetpub]\wwwroot\AppNet"

Selected SiteMinder option

• Oracle WebGate is setup inside IIS to protect URLS below https://webserver/AppNet/

This successfully redirects to the Dartmouth SSO page, accepts credentials, and returns the call to AppNet.

 WebGate is adding these headers (verified by our custom /AppNet/headertest.aspx).  We can add any additional headers we desire in WebGate configuration:

USERID = "domain ID"

HTTP_USERNAME = "domain ID"

HTTP_USERID = "domain ID"

IMPERSONATE = "domain ID"

OAM_REMOTE_USER = "domain ID"

OAM_IDENTITY_DOMAIN = DartOVD

• Current settings (that I know to be relevant) are:

AppPool for AppNet running as a domain service account

AppNet Authorization

Anonymous Enabled

Impersonating domain service account

Windows Auth disabled

AppNet web.config

SectionHyland.Services.Client

Using SOAP, not .Net Remoting

(Based on SSO MRG page 14)

AllowNTAuthenticationOnForwarding = False

(Baed on web.config stock comment "Only turn this flag on when using NT authentication"                                                        

appSettings:

EnableAutoLogin = True

forceSSOAutoLoginOverDomain = True

(based on SSO MRG Page 6)

CustomSSOAuthenticationFailurePage =  "http://server/AppNet/ssofailure.html"

Section Hyland.Authentication:

<Hyland.Authentication Type="Hyland.Authentication.SiteMinderAuthenticationProvider, Hyland.Authentication.SiteMinder, Version=2.3.0.0, Culture=neutral, PublicKeyToken=c02e21dc39c53bb0">

    <properties>

      <add key="userIdHeader" value="HTTP_USERID" />

      <add key="logServerVariables" value="true" />

      <add key="logUserName" value="true" />

    </properties>

  </Hyland.Authentication>

• Results:
• Load /AppNet/login.apsx
• Redirected to Dartmouth SSO page, login, redirects to /AppNet/login.aspx
• Re-directs to /AppNet/ssofailure.html
• Nothing in Diagnostics Console on web server

• Hit /AppNet/login.aspx AGAIN
• Returns standard Web Client error page with "An unknown login error occurred."
• Diagnostics console says:

Invalid value for 'encryptedTicket' parameter	System.Web	Systems.Web.Security.FormsAuthentication	Decrypt
	Hyland.Applications.Web	Hyland.Applications.Web.Login	ValidateOBLoginTicket
Login Ticket invalid	Hyland.Application.Web	Login	ValidateOBLoginTicket

• Hit /AppNet/login.aspx AGAIN
• SSO failure page
• Alternates between SSO failure page and "An unknown login error occurred" each time I hit /AppNet/login.aspx

[Ian_Cordova]

Hi Kevin,

I hope all is well!

Right off the bat but this is not the issue with the <Hyland.Authentication> section.  By default, the SiteMinder SSO is already configured to look for the header "HTTP_USERID".  Basically you do not have to specify userIdHeader property.  The only time you will need this if the header name is different than the default "HTTP_USERID" name.

I do have other questions for you.
1) Did you run and configure the OnBase AppServer for Single Sign-On?  Both the Web Server and AppServer need to be configured.
2) Are there any errors in the Diagnostic Console on the AppServer? 
3) Do you see any entries within the Web Server's Windows Event Log?  You have logServerVariables and logUsername set to true.  When these are set to true, the SSO provider attempts to write to the Application log on the Web Server where the Hyland.Authentication.SiteMinder.dll is hosted.  It will attempt to write to a Source Name of "Hyland.Authentication.SiteMinder.dll".  If you see the expected username then at least we know that the SSO provider is properly being called and executed.  If you don't see anything, I would search the Windows Event log to make sure that there was not a Permission Error writing to the Event Log. 

4) Can you verify that the Domain Account running the Web Server AppPool has read rights to the following registry keys
- HKEY_LOCAL_MACHINE\SOFTWARE\Hyland\HylandWebServices
- HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Hyland\HylandWebServices

On certain occasions I had to first provide Read rights to everyone and then narrow it down from there.

5) Another option to do and will help us determine if the SSO framework is being called properly is to add the following switch to the Web Server web.config file under <system.diagnostics>

<add name="Hyland.Authentication.TraceSwitch" value="4" />

Then make sure you have the trace mailslot enabled within the Web Server web.config file in the <Hyland.Diagnostics> section.  Run the OnBase Diagnostic Console on the Web Server and ensure the Trace tab is enabled/visable.  If configured correctly, you will see XML outputted to the Trace tab when you hit Login.aspx.  

Let's start here and based on your answers, I can provide next steps on troubleshooting.

Please let me know if you have any questions.

Thanks,
Ian Cordova
Custom Solutions Group - Team Lead

[Kevin_Perron]

Thanks Ian - things are going great!

We will research this information it looks very helpful.  We also never received the dll files you referenced so we will look into getting those. 

Regards,

Kevin

[#4]

Hi Kevin,

One item I wanted to throw in as well is that Anonymous Authentication should be disabled on the Appnet application pool. On the AppServer application pool, this setting can be on or off based on your own preferences. Everything else I could think to mention was already covered by Ian.

With the alternating errors you're seeing (between the SSO failure and "unknown login error" message coming from the same page) I can't help but wonder if you're using a load balancer. If you are, as a troubleshooting step, you may want to try removing the load balancing configuration by changing dmsVirtualRoot in your Appnet web.config to point to the IP address of one web server. This can remove a lot of headache, and once you have it working, you have a known good configuration you can copy to any subsequent web servers in the pool.

Lastly, if you no longer want to commit the time to figuring this issue out on your own and/or want a really fast response, I'd recommend contacting your reseller for support. And, if your reseller needs further assistance, they can escalate to Hyland's technical support team, and those guys and gals are sharp.

[Kevin_Perron]

Hi Steve

We have already worked through our reseller to get to the point we are.