SOLR search not working with 3rd party CA cert

[tullo]

I decided to invest in a SSL cert issued by an official CA.

For some reason I now cannot search the repository even though SOLR indexed the content after I deleted them.

The server starts without throwing exceptions and I can login and navigate the sites and document library but not search.

Maybe my visualization of the keystore and truststore kan help pinpoint the problem? https://tullo.net/share/s/VjojxGcKQEmeNtMNtSTTLQ

Thanks

alfresco-4.2.c on ubuntu

[mitpatoliya]

Are you trying to search the deleted content?
It will be moved to archive store and it will be visible to admin only (just to give you idea).
Are you getting any error in the logs?
and Are you able to search that content through node browser?

[tullo]

When I try to search via alfresco explorer and the nodebrowser on file name "HTML5 in Action.pdf" I get

Search failed due to system error: 03180070 Request failed 401 /solr/alfresco/alfresco?wt=json&fl=DBID%2Cscore&rows=500&df=TEXT&start=0&locale=en_US&fq=%7B%21afts%7DAUTHORITY_FILTER_FROM_JSON&fq=%7B%21afts%7DTENANT_FILTER_FROM_JSON
org.alfresco.repo.search.impl.lucene.LuceneQueryParserException: 03180070 Request failed 401 /solr/alfresco/alfresco?wt=json&fl=DBID%2Cscore&rows=500&df=TEXT&start=0&locale=en_US&fq=%7B%21afts%7DAUTHORITY_FILTER_FROM_JSON&fq=%7B%21afts%7DTENANT_FILTER_FROM_JSON
   at org.alfresco.repo.search.impl.solr.SolrQueryHTTPClient.executeQuery(SolrQueryHTTPClient.java:420)
   at org.alfresco.repo.search.impl.solr.SolrQueryLanguage.executeQuery(SolrQueryLanguage.java:49)
   at org.alfresco.repo.search.impl.solr.SolrSearchService.query(SolrSearchService.java:348)
   at org.alfresco.repo.search.SearcherComponent.query(SearcherComponent.java:78)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   at java.lang.reflect.Method.invoke(Method.java:601)
   at org.alfresco.repo.management.subsystems.SubsystemProxyFactory$1.invoke(SubsystemProxyFactory.java:72)
   at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
   at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202)
   at $Proxy17.query(Unknown Source)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   at java.lang.reflect.Method.invoke(Method.java:601)
   at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:309)
   at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
   at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
   at net.sf.acegisecurity.intercept.method.aopalliance.MethodSecurityInterceptor.invoke(MethodSecurityInterceptor.java:80)
   at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
   at org.alfresco.repo.security.permissions.impl.ExceptionTranslatorMethodInterceptor.invoke(ExceptionTranslatorMethodInterceptor.java:46)
   at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
   at org.alfresco.repo.audit.AuditMethodInterceptor.invoke(AuditMethodInterceptor.java:159)
   at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
   at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:110)
   at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
   at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202)
   at $Proxy17.query(Unknown Source)
   at org.alfresco.web.bean.BrowseBean.searchBrowseNodes(BrowseBean.java:1096)
   at org.alfresco.web.bean.BrowseBean.getNodes(BrowseBean.java:529)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   at java.lang.reflect.Method.invoke(Method.java:601)
   at org.apache.myfaces.el.PropertyResolverImpl.getProperty(PropertyResolverImpl.java:459)
   at org.apache.myfaces.el.PropertyResolverImpl.getValue(PropertyResolverImpl.java:85)
   at org.apache.myfaces.el.ELParserHelper$MyPropertySuffix.evaluate(ELParserHelper.java:539)
   at org.apache.commons.el.ComplexValue.evaluate(ComplexValue.java:145)
   at org.apache.myfaces.el.ValueBindingImpl.getValue(ValueBindingImpl.java:386)
   at org.alfresco.web.ui.common.component.data.UIRichList.getValue(UIRichList.java:151)
   at org.alfresco.web.ui.common.component.data.UIRichList.getDataModel(UIRichList.java:512)
   at org.alfresco.web.ui.common.component.data.UIRichList.bind(UIRichList.java:448)
   at org.alfresco.web.ui.common.renderer.data.RichListRenderer.encodeChildren(RichListRenderer.java:77)
   at javax.faces.component.UIComponentBase.encodeChildren(UIComponentBase.java:776)
   at javax.faces.webapp.UIComponentTag.encodeChildren(UIComponentTag.java:663)
   at javax.faces.webapp.UIComponentTag.doEndTag(UIComponentTag.java:544)
   at org.apache.jsp.jsp.browse.browse_jsp._jspx_meth_a_005frichList_005f0(browse_jsp.java:2833)
   at org.apache.jsp.jsp.browse.browse_jsp._jspx_meth_a_005fpanel_005f5(browse_jsp.java:2721)
   at org.apache.jsp.jsp.browse.browse_jsp._jspService(browse_jsp.java:878)
   at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:70)
   at javax.servlet.http.HttpServlet.service(HttpServlet.java:722)
   at org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:432)
   at org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:390)
   at org.apache.jasper.servlet.JspServlet.service(JspServlet.java:334)
   at javax.servlet.http.HttpServlet.service(HttpServlet.java:722)
   at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:305)
   at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
   at org.apache.catalina.core.ApplicationDispatcher.invoke(ApplicationDispatcher.java:749)
   at org.apache.catalina.core.ApplicationDispatcher.processRequest(ApplicationDispatcher.java:487)
   at org.apache.catalina.core.ApplicationDispatcher.doForward(ApplicationDispatcher.java:412)
   at org.apache.catalina.core.ApplicationDispatcher.forward(ApplicationDispatcher.java:339)
   at org.apache.myfaces.context.servlet.ServletExternalContextImpl.dispatch(ServletExternalContextImpl.java:426)
   at org.apache.myfaces.application.jsp.JspViewHandlerImpl.renderView(JspViewHandlerImpl.java:255)
   at org.apache.myfaces.lifecycle.RenderResponseExecutor.execute(RenderResponseExecutor.java:41)
   at org.apache.myfaces.lifecycle.LifecycleImpl.render(LifecycleImpl.java:146)
   at javax.faces.webapp.FacesServlet.service(FacesServlet.java:147)
   at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:305)
   at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
   at org.alfresco.repo.web.filter.beans.SessionSynchronizedFilter.doFilter(SessionSynchronizedFilter.java:67)
   at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
   at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
   at org.alfresco.web.app.servlet.AuthenticationFilter.doFilter(AuthenticationFilter.java:104)
   at sun.reflect.GeneratedMethodAccessor973.invoke(Unknown Source)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   at java.lang.reflect.Method.invoke(Method.java:601)
   at org.alfresco.repo.management.subsystems.ChainingSubsystemProxyFactory$1.invoke(ChainingSubsystemProxyFactory.java:116)
   at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
   at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202)
   at $Proxy240.doFilter(Unknown Source)
   at org.alfresco.repo.web.filter.beans.BeanProxyFilter.doFilter(BeanProxyFilter.java:82)
   at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
   at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
   at org.alfresco.repo.web.filter.beans.NullFilter.doFilter(NullFilter.java:68)
   at sun.reflect.GeneratedMethodAccessor973.invoke(Unknown Source)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   at java.lang.reflect.Method.invoke(Method.java:601)
   at org.alfresco.repo.management.subsystems.ChainingSubsystemProxyFactory$1.invoke(ChainingSubsystemProxyFactory.java:116)
   at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
   at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202)
   at $Proxy240.doFilter(Unknown Source)
   at org.alfresco.repo.web.filter.beans.BeanProxyFilter.doFilter(BeanProxyFilter.java:82)
   at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
   at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
   at org.alfresco.web.app.servlet.GlobalLocalizationFilter.doFilter(GlobalLocalizationFilter.java:61)
   at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
   at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
   at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)
   at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
   at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:472)
   at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:168)
   at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99)
   at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:929)
   at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
   at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:407)
   at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1002)
   at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:585)
   at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:312)
   at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
   at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
   at java.lang.Thread.run(Thread.java:722)



But then I also see:

Apr 18, 2013 9:35:00 AM org.apache.catalina.realm.LockOutRealm authenticate
WARNING: An attempt was made to authenticate the locked user "CN=tullo.net, O=Java Consulting, OU=Web Security, L=Copenhagen, C=DK"
‍‍‍‍

which is the repository user defined in tomcat-users.xml
<code>
<tomcat-users>
  <user username="CN=tullo.net,OU=Web Security, O=Java Consulting, L=Copenhagen, ST=, C=DK" roles="repository" password="null"/>
  <!–<user username="CN=Alfresco Repository, OU=Unknown, O=Alfresco Software Ltd., L=Maidenhead, ST=UK, C=GB" roles="repository" password="null"/>–>
  <user username="CN=Alfresco Repository Client, OU=Unknown, O=Alfresco Software Ltd., L=Maidenhead, ST=UK, C=GB" roles="repoclient" password="null"/>
</tomcat-users>
<code>

[tullo]

Turing on SSL debug information gives a hint on the repository client not sending a valid cert to the server. Not sure how I would fix that though!

*** CertificateRequest
Cert Types: RSA, DSS, ECDSA
Cert Authorities:
<CN=Class 2 Primary CA, O=Certplus, C=FR>
<CN=Alfresco Repository Client, OU=Unknown, O=Alfresco Software Ltd., L=Maidenhead, ST=UK, C=GB>
<CN=CLASS 2 KEYNECTIS CA, O=KEYNECTIS, C=FR>
<CN=tullo.net, O=Java Consulting, OU=Web Security, L=Copenhagen, C=DK>
*** ServerHelloDone
http-bio-9443-exec-9, WRITE: TLSv1 Handshake, length = 4055
http-bio-9443-exec-9, READ: TLSv1 Handshake, length = 97
*** Certificate chain
***
%% Invalidated:  [Session-13, TLS_ECDHE_RSA_WITH_RC4_128_SHA]
%% Invalidated:  [Session-14, TLS_ECDHE_RSA_WITH_RC4_128_SHA]
http-bio-9443-exec-9, SEND TLSv1 ALERT:  fatal, description = bad_certificate
http-bio-9443-exec-9, WRITE: TLSv1 Alert, length = 22
http-bio-9443-exec-9, called closeSocket()
http-bio-9443-exec-9, handling exception: javax.net.ssl.SSLHandshakeException: null cert chain
Apr 19, 2013 12:47:09 AM org.apache.tomcat.util.net.jsse.JSSESupport handShake
INFO: Error trying to obtain a certificate from the client
javax.net.ssl.SSLHandshakeException: null cert chain
        at sun.security.ssl.Alerts.getSSLException(Alerts.java:192)
        at sun.security.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1902)
        at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:276)
        at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:266)
        at sun.security.ssl.ServerHandshaker.clientCertificate(ServerHandshaker.java:1629)
        at sun.security.ssl.ServerHandshaker.processMessage(ServerHandshaker.java:176)
        at sun.security.ssl.Handshaker.processLoop(Handshaker.java:868)
        at sun.security.ssl.Handshaker.process_record(Handshaker.java:804)
        at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1032)
        at sun.security.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:884)
        at sun.security.ssl.AppInputStream.read(AppInputStream.java:102)
        at java.io.InputStream.read(InputStream.java:101)
        at org.apache.tomcat.util.net.jsse.JSSESupport.handShake(JSSESupport.java:186)
        at org.apache.tomcat.util.net.jsse.JSSESupport.getPeerCertificateChain(JSSESupport.java:153)
        at org.apache.coyote.http11.Http11Processor.actionInternal(Http11Processor.java:344)
        at org.apache.coyote.http11.AbstractHttp11Processor.action(AbstractHttp11Processor.java:847)
        at org.apache.coyote.Request.action(Request.java:344)
        at org.apache.catalina.authenticator.SSLAuthenticator.authenticate(SSLAuthenticator.java:137)
‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

[tullo]

I'm giving up on this. This solr stuff is way to fragile. Switched to a reverse proxy instead. Seems to be the advised solution anyway.