Web Service invocations of repo service fail 90-95%

[jc]

We are using Alfresco running on WAS 6.1 against DB2. We're using the Web Service Client JDK 1.4, and things work alright on my local instance. We are using LTPA Authentication (WebSphere) and are able to call web services successfully. But on our development environment with everything setup exactly as on my local box, I am experiencing the following when I run the below (pseudocode shown):
AuthenticationUtils.startSession(); // runs fine          
…
…
RepositoryServiceSoapBindingStub repositoryService = WebServiceFactory.getRepositoryService(); // runs ok
…
…
QueryResult queryResult = repositoryService.query(STORE, query, true); // runs successfully once in a while, but mostly causes an error with exception shown below.
…
…
Authenticationutils.endSession()

the only way I see the error occurring during the query invocation is if I use a TCP/IP monitor to examine the request header. When the query invocation runs, it causes this exception to come back in the reponse header.

<?xml version="1.0" encoding="utf-8"?><soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"><soapenv:Body><soapenv:Fault><faultcode>soapenv:Server.generalException</faultcode><faultstring></faultstring><detail><ns1:RepositoryFault xmlns:ns1="http://www.alfresco.org/ws/service/repository/1.0"><ns1:errorCode>0</ns1:errorCode><ns1:message>net.sf.acegisecurity.AuthenticationCredentialsNotFoundException: A valid SecureContext was not provided in the RequestContext</ns1:message></ns1:RepositoryFault><ns2:exceptionName xmlns:ns2="http://xml.apache.org/axis/">org.alfresco.repo.webservice.repository.RepositoryFault</ns2:exceptionName><ns3:stackTrace xmlns:ns3="http://xml.apache.org/axis/">
   at org.alfresco.repo.webservice.repository.RepositoryWebService.query(RepositoryWebService.java:234)
   at sun.reflect.GeneratedMethodAccessor274.invoke(Unknown Source)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   at java.lang.reflect.Method.invoke(Method.java:615)
   at org.apache.axis.providers.java.RPCProvider.invokeMethod(RPCProvider.java:397)
   at org.apache.axis.providers.java.RPCProvider.processMessage(RPCProvider.java:186)
   at org.apache.axis.providers.java.JavaProvider.invoke(JavaProvider.java:323)
   at org.apache.axis.strategies.InvocationStrategy.visit(InvocationStrategy.java:32)
   at org.apache.axis.SimpleChain.doVisiting(SimpleChain.java:118)
   at org.apache.axis.SimpleChain.invoke(SimpleChain.java:83)
   at org.apache.axis.handlers.soap.SOAPService.invoke(SOAPService.java:454)
   at org.apache.axis.server.AxisServer.invoke(AxisServer.java:281)
   at org.apache.axis.transport.http.AxisServlet.doPost(AxisServlet.java:699)
   at javax.servlet.http.HttpServlet.service(HttpServlet.java:763)
   at org.apache.axis.transport.http.AxisServletBase.service(AxisServletBase.java:327)
   at javax.servlet.http.HttpServlet.service(HttpServlet.java:856)
   at com.ibm.ws.webcontainer.servlet.ServletWrapper.service(ServletWrapper.java:989)
   at com.ibm.ws.webcontainer.servlet.ServletWrapper.service(ServletWrapper.java:930)
   at com.ibm.ws.webcontainer.filter.WebAppFilterChain.doFilter(WebAppFilterChain.java:118)
   at com.ibm.ws.webcontainer.filter.WebAppFilterChain._doFilter(WebAppFilterChain.java:87)
   at com.ibm.ws.webcontainer.filter.WebAppFilterManager.doFilter(WebAppFilterManager.java:761)
   at com.ibm.ws.webcontainer.filter.WebAppFilterManager.doFilter(WebAppFilterManager.java:673)
   at com.ibm.ws.webcontainer.servlet.ServletWrapper.handleRequest(ServletWrapper.java:498)
   at com.ibm.ws.wswebcontainer.servlet.ServletWrapper.handleRequest(ServletWrapper.java:464)
   at com.ibm.ws.webcontainer.servlet.CacheServletWrapper.handleRequest(CacheServletWrapper.java:90)
   at com.ibm.ws.webcontainer.WebContainer.handleRequest(WebContainer.java:744)
   at com.ibm.ws.wswebcontainer.WebContainer.handleRequest(WebContainer.java:1433)
   at com.ibm.ws.webcontainer.channel.WCChannelLink.ready(WCChannelLink.java:111)
   at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.handleDiscrimination(HttpInboundLink.java:454)
   at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.handleNewInformation(HttpInboundLink.java:383)
   at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.ready(HttpInboundLink.java:263)
   at com.ibm.ws.tcp.channel.impl.NewConnectionInitialReadCallback.sendToDiscriminators(NewConnectionInitialReadCallback.java:214)
   at com.ibm.ws.tcp.channel.impl.NewConnectionInitialReadCallback.complete(NewConnectionInitialReadCallback.java:113)
   at com.ibm.ws.tcp.channel.impl.AioReadCompletionListener.futureCompleted(AioReadCompletionListener.java:165)
   at com.ibm.io.async.AbstractAsyncFuture.invokeCallback(AbstractAsyncFuture.java:217)
   at com.ibm.io.async.AsyncChannelFuture.fireCompletionActions(AsyncChannelFuture.java:161)
   at com.ibm.io.async.AsyncFuture.completed(AsyncFuture.java:136)
   at com.ibm.io.async.ResultHandler.complete(ResultHandler.java:195)
   at com.ibm.io.async.ResultHandler.runEventProcessingLoop(ResultHandler.java:743)
   at com.ibm.io.async.ResultHandler$2.run(ResultHandler.java:873)
   at com.ibm.ws.util.ThreadPool$Worker.run(ThreadPool.java:1469)
</ns3:stackTrace><ns4:hostname xmlns:ns4="http://xml.apache.org/axis/">aix6</ns4:hostname></detail></soapenv:Fault></soapenv:Body></soapenv:Envelope>
‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

Two leads I have been trying to follow:
(1) Our dev environment is a quad code machine, and I read something somewhere about the way ThreadLocal variables are handled, and was wondering if that had anything to do with it.
(2) I read that ACEGI stores the userdetails in a cache - is that somehow relevant to this discussion?

This issue is driving me nuts for two whole days now and its starting to put our use of alfresco in jeopardy ;-(

[rwetherall]

Hi,

The error seems to be telling us that the correct security information is not being sent in the header when requests are being made.  The fact that it is sporadic tells us that code does at least work, but it is being effected by some environmental factor.

I'm guessing that either the ticket is not, on occasion, being sent or that the ticket, for what ever reason, has been invalidated.

I can't think how a ticket could be invalidated without you knowing so perhaps the first case is the best place to start.

Have you traced the information being sent to and from Tomcat.  It would probably help to see the requests being made, this will tell you quickly whether the correct ticket is always being sent indicating whether the issue is on the client or the server.

Also, turning on debug in the repository may give you a better idea of where things are getting to before the failure happens.

Cheers,
Roy

[jc]

Thanks Roy. I was returning the wrong ticket (leftover code change) in AuthenticationUtils.getTicket() that is called by the handle() method, fixed that and it works now.