Virtualisation server doesn't retry connection

[sacco]

Alfresco versions: any from 2.1.0RC1 through to latest nightly build of 2.1.0

Using  alfresco-community-tomcat  to eliminate tomcat version problems.

IBM JVM 1.5.0

RHEL 4

The virtualisation server is clearly intended to retry the JMX remote connection until the
main Alfresco server comes up, but it doesn't!

It seems that the authenticationService fails to connect and then simply gives up. 
If I wait until the main server is up and then start it manually, then the Virtualisation
server can be started.

The advice given here:
http://forums.alfresco.com/viewtopic.php?t=5343&start=14
doesn't seem to hold: The virtualization server *does* gives up.

The point at which it gives up is just when it finds the remote port not yet open:
if I put the following into my virtual_alf.sh script to poll the port just before the line
which calls  startup.sh for the virtual-tomcat, then the Virtualisation server now sees
an open port and does not immediately give up.

while  ! 4</dev/tcp/${Alf_Server:=localhost}/${Alf_JMX_port:=50500} true \
   2>/dev/null  4<&- ; 
  do  4<&-  sleep 1s ; done  4<&-  2>/dev/null
‍‍‍‍

Here's a typical stack-trace:

INFO: The Apache Tomcat Native library which allows optimal performance in production environments was not found on the java.library.path: /usr/lib/jvm/java-1.5.0-ibm-1.5.0.5/jre/bin:/usr/lib/jvm/java-1.5.0-ibm-1.5.0.5/jre/bin:/usr/lib/jvm/java-1.5.0-ibm-1.5.0.5/jre/bin/classic:/usr/lib/jvm/java-1.5.0-ibm-1.5.0.5/jre/bin:/usr/lib
03-Aug-2007 01:34:52 org.apache.coyote.http11.Http11BaseProtocol init
INFO: Initializing Coyote HTTP/1.1 on http-8180
03-Aug-2007 01:34:52 org.apache.catalina.startup.Catalina load
INFO: Initialization processed in 3337 ms
03-Aug-2007 01:34:53 org.apache.catalina.core.StandardService start
INFO: Starting service Catalina
03-Aug-2007 01:34:53 org.apache.catalina.core.StandardEngine start
INFO: Starting Servlet Engine: Apache Tomcat/5.5.23
03-Aug-2007 01:34:53 org.apache.catalina.core.StandardHost start
INFO: XML validation disabled
03-Aug-2007 01:34:56 org.apache.catalina.core.StandardContext start
SEVERE: Error filterStart
03-Aug-2007 01:34:56 org.apache.catalina.core.StandardContext start
SEVERE: Context [/manager] startup failed due to previous errors
03-Aug-2007 01:34:56 org.apache.catalina.core.StandardContext start
SEVERE: Error filterStart
03-Aug-2007 01:34:56 org.apache.catalina.core.StandardContext start
SEVERE: Context [/host-manager] startup failed due to previous errors
03-Aug-2007 01:34:56 org.apache.catalina.core.StandardContext start
SEVERE: Error filterStart
03-Aug-2007 01:34:56 org.apache.catalina.core.StandardContext start
SEVERE: Context [] startup failed due to previous errors
03-Aug-2007 01:34:56 org.springframework.core.CollectionFactory <clinit>
INFO: JDK 1.4+ collections available
03-Aug-2007 01:34:57 org.springframework.beans.factory.xml.XmlBeanDefinitionReader loadBeanDefinitions
INFO: Loading XML bean definitions from URL [file:/opt/alfresco/2.1-a/virtual-tomcat/conf/alfresco-virtserver-context.xml]
03-Aug-2007 01:34:57 org.springframework.context.support.AbstractRefreshableApplicationContext refreshBeanFactory
INFO: Bean factory for application context [org.springframework.context.support.FileSystemXmlApplicationContext;hashCode=1720215176]: org.springframework.beans.factory.support.DefaultListableBeanFactory defining beans [virtserver-properties,avmReauthenticatingAdvisor,reauthenticatingAdvice,clientTicketHolder,authenticationService,avmRemote,avmRemoteTransport,avmRemoteTransportRaw,attributeService,attributeServiceTransport,attributeServiceTransportRaw,jndiInfoBean,virtServerInfo,virtServerMBeanServer,virtServerRegistry,serverConnector,virtServerExporter,virtWebappRegistry,cacheControlInfo]; root of BeanFactory hierarchy
03-Aug-2007 01:34:57 org.springframework.context.support.AbstractApplicationContext refresh
INFO: 19 beans defined in application context [org.springframework.context.support.FileSystemXmlApplicationContext;hashCode=1720215176]
03-Aug-2007 01:34:58 org.springframework.core.io.support.PropertiesLoaderSupport loadProperties
INFO: Loading properties file from URL [file:/opt/alfresco/2.1-a/virtual-tomcat/conf/alfresco-shared.properties]
03-Aug-2007 01:34:58 org.springframework.core.io.support.PropertiesLoaderSupport loadProperties
INFO: Loading properties file from URL [file:/opt/alfresco/2.1-a/virtual-tomcat/conf/alfresco-virtserver.properties]
03-Aug-2007 01:34:58 org.springframework.context.support.AbstractApplicationContext initMessageSource
INFO: Unable to locate MessageSource with name 'messageSource': using default [org.springframework.context.support.DelegatingMessageSource@2fd42fd4]
03-Aug-2007 01:34:58 org.springframework.context.support.AbstractApplicationContext initApplicationEventMulticaster
INFO: Unable to locate ApplicationEventMulticaster with name 'applicationEventMulticaster': using default [org.springframework.context.event.SimpleApplicationEventMulticaster@7b287b28]
03-Aug-2007 01:34:58 org.springframework.beans.factory.support.DefaultListableBeanFactory preInstantiateSingletons
INFO: Pre-instantiating singletons in factory [org.springframework.beans.factory.support.DefaultListableBeanFactory defining beans [virtserver-properties,avmReauthenticatingAdvisor,reauthenticatingAdvice,clientTicketHolder,authenticationService,avmRemote,avmRemoteTransport,avmRemoteTransportRaw,attributeService,attributeServiceTransport,attributeServiceTransportRaw,jndiInfoBean,virtServerInfo,virtServerMBeanServer,virtServerRegistry,serverConnector,virtServerExporter,virtWebappRegistry,cacheControlInfo]; root of BeanFactory hierarchy]
03-Aug-2007 01:34:59 org.springframework.beans.factory.support.DefaultSingletonBeanRegistry destroySingletons
INFO: Destroying singletons in {org.springframework.beans.factory.support.DefaultListableBeanFactory defining beans [virtserver-properties,avmReauthenticatingAdvisor,reauthenticatingAdvice,clientTicketHolder,authenticationService,avmRemote,avmRemoteTransport,avmRemoteTransportRaw,attributeService,attributeServiceTransport,attributeServiceTransportRaw,jndiInfoBean,virtServerInfo,virtServerMBeanServer,virtServerRegistry,serverConnector,virtServerExporter,virtWebappRegistry,cacheControlInfo]; root of BeanFactory hierarchy}
java.lang.reflect.InvocationTargetException
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:64)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   at java.lang.reflect.Method.invoke(Method.java:615)
   at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:295)
   at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:433)
Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'avmReauthenticatingAdvisor' defined in URL [file:/opt/alfresco/2.1-a/virtual-tomcat/conf/alfresco-virtserver-context.xml]: Cannot resolve reference to bean 'reauthenticatingAdvice' while setting bean property 'advice'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'reauthenticatingAdvice' defined in URL [file:/opt/alfresco/2.1-a/virtual-tomcat/conf/alfresco-virtserver-context.xml]: Cannot resolve reference to bean 'authenticationService' while setting bean property 'authenticationService'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'authenticationService' defined in URL [file:/opt/alfresco/2.1-a/virtual-tomcat/conf/alfresco-virtserver-context.xml]: Invocation of init method failed; nested exception is org.springframework.remoting.RemoteLookupFailureException: Lookup of RMI stub failed; nested exception is java.rmi.ConnectException: Connection refused to host: localhost; nested exception is: 
   java.net.ConnectException: Connection refused
   at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveReference(BeanDefinitionValueResolver.java:253)
   at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveValueIfNecessary(BeanDefinitionValueResolver.java:124)
   at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyPropertyValues(AbstractAutowireCapableBeanFactory.java:1019)
   at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:809)
   at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:425)
   at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:250)
   at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:141)
   at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:247)
   at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:161)
   at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:273)
   at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:346)
   at org.springframework.context.support.FileSystemXmlApplicationContext.<init>(FileSystemXmlApplicationContext.java:89)
   at org.springframework.context.support.FileSystemXmlApplicationContext.<init>(FileSystemXmlApplicationContext.java:74)
   at org.springframework.context.support.FileSystemXmlApplicationContext.<init>(FileSystemXmlApplicationContext.java:65)
   at org.alfresco.catalina.host.AVMHost.init(AVMHost.java:376)
   at org.alfresco.catalina.host.AVMHost.start(AVMHost.java:642)
   at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1014)
   at org.apache.catalina.core.StandardEngine.start(StandardEngine.java:443)
   at org.apache.catalina.core.StandardService.start(StandardService.java:448)
   at org.apache.catalina.core.StandardServer.start(StandardServer.java:700)
   at org.apache.catalina.startup.Catalina.start(Catalina.java:552)
   … 6 more
Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'reauthenticatingAdvice' defined in URL [file:/opt/alfresco/2.1-a/virtual-tomcat/conf/alfresco-virtserver-context.xml]: Cannot resolve reference to bean 'authenticationService' while setting bean property 'authenticationService'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'authenticationService' defined in URL [file:/opt/alfresco/2.1-a/virtual-tomcat/conf/alfresco-virtserver-context.xml]: Invocation of init method failed; nested exception is org.springframework.remoting.RemoteLookupFailureException: Lookup of RMI stub failed; nested exception is java.rmi.ConnectException: Connection refused to host: localhost; nested exception is: 
   java.net.ConnectException: Connection refused
   at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveReference(BeanDefinitionValueResolver.java:253)
   at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveValueIfNecessary(BeanDefinitionValueResolver.java:124)
   at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyPropertyValues(AbstractAutowireCapableBeanFactory.java:1019)
   at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:809)
   at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:425)
   at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:250)
   at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:141)
   at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:247)
   at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:161)
   at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveReference(BeanDefinitionValueResolver.java:245)
   … 26 more
Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'authenticationService' defined in URL [file:/opt/alfresco/2.1-a/virtual-tomcat/conf/alfresco-virtserver-context.xml]: Invocation of init method failed; nested exception is org.springframework.remoting.RemoteLookupFailureException: Lookup of RMI stub failed; nested exception is java.rmi.ConnectException: Connection refused to host: localhost; nested exception is: 
   java.net.ConnectException: Connection refused
   at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1088)
   at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:429)
   at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:250)
   at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:141)
   at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:247)
   at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:161)
   at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveReference(BeanDefinitionValueResolver.java:245)
   … 35 more
Caused by: org.springframework.remoting.RemoteLookupFailureException: Lookup of RMI stub failed; nested exception is java.rmi.ConnectException: Connection refused to host: localhost; nested exception is: 
   java.net.ConnectException: Connection refused
   at org.springframework.remoting.rmi.RmiClientInterceptor.lookupStub(RmiClientInterceptor.java:214)
   at org.springframework.remoting.rmi.RmiClientInterceptor.prepare(RmiClientInterceptor.java:145)
   at org.springframework.remoting.rmi.RmiClientInterceptor.afterPropertiesSet(RmiClientInterceptor.java:133)
   at org.springframework.remoting.rmi.RmiProxyFactoryBean.afterPropertiesSet(RmiProxyFactoryBean.java:66)
   at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1118)
   at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1085)
   … 41 more
Caused by: java.rmi.ConnectException: Connection refused to host: localhost; nested exception is: 
   java.net.ConnectException: Connection refused
   at sun.rmi.transport.tcp.TCPEndpoint.newSocket(TCPEndpoint.java:590)
   at sun.rmi.transport.tcp.TCPChannel.createConnection(TCPChannel.java:204)
   at sun.rmi.transport.tcp.TCPChannel.newConnection(TCPChannel.java:190)
   at sun.rmi.server.UnicastRef.newCall(UnicastRef.java:321)
   at sun.rmi.registry.RegistryImpl_Stub.lookup(RegistryImpl_Stub.java:88)
   at java.rmi.Naming.lookup(Naming.java:98)
   at org.springframework.remoting.rmi.RmiClientInterceptor.lookupStub(RmiClientInterceptor.java:199)
   … 46 more
Caused by: java.net.ConnectException: Connection refused
   at java.net.PlainSocketImpl.socketConnect(Native Method)
   at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:372)
   at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:233)
   at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:220)
   at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:385)
   at java.net.Socket.connect(Socket.java:541)
   at java.net.Socket.connect(Socket.java:491)
   at java.net.Socket.<init>(Socket.java:399)
   at java.net.Socket.<init>(Socket.java:208)
   at sun.rmi.transport.proxy.RMIDirectSocketFactory.createSocket(RMIDirectSocketFactory.java:41)
   at sun.rmi.transport.proxy.RMIMasterSocketFactory.createSocket(RMIMasterSocketFactory.java:145)
   at sun.rmi.transport.tcp.TCPEndpoint.newSocket(TCPEndpoint.java:585)
   … 52 more
‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

[jcox]

This issue has just been resolved in 2.1 ENTERPRISE.
For details, see:  http://issues.alfresco.com/browse/WCM-750

[dave_whyte]

I am just starting with alfresco community 2.1 and I have this issue

I can see the post from jcox saying that the issue is fixed but only in 2.1E.

I am using 2.1 because that has things like the wsf project in alfresco forge don't support 3  yet.  The help\content online for WCM 2.1 is plentiful.

So is there a way to access the fixed version of 2.1 - or at least find out how to fix this. 

Can anyone suggest how to proceed?


FYI, I am working on MAC OSX 10.5.

[sacco]

Your choices are either to upgrade to a more recent build (the 2.9 builds are pretty much the same as 2.1, but may have some issues of their own), or to put the following is your script to delay starting the virtualisation server until the JMX port is open:

while  ! 4</dev/tcp/${Alf_Server:=localhost}/${Alf_JMX_port:=50500} true \
   2>/dev/null  4<&- ;
  do  4<&-  sleep 1s ; done  4<&-  2>/dev/null‍‍‍

Personally I'd go with 2.9