MySQL database connectivity issue

[zladuric]

Hello,
I have a working  alfresco (3.0 I believe) and I've tried to do the backup_and_restore procedure to make a copy of alfresco on the development box. I'm constantly hitting the wall here with database connectivity.
My problem is, I believe, that after hibernate (or whatever) makes its 10 connections, it doesn't use them:
I keep getting "java.net.ConnectException: Connection timed out", right after  "Registered Task Component 'jbpm'". Then later it complains about dir.root not pointing to the correct location.

The devel server setup and the production one are the same (I even copied all of the tomcat directory and everything else).

I don't know why, the connections are there (I see those in mysqladmin processlist).

Can somebody give me a hint here on where to look?

Here's my catalina.out log:

Dec 1, 2009 6:19:10 PM org.apache.catalina.core.StandardEngine start
INFO: Starting Servlet Engine: Apache Tomcat/6.0.18
Dec 1, 2009 6:19:23 PM org.apache.catalina.core.AprLifecycleListener init
INFO: The APR based Apache Tomcat Native library which allows optimal performance in production environments was not found on the java.library.path: /usr/lib/jvm/java-6-openjdk/jre/lib/amd64/server:/usr/lib/jvm/java-6-openjdk/jre/lib/amd64:/usr/lib/jvm/java-6-openjdk/jre/../lib/amd64:/usr/java/packages/lib/amd64:/usr/lib/jni:/lib:/usr/lib
Dec 1, 2009 6:19:23 PM org.apache.coyote.http11.Http11Protocol init
INFO: Initializing Coyote HTTP/1.1 on http-8080
Dec 1, 2009 6:19:23 PM org.apache.catalina.startup.Catalina load
INFO: Initialization processed in 626 ms
Dec 1, 2009 6:19:23 PM org.apache.catalina.core.StandardService start
INFO: Starting service Catalina
Dec 1, 2009 6:19:23 PM org.apache.catalina.core.StandardEngine start
INFO: Starting Servlet Engine: Apache Tomcat/6.0.18
Dec 1, 2009 6:19:24 PM org.apache.catalina.core.StandardContext addApplicationListener
INFO: The listener "org.apache.myfaces.webapp.StartupServletContextListener" is already configured for this context. The duplicate definition has been ignored.
[GC 196608K->5554K(733184K), 0.0664680 secs]
[GC 202162K->16457K(820544K), 0.0567970 secs]
[GC 300425K->31633K(820544K), 0.0754170 secs]
[GC 195546K->42662K(820544K), 0.1954050 secs]
[Full GC 42662K->42450K(820544K), 0.4309940 secs]
18:19:47,419  INFO  [config.xml.XMLConfigService$PropertyConfigurer] Loading properties file from class path resource [alfresco/file-servers.properties]
18:19:48,517  DEBUG [workflow.jbpm.JBPMTransactionTemplate] jBPM persistence service present
18:19:48,517  DEBUG [workflow.jbpm.JBPMTransactionTemplate] creating hibernateTemplate based on jBPM SessionFactory
18:19:52,115  INFO  [alfresco.repo.workflow] Registered Workflow Component 'jbpm' (class org.alfresco.repo.workflow.jbpm.JBPMEngine)
18:19:52,115  INFO  [alfresco.repo.workflow] Registered Task Component 'jbpm' (class org.alfresco.repo.workflow.jbpm.JBPMEngine)
[GC 326418K->62388K(820544K), 0.0805730 secs]
java.net.ConnectException: Connection timed out
   at java.net.PlainSocketImpl.socketConnect(Native Method)
   at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:310)
   at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:176)
   at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:163)
   at java.net.Socket.connect(Socket.java:542)
   at java.net.Socket.connect(Socket.java:492)
   at sun.net.NetworkClient.doConnect(NetworkClient.java:174)
   at sun.net.NetworkClient.openServer(NetworkClient.java:135)
   at sun.net.ftp.FtpClient.openServer(FtpClient.java:505)
   at sun.net.ftp.FtpClient.openServer(FtpClient.java:492)
   at sun.net.www.protocol.ftp.FtpURLConnection.connect(FtpURLConnection.java:288)
   at sun.net.www.protocol.ftp.FtpURLConnection.getInputStream(FtpURLConnection.java:375)
   at java.net.URL.openStream(URL.java:1029)
   at org.apache.xml.resolver.Catalog.parseCatalogFile(Unknown Source)
   at org.apache.xml.resolver.Catalog.parsePendingCatalogs(Unknown Source)
   at org.apache.xml.resolver.Catalog.parseCatalog(Unknown Source)
   at org.apache.xml.resolver.Catalog.loadSystemCatalogs(Unknown Source)
   at org.apache.xml.resolver.CatalogManager.getPrivateCatalog(Unknown Source)
   at org.apache.xml.resolver.CatalogManager.getCatalog(Unknown Source)
   at org.apache.xml.resolver.tools.CatalogResolver.initializeCatalogs(Unknown Source)
   at org.apache.xml.resolver.tools.CatalogResolver.<init>(Unknown Source)
   at org.apache.cxf.catalog.OASISCatalogManager.<init>(OASISCatalogManager.java:60)
   at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
   at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
   at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
   at java.lang.reflect.Constructor.newInstance(Constructor.java:532)
   at org.springframework.beans.BeanUtils.instantiateClass(BeanUtils.java:85)
   at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:61)
   at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.instantiateBean(AbstractAutowireCapableBeanFactory.java:757)
   at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:722)
   at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:386)
   at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:249)
   at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:155)
   at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:246)
   at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:160)
   at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:291)
   at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:352)
   at org.springframework.web.context.ContextLoader.createWebApplicationContext(ContextLoader.java:246)
   at org.springframework.web.context.ContextLoader.initWebApplicationContext(ContextLoader.java:189)
   at org.springframework.web.context.ContextLoaderListener.contextInitialized(ContextLoaderListener.java:49)
   at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:3843)
   at org.apache.catalina.core.StandardContext.start(StandardContext.java:4342)
   at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:791)
   at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:771)
   at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:525)
   at org.apache.catalina.startup.HostConfig.deployDirectory(HostConfig.java:926)
   at org.apache.catalina.startup.HostConfig.deployDirectories(HostConfig.java:889)
   at org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:492)
   at org.apache.catalina.startup.HostConfig.start(HostConfig.java:1149)
   at org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:311)
   at org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:117)
   at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1053)
   at org.apache.catalina.core.StandardHost.start(StandardHost.java:719)
   at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1045)
   at org.apache.catalina.core.StandardEngine.start(StandardEngine.java:443)
   at org.apache.catalina.core.StandardService.start(StandardService.java:516)
   at org.apache.catalina.core.StandardServer.start(StandardServer.java:710)
   at org.apache.catalina.startup.Catalina.start(Catalina.java:578)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   at java.lang.reflect.Method.invoke(Method.java:616)
   at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:288)
   at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:413)
[GC 346356K->91743K(768832K), 0.1303770 secs]
18:23:09,187  INFO  [domain.schema.SchemaBootstrap] Schema managed by database dialect org.hibernate.dialect.MySQLInnoDBDialect.
18:23:14,448  INFO  [domain.schema.SchemaBootstrap] No changes were made to the schema.
18:23:15,595 Userystem DEBUG [alfresco.repo.workflow] Attached JBPM Context to transaction b4c3af77-ac44-495f-bf1c-88fae1871399
18:23:15,751 Userystem DEBUG [alfresco.repo.workflow] Workflow deployer: Definition 'alfresco/workflow/review_processdefinition.xml' already deployed
18:23:15,774 Userystem DEBUG [alfresco.repo.workflow] Workflow deployer: Definition 'alfresco/workflow/adhoc_processdefinition.xml' already deployed
18:23:15,803 Userystem DEBUG [alfresco.repo.workflow] Workflow deployer: Definition 'alfresco/workflow/submit_processdefinition.xml' already deployed
18:23:15,828 Userystem DEBUG [alfresco.repo.workflow] Workflow deployer: Definition 'alfresco/workflow/changerequest_processdefinition.xml' already deployed
18:23:15,849 Userystem DEBUG [alfresco.repo.workflow] Workflow deployer: Definition 'alfresco/workflow/submitdirect_processdefinition.xml' already deployed
18:23:15,868 Userystem DEBUG [alfresco.repo.workflow] Workflow deployer: Definition 'alfresco/workflow/invitation-nominated_processdefinition.xml' already deployed
18:23:15,884 Userystem DEBUG [alfresco.repo.workflow] Workflow deployer: Definition 'alfresco/workflow/invitation-moderated_processdefinition.xml' already deployed
18:23:15,963 Userystem DEBUG [alfresco.repo.workflow] Detached (commit) JBPM Context from transaction b4c3af77-ac44-495f-bf1c-88fae1871399
18:23:16,715 Userystem WARN  [node.index.FullIndexRecoveryComponent] The indexes are not synchronized with the database.
18:23:16,850 Userystem WARN  [node.index.AVMFullIndexRecoveryComponent]     Index for avm store sitestore is out of date
18:23:17,421 Userystem INFO  [repo.admin.ConfigurationChecker] The Alfresco root data directory ('dir.root') is: /opt/alf_data
18:23:17,463 Userystem ERROR [repo.admin.ConfigurationChecker] CONTENT INTEGRITY ERROR: Indexes not found for 7 stores.
18:23:17,463 Userystem INFO  [repo.admin.ConfigurationChecker] You may set 'index.recovery.mode=FULL' if you need to rebuild the indexes.
18:23:17,463 Userystem ERROR [repo.admin.ConfigurationChecker] CONTENT INTEGRITY ERROR: System content not found in content store.
18:23:17,463 Userystem ERROR [repo.admin.ConfigurationChecker] Ensure that the 'dir.root' property is pointing to the correct data location.
18:23:17,464 Userystem DEBUG [repo.transaction.RetryingTransactionHelper]
Transaction commit failed:
   Thread: main
   Txn:    UserTransaction[object=org.alfresco.util.transaction.SpringAwareUserTransaction@15726ac1, status=0]
   Iteration: 0
   Exception follows:
org.alfresco.error.AlfrescoRuntimeException: Ensure that the 'dir.root' property is pointing to the correct data location.
   at org.alfresco.repo.admin.ConfigurationChecker.check(ConfigurationChecker.java:312)
   at org.alfresco.repo.admin.ConfigurationChecker.access$000(ConfigurationChecker.java:72)
   at org.alfresco.repo.admin.ConfigurationChecker$1.execute(ConfigurationChecker.java:178)
   at org.alfresco.repo.transaction.RetryingTransactionHelper.doInTransaction(RetryingTransactionHelper.java:322)
   at org.alfresco.repo.transaction.RetryingTransactionHelper.doInTransaction(RetryingTransactionHelper.java:248)
   at org.alfresco.repo.admin.ConfigurationChecker.onBootstrap(ConfigurationChecker.java:182)
   at org.alfresco.util.AbstractLifecycleBean.onApplicationEvent(AbstractLifecycleBean.java:62)
   at org.springframework.context.event.SimpleApplicationEventMulticaster$1.run(SimpleApplicationEventMulticaster.java:77)
   at org.springframework.core.task.SyncTaskExecutor.execute(SyncTaskExecutor.java:49)
   at org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMulticaster.java:75)
   at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:246)
   at org.springframework.context.support.AbstractApplicationContext.finishRefresh(AbstractApplicationContext.java:617)
   at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:355)
   at org.springframework.web.context.ContextLoader.createWebApplicationContext(ContextLoader.java:246)
   at org.springframework.web.context.ContextLoader.initWebApplicationContext(ContextLoader.java:189)
   at org.springframework.web.context.ContextLoaderListener.contextInitialized(ContextLoaderListener.java:49)
   at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:3843)
   at org.apache.catalina.core.StandardContext.start(StandardContext.java:4342)
   at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:791)
   at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:771)
   at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:525)
   at org.apache.catalina.startup.HostConfig.deployDirectory(HostConfig.java:926)
   at org.apache.catalina.startup.HostConfig.deployDirectories(HostConfig.java:889)
   at org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:492)
   at org.apache.catalina.startup.HostConfig.start(HostConfig.java:1149)
   at org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:311)
   at org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:117)
   at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1053)
   at org.apache.catalina.core.StandardHost.start(StandardHost.java:719)
   at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1045)
   at org.apache.catalina.core.StandardEngine.start(StandardEngine.java:443)
   at org.apache.catalina.core.StandardService.start(StandardService.java:516)
   at org.apache.catalina.core.StandardServer.start(StandardServer.java:710)
   at org.apache.catalina.startup.Catalina.start(Catalina.java:578)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   at java.lang.reflect.Method.invoke(Method.java:616)
   at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:288)
   at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:413)
18:23:17,467 Userystem ERROR [web.context.ContextLoader] Context initialization failed
org.alfresco.error.AlfrescoRuntimeException: Ensure that the 'dir.root' property is pointing to the correct data location.
   at org.alfresco.repo.admin.ConfigurationChecker.check(ConfigurationChecker.java:312)
   at org.alfresco.repo.admin.ConfigurationChecker.access$000(ConfigurationChecker.java:72)
   at org.alfresco.repo.admin.ConfigurationChecker$1.execute(ConfigurationChecker.java:178)
   at org.alfresco.repo.transaction.RetryingTransactionHelper.doInTransaction(RetryingTransactionHelper.java:322)
   at org.alfresco.repo.transaction.RetryingTransactionHelper.doInTransaction(RetryingTransactionHelper.java:248)
   at org.alfresco.repo.admin.ConfigurationChecker.onBootstrap(ConfigurationChecker.java:182)
   at org.alfresco.util.AbstractLifecycleBean.onApplicationEvent(AbstractLifecycleBean.java:62)
   at org.springframework.context.event.SimpleApplicationEventMulticaster$1.run(SimpleApplicationEventMulticaster.java:77)
   at org.springframework.core.task.SyncTaskExecutor.execute(SyncTaskExecutor.java:49)
   at org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMulticaster.java:75)
   at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:246)
   at org.springframework.context.support.AbstractApplicationContext.finishRefresh(AbstractApplicationContext.java:617)
   at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:355)
   at org.springframework.web.context.ContextLoader.createWebApplicationContext(ContextLoader.java:246)
   at org.springframework.web.context.ContextLoader.initWebApplicationContext(ContextLoader.java:189)
   at org.springframework.web.context.ContextLoaderListener.contextInitialized(ContextLoaderListener.java:49)
   at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:3843)
   at org.apache.catalina.core.StandardContext.start(StandardContext.java:4342)
   at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:791)
   at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:771)
   at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:525)
   at org.apache.catalina.startup.HostConfig.deployDirectory(HostConfig.java:926)
   at org.apache.catalina.startup.HostConfig.deployDirectories(HostConfig.java:889)
   at org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:492)
   at org.apache.catalina.startup.HostConfig.start(HostConfig.java:1149)
   at org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:311)
   at org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:117)
   at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1053)
   at org.apache.catalina.core.StandardHost.start(StandardHost.java:719)
   at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1045)
   at org.apache.catalina.core.StandardEngine.start(StandardEngine.java:443)
   at org.apache.catalina.core.StandardService.start(StandardService.java:516)
   at org.apache.catalina.core.StandardServer.start(StandardServer.java:710)
   at org.apache.catalina.startup.Catalina.start(Catalina.java:578)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   at java.lang.reflect.Method.invoke(Method.java:616)
   at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:288)
   at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:413)
Dec 1, 2009 6:23:17 PM org.apache.catalina.core.StandardContext start
SEVERE: Error listenerStart
Dec 1, 2009 6:23:17 PM org.apache.catalina.core.StandardContext start
SEVERE: Context [/alfresco] startup failed due to previous errors
log4j:ERROR LogMananger.repositorySelector was null likely due to error in class reloading, using NOPLoggerRepository.
[GC 323999K->108302K(779136K), 0.0974900 secs]
18:23:23,790  INFO  [web.site.FrameworkHelper] Successfully Initialized Web Framework
Dec 1, 2009 6:23:23 PM org.apache.coyote.http11.Http11Protocol start
INFO: Starting Coyote HTTP/1.1 on http-8080
Dec 1, 2009 6:23:24 PM org.apache.jk.common.ChannelSocket init
INFO: JK: ajp13 listening on /0.0.0.0:8009
Dec 1, 2009 6:23:24 PM org.apache.jk.server.JkMain start
INFO: Jk running ID=0 time=0/32  config=null
Dec 1, 2009 6:23:24 PM org.apache.catalina.startup.Catalina start
INFO: Server startup in 240651 ms


–
Zlatko

[sebp]

I would concentrate on these lines in the error log:

18:23:16,715 Userystem WARN [node.index.FullIndexRecoveryComponent] The indexes are not synchronized with the database.
18:23:16,850 Userystem WARN [node.index.AVMFullIndexRecoveryComponent] Index for avm store sitestore is out of date
18:23:17,421 Userystem INFO [repo.admin.ConfigurationChecker] The Alfresco root data directory ('dir.root') is: /opt/alf_data
18:23:17,463 Userystem ERROR [repo.admin.ConfigurationChecker] CONTENT INTEGRITY ERROR: Indexes not found for 7 stores.
18:23:17,463 Userystem INFO [repo.admin.ConfigurationChecker] You may set 'index.recovery.mode=FULL' if you need to rebuild the indexes.
18:23:17,463 Userystem ERROR [repo.admin.ConfigurationChecker] CONTENT INTEGRITY ERROR: System content not found in content store.
18:23:17,463 Userystem ERROR [repo.admin.ConfigurationChecker] Ensure that the 'dir.root' property is pointing to the correct data location.

You have the hint "Ensure that the 'dir.root' property is pointing to the correct data location." in your error logs. This hint may also appear if the content of the dir.root directory does not match the contents of the database. Maybe you made some mistake while copying the dir.root directory or the database. Besides did you try to do a full index recovery?

[zladuric]

Bah, I don't get this forum, I thought I posted the answer already!

Anyway, thanks for the assistance. I'll try with reindexing, hopefully it'll work (already did try, but got some OOM's on my devel machine).