JDBC Error every day at 3:00 am

[tonytony]

I get this error every day for the last two weeks or so.

03:01:15,406 ERROR [org.hibernate.transaction.JDBCTransaction] JDBC commit failed
com.mysql.jdbc.exceptions.MySQLNonTransientConnectionException: Communications link failure during commit(). Transaction resolution unknown.
   at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:888)
   at com.mysql.jdbc.Connection.commit(Connection.java:2259)
   at org.apache.commons.dbcp.DelegatingConnection.commit(DelegatingConnection.java:238)
   at org.apache.commons.dbcp.PoolingDataSource$PoolGuardConnectionWrapper.commit(PoolingDataSource.java:199)
   at org.hibernate.transaction.JDBCTransaction.commitAndResetAutoCommit(JDBCTransaction.java:139)
   at org.hibernate.transaction.JDBCTransaction.commit(JDBCTransaction.java:115)
   at org.springframework.orm.hibernate3.HibernateTransactionManager.doCommit(HibernateTransactionManager.java:562)
   at org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:654)
   at org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:624)
   at org.springframework.transaction.interceptor.TransactionAspectSupport.commitTransactionAfterReturning(TransactionAspectSupport.java:307)
   at org.alfresco.util.transaction.SpringAwareUserTransaction.commit(SpringAwareUserTransaction.java:420)
   at org.alfresco.repo.transaction.RetryingTransactionHelper.doInTransaction(RetryingTransactionHelper.java:239)
   at org.alfresco.repo.transaction.RetryingTransactionHelper.doInTransaction(RetryingTransactionHelper.java:155)
   at org.alfresco.repo.search.impl.lucene.AbstractLuceneIndexerAndSearcherFactory$LuceneIndexBackupComponent.backup(AbstractLuceneIndexerAndSearcherFactory.java:974)
   at org.alfresco.repo.search.impl.lucene.AbstractLuceneIndexerAndSearcherFactory$LuceneIndexBackupJob.execute(AbstractLuceneIndexerAndSearcherFactory.java:1102)
   at org.quartz.core.JobRunShell.run(JobRunShell.java:202)
   at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:529)‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

This repeats every minute for 2 to 6 retries.

After that, the Alfresco web UI and FTP server freezr up.

Restarting mysql dosen't work, I have to restart Tomcat.

Any idea what I'm experiencing?

TIA

[mikeh]

Hi,

Bit of a guess, but Windows Update tends to schedule itself for 3am. Have you also got anything like a firewall or anti-virus that might be updating overnight too? i.e. something that might reset the network connection.

Thanks,
Mike

[tonytony]

I thought I had it licked, but I'm still getting this error.

My question is, what exactly is Alfresco doing at exactly 3am every night? rebuilding indexes?

[mikeh]

The indexes get backed up at 3am, but I wonder if this is clashing with something else happening on your system?

You can try changing the time this happens by overriding/editing the job in scheduled-jobs-context.xml

Thanks,
Mike

[tonytony]

mike:


I traced this to a flaky mySQL implementation.
Thanks for your help and insight.

My other question is, should an intermittent loss of the database be fatal to the application? (It appears it only crashes alfresco when backing up the indexes)

Thanks again for your help.

[mikeh]

My other question is, should an intermittent loss of the database be fatal to the application? (It appears it only crashes alfresco when backing up the indexes)

Losing the database connection temporarily shouldn't be fatal to the application during normal use, as we use transactions throughout (you may see an exception in the web client).

If you're seeing the whole application crash during the backup procedure, then that sound like a problem that should be raised in JIRA

thanks,
Mike

[tonytony]

Tomcat and MySQL are running on the same partition (server).

The alfresco repository is on another larger partition.

It looks like during the backup procedure, the 'server' partition ran out of disk space.

Solution: move the mysql data folders to the larger partition.

seems to be working ok now. Looks like it was specific to my environment.

Thanks for all your help troubleshooting.