File upload fails because of UTF8 strings in metadata

[don_eros]

Hi there,

I'm having trouble uploading files (via the "Add content" button and via webdev). I think the problem is the metadata included in the files: I have the same problem with DOC, PDF and HTML files.

In all cases I managed to correct the problem by simply removing non-ASCII characters from the file's metadata, the problem seems to be in the "title" element and "description" meta for HTML files, in "Title" for PDF files and again in "Title" for DOC files.

Attached to this post you'll find the log file of a very simple case, I fixed this HTML file (a newspaper article) by removing the "è" just before the string "clausola di salvaguardia" (you'll see it on line 5 of the log).

The only workaround I found is precisely this: removing troublesome characters from meta attributes, but needless to say this is not a viable solutions, because a) it's a lot of work which I cannot ask my users to do and b) my repository contains files in dozens of different languages (including Hebrew, Russian and Arabic) and I would like to keep the metadata intact.

Is there any other more obvious solution I'm overlooking?

The specs of my system are:

<ul>
<li>Ubuntu 12.04.2 LTS</li>
<li>Alfresco 4.2.c (binary installer)</li>
<li>MySQL 5.5.31 (instead of the included PostGre)</li>
<li>default locale for the OS is en_US.UTF-8</li>
<li>most tables in the MySQL database are UTF8, but some of them are latin1, default for the system is UTF8, I'm not sure what happened</li>
</ul>

Thanks,
Eros

[don_eros]

… and here's the log:

2013-05-22 15:51:10,270  ERROR [alfresco.webdav.protocol] [http-apr-8080-exec-9] java.io.PrintWriter@222bd287
 HTTP Status Code: 500 caused by: org.alfresco.error.AlfrescoRuntimeException: 042219785 Failed to write property deltas: 
  Node:          11221
  Old:           {{http://www.alfresco.org/model/content/1.0}name=test_firefox3.htm, {http://www.alfresco.org/model/content/1.0}content=contentUrl=store://2013/5/22/15/51/8586a510-d357-4...}
  New:           {{http://www.alfresco.org/model/content/1.0}name=test_firefox3.htm, {http://www.alfresco.org/model/content/1.0}content=contentUrl=store://2013/5/22/15/51/8586a510-d357-4..., {http://www.alfresco.org/model/content/1.0}description={en_US=Franceschini: �C'� clausola di salvaguardia per non tornare a votare con questa legge�}, {http://www.alfresco.org/model/content/1.0}title={en_US=Vertice governo-maggioranza: �Ok a correzione porcellum entro l'estate� - Corriere.it}, {http://www.alfresco.org/model/content/1.0}author=Redazione Online}
  Diff:          {{http://www.alfresco.org/model/content/1.0}name=EQUAL, {http://www.alfresco.org/model/content/1.0}content=EQUAL, {http://www.alfresco.org/model/content/1.0}description=RIGHT_ONLY, {http://www.alfresco.org/model/content/1.0}title=RIGHT_ONLY, {http://www.alfresco.org/model/content/1.0}author=RIGHT_ONLY}
  Delete Tried:  []
  Add Tried:     {{http://www.alfresco.org/model/content/1.0}title={en_US=Vertice governo-maggioranza: �Ok a correzione porcellum entro l'estate� - Corriere.it}, {http://www.alfresco.org/model/content/1.0}author=Redazione Online, {http://www.alfresco.org/model/content/1.0}description={en_US=Franceschini: �C'� clausola di salvaguardia per non tornare a votare con questa legge�}}
   at org.alfresco.repo.webdav.PutMethod.executeImpl(PutMethod.java:308)
   at org.alfresco.repo.webdav.WebDAVMethod$2.execute(WebDAVMethod.java:391)
   at org.alfresco.repo.transaction.RetryingTransactionHelper.doInTransaction(RetryingTransactionHelper.java:433)
   at org.alfresco.repo.transaction.RetryingTransactionHelper.doInTransaction(RetryingTransactionHelper.java:323)
   at org.alfresco.repo.webdav.WebDAVMethod.execute(WebDAVMethod.java:399)
   at org.alfresco.repo.webdav.WebDAVServlet.service(WebDAVServlet.java:140)
   at javax.servlet.http.HttpServlet.service(HttpServlet.java:722)
   at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:305)
   at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
   at org.alfresco.repo.webdav.auth.AuthenticationFilter.doFilter(AuthenticationFilter.java:233)
   at sun.reflect.GeneratedMethodAccessor599.invoke(Unknown Source)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   at java.lang.reflect.Method.invoke(Method.java:601)
   at org.alfresco.repo.management.subsystems.ChainingSubsystemProxyFactory$1.invoke(ChainingSubsystemProxyFactory.java:116)
   at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
   at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202)
   at $Proxy242.doFilter(Unknown Source)
   at org.alfresco.repo.web.filter.beans.BeanProxyFilter.doFilter(BeanProxyFilter.java:82)
   at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
   at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
   at org.alfresco.web.app.servlet.GlobalLocalizationFilter.doFilter(GlobalLocalizationFilter.java:61)
   at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
   at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
   at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)
   at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
   at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:472)
   at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:168)
   at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99)
   at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:929)
   at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
   at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:407)
   at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1002)
   at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:585)
   at org.apache.tomcat.util.net.AprEndpoint$SocketWithOptionsProcessor.run(AprEndpoint.java:1771)
   at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
   at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
   at java.lang.Thread.run(Thread.java:722)
Caused by: org.alfresco.error.AlfrescoRuntimeException: 042219785 Failed to write property deltas: 
  Node:          11221
  Old:           {{http://www.alfresco.org/model/content/1.0}name=test_firefox3.htm, {http://www.alfresco.org/model/content/1.0}content=contentUrl=store://2013/5/22/15/51/8586a510-d357-4...}
  New:           {{http://www.alfresco.org/model/content/1.0}name=test_firefox3.htm, {http://www.alfresco.org/model/content/1.0}content=contentUrl=store://2013/5/22/15/51/8586a510-d357-4..., {http://www.alfresco.org/model/content/1.0}description={en_US=Franceschini: �C'� clausola di salvaguardia per non tornare a votare con questa legge�}, {http://www.alfresco.org/model/content/1.0}title={en_US=Vertice governo-maggioranza: �Ok a correzione porcellum entro l'estate� - Corriere.it}, {http://www.alfresco.org/model/content/1.0}author=Redazione Online}
  Diff:          {{http://www.alfresco.org/model/content/1.0}name=EQUAL, {http://www.alfresco.org/model/content/1.0}content=EQUAL, {http://www.alfresco.org/model/content/1.0}description=RIGHT_ONLY, {http://www.alfresco.org/model/content/1.0}title=RIGHT_ONLY, {http://www.alfresco.org/model/content/1.0}author=RIGHT_ONLY}
  Delete Tried:  []
  Add Tried:     {{http://www.alfresco.org/model/content/1.0}title={en_US=Vertice governo-maggioranza: �Ok a correzione porcellum entro l'estate� - Corriere.it}, {http://www.alfresco.org/model/content/1.0}author=Redazione Online, {http://www.alfresco.org/model/content/1.0}description={en_US=Franceschini: �C'� clausola di salvaguardia per non tornare a votare con questa legge�}}
   at org.alfresco.repo.domain.node.AbstractNodeDAOImpl.setNodePropertiesImpl(AbstractNodeDAOImpl.java:2281)
   at org.alfresco.repo.domain.node.AbstractNodeDAOImpl.setNodeProperties(AbstractNodeDAOImpl.java:2326)
   at org.alfresco.repo.node.db.DbNodeServiceImpl.addAspectsAndProperties(DbNodeServiceImpl.java:528)
   at org.alfresco.repo.node.db.DbNodeServiceImpl.addAspectsAndProperties(DbNodeServiceImpl.java:435)
   at org.alfresco.repo.node.db.DbNodeServiceImpl.setProperties(DbNodeServiceImpl.java:1621)
   at sun.reflect.GeneratedMethodAccessor299.invoke(Unknown Source)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   at java.lang.reflect.Method.invoke(Method.java:601)
   at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:309)
   at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
   at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
   at org.alfresco.repo.tenant.MultiTNodeServiceInterceptor.invoke(MultiTNodeServiceInterceptor.java:105)
   at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
   at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202)
   at $Proxy9.setProperties(Unknown Source)
   at sun.reflect.GeneratedMethodAccessor299.invoke(Unknown Source)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   at java.lang.reflect.Method.invoke(Method.java:601)
   at org.alfresco.repo.service.StoreRedirectorProxyFactory$RedirectorInvocationHandler.invoke(StoreRedirectorProxyFactory.java:215)
   at $Proxy37.setProperties(Unknown Source)
   at org.alfresco.repo.node.MLPropertyInterceptor.invoke(MLPropertyInterceptor.java:215)
   at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
   at org.alfresco.repo.node.NodeRefPropertyMethodInterceptor.invoke(NodeRefPropertyMethodInterceptor.java:244)
   at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
   at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202)
   at $Proxy9.setProperties(Unknown Source)
   at sun.reflect.GeneratedMethodAccessor299.invoke(Unknown Source)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   at java.lang.reflect.Method.invoke(Method.java:601)
   at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:309)
   at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:196)
   at $Proxy9.setProperties(Unknown Source)
   at sun.reflect.GeneratedMethodAccessor299.invoke(Unknown Source)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   at java.lang.reflect.Method.invoke(Method.java:601)
   at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:309)
   at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
   at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
   at org.alfresco.repo.audit.DisableAuditableBehaviourInterceptor.invoke(DisableAuditableBehaviourInterceptor.java:113)
   at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
   at net.sf.acegisecurity.intercept.method.aopalliance.MethodSecurityInterceptor.invoke(MethodSecurityInterceptor.java:80)
   at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
   at org.alfresco.repo.security.permissions.impl.ExceptionTranslatorMethodInterceptor.invoke(ExceptionTranslatorMethodInterceptor.java:46)
   at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
   at org.alfresco.repo.audit.AuditMethodInterceptor.invoke(AuditMethodInterceptor.java:159)
   at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
   at org.alfresco.repo.transaction.RetryingTransactionInterceptor$1.execute(RetryingTransactionInterceptor.java:69)
   at org.alfresco.repo.transaction.RetryingTransactionHelper.doInTransaction(RetryingTransactionHelper.java:433)
   at org.alfresco.repo.transaction.RetryingTransactionInterceptor.invoke(RetryingTransactionInterceptor.java:59)
   at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
   at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202)
   at $Proxy9.setProperties(Unknown Source)
   at org.alfresco.repo.action.executer.ContentMetadataExtracter.executeImpl(ContentMetadataExtracter.java:357)
   at org.alfresco.repo.action.executer.ActionExecuterAbstractBase.execute(ActionExecuterAbstractBase.java:241)
   at org.alfresco.repo.action.ActionServiceImpl.directActionExecution(ActionServiceImpl.java:822)
   at org.alfresco.repo.action.ActionServiceImpl.executeActionImpl(ActionServiceImpl.java:723)
   at org.alfresco.repo.action.ActionServiceImpl.executeAction(ActionServiceImpl.java:557)
   at org.alfresco.repo.action.ActionServiceImpl.executeAction(ActionServiceImpl.java:543)
   at org.alfresco.repo.action.ActionServiceImpl.executeAction(ActionServiceImpl.java:831)
   at sun.reflect.GeneratedMethodAccessor958.invoke(Unknown Source)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   at java.lang.reflect.Method.invoke(Method.java:601)
   at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:309)
   at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
   at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
   at org.alfresco.repo.security.permissions.impl.AlwaysProceedMethodInterceptor.invoke(AlwaysProceedMethodInterceptor.java:34)
   at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
   at org.alfresco.repo.security.permissions.impl.ExceptionTranslatorMethodInterceptor.invoke(ExceptionTranslatorMethodInterceptor.java:46)
   at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
   at org.alfresco.repo.audit.AuditMethodInterceptor.invoke(AuditMethodInterceptor.java:159)
   at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
   at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:110)
   at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
   at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202)
   at $Proxy42.executeAction(Unknown Source)
   at org.alfresco.repo.webdav.PutMethod.executeImpl(PutMethod.java:264)
   … 36 more
Caused by: org.springframework.jdbc.UncategorizedSQLException: 
### Error updating database.  Cause: java.sql.SQLException: Incorrect string value: '\xEF\xBF\xBDOk …' for column 'string_value' at row 1
### The error may involve alfresco.node.parameter_NodeProperty
### The error occurred while setting parameters
### Cause: java.sql.SQLException: Incorrect string value: '\xEF\xBF\xBDOk …' for column 'string_value' at row 1
; uncategorized SQLException for SQL []; SQL state [HY000]; error code [1366]; Incorrect string value: '\xEF\xBF\xBDOk …' for column 'string_value' at row 1; nested exception is java.sql.SQLException: Incorrect string value: '\xEF\xBF\xBDOk …' for column 'string_value' at row 1
   at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:83)
   at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80)
   at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80)
   at org.mybatis.spring.MyBatisExceptionTranslator.translateExceptionIfPossible(MyBatisExceptionTranslator.java:71)
   at org.mybatis.spring.SqlSessionTemplate$SqlSessionInterceptor.invoke(SqlSessionTemplate.java:346)
   at $Proxy7.insert(Unknown Source)
   at org.mybatis.spring.SqlSessionTemplate.insert(SqlSessionTemplate.java:231)
   at org.alfresco.repo.domain.node.ibatis.NodeDAOImpl.insertNodeProperties(NodeDAOImpl.java:596)
   at org.alfresco.repo.domain.node.AbstractNodeDAOImpl.setNodePropertiesImpl(AbstractNodeDAOImpl.java:2274)
   … 111 more
Caused by: java.sql.SQLException: Incorrect string value: '\xEF\xBF\xBDOk …' for column 'string_value' at row 1
   at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:1078)
   at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:4187)
   at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:4119)
   at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2570)
   at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2731)
   at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2815)
   at com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:2155)
   at com.mysql.jdbc.PreparedStatement.execute(PreparedStatement.java:1379)
   at org.apache.commons.dbcp.DelegatingPreparedStatement.execute(DelegatingPreparedStatement.java:172)
   at org.apache.commons.dbcp.DelegatingPreparedStatement.execute(DelegatingPreparedStatement.java:172)
   at org.apache.commons.dbcp.DelegatingPreparedStatement.execute(DelegatingPreparedStatement.java:172)
   at org.apache.ibatis.executor.statement.PreparedStatementHandler.update(PreparedStatementHandler.java:22)
   at org.apache.ibatis.executor.statement.RoutingStatementHandler.update(RoutingStatementHandler.java:51)
   at org.apache.ibatis.executor.SimpleExecutor.doUpdate(SimpleExecutor.java:29)
   at org.apache.ibatis.executor.BaseExecutor.update(BaseExecutor.java:87)
   at org.apache.ibatis.executor.CachingExecutor.update(CachingExecutor.java:46)
   at org.apache.ibatis.session.defaults.DefaultSqlSession.update(DefaultSqlSession.java:118)
   at org.apache.ibatis.session.defaults.DefaultSqlSession.insert(DefaultSqlSession.java:107)
   at sun.reflect.GeneratedMethodAccessor484.invoke(Unknown Source)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   at java.lang.reflect.Method.invoke(Method.java:601)
   at org.mybatis.spring.SqlSessionTemplate$SqlSessionInterceptor.invoke(SqlSessionTemplate.java:338)
   … 115 more
‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

[don_eros]

It turns out the problem was probably MySQL, I reinstalled using the default DB engine and everything works perfectly now.