Content Transformations not available

[enigma969]

Hi, 

I have alfresco-5.0.a installed on a Ubuntu VPS. Now I want to transform various documents (e.g. docx, .xlsx) to PDF, but I get errors like

Transformer for 'application/vnd.openxmlformats-officedocument.spreadsheetml.sheet' source mime type and 'application/pdf' target mime type was not found. Operation can't be performed

When I look at Available transform options | Alfresco Documentation  it says application/vnd.openxmlformats-officedocument.spreadsheetml.sheet should be convertible to application/pdf. 

/opt/alfresco-5.0.a/libreoffice/program/.soffice.bin is running on the VPS.

alfresco-global.properties contains following:

###############################
## Common Alfresco Properties #
###############################

dir.root=/opt/Alfresco/alf_data

alfresco.context=alfresco
alfresco.host=127.0.0.1
alfresco.port=8080
alfresco.protocol=http

share.context=share
share.host=127.0.0.1
share.port=8080
share.protocol=http

### database connection properties ###
db.driver=org.postgresql.Driver
db.username=alfresco
db.password=XXX
db.name=alfresco
db.url=jdbcostgresql://localhost:5432/${db.name}

# The server mode. Set value here
# UNKNOWN | TEST | BACKUP | PRODUCTION
system.serverMode=UNKNOWN

### FTP Server Configuration ###
ftp.enabled=true
ftp.port=23

### RMI service ports ###
alfresco.rmi.services.port=50500
avm.rmi.service.port=0
avmsync.rmi.service.port=0
attribute.rmi.service.port=0
authentication.rmi.service.port=0
repo.rmi.service.port=0
action.rmi.service.port=0
deployment.rmi.service.port=0

### External executable locations ###
ooo.exe=/opt/Alfresco/libreoffice/program/soffice.bin
ooo.enabled=true
ooo.port=8100
img.root=/opt/alfresco-5.0.a/common
img.dyn=${img.root}/lib
img.exe=${img.root}/bin/convert
swf.exe=/opt/Alfresco/common/bin/pdf2swf
swf.languagedir=/opt/Alfresco/common/japanese

jodconverter.enabled=true
jodconverter.officeHome=/opt/Alfresco/libreoffice
jodconverter.portNumbers=8100

### Initial admin password ###
alfresco_user_store.adminpassword=XXX

### E-mail site invitation setting ###
notification.email.siteinvite=false

### License location ###
dir.license.external=/opt/alfresco-5.0.a

### Solr indexing ###
index.subsystem.name=solr
dir.keystore=${dir.root}/keystore
solr.port.ssl=8443

### BPM Engine ###
system.workflow.engine.jbpm.enabled=false

### Allow extended ResultSet processing
security.anyDenyDenies=false

Thanks for your help!

[briant]

I'm unaware of a log file specific to LO by default. There is a config for transformations. I haven't used it.

In my environment /opt/alfresco/tomcat/shared/classes/alfresco/extension/custom-log4j.properties.sample contains:

#log4j.logger.org.alfresco.repo.content.transform.TransformerDebug=debug

Are you able to post the jvm commandline for us to see the mx/ms/etc settings (ala "ps ax | grep java"). The default jvm settings may not be optimal in this instance and it canot use more than those settings allow, irrespective of the memory available to the OS.

and more than enough is free

This could actually be the problem if mx, etc too small - iirc the default is Xmx4096 ? which is quite limited for that document - it needs more.

[enigma969]

Enabling log4j.logger.org.alfresco.repo.content.transform.TransformerDebug=debug hasn't resulted in more outputs. 

Alfresco Process: 

/opt/Alfresco/java/bin/java -Djava.util.logging.config.file=/opt/Alfresco/tomcat/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -XX:MaxPermSize=512m -Xms128m -Xmx2G -XX:+DisableExplicitGC -Djava.awt.headless=true -Dalfresco.home=/opt/Alfresco -Dcom.sun.management.jmxremote -Dsun.security.ssl.allowUnsafeRenegotiation=true -XX:ReservedCodeCacheSize=128m [...]

Any idea how to start it with more memory? 

[briant]

I did some testing just now and on 5.1 I needed 2 extra lines in that file to make it work (mentioned here for v5.0 Troubleshooting OpenOffice subsystems | Alfresco Documentation 😞

log4j.logger.org.alfresco.enterprise.repo.content=DEBUG

log4j.logger.org.artofsolving.jodconverter=DEBUG

You will need to bounce tomcat afterwards I believe. Then you'll get info like so in catalina.out/alfresco.log:

2016-12-17 15:27:32,722 DEBUG [content.transform.TransformerDebug] [pool-13-thread-1] 2 ods pdf MyWhois-v7.2.ods 678.7 KB -- pdf -- ContentService.transform(...)
2016-12-17 15:27:32,724 DEBUG [content.transform.TransformerDebug] [pool-13-thread-1] 2 workspace://SpacesStore/92d8f49a-ae9d-4007-ad4c-7cc68412fa50
2016-12-17 15:27:32,724 DEBUG [content.transform.TransformerDebug] [pool-13-thread-1] 2 **a) [110] OpenOffice<<Proxy>> 0 ms
2016-12-17 15:27:32,725 DEBUG [content.transform.TransformerDebug] [pool-13-thread-1] 2.1 ods pdf MyWhois-v7.2.ods 678.7 KB OpenOffice<<Proxy>>
2016-12-17 15:28:05,043 DEBUG [content.transform.TransformerDebug] [pool-13-thread-1] 2 Finished in 32,323 ms

For your memory shortage I would start with changing Xmx2G => Xmx8G (or 4, but it seems there is "plenty" of free memory on this system), You can change it in /opt/alfresco/tomcat/bin/setenv.sh in the JAVA_OPTS line on my system. Yours may start using different scripts and could be located elsewhere.

Actually, can you confirm it is running a 64-bit jvm (otherwise going over 4G is nonsensical):

alfresco:/opt/alfresco/tomcat/shared/classes # java -version
...
OpenJDK 64-Bit Server VM (build ******, mixed mode)

It should be the 64-bit one if you installed from the Linux binary installer I believe, but I may be incorrect on that point.

Per the tuning guide I linked above:

For most cases, tuning the JVM is done in three steps:

1. Throw as much RAM as possible at the JVM (-Xmx32G),
2. Set the perm gen to 256M (-XX:MaxPermSize:256m), NB: ignored with java > 8
3. Don't add any other configuration settings.

Xmx32G would exceed your instance's capability leaving nothing for the OS, LO, etc => huge swapping, which is to be avoided at all costs. (Timeouts during complex conversions are much more likely [almost a given] when swap is being thrashed).

I am unaware if the PermGen needs changing - I have done so in the past on earlier alfresco versions (3.x, 4.x) but they were 32-bit java not 64 like I have now. Suggest monitor for those OutOfPermGen space messages, but I seem to recall that some work was done in this area recently - not certain though.

Edit: At startup I see this so I guess not: OpenJDK 64-Bit Server VM warning: ignoring option MaxPermSize=256M; support was removed in 8.0

Personally, I would jump straight to mx8G, monitor the system while it processes that document (and any others that fail), and if there is still plenty of free memory *and* conversions are still failing, up it again to 12, then 16. Provided the machine is not swapping, keep giving it more.

Lastly, the memory requirement for LO (at least 1G) and the OS must be taken into consideration and remember to leave some memory for the database also - I see your config indicates a local postgres instance.

Edits: fix message re permgen and java 8, add debug info and location of JAVA_OPTS

[enigma969]

I found this errors in alfresco.log before the other main exception I posted above: 

14:23:14,964 ERROR [org.alfresco.repo.content.transform.TransformerConfigDynamicTransformers] [http-apr-8080-exec-1] Cannot create dynamic transformer transformer.complex.JodConverter.Image as sub transformers could not be found or created ("JodConverter.2Pdf|pdf|complex.PDF.Image").
14:23:14,964 ERROR [org.alfresco.repo.content.transform.TransformerConfigDynamicTransformers] [http-apr-8080-exec-1] Cannot create dynamic transformer transformer.complex.JodConverter.Pdf2swf as sub transformers could not be found or created ("JodConverter.2Pdf|pdf|Pdf2swf").
14:23:14,964 ERROR [org.alfresco.repo.content.transform.TransformerConfigDynamicTransformers] [http-apr-8080-exec-1] Cannot create dynamic transformer transformer.JodConverter.Html2Pdf as sub transformers could not be found or created ("JodConverter|odt|JodConverter").
14:23:14,964 ERROR [org.alfresco.repo.content.transform.TransformerConfigDynamicTransformers] [http-apr-8080-exec-1] Cannot create dynamic transformer transformer.JodConverter.2Pdf as sub transformers could not be found or created ("JodConverter|JodConverter.Html2Pdf").
14:23:14,964 ERROR [org.alfresco.repo.content.transform.TransformerConfigDynamicTransformers] [http-apr-8080-exec-1] Cannot create dynamic transformer transformer.complex.JodConverter.PdfBox as sub transformers could not be found or created ("JodConverter.2Pdf|pdf|PdfBox").
14:23:15,118 INFO [org.alfresco.repo.management.subsystems.ChildApplicationContextFactory] [http-apr-8080-exec-1] Startup of 'Transformers' subsystem, ID: [Transformers, default] complete

Adding the two lines in the log4j file resulted in no error output at all. 

Setting max. Memory to 8 / 16GB lead to an "OutOfMemoryError: PermGen space" Exception and Tomcat didn't start at all.

setenv.conf

JAVA_HOME=/opt/Alfresco/java
JRE_HOME=/opt/Alfresco/java
JAVA_OPTS="-Xmx16G"
export JAVA_HOME
export JRE_HOME
export JAVA_OPTS

Java version 

java version "1.8.0_72"
Java(TM) SE Runtime Environment (build 1.8.0_72-b15)
Java HotSpot(TM) 64-Bit Server VM (build 25.72-b15, mixed mode)

Are you able to convert my document to a PDF with Alfresco? Maybe I just have to upgrade my Version? 

Why is JODConverter not working with my setup? 

[briant]

Why is JODConverter not working with my setup?

We don't have enough information yet to answer that question.

Did you install Java manually in /opt/Alfresco and Alfresco itself in /opt/alfresco-5.0.a ?  Better question: How was this system installed - using the Linux All-In-One installer, or manually?

I ask because you specified a non-default java location:

JAVA_HOME=/opt/Alfresco/java
JRE_HOME=/opt/Alfresco/java

..and yet in the global properties file the default installer locations are specified:

dir.root=/opt/alfresco-5.0.a/alf_data

...

jodconverter.officeHome=/opt/alfresco-5.0.a/libreoffice

It is perfectly OK to have java installed elsewhere (on my FreeBSD systems it lives under /usr...) - I am just trying to understand your setup and eliminate gotchas like incorrect paths and typos, running the default alfresco JVM instead of the system installed one, etc

Setting max. Memory to 8 / 16GB lead to an "OutOfMemoryError: PermGen space" Exception and Tomcat didn't start at all.

..

JAVA_OPTS="-Xmx16G"

EDIT: If it complained about PermGen space at startup then it possibly (probably?) is using the JDK that would have been installed by the all-in-one installer under /opt/alfresco-5.0.a/java/ - which (given that it produces that error) is most likely a Java 7 JDK - in which case the -XX:MaxPermSize=256M (or 512M since you have plenty of memory and complex docs) is a required entry in JAVA_OPTS or it exhibits exactly the behaviour you describe - failure at startup with an OutOfMemory: PermGen space exception.

With that in mind, can you confirm the JAVA_HOME location from a script that is used to start alfresco?  It can be overridden in several locations so you may need to check them all, depending how this system was installed... It was my bad before to suggest you run java -version at the command line - that will find and run the one on your path, which is not necessarily the same one used by alfresco.

End EDIT.

Our JAVA_OPTS variable has much more than just the Xmx entry, and I believe you will have problems without an Xms###M or -Dalfresco.home entries, etc etc.  It is no surprise to me that it failed to start with just an Xmx entry - or did you do something else in addition to the Xmx line and did not include that info?  Remember we cannot see what you did unless you tell us

In our config (from our 5.0.c backup on the old Atom server), /opt/alfresco/tomcat/bin/setenv.sh contains a JAVA_OPTS entry as below:

JAVA_OPTS="-XX:+DisableExplicitGC -Djava.awt.headless=true -Dalfresco.home=/opt/alfresco -Dcom.sun.management.jmxremote -XX:ReservedCodeCacheSize=128m -Xms512M -Xmx2048M -XX:MaxPermSize=256M $JAVA_OPTS"

(It was upgraded from Java7 -> 8, hence the MaxPermSize entry is still there. Memory constrained Atom -> Xmx2048M)

On another Ubuntu running 5.0.d, we have upstart script /etc/init/alfresco.conf which contains the JAVA_HOME and JAVA_OPTS definition.

Adding the two lines in the log4j file resulted in no error output at all. 

Please can you post your /opt/alfresco-5.0.a/tomcat/shared/classes/alfresco/extension/custom-log4j.properties here.  Mine configured as below works perfectly (*AND* noisily - you will not want to leave it on once this issue is solved, otherwise very many log entries will be written). The 2 RunTime entries are optional - it worked both with and without them.

log4j.logger.org.alfresco.repo.content.transform.TransformerDebug=debug

log4j.logger.org.alfresco.enterprise.repo.content=DEBUG

#log4j.logger.org.artofsolving.jodconverter=DEBUG We use OODirect so this entry is not needed for us
log4j.logger.org.alfresco.util.exec.RuntimeExecBootstrapBean=debug
log4j.logger.org.alfresco.util.exec.RuntimeExec=debug

Remember to always restart tomcat after making any changes to this file or the startup scripts.

Alterntively, why not spin up another VPS instance and install 5.0.d or 5.1 GA - as it is only for dev and test?  I feel that may be quicker, solves any bugs you will find running a "3 versions behind" 5.0.a system, and ensures the installer puts everything where it should be, including all startup scripts. Personally I would go directly to 5.1 GA.

Edits: Many

[enigma969]

Did you install Java manually in /opt/Alfresco and Alfresco itself in /opt/alfresco-5.0.a ?  Better question: How was this system installed - using the Linux All-In-One installer, or manually?

I ask because you specified a non-default java location:

JAVA_HOME=/opt/Alfresco/java
JRE_HOME=/opt/Alfresco/java

..and yet in the global properties file the default installer locations are specified:

dir.root=/opt/alfresco-5.0.a/alf_data

...

jodconverter.officeHome=/opt/alfresco-5.0.a/libreoffice

It is perfectly OK to have java installed elsewhere (on my FreeBSD systems it lives under /usr...) - I am just trying to understand your setup and eliminate gotchas like incorrect paths and typos, running the default alfresco JVM instead of the system installed one, etc

Sorry for the confusion, it is /opt/Alfresco of course, the file here is from another machine I tried using exactly the same configuration, except this alfresco path. Same alfresco version, same errors. This path shouldn't be the cause. I updated the config file.  

Our JAVA_OPTS variable has much more than just the Xmx entry, and I believe you will have problems without an Xms###M or -Dalfresco.home entries, etc etc.  It is no surprise to me that it failed to start with just an Xmx entry - or did you do something else in addition to the Xmx line and did not include that info?  Remember we cannot see what you did unless you tell us

In our config (from our 5.0.c backup on the old Atom server), /opt/alfresco/tomcat/bin/setenv.sh contains a JAVA_OPTS entry as below:

JAVA_OPTS="-XX:+DisableExplicitGC -Djava.awt.headless=true -Dalfresco.home=/opt/alfresco -Dcom.sun.management.jmxremote -XX:ReservedCodeCacheSize=128m -Xms512M -Xmx2048M -XX:MaxPermSize=256M $JAVA_OPTS"

(It was upgraded from Java7 -> 8, hence the MaxPermSize entry is still there. Memory constrained Atom -> Xmx2048M)

Originally, I just had "JAVA_OPTS="$JAVA_OPTS " in setenv.sh. Now I set JAVA_OPTS as in your file just with more memory: 

JAVA_HOME=/opt/Alfresco/java
JRE_HOME=/opt/Alfresco/java
JAVA_OPTS="-XX:+DisableExplicitGC -Djava.awt.headless=true -Dalfresco.home=/opt/Alfresco -Dcom.sun.management.jmxremote -XX:ReservedCodeCacheSize=128m -Xms2048M -Xmx16G -XX:MaxPermSize=2048M $JAVA_OPTS"
export JAVA_HOME
export JRE_HOME
export JAVA_OPTS

and now Alfresco starts without problems. But the error for this document still appears. 

Please can you post your /opt/alfresco-5.0.a/tomcat/shared/classes/alfresco/extension/custom-log4j.properties here.  Mine configured as below works perfectly (*AND* noisily - you will not want to leave it on once this issue is solved, otherwise very many log entries will be written). The 2 RunTime entries are optional - it worked both with and without them.

The file looks exactly the same as yours. No additional error logs as I already posted. 

Alterntively, why not spin up another VPS instance and install 5.0.d or 5.1 GA - as it is only for dev and test?  I feel that may be quicker, solves any bugs you will find running a "3 versions behind" 5.0.a system, and ensures the installer puts everything where it should be, including all startup scripts. Personally I would go directly to 5.1 GA.

I set up a 5.0.d instance and got exactly the same errors. 

Have you tried converting this document on one of your machines? Maybe it is just a document that can't be converted with Alfresco/LO at all? 

Thanks for your support.

[briant]

Is the custom-log4j.properties file readable by the user (usually alfresco) running tomcat?  That's the only reason I can think of unless logs have been redirected elsewhere or logging disabled somehow.

I added the debug lines to our 5.0.d and it produced debug entries in /var/log/upstart/alfresco.log when adding content. I am at a loss as to why it doesn't work on your systems.

Have you tried converting this document on one of your machines?

I don't have a dev machine to test it on unfortunately.

Wish I could help more but I am out of ideas now. 5.1 install perhaps? Up the memory and try again?

[michael_s1]

I was of the impression that there was a 1MB limit on conversion. I could be wrong. 

Michael

[enigma969]

No, if I save the document as application/msword it has 8mb and it gets converted without problems.

[mrks_js1 · Accepted solution]

these are the defaults (taken from https://svn.alfresco.com/repos/alfresco-open-mirror/alfresco/HEAD/root/projects/repository/config/al... 😞

content.transformer.OpenOffice.extensions.docx.pdf.maxSourceSizeKBytes=768
content.transformer.OpenOffice.extensions.doc.pdf.maxSourceSizeKBytes=10240‍‍‍‍

application/msword = .doc

!=

application/vnd.openxmlformats-officedocument.wordprocessingml.document = .docx

try adding 

content.transformer.OpenOffice.extensions.docx.pdf.maxSourceSizeKBytes=10240‍‍‍‍‍