Complex pipeline transformation not recognized

[jpotts]

I have created the following transformers:

* JSON-to-XML
* XML-to-XSLFO
* XSLFO-to-PDF

Each of these work individually. In other words, I can create folders with rules that transform from one format to another using each of these transformations.

Now I'd like to create a complex or pipeline transformation that will go from JSON to PDF in one step.

To do this, I've added a pipeline config to my existing tranformer properties file in subsystems/Transformers/default/default/someco_transformers.properties as follows:

# FO to PDF
content.transformer.FOP.PDF.priority=50
content.transformer.FOP.PDF.extensions.fo.pdf.supported=true
content.transformer.FOP.PDF.extensions.fo.pdf.priority=50

# JSON to XML
content.transformer.JSON.XML.priority=50
content.transformer.JSON.XML.extensions.json.xml.supported=true
content.transformer.JSON.XML.extensions.json.xml.priority=50

# XML to XSL-FO
content.transformer.XML.XSL.priority=50
content.transformer.XML.XSL.extensions.xml.fo.supported=true
content.transformer.XML.XSL.extensions.xml.fo.priority=50

# JSON to PDF
content.transformer.complex.JSON.PDF.pipeline=JSON.XML|xml|XML.XSL|fo|FOP.PDF
content.transformer.complex.JSON.PDF.extensions.json.pdf.priority=50
content.transformer.complex.JSON.PDF.extensions.json.pdf.supported=true
‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

When I then upload my JSON document to a folder with a rule that transforms from JSON to PDF I get the following:

 2015-11-30 20:58:59,004  ERROR [extensions.webscripts.AbstractRuntime] [http-bio-8080-exec-5] Exception from executeScript - redirecting to status template error: 10300028 Transformer for 'application/json' source mime type and 'application/pdf' target mime type was not found. Operation can't be performed
 org.alfresco.service.cmr.rule.RuleServiceException: 10300028 Transformer for 'application/json' source mime type and 'application/pdf' target mime type was not found. Operation can't be performed
‍‍‍‍

I have also tried adding:

content.transformer.complex.JSON.PDF.mimetypes.application/json.application/pdf.supported=true
‍‍‍

But that does not seem to make a difference.

Do I need anything else to enable my pipeline?

Are the pipeline properties set properly?

Any tips appreciated.

Jeff

[afaust]

Hello Jeff,

the transformer properties look ok to me. You can actually drop the "complex" name fragment unless you want this as part of a naming convention. The variant with mimetype configuration is not doing anything special as the evaluation of the configuration just resolves mimetypes to the corresponding extensions anyway.

By any chance do you have (global) limits to source size / read time in place? I have on occassion been trumped by size limits, especially with failover transformers in pipelines that need to account for intermediary (potentially bloated)sizes, e.g. in OCR when a PBM image file can be several hundred megabytes.

Have you tried getting any insight from TransformerDebug logging?

Regards
Axel

[jpotts]

Hi Axel,

Here is the output from the transform debugger:

2015-12-01 21:06:25,608  INFO  [management.subsystems.ChildApplicationContextFactory] [http-bio-8080-exec-1] Starting 'Transformers' subsystem, ID: [Transformers, default]
 2015-12-01 21:06:25,677  ERROR [content.transform.TransformerConfigDynamicTransformers] [http-bio-8080-exec-1] Cannot create dynamic transformer transformer.complex.JSON.PDF as sub transformers could not be found or created ("JSON.XML|xml|XML.XSL|fo|FOP.PDF").
 2015-12-01 21:06:25,779  INFO  [management.subsystems.ChildApplicationContextFactory] [http-bio-8080-exec-1] Startup of 'Transformers' subsystem, ID: [Transformers, default] complete
 2015-12-01 21:06:25,789  DEBUG [content.transform.TransformerDebug] [http-bio-8080-exec-1] 0             json pdf  source2.json 82 bytes – syncRule – ContentService.getTransformer(…) NO transformers
 2015-12-01 21:06:25,789  DEBUG [content.transform.TransformerDebug] [http-bio-8080-exec-1] 0             Finished in 181 ms Transformer NOT called

 2015-12-01 21:06:25,807  ERROR [extensions.webscripts.AbstractRuntime] [http-bio-8080-exec-1] Exception from executeScript - redirecting to status template error: 11010023 Transformer for 'application/json' source mime type and 'application/pdf' target mime type was not found. Operation can't be performed
 org.alfresco.service.cmr.rule.RuleServiceException: 11010023 Transformer for 'application/json' source mime type and 'application/pdf' target mime type was not found. Operation can't be performed
‍‍‍‍‍‍‍‍‍‍

This is really frustrating. The log message indicates that the sub-transformers could not be found, but they work individually.

I'm really close to scrapping the transformation framework and just doing it all within an action, but I hate to do that.

Jeff

[jpotts]

Interestingly, this works:

content.transformer.complex.JSON.PDF.pipeline=*|xml|*|fo|*
‍‍‍

And outputs the following transform debug:

 2015-12-01 22:03:24,331  INFO  [management.subsystems.ChildApplicationContextFactory] [http-bio-8080-exec-10] Startup of 'Transformers' subsystem, ID: [Transformers, default] complete
 2015-12-01 22:03:24,345  DEBUG [content.transform.TransformerDebug] [http-bio-8080-exec-10] 0             json pdf  source2.json 82 bytes – syncRule – ContentService.getTransformer(…)
 2015-12-01 22:03:24,347  DEBUG [content.transform.TransformerDebug] [http-bio-8080-exec-10] 0             **a)  [50] complex.JSON.PDF<<Complex>>           0 ms
 2015-12-01 22:03:24,347  DEBUG [content.transform.TransformerDebug] [http-bio-8080-exec-10] 0             Finished in 186 ms Transformer NOT called

 2015-12-01 22:03:24,382  DEBUG [content.transform.TransformerDebug] [http-bio-8080-exec-10] 1             json pdf  source2.json 82 bytes – syncRule – ContentService.transform(…)
 2015-12-01 22:03:24,383  DEBUG [content.transform.TransformerDebug] [http-bio-8080-exec-10] 1             **a)  [50] complex.JSON.PDF<<Complex>>           0 ms
 2015-12-01 22:03:24,383  DEBUG [content.transform.TransformerDebug] [http-bio-8080-exec-10] 1.1           json pdf  source2.json 82 bytes complex.JSON.PDF<<Complex>>
 2015-12-01 22:03:24,386  DEBUG [content.transform.TransformerDebug] [http-bio-8080-exec-10] 1.1.1         json xml  source2.json 82 bytes ContentService.transform(…)
 2015-12-01 22:03:24,386  DEBUG [content.transform.TransformerDebug] [http-bio-8080-exec-10] 1.1.1         **a)  [50] JSON.XML           0 ms
 2015-12-01 22:03:24,386  DEBUG [content.transform.TransformerDebug] [http-bio-8080-exec-10] 1.1.1.1       json xml  source2.json 82 bytes JSON.XML
 2015-12-01 22:03:24,394  DEBUG [content.transform.TransformerDebug] [http-bio-8080-exec-10] 1.1.2         xml  fo   <<TemporaryFile>> 117 bytes ContentService.transform(…)
 2015-12-01 22:03:24,394  DEBUG [content.transform.TransformerDebug] [http-bio-8080-exec-10] 1.1.2         **a) [100] XML.FO           0 ms
 2015-12-01 22:03:24,394  DEBUG [content.transform.TransformerDebug] [http-bio-8080-exec-10] 1.1.2.1       xml  fo   <<TemporaryFile>> 117 bytes XML.FO
 2015-12-01 22:03:24,560  DEBUG [content.transform.TransformerDebug] [http-bio-8080-exec-10] 1.1.3         fo   pdf  <<TemporaryFile>> 696 bytes ContentService.transform(…)
 2015-12-01 22:03:24,561  DEBUG [content.transform.TransformerDebug] [http-bio-8080-exec-10] 1.1.3         **a)  [50] FOP.PDF           0 ms
 2015-12-01 22:03:24,561  DEBUG [content.transform.TransformerDebug] [http-bio-8080-exec-10] 1.1.3.1       fo   pdf  <<TemporaryFile>> 696 bytes FOP.PDF
 2015-12-01 22:03:25,350  DEBUG [content.transform.TransformerDebug] [http-bio-8080-exec-10] 1             Finished in 971 ms
‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

So something is fishy with the names of my transformers I guess.

Jeff

[afaust]

Hello Jeff,

with the logger for TransformerConfigDynamicTransformers set to trace you can verify if the name lookup for the pipeline transformers is the cause. Technically, there can be other kind of "IllegalArgumentException" issues but that type of error is silently swallowed and I only found this one trace log statement.

In what location / Spring context file did you define your simple transformer beans?  Maybe this is a simple timing issue, i.e. the *.properties file is evaluated before your transformer beans are registered and available. Using the wildccard, reslution will take place at transformation time, so transformers would be guaranteed to have been registered by then.

At least I learned something new today while looking through the source: It is possible to define edition specific transformers as well as associate transformers with AMPs so they are only available when the AMP is installed.

Regards
Axel