topic Re: "OCR Extract" action doesn't work well (alfresco-simple-ocr + pdfsandwich) in Alfresco Forum

"OCR Extract" action doesn't work well (alfresco-simple-ocr + pdfsandwich)

hisayo-s — Thu, 16 Aug 2018 02:51:03 GMT

Hello,I'm using Alfresco 5.2 community edition on CentOS7.5 and it works well itself.Now I trying to add OCR function to Alfresco, so I installed alfresco-simple-ocr (simple-ocr-repo-2.3.1.jar) and pdfsandwich to add function.When I install pdfsandwich version 1.4, ruled "Extract OCR" action do work

Re: "OCR Extract" action doesn't work well (alfresco-simple-ocr + pdfsandwich)

douglascrp — Fri, 17 Aug 2018 17:30:42 GMT

Hello.

Check the following link FAQ · keensoft/alfresco-simple-ocr Wiki · GitHub

Maybe that can help you.

Re: "OCR Extract" action doesn't work well (alfresco-simple-ocr + pdfsandwich)

hisayo-s — Tue, 21 Aug 2018 00:33:55 GMT

Hi,

Thank you for your information.

I haven't read the FAQ page, so I will read the FAQ carefully and try to improve my environment.

I hope good result.

Best regards,

Re: "OCR Extract" action doesn't work well (alfresco-simple-ocr + pdfsandwich)

hisayo-s — Tue, 21 Aug 2018 05:28:45 GMT

Hello,

My problems has been partly solved.

Reading FAQ, I installed 2 jar files (simple-ocr-repo-2.3.1.jar and simple-ocr-share-2.3.1.jar) insted of simple-ocr-repo.amp. (I had used amp file)

After restarted alfresco, the "Extract PDF action" sometimes works well, and sometimes not.

When action(conversion) succeed, "tesseract" ".convers.b+" "unpaper" processes are running on "top" view.

Otherwise when action(conversion) fails, their processes appears shortly and soon disappears.

It seems that file size and number of page are unrelated.

I have no idea how to solve this problem.

Anyone know the solution. Please let me know!

------

- CentOS 7.5

- Alfresco 5.2 - community edition

- alfresco-simple-ocr 2.3.1

- pdfsandwich is 1.6 (*1)

- tesseract 3.04

(*1)

When I tested version 1.7 again, pdfsandwich says following message as before.

> "Fatal error: exception Unix.Unix_error(Unix.ENOTEMPTY, "rmdir", "/tmp/pdfsandwich_tmp2d3ca3")"

Such being the case, I use version 1.7 with "-debug" option to avoid error. (temp files should be erased manually...)

Re: "OCR Extract" action doesn't work well (alfresco-simple-ocr + pdfsandwich)

angelborroy — Wed, 22 Aug 2018 10:04:28 GMT

Try using OCRmyPDF (https://github.com/jbarlow83/OCRmyPDF) instead of pdfsandwich.

Both pdfsandwich and OCRmyPDF have some issues on CentOS (they are developed for Ubuntu), but you can use the Docker Image for OCRmyPDF available at https://ocrmypdf.readthedocs.io/en/latest/installation.html#installing-the-docker-image

Re: "OCR Extract" action doesn't work well (alfresco-simple-ocr + pdfsandwich)

hisayo-s — Thu, 23 Aug 2018 01:48:16 GMT

Hello,

Thank you for your suggestion.

Unfortunately I'm not familiar with Docker.

I tried to install OCRmyPDF, but I could't.

So, I'm going to continue struggling to use pdfsandwich.

Re: "OCR Extract" action doesn't work well (alfresco-simple-ocr + pdfsandwich)

angelborroy — Thu, 23 Aug 2018 05:23:24 GMT

I don’t know if this still works, as I haven’t tested it recently, but you can find a reference for installing pdfsandwich at CentOS 7 at https://github.com/keensoft/alfresco-simple-ocr/blob/master/docker/pdfsandwich-1.6-centos-7/Dockerfile

Re: "OCR Extract" action doesn't work well (alfresco-simple-ocr + pdfsandwich)

hisayo-s — Tue, 28 Aug 2018 07:07:20 GMT

I tried to install "pdfsandwich" and "OCRmyPDF"(with docker), however I couldn't set up propery.

It is a pitty that I give up to try.

Thank you very much for giving suggestions and informations.

Re: "OCR Extract" action doesn't work well (alfresco-simple-ocr + pdfsandwich)

hisayo-s — Tue, 28 Aug 2018 07:53:25 GMT

Thank you for your information.

I tried to install "OCRmyPDF" using Docker and partly successed to install it.

On the command line, and at the directory where the inputfile exist, conversion successfully done.

However at the othe directory, it does not work.

> ERROR - File not found - /home/hisayo-s/AAAAA.pdf

I give up my challenge.

Thanks a lot.

Re: "OCR Extract" action doesn't work well (alfresco-simple-ocr + pdfsandwich)

angelborroy — Tue, 28 Aug 2018 08:17:37 GMT

I'm currently using Docker Compose as base for my installations, so I only can give you some tips on how to configure the whole thing with Docker.

OCRmyPDF Dockerfile

FROM jbarlow83/ocrmypdf:v7.0.0
USER root

RUN apt-get update && apt-get install -y openssh-server
RUN mkdir /var/run/sshd
RUN echo 'root:screencast' | chpasswd
RUN sed -i 's/PermitRootLogin prohibit-password/PermitRootLogin yes/' /etc/ssh/sshd_config

# SSH login fix. Otherwise user is kicked off after login
RUN sed 's@session\s*required\s*pam_loginuid.so@session optional pam_loginuid.so@g' -i /etc/pam.d/sshd

ENV NOTVISIBLE "in users profile"
RUN echo "export VISIBLE=now" >> /etc/profile

COPY assets/ssh/id_rsa.pub /root/.ssh/id_rsa.pub
COPY assets/ocr.sh /usr/bin/ocr.sh
RUN cat /root/.ssh/id_rsa.pub >> /root/.ssh/authorized_keys \
 && chmod 0600 /root/.ssh/authorized_keys \
 && chmod +x /usr/bin/ocr.sh

EXPOSE 22
ENTRYPOINT ["/usr/sbin/sshd", "-D"]‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

assets/ocr.sh

#!/bin/bash

export LC_ALL=C.UTF-8
export LANG=C.UTF-8

/usr/bin/ocrmypdf $@‍‍‍‍‍‍

Alfresco Dockerfile

FROM alfresco/alfresco-content-repository-community:6.0.7-ga

ENV LC_ALL C.UTF-8
ENV LANG C.UTF-8

# Extra software
RUN set -x \
     && yum install -y \
     wget \
     unzip \
     && yum clean all

# Install api-explorer webapp for REST API
RUN set -x \
     && wget https://artifacts.alfresco.com/nexus/service/local/repositories/releases/content/org/alfresco/api-explorer/6.0.7-ga/api-explorer-6.0.7-ga.war -O /usr/local/tomcat/webapps/api-explorer.war

ARG TOMCAT_DIR=/usr/local/tomcat

RUN mkdir -p $TOMCAT_DIR/amps

# Install AOS
RUN set -x \
        && mkdir /tmp/aos \
        && wget --no-check-certificate https://download.alfresco.com/cloudfront/release/community/201806-GA-build-00113/alfresco-aos-module-distributionzip-1.2.0.zip \
        && unzip alfresco-aos-module-distributionzip-1.2.0.zip -d /tmp/aos \
        && mv /tmp/aos/extension/* /usr/local/tomcat/shared/classes/alfresco/extension \
        && mv /tmp/aos/alfresco-aos-module-1.2.0.amp amps \
        && mv /tmp/aos/aos-module-license.txt licenses \
        && mv /tmp/aos/_vti_bin.war /usr/local/tomcat/webapps \
        && rm -rf /tmp/aos alfresco-aos-module-distributionzip-1.2.0.zip

# SSH keys for ocrmypdf
COPY ssh/ /root/.ssh/

# Install OCR
COPY bin/ /opt/alfresco/bin/

# Configure SSH Client
RUN set -x && \
    chmod +x /opt/alfresco/bin/ocrmypdf.sh && \
    # Configure ssh    
    yum install -y openssh-clients && \
    echo "StrictHostKeyChecking no" >> /etc/ssh/ssh_config && \
    # Alfresco Image is using POSIX as Locale (!)
    sed -i '/^\s*SendEnv/ d' /etc/ssh/ssh_config && \
    chmod 600 /root/.ssh/id_rsa

# Install modules and addons
COPY modules/amps $TOMCAT_DIR/amps
COPY modules/jars $TOMCAT_DIR/webapps/alfresco/WEB-INF/lib

RUN java -jar $TOMCAT_DIR/alfresco-mmt/alfresco-mmt*.jar install \
            $TOMCAT_DIR/amps $TOMCAT_DIR/webapps/alfresco -directory -nobackup -force

# Add services configuration to alfresco-global.properties
COPY conf/alfresco-global.properties /usr/local/tomcat/shared/classes/alfresco-global.properties

EXPOSE 21 143 25 445 137/udp 138/udp 139‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

bin/ocrmypdf.sh

#!/bin/bash

INPUT_DIR=/ocr_input
OUTPUT_DIR=/ocr_output

# ocrmypdf hostname
OCRMYPDF_SERVER="ocrmypdf"

# identify parameters, input and output file
array=( "$@" )
len=${#array[@]}
ARGS=${array[@]:0:$len-2}

LAST_ARGS="${@: -2}"
INPUT_FILE_PARAM=`echo "$LAST_ARGS" | cut -d ' ' -f 1`
OUTPUT_FILE_PARAM=`echo "$LAST_ARGS" | cut -d ' ' -f 2`

# extract filenames
INPUT_FILE=$(basename "$INPUT_FILE_PARAM")
OUTPUT_FILE=$(basename "$OUTPUT_FILE_PARAM")

# SSH parameters
SCP=cp
SSH=ssh
USER=root

# copy original pdf to ocrmypdf server
$SCP $INPUT_FILE_PARAM $INPUT_DIR

# execute ocrmypdf program
$SSH $USER@$OCRMYPDF_SERVER "/usr/bin/ocr.sh $ARGS $INPUT_DIR/$INPUT_FILE $OUTPUT_DIR/$OUTPUT_FILE"

# copy transformed pdf back to alfresco path
$SCP $OUTPUT_DIR/$OUTPUT_FILE ${OUTPUT_FILE_PARAM}

# remove temporal files
rm -f $INPUT_DIR/$INPUT_FILE $OUTPUT_DIR/$OUTPUT_FILE‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

conf/alfresco-global.properties

(Only OCRmyPDF section)

## simple-ocr
# https://github.com/keensoft/alfresco-simple-ocr
ocr.command=/opt/alfresco/bin/ocrmypdf.sh
ocr.output.verbose=true
ocr.output.file.prefix.command=
# https://github.com/jbarlow83/OCRmyPDF/issues/124
ocr.extra.commands=-j1 --author keensoft --rotate-pages -l spa+eng+fra --deskew --clean --skip-text
ocr.server.os=linux‍‍‍‍‍‍‍‍

Re: "OCR Extract" action doesn't work well (alfresco-simple-ocr + pdfsandwich)

hisayo-s — Thu, 30 Aug 2018 00:20:18 GMT

Thanks you for your kindness.

However, my environment is consist of CentOS7 and Alfresco5.2 and OCRmyPDF(docker).

The scripts you have posted aren't match my environment.

As I am very new to docker, I don't know how to change the scripts.

Re: "OCR Extract" action doesn't work well (alfresco-simple-ocr + pdfsandwich)

hisayo-s — Thu, 30 Aug 2018 01:25:29 GMT

Comparing pdfsandwich to OCRmyPDF, pdfsandwich's quality for letter recognition is better than OCRmyPDF in Japanese.

So I will focused on using pdfsandwich.

Thank you very much for your help.

Re: "OCR Extract" action doesn't work well (alfresco-simple-ocr + pdfsandwich)

angelborroy — Thu, 30 Aug 2018 06:53:25 GMT

Did you test with these instructions?

https://github.com/keensoft/alfresco-simple-ocr/blob/master/docker/pdfsandwich-1.6-centos-7/Dockerfile

I don't know if they are still working with latest CentOS releases, but it can be an starting point.

Re: "OCR Extract" action doesn't work well (alfresco-simple-ocr + pdfsandwich)

fedorow — Thu, 27 Jun 2019 15:33:25 GMT

I try go over this solution. My deployment:

Alfresco 6.1.2-ga / Share 6.1.0

jbarlow83/ocrmypdf:v8.2.3 or v7.0.0

api-explorer-6.1.0-ea.war or 6.0.7-ga

And I have got "failed to copy".

I had file /usr/local/tomcat/temp/Alfresco/OCRTransformWorker_source_5503547424193883468.pdf but /usr/local/tomcat/temp/Alfresco/OCRTransformWorker_source_5503547424193883468_ocr.pdf don't.

My thought, I should change

INPUT_DIR=/ocr_input
OUTPUT_DIR=/ocr_output

but i don't understand how. "ocrmypdf" container don't contain this directories.

Log:

alfresco_1 | Exception in thread "defaultAsyncAction1" java.lang.RuntimeException: java.lang.RuntimeException: org.alfresco.service.cmr.repository.ContentIOException: 05270018 Failed to copy content from file:
alfresco_1 | writer: ContentAccessor[ contentUrl=store://2019/6/27/18/13/0081dc19-8750-4ddb-ac3c-396b4ba1a859.bin, mimetype=application/pdf, size=0, encoding=UTF-8, locale=en_US]
alfresco_1 | file: /usr/local/tomcat/temp/Alfresco/OCRTransformWorker_source_5503547424193883468_ocr.pdf
alfresco_1 | at es.keensoft.alfresco.ocr.OCRExtractAction.executeImplInternal(OCRExtractAction.java:183)
alfresco_1 | at es.keensoft.alfresco.ocr.OCRExtractAction.access$200(OCRExtractAction.java:38)
alfresco_1 | at es.keensoft.alfresco.ocr.OCRExtractAction$1.execute(OCRExtractAction.java:164)
alfresco_1 | at es.keensoft.alfresco.ocr.OCRExtractAction$1.execute(OCRExtractAction.java:161)
alfresco_1 | at org.alfresco.repo.transaction.RetryingTransactionHelper.doInTransaction(RetryingTransactionHelper.java:450)
alfresco_1 | at es.keensoft.alfresco.ocr.OCRExtractAction.executeInNewTransaction(OCRExtractAction.java:169)
alfresco_1 | at es.keensoft.alfresco.ocr.OCRExtractAction.access$100(OCRExtractAction.java:38)
alfresco_1 | at es.keensoft.alfresco.ocr.OCRExtractAction$ExtractOCRTask.run(OCRExtractAction.java:151)
alfresco_1 | at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
alfresco_1 | at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
alfresco_1 | at java.base/java.lang.Thread.run(Thread.java:834)
alfresco_1 | Caused by: java.lang.RuntimeException: org.alfresco.service.cmr.repository.ContentIOException: 05270018 Failed to copy content from file:
alfresco_1 | writer: ContentAccessor[ contentUrl=store://2019/6/27/18/13/0081dc19-8750-4ddb-ac3c-396b4ba1a859.bin, mimetype=application/pdf, size=0, encoding=UTF-8, locale=en_US]
alfresco_1 | file: /usr/local/tomcat/temp/Alfresco/OCRTransformWorker_source_5503547424193883468_ocr.pdf
alfresco_1 | at es.keensoft.alfresco.ocr.OCRTransformWorker.transform(OCRTransformWorker.java:86)
alfresco_1 | at es.keensoft.alfresco.ocr.OCRExtractAction.executeImplInternal(OCRExtractAction.java:181)
alfresco_1 | ... 10 more
alfresco_1 | Caused by: org.alfresco.service.cmr.repository.ContentIOException: 05270018 Failed to copy content from file:
alfresco_1 | writer: ContentAccessor[ contentUrl=store://2019/6/27/18/13/0081dc19-8750-4ddb-ac3c-396b4ba1a859.bin, mimetype=application/pdf, size=0, encoding=UTF-8, locale=en_US]
alfresco_1 | file: /usr/local/tomcat/temp/Alfresco/OCRTransformWorker_source_5503547424193883468_ocr.pdf
alfresco_1 | at org.alfresco.repo.content.AbstractContentWriter.putContent(AbstractContentWriter.java:491)
alfresco_1 | at es.keensoft.alfresco.ocr.OCRTransformWorker.transform(OCRTransformWorker.java:83)
alfresco_1 | ... 11 more
alfresco_1 | Caused by: java.io.FileNotFoundException: /usr/local/tomcat/temp/Alfresco/OCRTransformWorker_source_5503547424193883468_ocr.pdf (No such file or directory)
alfresco_1 | at java.base/java.io.FileInputStream.open0(Native Method)
alfresco_1 | at java.base/java.io.FileInputStream.open(FileInputStream.java:219)
alfresco_1 | at java.base/java.io.FileInputStream.<init>(FileInputStream.java:157)
alfresco_1 | at org.alfresco.repo.content.AbstractContentWriter.putContent(AbstractContentWriter.java:485)
alfresco_1 | ... 12 more

Re: "OCR Extract" action doesn't work well (alfresco-simple-ocr + pdfsandwich)

fedorow — Tue, 02 Jul 2019 13:20:07 GMT

So, to make it works on Alfresco/Share CE 6.1.2-ga/6.1.0 I made shared volume between alfresco and ocrmypdf containers. I replace /ocr_input and /ocr_output to one directory /ocr and map it as volume for both containers.

Only one problem, asynchronous mode for rule gives me error. So I turn it off.

Angel thanks!

docker-compose.yml

...
services:
   alfresco:
      ...
      volumes:
         - ocr:/ocr
      ...

   ocrmypdf:
      ...
      volumes:
         - ocr:/ocr
   ...
volumes:
   ...
   ocr:
      driver: local
...

bin/ocrmypdf.sh

(and remove {} from $OUTPUT_FILE_PARAM in copy output file command)

#!/bin/bash

INPUT_DIR=/ocr
OUTPUT_DIR=/ocr

# ocrmypdf hostname
OCRMYPDF_SERVER="ocrmypdf"

# identify parameters, input and output file
array=( "$@" )
len=${#array[@]}
ARGS=${array[@]:0:$len-2}

LAST_ARGS="${@: -2}"
INPUT_FILE_PARAM=`echo "$LAST_ARGS" | cut -d ' ' -f 1`
OUTPUT_FILE_PARAM=`echo "$LAST_ARGS" | cut -d ' ' -f 2`

# extract filenames
INPUT_FILE=$(basename "$INPUT_FILE_PARAM")
OUTPUT_FILE=$(basename "$OUTPUT_FILE_PARAM")

# SSH parameters
SCP=cp
SSH=ssh
USER=root

# copy original pdf to ocrmypdf server
$SCP $INPUT_FILE_PARAM $INPUT_DIR

# execute ocrmypdf program
$SSH $USER@$OCRMYPDF_SERVER "/usr/bin/ocr.sh $ARGS $INPUT_DIR/$INPUT_FILE $OUTPUT_DIR/$OUTPUT_FILE"

# copy transformed pdf back to alfresco path
$SCP $OUTPUT_DIR/$OUTPUT_FILE $OUTPUT_FILE_PARAM

# remove temporal files
rm -f $INPUT_DIR/$INPUT_FILE $OUTPUT_DIR/$OUTPUT_FILE

Re: "OCR Extract" action doesn't work well (alfresco-simple-ocr + pdfsandwich)

SriramG — Thu, 18 Jun 2020 21:30:16 GMT

With the approach suggested by Fedorow, I was able to make OCR work with Alfresco 6.1.0. I update ocr_input and /ocr_output to /usr/local/tomcat/ocr_input and /usr/local/tomcat/ocr_out so that alfresco container can access these folders without any access issues.

Thanks Fedorow

docker-compose.yml

...
services:
   alfresco:
      ...
      volumes:
         - ocr-input:/usr/local/tomcat/ocr_input
         - ocr-output:/usr/local/tomcat/ocr_output
      ...

   ocrmypdf:
      ...
      volumes:
             - ocr-input:/usr/local/tomcat/ocr_input
             - ocr-output:/usr/local/tomcat/ocr_output
   ...
volumes:
   ...
  ocr-input:
       external: true
  ocr-output:
       external: true
...

bin/ocrmypdf.sh

#!/bin/bash

INPUT_DIR=/usr/local/tomcat/ocr_input
OUTPUT_DIR=/usr/local/tomcat/ocr_output

# ocrmypdf hostname
OCRMYPDF_SERVER="ocrmypdf"

# identify parameters, input and output file
array=( "$@" )
len=${#array[@]}
ARGS=${array[@]:0:$len-2}

LAST_ARGS="${@: -2}"
INPUT_FILE_PARAM=`echo "$LAST_ARGS" | cut -d ' ' -f 1`
OUTPUT_FILE_PARAM=`echo "$LAST_ARGS" | cut -d ' ' -f 2`

# extract filenames
INPUT_FILE=$(basename "$INPUT_FILE_PARAM")
OUTPUT_FILE=$(basename "$OUTPUT_FILE_PARAM")

# SSH parameters
SCP=cp
SSH=ssh
USER=root

# copy original pdf to ocrmypdf server
$SCP $INPUT_FILE_PARAM $INPUT_DIR

# execute ocrmypdf program
$SSH $USER@$OCRMYPDF_SERVER "/usr/bin/ocr.sh $ARGS $INPUT_DIR/$INPUT_FILE $OUTPUT_DIR/$OUTPUT_FILE"

# copy transformed pdf back to alfresco path
$SCP $OUTPUT_DIR/$OUTPUT_FILE $OUTPUT_FILE_PARAM

# remove temporal files
rm -f $INPUT_DIR/$INPUT_FILE $OUTPUT_DIR/$OUTPUT_FILE

After the above changes I was able to successfully run OCR with Alfresco 6.1.

As we are running our Alfresco instance on Kubernetes and using HELM deployment, I need to configure the volumes in values.yaml file but I am not sure how to configure the volumes in values.yaml file. Any one has idea on how we need to make similar configuration in kubernetes.

Any help apprecaited.

Re: "OCR Extract" action doesn't work well (alfresco-simple-ocr + pdfsandwich)

EddieMay — Mon, 22 Jun 2020 11:00:25 GMT

Hi @SriramG,

Thanks for updating us on how you resolved your issue - really helpful.

Maybe start a new thread for your question about configuring volumes?

Cheers,