Open the PDF Form using Acrobat Pro X
Create the Custom Properties
File => Properties, Custom Tab: add name/value pairs
Add JavaScript to PDF Forms
Change to Forms Edit (Edit=>Tools=>Form then Edit)
Select a field, right-click, Properties
Select the Actions tab, trigger: Mouse Up, action: Run a JavaScript
Edit the JavaScript and enter:
Caveat: While this will copy data from fields to properties when using Acrobat Pro X it does not appear to work when using Acrobat Reader
reference: http://forums.adobe.com/thread/859315
We have not found a solution at this time.
Create the Custom Properties
File => Properties, Custom Tab: add name/value pairs
Add JavaScript to PDF Forms
Change to Forms Edit (Edit=>Tools=>Form then Edit)
Select a field, right-click, Properties
Select the Actions tab, trigger: Mouse Up, action: Run a JavaScript
Edit the JavaScript and enter:
function writeToProperty() {
var fld = this.getField("sourceFieldName");
this.info.targetPropertyName = fld.value;
}
writeToProperty(); // call my functionCaveat: While this will copy data from fields to properties when using Acrobat Pro X it does not appear to work when using Acrobat Reader
reference: http://forums.adobe.com/thread/859315
We have not found a solution at this time.
After much learning I believe I understand the problem and have a version 1.0 solution
Problem
PDFBox 1.6.0 (Alfresco 4.x) is not parsing all the objects in the PDF Form. Specifically it is not parsing the form fields that have been filled out using Acrobat Reader. As a result the PDF Form fields (filled out using Acrobat Reader) cannot be extracted by Alfresco using Tika and PDFBox.
Analysis
The parser in PDFBox 1.7.0 is being improved to handle stream objects in a more complete manner
https://issues.apache.org/jira/browse/PDFBOX-1199
This new code is contained in Rev 1333582 ==> NonSequentialPDFParser.java
http://svn.apache.org/repos/asf/pdfbox/trunk/pdfbox/src/main/java/org/apache/pdfbox/pdfparser/ and testing shows that the new PDDocument.loadNonSeq(File, RandomAccess) method and form field values created by Acrobat Reader are now readable.
Tika 1.1 currently calls org.apache.pdfbox.pdmodel.PDDocument.load() which correctly parses the metadata of the document but fails to parse the PDF Form fields. Furthermore the new org.apache.pdfbox.pdmodel.PDDocument.loadNonSeq() method will correctly parse the PDF form fields but is not able to parse the metadata. Evidently you can't have it both ways.
Solution
- Checkout, build and deploy PDFBox-1.7.0-SNAPSHOT.jar containing the loadNonSeq() code
- Modify and deploy the Tika 1.1 package to use the new PDFBox code
The InputStream needs to be processed twice, once for metadata and once for form field data so a temp file is used instead.
The org.apache.tika.parser.pdf.PDFParser class was edited as follows:
Perhaps the Tika and PDFBox developers will consider this problem as the two projects evolve.
I hope that this posting helps others.
Problem
PDFBox 1.6.0 (Alfresco 4.x) is not parsing all the objects in the PDF Form. Specifically it is not parsing the form fields that have been filled out using Acrobat Reader. As a result the PDF Form fields (filled out using Acrobat Reader) cannot be extracted by Alfresco using Tika and PDFBox.
Analysis
The parser in PDFBox 1.7.0 is being improved to handle stream objects in a more complete manner
https://issues.apache.org/jira/browse/PDFBOX-1199
This new code is contained in Rev 1333582 ==> NonSequentialPDFParser.java
http://svn.apache.org/repos/asf/pdfbox/trunk/pdfbox/src/main/java/org/apache/pdfbox/pdfparser/ and testing shows that the new PDDocument.loadNonSeq(File, RandomAccess) method and form field values created by Acrobat Reader are now readable.
Tika 1.1 currently calls org.apache.pdfbox.pdmodel.PDDocument.load() which correctly parses the metadata of the document but fails to parse the PDF Form fields. Furthermore the new org.apache.pdfbox.pdmodel.PDDocument.loadNonSeq() method will correctly parse the PDF form fields but is not able to parse the metadata. Evidently you can't have it both ways.
Solution
- Checkout, build and deploy PDFBox-1.7.0-SNAPSHOT.jar containing the loadNonSeq() code
- Modify and deploy the Tika 1.1 package to use the new PDFBox code
The InputStream needs to be processed twice, once for metadata and once for form field data so a temp file is used instead.
The org.apache.tika.parser.pdf.PDFParser class was edited as follows:
try {
// New - Use a temp file so it can be parsed twice
tstream = TikaInputStream.get(stream, tmp);
tsFile = tstream.getFile();
// PDFBox can process entirely in memory, or can use a temp file
// for unpacked / processed resources
// Decide which to do based on if we're reading from a file or not already
if (tstream != null && tstream.hasFile()) {
// File based, take that as a cue to use a temporary file
scratchFile = new RandomAccessFile(tmp.createTemporaryFile(), "rw");
pdfDocument = PDDocument.load(tsFile, scratchFile);
} else {
// Go for the normal, stream based in-memory parsing
pdfDocument = PDDocument.load(tsFile);
}
…snip code to cope with encrypted files…
metadata.set(Metadata.CONTENT_TYPE, "application/pdf");
extractMetadata(pdfDocument, metadata);
// New - Now parse again but non-sequentially to retrieve any form field data
pdfFormDoc = PDDocument.loadNonSeq(tsFile, scratchFile);
extractFormFieldData(pdfFormDoc, metadata);
PDF2XHTML.process(pdfDocument, handler, metadata,
extractAnnotationText, enableAutoSpace,
suppressDuplicateOverlappingText, sortByPosition);
private void extractFormFieldData(PDDocument document, Metadata metadata)
throws TikaException, IOException {
PDDocumentCatalog docCatalog = document.getDocumentCatalog();
PDAcroForm acroForm = docCatalog.getAcroForm();
if (acroForm != null) {
List fldList = acroForm.getFields();
Iterator fIter = fldList.iterator();
while(fIter.hasNext()){
PDField field = (PDField)fIter.next();
addMetadata(metadata, field.getFullyQualifiedName(), field.getValue());
if (logger.isDebugEnabled())
{
String logMsg = "extracting: " + field.getFullyQualifiedName();
logMsg += " value: " + field.getValue();
logger.debug(logMsg);
}
}
}
}
Perhaps the Tika and PDFBox developers will consider this problem as the two projects evolve.
I hope that this posting helps others.
Chris,
It's been a couple of months since I worked on this so my memory isn't too fresh.
But at a very high level here's how I built the jar files:
Using eclipse Indigo (3.7) with subversion and Maven plugins
- Checkout the Tika 1.1 project from: http://svn.apache.org/repos/asf/tika/tags/1.1
- Checkout the PDFBox 1.7.0 SNAPSHOT (or higher) http://svn.apache.org/repos/asf/pdfbox/tags/1.7.1
Resolve dependencies (tika is dependent on pdfbox)
This is where I learned how to use Maven with eclipse
- Use Maven to build PDFBox with maven goals of "clean and install" (Hint: eclipse Run Configurations…)
- Modify the Tika code org.apache.tika.parser.pdf.PDFParser
- Using the eclipse editor modify the class as described earlier in this thread
- change the parse() method
- add the extractFormFieldData() method
- Use Maven to build Tika with maven goals of "clean and install" (This was new to me, since I've been using Ant).
If you're new to these tools and the language it will require learning but that's the fun of it
I hope this helps.
Steve
It's been a couple of months since I worked on this so my memory isn't too fresh.
But at a very high level here's how I built the jar files:
Using eclipse Indigo (3.7) with subversion and Maven plugins
- Checkout the Tika 1.1 project from: http://svn.apache.org/repos/asf/tika/tags/1.1
- Checkout the PDFBox 1.7.0 SNAPSHOT (or higher) http://svn.apache.org/repos/asf/pdfbox/tags/1.7.1
Resolve dependencies (tika is dependent on pdfbox)
This is where I learned how to use Maven with eclipse
- Use Maven to build PDFBox with maven goals of "clean and install" (Hint: eclipse Run Configurations…)
- Modify the Tika code org.apache.tika.parser.pdf.PDFParser
- Using the eclipse editor modify the class as described earlier in this thread
- change the parse() method
public void parse(
InputStream stream, ContentHandler handler,
Metadata metadata, ParseContext context)
throws IOException, SAXException, TikaException {
PDDocument pdfDocument = null;
PDDocument pdfFormDoc = null;
TikaInputStream tstream = null;
File tsFile = null;
TemporaryResources tmp = new TemporaryResources();
RandomAccess scratchFile = null;
try {
// SMD - Use a temp file so it can be parsed twice
tstream = TikaInputStream.get(stream, tmp);
tsFile = tstream.getFile();
// PDFBox can process entirely in memory, or can use a temp file
// for unpacked / processed resources
// Decide which to do based on if we're reading from a file or not already
if (tstream != null && tstream.hasFile()) {
// File based, take that as a cue to use a temporary file
scratchFile = new RandomAccessFile(tmp.createTemporaryFile(), "rw");
// pdfDocument = PDDocument.load(new CloseShieldInputStream(stream), scratchFile, true);
pdfDocument = PDDocument.load(tsFile, scratchFile);
} else {
// Go for the normal, stream based in-memory parsing
// pdfDocument = PDDocument.load(new CloseShieldInputStream(stream), true);
pdfDocument = PDDocument.load(tsFile);
}
if (pdfDocument.isEncrypted()) {
String password = null;
// Did they supply a new style Password Provider?
PasswordProvider passwordProvider = context.get(PasswordProvider.class);
if (passwordProvider != null) {
password = passwordProvider.getPassword(metadata);
}
// Fall back on the old style metadata if set
if (password == null && metadata.get(PASSWORD) != null) {
password = metadata.get(PASSWORD);
}
// If no password is given, use an empty string as the default
if (password == null) {
password = "";
}
try {
pdfDocument.decrypt(password);
} catch (Exception e) {
// Ignore
}
}
metadata.set(Metadata.CONTENT_TYPE, "application/pdf");
extractMetadata(pdfDocument, metadata);
// SMD - Now parse non-sequentially to retrieve any form field data
pdfFormDoc = PDDocument.loadNonSeq(tsFile, scratchFile);
extractFormFieldData(pdfFormDoc, metadata);
PDF2XHTML.process(pdfDocument, handler, metadata,
extractAnnotationText, enableAutoSpace,
suppressDuplicateOverlappingText, sortByPosition);
} finally {
if (pdfDocument != null) {
pdfDocument.close();
pdfFormDoc.close();
}
tmp.dispose();
}
}
- add the extractFormFieldData() method
/**
* Steve Deal - Added to parse PDF Form fields
*
* @param document
* @param metadata
* @throws TikaException
*/
private void extractFormFieldData(PDDocument document, Metadata metadata)
throws TikaException, IOException {
PDDocumentCatalog docCatalog = document.getDocumentCatalog();
PDAcroForm acroForm = docCatalog.getAcroForm();
if (acroForm != null) {
List fldList = acroForm.getFields();
Iterator fIter = fldList.iterator();
while(fIter.hasNext()){
PDField field = (PDField)fIter.next();
addMetadata(metadata, field.getFullyQualifiedName(), field.getValue());
if (logger.isDebugEnabled())
{
String logMsg = "extracting: " + field.getFullyQualifiedName();
logMsg += " value: " + field.getValue();
logger.debug(logMsg);
}
}
}
}
- Use Maven to build Tika with maven goals of "clean and install" (This was new to me, since I've been using Ant).
If you're new to these tools and the language it will require learning but that's the fun of it
I hope this helps.
Steve
Well that sure was an adventure. I've pretty much managed to follow your steps (woohoo!) but I just need a leeedle bit more help to get over the line. Hope it doesn't put you out at all and thanks very much for the help thusfar.
In case someone else sees this post looking for the same thing, I'll just go over my steps here:
-For starters, I tried to checkout the original projects using File>New>Project…>Maven>Checkout Maven Projects from SCM, but had some issues, none of my SVN connectors were showing up in the SCM type field. I spent almost an hour trying to troubleshoot this issue (which appears to be prevalent among eclipse indigo users) before I gave up, uninstalled indigo and installed Eclipse Juno (which looks a little more fancy anyway
).
after installing juno, m2e and subversive, as well as the Maven SCM Handler for Subversive, I was able to use File>New>Project…>Maven>Checkout Maven Projects from SCM Normally to check out the two URI's you posted.
-I had some trouble resolving dependencies due to two issues. The first was a basic misunderstanding on my part of the difference between a dependency and a folder on the classpath. Once I figured out how to work with the POM.xml files this was sorted. The second was because eclipse took what I expected to be 2 projects and turned them into 14. For example, I had pdfbox-ant, pdfbox-app etc etc, as well as pdfbox and pdfbox-parent. The same was true in the tika files.
- Spent most of my day trying to figure out why I couldn't compile tika. I compiled pdfbox no issues, and made the changes to the pdf parser class, however I ran into a bunch of "error: cannot find symbol"'s when trying to compile. eventually I had to learn what an import statement was and add a few of these (still not sure why I had to import java.io.File, it seems like something that would have been in the file already if it was needed, but I spose that's why I'm not a Java dev).
So now I have managed to build both of these with the changes. I have 5 jar files:
Secondly, I know that the filenames are not exactly the same - the version numbers are different. So my second question is - should I delete/move the originals, should I change the names of the jar's I have built so that they will override/overwrite?
In case someone else sees this post looking for the same thing, I'll just go over my steps here:
-For starters, I tried to checkout the original projects using File>New>Project…>Maven>Checkout Maven Projects from SCM, but had some issues, none of my SVN connectors were showing up in the SCM type field. I spent almost an hour trying to troubleshoot this issue (which appears to be prevalent among eclipse indigo users) before I gave up, uninstalled indigo and installed Eclipse Juno (which looks a little more fancy anyway
).after installing juno, m2e and subversive, as well as the Maven SCM Handler for Subversive, I was able to use File>New>Project…>Maven>Checkout Maven Projects from SCM Normally to check out the two URI's you posted.
-I had some trouble resolving dependencies due to two issues. The first was a basic misunderstanding on my part of the difference between a dependency and a folder on the classpath. Once I figured out how to work with the POM.xml files this was sorted. The second was because eclipse took what I expected to be 2 projects and turned them into 14. For example, I had pdfbox-ant, pdfbox-app etc etc, as well as pdfbox and pdfbox-parent. The same was true in the tika files.
- Spent most of my day trying to figure out why I couldn't compile tika. I compiled pdfbox no issues, and made the changes to the pdf parser class, however I ran into a bunch of "error: cannot find symbol"'s when trying to compile. eventually I had to learn what an import statement was and add a few of these (still not sure why I had to import java.io.File, it seems like something that would have been in the file already if it was needed, but I spose that's why I'm not a Java dev).
So now I have managed to build both of these with the changes. I have 5 jar files:
- pdfbox-1.7.1.jar
- tika-app-1.1.jar
- tika-bundle-1.1.jar
- tika-core-1.1
- tika-parsers-1.1.jar
Secondly, I know that the filenames are not exactly the same - the version numbers are different. So my second question is - should I delete/move the originals, should I change the names of the jar's I have built so that they will override/overwrite?
