AAAR Extract finishing with error messages

[senzod]

Hi

First I should say, great work you have done with A.A.A.R. - Alfresco Audit Analysis and Reporting.

I have been able to install it and have been able to get some data
extraction but it finishes with errors.

Can you please help and look at the attached .txt log file?


Thanks

[fcorti]

HI Senzo,

Thank you for the feedback. 🙂

The problem is that you don't have enough space available for you java virtual machine.
To solve it you have two different solutions: extend the resources to the JVM, reduce the chunk of audit actions extracted per time.

To give more resources to the JVM you have to modify the JAVA_OPT variable.
Take a look here for a suggestion:
http://djugal.blogspot.it/2011/07/increase-java-memory-for-pentaho-data.html

To reduce the number of audit actions extracted per time, you can update the 'limit' parameter at 10000 (or less) in the 'AAAR_DataMart.dm_dim_alfresco' table.
Below an example of standard configuration.
http://francescocorti.files.wordpress.com/2013/03/dm_dim_alfresco.png

Cheers.

[senzod]

Thanks Francesco

I have increased the JVM resource and also reduced the 'limit' parameter. It worked but was only able to pull data until the march 2013. For the past 2 weeks I have been running the extract command hoping it will update the database but I have been getting the same error message and and I have attached the log file of that. . .Can you please advise where am I missing things here.

Thanking you in advance

Senzo

[fcorti]

Hi Senzo,

I'm happy you find the correct settings for your environment.
The log you share tells us two things.

The first is that the audit data are correctly imported into the system.
Please, take a look at:
http://???.???.???.???:8080/alfresco/service/api/audit/query/alfresco-access?verbose=true&limit=5000...
This row tells us you are importing 50K rows per time, starting from the audit action number 1,728,149.
You probably have lot of audit data (and this is not a problem for the audit Data Mart) and you will need several import execution to retrieve all the history.
Please remember: you are going to import 50K audit action per execution!
You could take a look here for the detailed explanation of the reason:
<a href="http://fcorti.com/alfresco-audit-analysis-reporting/aaar-faq/#q6">http://fcorti.com/alfresco-audit-analysis-reporting/aaar-faq/#q6</a>

The second thing that your log tells us, is about an error during the retrieving of the repository structure.
Due to this error:
2014/03/30 08:21:05 - Get nodes - The last packet successfully received from the server was 57,856,969 milliseconds ago.  The last packet sent successfully to the server was 57,856,970 milliseconds ago. is longer than the server configured value of 'wait_timeout'. You should consider either expiring and/or testing connection validity before use in your application, increasing the server configured values for client timeouts, or using the Connector/J connection property 'autoReconnect=true' to avoid this problem.
The import process cannot have an answer to the CMIS query and a timeout is thrown.
I suggest you to check if you have a good connection (in terms of speed) between the Alfresco server and the AAAR server.
In every case, the problem is due to this timeout.

Let us know how it goes.

[senzod]

Hi Francesco

I have reduced the 'limit' parameter to 20000 now and now I get the following error
org.pentaho.di.core.exception.KettleStepException:
Error in step, asking everyone to stop because of:

Error inserting/updating row
Cannot add or update a child row: a foreign key constraint fails (`aaar_datamart`.`dm_dim_dates`, CONSTRAINT `fk_dm_dim_dates_dm_dim_months` FOREIGN KEY (`month_id`) REFERENCES `dm_dim_months` (`id`)
ON DELETE NO ACTION ON UPDATE NO ACTION)


I have attached the whole error log also.

I doubt its permissions, any comments on this?

Regards

Senzo

[fcorti]

Hi Senzo,

The limit param is not the reason of your error.
I don't know why with 20K it fails but I suggest you to set it to 50K.
I suggest you to check the connection between the two servers for the error.