topic Re: Indexing does not work in Alfresco Archive

Indexing does not work

ashwini — Tue, 04 Sep 2012 15:05:57 GMT

Hello There ,After configuring index.recovery.mode=FULL , reindexing hangs for a while and continue again with below exception. 2012-09-04 04:27:23,416 ERROR [index.AbstractReindexComponent.threads] [indexTrackerThread1] Reindexer 206 failed with error: Possible cache integrity issue during reind

Re: Indexing does not work

amitabhandari1 — Wed, 05 Sep 2012 10:49:47 GMT

Hi Ashwini,

when you do any activity on server like creation of content or modify metadata ,it indexes in background.There are option available to whether index in background or foreground.
. But when you are doing Full Indexing , you will have to wait until and unless server gets Up.

Thanks

Re: Indexing does not work

ashwini — Wed, 05 Sep 2012 11:45:42 GMT

Thanks Amita for clarification.

Is it possible for you to give input on mentioned issue also ?

Regards,
Ashwini

Re: Indexing does not work

amitabhandari1 — Mon, 10 Sep 2012 12:23:23 GMT

Hi Ashwini,

Can you please check on nodeBrowser with this query
TX:927d124a-41ca-4b1e-9d9b-afd0067edd56

Also can , you fire query in database : select * from alf_node where uuid='16a0b417-6aca-4e03-96d2-97debcf93357'

Let me know result.

Thanks,

Re: Indexing does not work

ashwini — Mon, 10 Sep 2012 14:04:08 GMT

The result of select * from alf_node where uuid='16a0b417-6aca-4e03-96d2-97debcf93357' is as below:

id   version   store_id   uuid   transaction_id   node_deleted   type_qname_id   locale_id   acl_id   audit_creator   audit_created   audit_modifier   audit_modified   audit_accessed
266483   5   6   16a0b417-6aca-4e03-96d2-97debcf93357   230497   b'1'   633   5      Arsene_Woboyi   2012-05-18T14:37:04.560+02:00   Arsene_Woboyi   2012-05-18T14:37:39.102+02:00‍‍

Even I am getting some exception everyday around 21:00 , when scheduler runs and try to delete the contents

'>https://forums.alfresco.com/en/viewtopic.php?f=4&t=46214

So I tried to run below query
SELECT * FROM alf_child_assoc where parent_node_id = 266483
and the result is :

id   version   parent_node_id   type_qname_id   child_node_name_crc   child_node_name   child_node_id   qname_ns_id   qname_localname   qname_crc   is_primary   assoc_index
254898   1   266483   351   -493933189   e9e162fa-45c1-4f2f-92c8-52b450ae701b   266495   6   webpreview   1387062285   b'1'   -1‍‍

I am not getting whats failing here too . Might be this is the cause of exception which I am getting while full indexing.

And

Can you please check on nodeBrowser with this query
TX:927d124a-41ca-4b1e-9d9b-afd0067edd56

I tried searching with above query in share's nodeBrowser, but I didnt get which select store I have to use. Could you give me some more input.

Regards,
Ashwini

Re: Indexing does not work

amitabhandari1 — Tue, 11 Sep 2012 04:42:26 GMT

Hi Ashwini,

Use workspace//spacestore and lucene query. Wh
while executing this query :
select * from alf_node where uuid='16a0b417-6aca-4e03-96d2-97debcf93357'

The value of node_deleted column should be either t or f . Can you please confirm?

Thanks,

Re: Indexing does not work

ashwini — Tue, 11 Sep 2012 07:26:36 GMT

Hi Amita ,

After executing select * from alf_node where uuid='16a0b417-6aca-4e03-96d2-97debcf93357'

I found node_deleted = b'1'

Node browser query displayed No items found message

Regards,
Ashwini

Re: Indexing does not work

amitabhandari1 — Tue, 11 Sep 2012 07:40:49 GMT

Hi Ashwini,

node_deleted column stores value of Boolean types. I am not sure why you are getting this value.
Are you facing this issue on your development server. if yes, you can try delete node from database.
You can look at 3 tables : alf_node , alf_node_properties and alf_node_aspects . And delete all rows associated with this node.

Thanks,

Re: Indexing does not work

ashwini — Tue, 11 Sep 2012 08:54:04 GMT

Thanks Amita.

I just googled for value b'1' and found

BOOL, BOOLEAN :These types are synonyms for TINYINT(1). A value of zero is considered false. Nonzero values are considered true.
TINYINT[(M)] [UNSIGNED] [ZEROFILL] :A very small integer. The signed range is -128 to 127. The unsigned range is 0 to 255.

in result I found : node_deleted = b'1' where b'1' is nothing but TINYINT value.

it means node_deleted= true

based on id values which is in result of query

select * from alf_node where uuid='16a0b417-6aca-4e03-96d2-97debcf93357'‍

tried to look into alf_node_properties(node_id), alf_node_aspects (node_id) , which returned nothing.

Re: Indexing does not work

amitabhandari1 — Tue, 11 Sep 2012 08:59:18 GMT

Ok , in that case can u try to delete this node from alf_node and check if you can run full indexing.

Re: Indexing does not work

ashwini — Tue, 11 Sep 2012 09:28:55 GMT

Thanks a lot amitha .

I tried deleting entry from alf_node and found foreign key constraint violation with alf_child_assoc error message. So I deleted first entry from alf_child_assoc and then from alf_node. And started server and found another exception :

  2012-09-11 11:09:31,603  ERROR
[index.AbstractReindexComponent.threads] [indexTrackerThread1]  
Reindexer   206 failed with error: null.
  2012-09-11 11:09:31,604  WARN   
[index.AbstractReindexComponent.threads] [indexTrackerThread1] java.lang.NullPointerException
   at
org.alfresco.repo.search.impl.lucene.ADMLuceneIndexerImpl.createDocumentsImpl(ADMLuceneIndexerImpl.java:803)
   at
org.alfresco.repo.search.impl.lucene.ADMLuceneIndexerImpl.createDocuments(ADMLuceneIndexerImpl.java:633)
   at
org.alfresco.repo.search.impl.lucene.ADMLuceneIndexerImpl$4.execute(ADMLuceneIndexerImpl.java:1985)
   at
org.alfresco.repo.search.impl.lucene.ADMLuceneIndexerImpl$4.execute(ADMLuceneIndexerImpl.java:1981)
   at
org.alfresco.repo.search.impl.lucene.AbstractLuceneIndexerImpl$1.execute(AbstractLuceneIndexerImpl.java:715)
   at
org.alfresco.repo.transaction.RetryingTransactionHelper.doInTransaction(RetryingTransactionHelper.java:388)
   at
org.alfresco.repo.search.impl.lucene.AbstractLuceneIndexerImpl.doInReadthroughTransaction(AbstractLuceneIndexerImpl.java:702)
   at
org.alfresco.repo.search.impl.lucene.ADMLuceneIndexerImpl.readDocuments(ADMLuceneIndexerImpl.java:1980)
   at
org.alfresco.repo.search.impl.lucene.ADMLuceneIndexerImpl.flushPending(ADMLuceneIndexerImpl.java:1044)
   at
org.alfresco.repo.search.impl.lucene.AbstractLuceneIndexerAndSearcherFactory.flush(AbstractLuceneIndexerAndSearcherFactory.java:874)
   at sun.reflect.GeneratedMethodAccessor338.invoke(Unknown Source)
   at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at
org.alfresco.repo.service.StoreRedirectorProxyFactory$RedirectorInvocationHandler.invoke(StoreRedirectorProxyFactory.java:215)
   at $Proxy215.flush(Unknown Source)
   at
org.alfresco.repo.search.IndexerComponent.flushPending(IndexerComponent.java:133)
   at
org.alfresco.repo.node.index.AbstractReindexComponent$ReindexWorkerRunnable.beforeCommit(AbstractReindexComponent.java:1086)
   at
org.alfresco.repo.transaction.AlfrescoTransactionSupport$TransactionSynchronizationImpl.doBeforeCommit(AlfrescoTransactionSupport.java:747)
   at
org.alfresco.repo.transaction.AlfrescoTransactionSupport$TransactionSynchronizationImpl.doBeforeCommit(AlfrescoTransactionSupport.java:727)
   at
org.alfresco.repo.transaction.AlfrescoTransactionSupport$TransactionSynchronizationImpl.beforeCommit(AlfrescoTransactionSupport.java:687)
   at
org.springframework.transaction.support.TransactionSynchronizationUtils.triggerBeforeCommit(TransactionSynchronizationUtils.java:95)
   at
org.springframework.transaction.support.AbstractPlatformTransactionManager.triggerBeforeCommit(AbstractPlatformTransactionManager.java:927)
   at
org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:737)
   at
org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:723)
   at
org.springframework.transaction.interceptor.TransactionAspectSupport.commitTransactionAfterReturning(TransactionAspectSupport.java:393)
   at
org.alfresco.util.transaction.SpringAwareUserTransaction.commit(SpringAwareUserTransaction.java:472)
   at
org.alfresco.repo.transaction.RetryingTransactionHelper.doInTransaction(RetryingTransactionHelper.java:410)
   at
org.alfresco.repo.node.index.AbstractReindexComponent$ReindexWorkerRunnable.run(AbstractReindexComponent.java:1008)
   at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
   at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
   at java.lang.Thread.run(Thread.java:662)

‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

before some days even I tried to delete the entry only from alf_child_assoc and got same exception. Do you have any idea why I am getting this exception.

Regards,
Ashwini

Re: Indexing does not work

amitabhandari1 — Tue, 11 Sep 2012 09:41:47 GMT

Hi Ashwini,

Did you try to start your server with AUTO/Validate mode . If not , can you try that .
If server start up , you can try indexing from web scripts. There are many web-scripts available for performing index.

Thanks,

Re: Indexing does not work

ashwini — Thu, 13 Sep 2012 11:54:59 GMT

Thanks Amita,

Now after removing some entries from database
1. I am not getting any exception when cleanup scheduler run(which runs everyday at 21:00)
2. indexing completes in around 20-30 min (which was taking before more than 16 hours only for 50000 transactions) with some exceptions like :

  2012-09-11 11:09:31,603  ERROR
[index.AbstractReindexComponent.threads] [indexTrackerThread1]  
Reindexer   206 failed with error: null.
  2012-09-11 11:09:31,604  WARN   
[index.AbstractReindexComponent.threads] [indexTrackerThread1] java.lang.NullPointerException
   at
org.alfresco.repo.search.impl.lucene.ADMLuceneIndexerImpl.createDocumentsImpl(ADMLuceneIndexerImpl.java:803)
   at
org.alfresco.repo.search.impl.lucene.ADMLuceneIndexerImpl.createDocuments(ADMLuceneIndexerImpl.java:633)
   at
org.alfresco.repo.search.impl.lucene.ADMLuceneIndexerImpl$4.execute(ADMLuceneIndexerImpl.java:1985)
   at
org.alfresco.repo.search.impl.lucene.ADMLuceneIndexerImpl$4.execute(ADMLuceneIndexerImpl.java:1981)
   at
org.alfresco.repo.search.impl.lucene.AbstractLuceneIndexerImpl$1.execute(AbstractLuceneIndexerImpl.java:715)
   at
org.alfresco.repo.transaction.RetryingTransactionHelper.doInTransaction(RetryingTransactionHelper.java:388)
   at
org.alfresco.repo.search.impl.lucene.AbstractLuceneIndexerImpl.doInReadthroughTransaction(AbstractLuceneIndexerImpl.java:702)
   at
org.alfresco.repo.search.impl.lucene.ADMLuceneIndexerImpl.readDocuments(ADMLuceneIndexerImpl.java:1980)
   at
org.alfresco.repo.search.impl.lucene.ADMLuceneIndexerImpl.flushPending(ADMLuceneIndexerImpl.java:1044)
   at
org.alfresco.repo.search.impl.lucene.AbstractLuceneIndexerAndSearcherFactory.flush(AbstractLuceneIndexerAndSearcherFactory.java:874)
   at sun.reflect.GeneratedMethodAccessor338.invoke(Unknown Source)
   at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at
org.alfresco.repo.service.StoreRedirectorProxyFactory$RedirectorInvocationHandler.invoke(StoreRedirectorProxyFactory.java:215)
   at $Proxy215.flush(Unknown Source)
   at
org.alfresco.repo.search.IndexerComponent.flushPending(IndexerComponent.java:133)
   at
org.alfresco.repo.node.index.AbstractReindexComponent$ReindexWorkerRunnable.beforeCommit(AbstractReindexComponent.java:1086)
   at
org.alfresco.repo.transaction.AlfrescoTransactionSupport$TransactionSynchronizationImpl.doBeforeCommit(AlfrescoTransactionSupport.java:747)
   at
org.alfresco.repo.transaction.AlfrescoTransactionSupport$TransactionSynchronizationImpl.doBeforeCommit(AlfrescoTransactionSupport.java:727)
   at
org.alfresco.repo.transaction.AlfrescoTransactionSupport$TransactionSynchronizationImpl.beforeCommit(AlfrescoTransactionSupport.java:687)
   at
org.springframework.transaction.support.TransactionSynchronizationUtils.triggerBeforeCommit(TransactionSynchronizationUtils.java:95)
   at
org.springframework.transaction.support.AbstractPlatformTransactionManager.triggerBeforeCommit(AbstractPlatformTransactionManager.java:927)
   at
org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:737)
   at
org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:723)
   at
org.springframework.transaction.interceptor.TransactionAspectSupport.commitTransactionAfterReturning(TransactionAspectSupport.java:393)
   at
org.alfresco.util.transaction.SpringAwareUserTransaction.commit(SpringAwareUserTransaction.java:472)
   at
org.alfresco.repo.transaction.RetryingTransactionHelper.doInTransaction(RetryingTransactionHelper.java:410)
   at
org.alfresco.repo.node.index.AbstractReindexComponent$ReindexWorkerRunnable.run(AbstractReindexComponent.java:1008)
   at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
   at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
   at java.lang.Thread.run(Thread.java:662)
‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

And then , when I restart my server with auto mode, in works fine.

Could you please tell me the web script which I can use for indexing.

Thanks a lot for your help.

Re: Indexing does not work

amitabhandari1 — Fri, 14 Sep 2012 09:32:21 GMT

Hi Ashwini,

Please check the following webscript:
http://<server-name>/alfresco/service/index/package/org/alfresco/enterprise/repository/admin/indexcheck

Thanks,

Re: Indexing does not work

ashwini — Fri, 14 Sep 2012 09:58:37 GMT

Hi Amita,

Please check the following webscript:
http://<server-name>/alfresco/service/index/package/org/alfresco/enterprise/repository/admin/indexcheck

I have looked for this before. Unfortunately we are not using enterprise edition. Do we have something for community edition.

Thanks ,
Ashwini

Re: Indexing does not work

shagan12 — Tue, 05 Mar 2013 15:48:00 GMT

I fixed this issue in 4.2b by opening table alf_activity_post and deleted all rows with status=Error and Pending

Re: Indexing does not work

dranakan — Mon, 10 Jun 2013 11:55:00 GMT

How can I get the node with problems ? (to manually delete records in DB)

I have done a FULL reindex but I don't get the node.

I have setted the log4j to warn (log4j.logger.org.alfresco.repo.node.cleanup=DEBUG), enable the profile (in alfresco-global.properties : db.url=jdbc:mysql://…&profileSQL=true) but I never get the node id.

I ever get this bug and I have done a FULL indexe, get the node and delete in the DB. This time I have not details when I do the FUL reindexe.
I have change scheduler (WEB-INF/classes/alfresco/scheduled-jobs-context.xml, bean "nodeServiceCleanupTrigger").

In my situation, deleting all rows (error and pending) in alf_activity_post has no effect.

Re: Indexing does not work

dranakan — Wed, 17 Jul 2013 11:28:00 GMT

instead of doing the full reindex, it's faster to do this :

Find the nodes with problems :


SELECT alf_node.id FROM alf_node INNER JOIN alf_child_assoc ON alf_node.id = alf_child_assoc.parent_node_id and alf_node.node_deleted = 1;
‍‍‍

Delete them :


delete from alf_child_assoc where parent_node_id in (1234,5678,…);
‍‍‍

After that, the cleaner can do its job (delete from alf_node where node_deleted = 1 and transaction_id <= (select max(txn.id) from alf_transaction txn where txn.commit_time_ms < 1368857400033))

Re: Indexing does not work

heiko_robert — Fri, 23 Aug 2013 15:13:33 GMT

Hi dranakan,

you shouldn't delete the parent nodes otherwise you have lots of lost nodes without parents and then will get trouble somewhen later and you have to find the lost nodes later!
The parents shoudn't be touched at all because they are just a maker for the DeletedNodeCleanupWorker.

I didn't find the source of the problem but it's allway related to webpreview, doclib. So it's maybe a problem in the removal of the hidden nodes? Maybe AndyH/DavidW reads this and gives a hint? The problem occurs not on a 4.0 system only it continues on a migrated system running 4.2.c.

Anyway - here is the way I just did it on a 4.2.c system (where deleted flag is moved to a doc type):


drop table if exists alf_remove_nodes;
create temporary table alf_remove_nodes as
    select alf_child_assoc.parent_node_id, 
           alf_child_assoc.child_node_id
    from alf_qname, alf_node, alf_child_assoc 
    where
    alf_node.id = alf_child_assoc.parent_node_id and 
    alf_node.type_qname_id = alf_qname.id and
    alf_qname.local_name = 'deleted'
;
select * from alf_remove_nodes;

– finally remove references and then the node itself
delete from alf_node_properties where node_id in (select child_node_id from alf_remove_nodes);
delete from alf_node_aspects where node_id in (select child_node_id from alf_remove_nodes);
delete from alf_child_assoc where child_node_id in (select child_node_id from alf_remove_nodes);
delete from alf_node where id in (select child_node_id from alf_remove_nodes);
– cleanup
drop table if exists alf_remove_nodes;
‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

Re: Indexing does not work

heiko_robert — Fri, 23 Aug 2013 15:19:43 GMT

if you do not allready intent to reindex you don't need to. In my case the childs to be deleted where allways just hidden thumbnails/previews.