Search web script + Transactional update cache full problem

[lista]

I have a Java backed web script that's in charge of searching; and a custom app on top of Alfresco.

I execute that search when visiting a certain tab, meaning, user doesn't get any chance to enter some variables to narrow down the search results. Query then ends up being only an aspect search. And since there's a fair amount of nodes in repository, I get "Transactional update cache full" warnings, query takes forever (minutes), and the whole user experience is awful.

How would you tackle this problem (apart from the obvious, setting a larger transactional cache limit)?

[mitpatoliya]

I did not get why user is not able to narrow down the search?
Also is your script flexible enough to search based on the various different criteria given by user?

[lista]

Because the search itself is triggered when visiting the tab (view) in question.
So user visits the tab - query executes - then user can narrow down the search with criteria - and execute the query as many times as he likes. The problem is the initial search, that happens automatically.

[mitpatoliya]

Are you using any kind of pagging?
Please explore that it could help you in reducing the overhead.
As it will fetch the results in the chunks based on the pagination designed.
There is batch size you can set during search.I think this should solve you problem.
may be this post will be helpful
https://forums.alfresco.com/en/viewtopic.php?t=5599

[lista]

The problem is in "Transactional update cache full" part.
Let me explain further.

If I do a basic query, I will get this kind of behavior:

13:45:04,723 User:admin WARN  [alfresco.cache.contentDataTransactionalCache] Transactional update cache 'org.alfresco.cache.contentDataTransactionalCache' is full (1000).
13:45:05,337 User:admin DEBUG [alfresco.scripts.MGSearch] Results: 1000 rows (limited: NUMBER_OF_PERMISSION_EVALUATIONS)‍‍

So, I have "Transactional update cache full", and results are limited by repository.properties NUMBER_OF_PERMISSION_EVALUATIONS.
If I setup my query like this:

parameters.setMaxPermissionChecks(10);
parameters.setLimitBy(LimitBy.NUMBER_OF_PERMISSION_EVALUATIONS);‍‍

I will get this kind of behavior:

13:45:04,723 User:admin WARN  [alfresco.cache.contentDataTransactionalCache] Transactional update cache 'org.alfresco.cache.contentDataTransactionalCache' is full (1000).
13:45:05,337 User:admin DEBUG [alfresco.scripts.MGSearch] Results: 10 rows (limited: NUMBER_OF_PERMISSION_EVALUATIONS)‍‍

So, I have "Transactional update cache full", and results are limited by NUMBER_OF_PERMISSION_EVALUATIONS defined in code.
And lastly, if I setup my query like this:

parameters.setLimitBy(LimitBy.FINAL_SIZE);
parameters.setLimit(10);‍‍

I will get this kind of behavior:

13:45:04,723 User:admin WARN  [alfresco.cache.contentDataTransactionalCache] Transactional update cache 'org.alfresco.cache.contentDataTransactionalCache' is full (1000).
13:45:05,337 User:admin DEBUG [alfresco.scripts.MGSearch] Results: 10 rows (limited: FINAL_SIZE)‍‍

So, I have "Transactional update cache full", and results are limited by FINAL_SIZE defined in code. But in each case, the "Transactional update cache full" is there. My conclusion is that I can't get around it whatever I try; any other suggestions?

[lista]

Using setMaxItems() appears to be working differently in comparison to setLimit().
Does anyone have any additional info on these two?

[mitpatoliya]

hey have you tried by changing the property inside repository.properties.
#
# Properties to limit resources spent on individual searches
#
# The maximum time spent pruning results
system.acl.maxPermissionCheckTimeMillis=10000

# The maximum number of results to perform permission checks against
system.acl.maxPermissionChecks=1000

# Properties to control read permission evaluation for acegi
system.readpermissions.optimise=true
system.readpermissions.bulkfetchsize=1000

try to increase the permission check size and remove all the limitation set from you code.

[lista]

I don't see any benefit from it at the moment.
Surely, I can set those number to a higher value, but since I have loads of nodes and they will keep on coming, I'll find myself in the same trouble soon enough.

I do appreciate your help!