timing between document.delete() and running query

[mcs130]

Hello

We are doing some POC work with AF4 (4.0.e) Community Edition and Apache Chemistry 0.8.0-SNAPSHOT source (built JARs from this)

Much is working fine.

We recently noticed when either running a couple of our unit tests or performing the 2 operations in series in our custom UI's business logic, we sometimes see the 2nd operation fail with an exception.  (I have asked the developer to get the detailed trace).  In the meantime, here is the failing scenario:

Performs a delete of a document selected from a list displayed in a dialog.  This works fine every time.

document.delete(true);‍

To refresh the list, the developer has the code then hitting the back-end AGAIN with the same query to get the new list (which should now be 1 less since the document object was deleted successfully). 

cmisSession.query(queryString, false);‍

However, the page has no list because the application throws an exception.

Then we noticed that when a breakpoint is used in Debug mode and you are stepping through the code to try and isolate the breakage, which naturally slows the process… it NEVER breaks - both operations then succeed.

This lead us to think it was a "timing" issue perhaps, but not likely? However, when the developer forced a sleep of say 500 ms between method calls being done in the UI… it works again.

Ironically, another ECM product being used for the POC work, an SP2103 preview, does NOT exhibit this behavior for THIS use-case - however, there have been other  challenges with SP that we have encountered which AF4 is far better at handling correctly.

Thanks

[mrogers]

It depends upon the search engine being used.   You are probably using solr which will have an asynchronous delay.

Need details of your exception to go forward.

[mcs130]

I will get the later today when I get in but you ARE CORRECT - our Community Edition is using SOLR.

Thanks

[mrogers]

So when you use SOLR for a search it may fail to find recently added content or find recently deleted content.    Its just a fact of life with the so called "eventual consistency",

I'm assuming that the error will be in your custom code.

There are a couple of alternate approaches
a) don't use solr as your working method to list contents of recently modified folders.   There are many other APIs available.    This is how Alfresco Share works.
b) Check that a node exists prior to using it, this will probably sort out your delete case.

[mcs130]

Thanks for the quick response…but when you say there are many other APIs available, we are constrained to make these calls through CMIS only.  We cannot use any product specific approach with what is being developed.  Are you referring to something AF-specific?

Mark

[mcs130]

here is last part of the stack trace from our logs involving the attempt to make the Query call:

I suppose this confirms what you expected?

[9/17/12 9:55:09:284 EDT] 0000001d SystemErr     R org.apache.chemistry.opencmis.commons.exceptions.CmisRuntimeException: Node does not exist: missing://missing/missing(null)
[9/17/12 9:55:09:284 EDT] 0000001d SystemErr     R    at org.apache.chemistry.opencmis.client.bindings.spi.atompub.AbstractAtomPubService.convertStatusCode(AbstractAtomPubService.java:452)
[9/17/12 9:55:09:284 EDT] 0000001d SystemErr     R    at org.apache.chemistry.opencmis.client.bindings.spi.atompub.AbstractAtomPubService.post(AbstractAtomPubService.java:570)
[9/17/12 9:55:09:285 EDT] 0000001d SystemErr     R    at org.apache.chemistry.opencmis.client.bindings.spi.atompub.DiscoveryServiceImpl.query(DiscoveryServiceImpl.java:142)
[9/17/12 9:55:09:285 EDT] 0000001d SystemErr     R    at org.apache.chemistry.opencmis.client.runtime.SessionImpl$3.fetchPage(SessionImpl.java:562)
[9/17/12 9:55:09:285 EDT] 0000001d SystemErr     R    at org.apache.chemistry.opencmis.client.runtime.util.AbstractIterator.getCurrentPage(AbstractIterator.java:132)
[9/17/12 9:55:09:285 EDT] 0000001d SystemErr     R    at org.apache.chemistry.opencmis.client.runtime.util.CollectionIterator.hasNext(CollectionIterator.java:48)


‍‍‍‍‍‍‍‍‍‍‍

[mrogers]

Sorry - I don't known about Open CMIS :mrgreen:  But I would have thought that there are list methods available that could be used in place of search.

[mcs130]

Not sure what you are saying about "Search" vs "list"

To recall:

We perform 3 basic calls (for brevity, focused solely on the calls made via CMIS Session object which we do everything through and which works for everything so far against Alfresco 4)

We build the queryString like anything else: (we do this dynamically w/ variable substitution, but to illustrate I am showing the captured String value)

Step 1:

String queryString = SELECT * FROM acme:documents  WHERE IN_FOLDER('workspace://SpacesStore/c98cd558-e159-44fa-927a-7e9b17241dff') AND CONTAINS ('engineering')

cmisSession.query(queryString, false);

‍‍‍‍‍‍

We get back 12 records - everything is exactly as expected.

Step 2:


One of the records in the list is displayed in the custom UI being built, is then marked (check box) for a delete.  The next call is then made:

document.delete(true);
‍‍‍

Here the document instance correlates to the selected item marked by the user for Delete.

This also works, the document is in fact deleted from the repository exactly as expected.

We then attempt to run this immediately AFTER the delete() call is made - same exact code as Step 1.

Step 3:

String queryString = SELECT * FROM acme:documents  WHERE IN_FOLDER('workspace://SpacesStore/c98cd558-e159-44fa-927a-7e9b17241dff') AND CONTAINS ('engineering')

cmisSession.query(queryString, false);
‍‍‍‍‍

We EXPECT to get back 11 records - INSTEAD we get the exception thrown UNLESS we purposefully hold with a sleep() between Step 2 and 3.

We are not using an Alfresco-specific APIs for this as we cannot for this work. We are adhering to the CMIS SQL only (a subset of SQL-92 as in the specification for CMIS)

Is this perhaps clearer?

[jpotts]

Mark is suggesting that you essentially have two options:
1. Keep Solr, but use a non-search call like Folder.getChildren(), Session.getObject(), or Session.getObjectByPath(). Granted this may not meet your needs because you are also doing a full-text search to get your result set.
2. Switch from Solr to Lucene. This is a simple configuration change, but if you are writing a tool that must work with Alfresco regardless of how it is configured, this may not be something you can depend on.

Jeff

[fpa1974]

I actually would like to ask a question on the eventual consistency theme here. Basically the sequence of events here is:
1. Perform a query
2. Delete a document
3. Perform a query
In mind, when I execute the query for the second time, I might get in the result set the document that was deleted or not, depending on the timing. But I would not expect to get an exception. This is what I understand as eventual consistency. Is there something that can be done so AF4 behaves that way with solr enabled?

Thanks,
Florian

P.S. I work with Mark on this project.