Missing .bin file after restore

[streetturtle]

After we did Alfresco restore some random users couldn't login to Alfresco. In the logs for each login attempt there is following stacktrace:

REPO_ 2015-11-16 16:46:12,569  ERROR [extensions.webscripts.AbstractRuntime] [http-apr-9443-exec-9] Exception from executeScript - redirecting to status template error: 10160432 Wrapped Exception (with status template): 10166691 Failed to execute script 'classpath*:alfresco/templates/webscripts/org/alfresco/slingshot/documentlibrary/doclist.get.js': 10166690 Failed to open stream onto channel:    accessor: ContentAccessor[ contentUrl=store://2015/7/28/10/17/5f8a8649-9f9e-472b-b6c1-4cc21d28155f.bin, mimetype=text/plain, size=0, encoding=UTF-8, locale=en_US]org.springframework.extensions.webscripts.WebScriptException: 10160432 Wrapped Exception (with status template): 10166691 Failed to execute script 'classpath*:alfresco/templates/webscripts/org/alfresco/slingshot/documentlibrary/doclist.get.js': 10166690 Failed to open stream onto channel:    accessor: ContentAccessor[ contentUrl=store://2015/7/28/10/17/5f8a8649-9f9e-472b-b6c1-4cc21d28155f.bin, mimetype=text/plain, size=0, encoding=UTF-8, locale=en_US]   at org.springframework.extensions.webscripts.AbstractWebScript.createStatusException(AbstractWebScript.java:   …   at java.lang.Thread.run(Thread.java:744)Caused by: org.alfresco.scripts.ScriptException: 10166691 Failed to execute script 'classpath*:alfresco/templates/webscripts/org/alfresco/slingshot/documentlibrary/doclist.get.js': 10166690 Failed to open stream onto channel:    accessor: ContentAccessor[ contentUrl=store://2015/7/28/10/17/5f8a8649-9f9e-472b-b6c1-4cc21d28155f.bin, mimetype=text/plain, size=0, encoding=UTF-8, locale=en_US]   at org.alfresco.repo.jscript.RhinoScriptProcessor.execute(RhinoScriptProcessor.java:204)        …    at org.springframework.extensions.webscripts.DeclarativeWebScript.execute(DeclarativeWebScript.java:86)   … 33 moreCaused by: org.alfresco.service.cmr.repository.ContentIOException: 10166690 Failed to open stream onto channel:    accessor: ContentAccessor[ contentUrl=store://2015/7/28/10/17/5f8a8649-9f9e-472b-b6c1-4cc21d28155f.bin, mimetype=text/plain, size=0, encoding=UTF-8, locale=en_US]   at org.alfresco.repo.content.AbstractContentReader.getContentInputStream(AbstractContentReader.java:401)   …   at org.mozilla.javascript.gen.classpath__alfresco_templates_webscripts_org_alfresco_slingshot_documentlibrary_doclist_get_js_6._c_Common_getFavourites_11(classpath*:alfresco/templates/webscripts/org/alfresco/slingshot/documentlibrary/doclist.get.js:666)   at org.mozilla.javascript.gen.classpath__alfresco_templates_webscripts_org_alfresco_slingshot_documentlibrary_doclist_get_js_6.call(classpath*:alfresco/templates/webscripts/org/alfresco/slingshot/documentlibrary/doclist.get.js)   at org.mozilla.javascript.optimizer.OptRuntime.callProp0(OptRuntime.java:85)   at org.mozilla.javascript.gen.classpath__alfresco_templates_webscripts_org_alfresco_slingshot_documentlibrary_doclist_get_js_6._c_getDoclist_20(classpath*:alfresco/templates/webscripts/org/alfresco/slingshot/documentlibrary/doclist.get.js:1111)   at org.mozilla.javascript.gen.classpath__alfresco_templates_webscripts_org_alfresco_slingshot_documentlibrary_doclist_get_js_6.call(classpath*:alfresco/templates/webscripts/org/alfresco/slingshot/documentlibrary/doclist.get.js)   at org.mozilla.javascript.optimizer.OptRuntime.callName0(OptRuntime.java:74)   at org.mozilla.javascript.gen.classpath__alfresco_templates_webscripts_org_alfresco_slingshot_documentlibrary_doclist_get_js_6._c_script_0(classpath*:alfresco/templates/webscripts/org/alfresco/slingshot/documentlibrary/doclist.get.js:1389)   at org.mozilla.javascript.gen.classpath__alfresco_templates_webscripts_org_alfresco_slingshot_documentlibrary_doclist_get_js_6.call(classpath*:alfresco/templates/webscripts/org/alfresco/slingshot/documentlibrary/doclist.get.js)   at org.mozilla.javascript.ContextFactory.doTopCall(ContextFactory.java:394)   at org.mozilla.javascript.ScriptRuntime.doTopCall(ScriptRuntime.java:3091)   at org.mozilla.javascript.gen.classpath__alfresco_templates_webscripts_org_alfresco_slingshot_documentlibrary_doclist_get_js_6.call(classpath*:alfresco/templates/webscripts/org/alfresco/slingshot/documentlibrary/doclist.get.js)   at org.mozilla.javascript.gen.classpath__alfresco_templates_webscripts_org_alfresco_slingshot_documentlibrary_doclist_get_js_6.exec(classpath*:alfresco/templates/webscripts/org/alfresco/slingshot/documentlibrary/doclist.get.js)   at org.alfresco.repo.jscript.RhinoScriptProcessor.executeScriptImpl(RhinoScriptProcessor.java:502)   at org.alfresco.repo.jscript.RhinoScriptProcessor.execute(RhinoScriptProcessor.java:200)   … 38 moreCaused by: org.springframework.dao.ConcurrencyFailureException: store://2015/7/28/10/17/5f8a8649-9f9e-472b-b6c1-4cc21d28155f.bin no longer exists   at org.alfresco.repo.content.EmptyContentReader.getDirectReadableChannel(EmptyContentReader.java:56)   at org.alfresco.repo.content.AbstractContentReader.getReadableChannel(AbstractContentReader.java:271)   at org.alfresco.repo.content.AbstractContentReader.getContentInputStream(AbstractContentReader.java:375)   … 62 more‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

This part looks interesting:

Caused by: org.springframework.dao.ConcurrencyFailureException: store://2015/7/28/10/17/5f8a8649-9f9e-472b-b6c1-4cc21d28155f.bin no longer exists‍‍‍

If I create this file in the

alf_data/contentstore/2015/7/28/10/17/5f8a8649-9f9e-472b-b6c1-4cc21d28155f.bin‍

with empty curly brackets (

{}‍

) user can login and it solves the problem. But we have to many users so it would be a total mess to create such file for each user who has this problem.

But in the stacktrace I found another interesting part:

org.mozilla.javascript.gen.classpath__alfresco_templates_webscripts_org_alfresco_slingshot_documentlibrary_doclist_get_js_6._c_Common_getFavourites_11(classpath*:alfresco/templates/webscripts/org/alfresco/slingshot/documentlibrary/doclist.get.js:666)‍‍‍

So I would guess that js method

getFavourites‍

in doclist.js is causing this error. Which means that when user logins to Alfresco main page should be shown, but it has dashlet with favorites, which couldn't be loaded due to missing file. This would explain the error.
I also tried to call a preference webscript which returns user preferences (and favourites files as well): https://someco.com/alfresco/service/api/people/someusername/preferences and for some user it failed with exactly the same error:

{  "status": {    "code": 500,    "name": "Internal Error",    "description": "An error inside the HTTP server which prevented it from fulfilling the request."  },  "message": "org.alfresco.service.cmr.repository.ContentIOException: 10166868 Failed to open stream onto channel: \n   accessor: ContentAccessor[ contentUrl=store:\/\/2015\/7\/28\/11\/18\/71f8a94e-2fbb-4ec1-a12a-56b8330ebd3c.bin, mimetype=text\/plain, size=0, encoding=UTF-8, locale=en_US]",  "exception": "",  "callstack": [  ],  "server": "Community v5.0.0 (c r91299-b145) schema 8,009",  "time": "Nov 16, 2015 6:28:40 PM"}‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

So I think that my guess with favourites was right: for some reason after restore all files in the contentstore which contained users' favourites disappeared.

<b>Now reasonable question is how it could be possible?</b>
Because apart from this problem Alfresco works fine, we have more than 60000 documents there and none of them are missed.

<b>And another question: how it could be solved?</b>
As a workaround I think could be useful to get list of all users who used favourites functionality, so we can tell which users will have similar problem with login. 

[aowian]

When we rolled back the database, the ALF_CONTENT_URL table was still pointing to content that wasn't there after having deleted the contentstore.    Basically any time the database is not in sync with the contentstore this problem could arise.  Orphaned pointers.

This is what I did to fix the same problem:
<blockcode>
#!/bin/sh

FAVS_PATH="${CONTENTSTORE_HOME}/alf_data/contentstore/2014/4/28/14/58/"
FAVS_FILE="${FAVS_PATH}0c2bf591-1138-4a33-bf8b-c28239d80b83.bin"

mkdir -p $FAVS_PATH
echo "{}" > $FAVS_FILE
chmod 755 $FAVS_FILE
</blockcode>

Here was my error:
<em>
…
Caused by: org.springframework.dao.ConcurrencyFailureException: store://2014/4/28/14/58/0c2bf591-1138-4a33-bf8b-c28239d80b83.bin no longer exists
…
</em>