Java backed webscript not working - upgrade 3.0 to 3.2r

[lynnders]

I'm upgrading from 3.0 to 3.2r and the only thing not working is a a Java backed webscript.

My webscript class Bloghunter extends PageableWebScript which in turn extends DeclarativeWebscript

Here is an extract from Bloghunter.getResults

        SearchParameters sp = new SearchParameters();        sp.addStore(new StoreRef(StoreRef.PROTOCOL_AVM, "someco"));        sp.setLanguage(SearchService.LANGUAGE_LUCENE);        sp.setQuery("@\\{http\\://www.alfresco.org/alfresco/blog\\}");               sp.setLimit(250);        sp.setLimitBy(LimitBy.FINAL_SIZE);        sp.addSort("@{http://www.alfresco.org/alfresco/blog}created", false);                     ResultSet results = null;        List<NodeRef> nodes = null;        try        {            results = serviceRegistry.getSearchService().query(sp);            nodes = results.getNodeRefs();        }        finally        {            if(results != null)            {                results.close();            }        }        logger.info("return nodes()");        return nodes == null ? null : nodes;‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

The problem seems to be with the sort parameter
        sp.addSort("@{http://www.alfresco.org/alfresco/blog}created", false);
I've tried
        setSort("created", false, sp);
this returned results, but sorted by last modified date and not by the created date

This used to work in 3.0, I can see that DeclarativeWebscript has changed from 3.0 to 3.2r and wondered would these changes affect my webscript's sort parameter?

Extract from my logs

15:55:10,253 User:admin DEBUG [web.scripts.RepositoryContainer] Authentication: authenticated as admin15:55:10,253 User:admin DEBUG [web.scripts.RepositoryContainer] Begin retry transaction block: required,readwrite15:55:10,253 User:admin DEBUG [web.scripts.RepositoryContainer] Creating Transactional Response for ReadWrite transaction; buffersize=409615:55:10,253 User:admin DEBUG [lucene.index.IndexInfo] Got /var/alfresco32/alf_data/lucene-indexes/avm/someco org.alfresco.repo.search.impl.lucene.index.IndexInfo@68d2d88 for /var/alfresco32/alf_data/lucene-indexes/avm/someco15:55:10,253 User:admin DEBUG [lucene.index.IndexInfo] Waiting for READ lock  - http-8080-115:55:10,253 User:admin DEBUG [lucene.index.IndexInfo] …GOT READ LOCK  - http-8080-1 -  in 1.374E-4 ms15:55:10,253 User:admin DEBUG [lucene.index.ReferenceCountingReadOnlyIndexReaderFactory$ReferenceCountingReadOnlyIndexReader] http-8080-1: Reader MainReader - increment - ref count is 1        … org.alfresco.repo.search.impl.lucene.index.ReferenceCountingReadOnlyIndexReaderFactory$ReferenceCountingReadOnlyIndexReader@49f11e6c15:55:10,253 User:admin DEBUG [lucene.index.IndexInfo] Main index reader references = 115:55:10,253 User:admin DEBUG [lucene.index.IndexInfo] RELEASED READ LOCK  - http-8080-115:55:10,259 User:admin DEBUG [web.scripts.DeclarativeWebScript] Caught exception; decorating with appropriate status template : java.lang.NullPointerException        at org.alfresco.repo.search.impl.lucene.ADMLuceneSearcherImpl.query(ADMLuceneSearcherImpl.java:330)        at org.alfresco.repo.search.SearcherComponent.query(SearcherComponent.java:84)        at sun.reflect.GeneratedMethodAccessor437.invoke(Unknown Source)        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)        at java.lang.reflect.Method.invoke(Method.java:597)        at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:304)        at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:182)        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:149)        at net.sf.acegisecurity.intercept.method.aopalliance.MethodSecurityInterceptor.invoke(MethodSecurityInterceptor.java:80)        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)        at org.alfresco.repo.security.permissions.impl.ExceptionTranslatorMethodInterceptor.invoke(ExceptionTranslatorMethodInterceptor.java:49)        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)        at org.alfresco.repo.audit.AuditMethodInterceptor.invoke(AuditMethodInterceptor.java:147)        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)        at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:106)        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)        at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)        at $Proxy24.query(Unknown Source)        at com.someco.webscript.content.Bloghunter.getResults(Bloghunter.java:52)        at com.someco.webscript.PageableWebScript.executeImpl(PageableWebScript.java:25)        at org.alfresco.web.scripts.DeclarativeWebScript.execute(DeclarativeWebScript.java:70)        at org.alfresco.repo.web.scripts.RepositoryContainer$2.execute(RepositoryContainer.java:372)        at org.alfresco.repo.transaction.RetryingTransactionHelper.doInTransaction(RetryingTransactionHelper.java:327)        at org.alfresco.repo.web.scripts.RepositoryContainer.transactionedExecute(RepositoryContainer.java:422)        at org.alfresco.repo.web.scripts.RepositoryContainer.transactionedExecuteAs(RepositoryContainer.java:439)        at org.alfresco.repo.web.scripts.RepositoryContainer.executeScript(RepositoryContainer.java:303)        at org.alfresco.web.scripts.AbstractRuntime.executeScript(AbstractRuntime.java:306)        at org.alfresco.web.scripts.AbstractRuntime.executeScript(AbstractRuntime.java:183)        at org.alfresco.web.scripts.servlet.WebScriptServlet.service(WebScriptServlet.java:122)        at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)        at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)        at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)        at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:128)        at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)        at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:286)        at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:845)        at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:583)        at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:447)        at java.lang.Thread.run(Thread.java:619)15:55:10,260 User:admin DEBUG [web.scripts.RepositoryContainer] Transaction exception: required: 04260001 Wrapped Exception (with status template): null15:55:10,261 User:admin DEBUG [web.scripts.RepositoryContainer] Transaction status: 115:55:10,261 User:admin DEBUG [web.scripts.RepositoryContainer] End retry transaction block: required,readwrite15:55:10,262  DEBUG [web.scripts.RepositoryContainer] Authentication reset: unauthenticated15:55:10,262  DEBUG [web.scripts.AbstractRuntime] Web Script com/someco/content/bloghunter.get executed in 14.612255ms15:55:10,264  ERROR [web.scripts.AbstractRuntime] Exception from executeScript - redirecting to status template error: 04260001 Wrapped Exception (with status template): nullorg.alfresco.web.scripts.WebScriptException: 04260001 Wrapped Exception (with status template): null        at org.alfresco.web.scripts.AbstractWebScript.createStatusException(AbstractWebScript.java:613)        at org.alfresco.web.scripts.DeclarativeWebScript.execute(DeclarativeWebScript.java:165)        at org.alfresco.repo.web.scripts.RepositoryContainer$2.execute(RepositoryContainer.java:372)        at org.alfresco.repo.transaction.RetryingTransactionHelper.doInTransaction(RetryingTransactionHelper.java:327)        at org.alfresco.repo.web.scripts.RepositoryContainer.transactionedExecute(RepositoryContainer.java:422)        at org.alfresco.repo.web.scripts.RepositoryContainer.transactionedExecuteAs(RepositoryContainer.java:439)        at org.alfresco.repo.web.scripts.RepositoryContainer.executeScript(RepositoryContainer.java:303)        at org.alfresco.web.scripts.AbstractRuntime.executeScript(AbstractRuntime.java:306)        at org.alfresco.web.scripts.AbstractRuntime.executeScript(AbstractRuntime.java:183)        at org.alfresco.web.scripts.servlet.WebScriptServlet.service(WebScriptServlet.java:122)        at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)        at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)        at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)        at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:128)        at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)        at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:286)        at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:845)        at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:583)        at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:447)        at java.lang.Thread.run(Thread.java:619)Caused by: java.lang.NullPointerException        at org.alfresco.repo.search.impl.lucene.ADMLuceneSearcherImpl.query(ADMLuceneSearcherImpl.java:330)        at org.alfresco.repo.search.SearcherComponent.query(SearcherComponent.java:84)        at sun.reflect.GeneratedMethodAccessor437.invoke(Unknown Source)        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)        at java.lang.reflect.Method.invoke(Method.java:597)        at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:304)        at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:182)        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:149)        at net.sf.acegisecurity.intercept.method.aopalliance.MethodSecurityInterceptor.invoke(MethodSecurityInterceptor.java:80)        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)        at org.alfresco.repo.security.permissions.impl.ExceptionTranslatorMethodInterceptor.invoke(ExceptionTranslatorMethodInterceptor.java:49)        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)        at org.alfresco.repo.audit.AuditMethodInterceptor.invoke(AuditMethodInterceptor.java:147)        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)        at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:106)        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)        at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)        at $Proxy24.query(Unknown Source)        at com.someco.webscript.content.Bloghunter.getResults(Bloghunter.java:52)        at com.someco.webscript.PageableWebScript.executeImpl(PageableWebScript.java:25)        at org.alfresco.web.scripts.DeclarativeWebScript.execute(DeclarativeWebScript.java:70)        … 21 more15:55:10,264  DEBUG [scripts.servlet.WebScriptServletRequest] Content Type: null‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

[lynnders]

The problem was in ADMLuceneSearcherImpl at line 330

The propertyDef is null so the call to propertyDef.getDataType throws an error

It used to work in 3.0 only because the guy who worked here before me had modified the alfresco code and added the following check
if (propertyDef != null)

So here's the code that works for me

    public ResultSet query(SearchParameters searchParameters)    {        if (searchParameters.getStores().size() != 1)        {            throw new IllegalStateException("Only one store can be searched at present");        }        ArrayList<StoreRef> stores = searchParameters.getStores();        stores.set(0, tenantService.getName(searchParameters.getStores().get(0)));        String parameterisedQueryString;        if (searchParameters.getQueryParameterDefinitions().size() > 0)        {            Map<QName, QueryParameterDefinition> map = new HashMap<QName, QueryParameterDefinition>();            for (QueryParameterDefinition qpd : searchParameters.getQueryParameterDefinitions())            {                map.put(qpd.getQName(), qpd);            }            parameterisedQueryString = parameterise(searchParameters.getQuery(), map, null, namespacePrefixResolver);        }        else        {            parameterisedQueryString = searchParameters.getQuery();        }        if (searchParameters.getLanguage().equalsIgnoreCase(SearchService.LANGUAGE_LUCENE))        {            try            {                Operator defaultOperator;                if (searchParameters.getDefaultOperator() == SearchParameters.AND)                {                    defaultOperator = LuceneQueryParser.AND_OPERATOR;                }                else                {                    defaultOperator = LuceneQueryParser.OR_OPERATOR;                }                ClosingIndexSearcher searcher = getSearcher(indexer);                Query query = LuceneQueryParser.parse(parameterisedQueryString, DEFAULT_FIELD, new LuceneAnalyser(getDictionaryService(),                        searchParameters.getMlAnalaysisMode() == null ? getLuceneConfig().getDefaultMLSearchAnalysisMode() : searchParameters.getMlAnalaysisMode()),                        namespacePrefixResolver, getDictionaryService(), tenantService, defaultOperator, searchParameters, getLuceneConfig(), searcher.getIndexReader());                if (s_logger.isDebugEnabled())                {                    s_logger.debug("Query is " + query.toString());                }                if (searcher == null)                {                    // no index return an empty result set                    return new EmptyResultSet();                }                Hits hits;                boolean requiresPostSort = false;                if (searchParameters.getSortDefinitions().size() > 0)                {                    int index = 0;                    SortField[] fields = new SortField[searchParameters.getSortDefinitions().size()];                    for (SearchParameters.SortDefinition sd : searchParameters.getSortDefinitions())                    {                        switch (sd.getSortType())                        {                        case FIELD:                            Locale sortLocale = null;                            String field = sd.getField();                            if (field.startsWith("@"))                            {                                field = expandAttributeFieldName(field);                                PropertyDefinition propertyDef = getDictionaryService().getProperty(QName.createQName(field.substring(1)));                                // ********************** 28/05/10 MODIFICATION - check for null propertyDef **********************                                if (propertyDef != null)                                {                                    if (propertyDef.getDataType().getName().equals(DataTypeDefinition.CONTENT))                                    {                                        throw new SearcherException("Order on content properties is not curently supported");                                    }                                    else if ((propertyDef.getDataType().getName().equals(DataTypeDefinition.MLTEXT))                                            || (propertyDef.getDataType().getName().equals(DataTypeDefinition.TEXT)))                                    {                                        List<Locale> locales = searchParameters.getLocales();                                        if (((locales == null) || (locales.size() == 0)))                                        {                                            locales = Collections.singletonList(I18NUtil.getLocale());                                        }                                            if (locales.size() > 1)                                        {                                            throw new SearcherException("Order on text/mltext properties with more than one locale is not curently supported");                                        }                                            sortLocale = locales.get(0);                                        // find best field match                                            MLAnalysisMode analysisMode = getLuceneConfig().getDefaultMLSearchAnalysisMode();                                        HashSet<String> allowableLocales = new HashSet<String>();                                        for (Locale l : MLAnalysisMode.getLocales(analysisMode, sortLocale, false))                                        {                                            allowableLocales.add(l.toString());                                        }                                            String sortField = field;                                            for (Object current : searcher.getReader().getFieldNames(FieldOption.INDEXED))                                        {                                            String currentString = (String) current;                                            if (currentString.startsWith(field) && currentString.endsWith(".sort"))                                            {                                                String fieldLocale = currentString.substring(field.length() + 1, currentString.length() - 5);                                                if (allowableLocales.contains(fieldLocale))                                                {                                                    if (fieldLocale.equals(sortLocale.toString()))                                                    {                                                        sortField = currentString;                                                        break;                                                    }                                                    else if (sortLocale.toString().startsWith(fieldLocale))                                                    {                                                        if (sortField.equals(field) || (currentString.length() < sortField.length()))                                                        {                                                            sortField = currentString;                                                        }                                                    }                                                    else if (fieldLocale.startsWith(sortLocale.toString()))                                                    {                                                        if (sortField.equals(field) || (currentString.length() < sortField.length()))                                                        {                                                            sortField = currentString;                                                        }                                                    }                                                }                                            }                                        }                                            field = sortField;                                        }                                    else if (propertyDef.getDataType().getName().equals(DataTypeDefinition.DATETIME))                                    {                                        DataTypeDefinition dataType = propertyDef.getDataType();                                        String analyserClassName = dataType.getAnalyserClassName();                                        if (analyserClassName.equals(DateTimeAnalyser.class.getCanonicalName()))                                        {                                            field = field + ".sort";                                        }                                        else                                        {                                            requiresPostSort = true;                                        }                                    }                                }                            }                            if (fieldHasTerm(searcher.getReader(), field))                            {                                fields[index++] = new SortField(field, sortLocale, !sd.isAscending());                            }                            else                            {                                fields[index++] = new SortField(null, SortField.DOC, !sd.isAscending());                            }                            break;                        case DOCUMENT:                            fields[index++] = new SortField(null, SortField.DOC, !sd.isAscending());                            break;                        case SCORE:                            fields[index++] = new SortField(null, SortField.SCORE, !sd.isAscending());                            break;                        }                    }                    hits = searcher.search(query, new Sort(fields));                }                else                {                    hits = searcher.search(query);                }                ResultSet rs = new LuceneResultSet(hits, searcher, nodeService, tenantService, searchParameters, getLuceneConfig());                rs = new PagingLuceneResultSet(rs, searchParameters, nodeService);                if (getLuceneConfig().getPostSortDateTime() && requiresPostSort)                {                    ResultSet sorted = new SortedResultSet(rs, nodeService, searchParameters, namespacePrefixResolver);                    return sorted;                }                else                {                    return rs;                }            }            catch (ParseException e)            {                throw new SearcherException("Failed to parse query: " + parameterisedQueryString, e);            }            catch (IOException e)            {                throw new SearcherException("IO exception during search", e);            }        }        else if (searchParameters.getLanguage().equalsIgnoreCase(SearchService.LANGUAGE_XPATH))        {            try            {                XPathReader reader = new XPathReader();                LuceneXPathHandler handler = new LuceneXPathHandler();                handler.setNamespacePrefixResolver(namespacePrefixResolver);                handler.setDictionaryService(getDictionaryService());                // TODO: Handler should have the query parameters to use in                // building its lucene query                // At the moment xpath style parameters in the PATH                // expression are not supported.                reader.setXPathHandler(handler);                reader.parse(parameterisedQueryString);                Query query = handler.getQuery();                Searcher searcher = getSearcher(null);                if (searcher == null)                {                    // no index return an empty result set                    return new EmptyResultSet();                }                Hits hits = searcher.search(query);                ResultSet rs = new LuceneResultSet(hits, searcher, nodeService, tenantService, searchParameters, getLuceneConfig());                rs = new PagingLuceneResultSet(rs, searchParameters, nodeService);                return rs;            }            catch (SAXPathException e)            {                throw new SearcherException("Failed to parse query: " + searchParameters.getQuery(), e);            }            catch (IOException e)            {                throw new SearcherException("IO exception during search", e);            }        }        else        {            LuceneQueryLanguageSPI language = queryLanguages.get(searchParameters.getLanguage().toLowerCase());            if (language != null)            {                return language.executQuery(searchParameters, this);            }            else            {                throw new SearcherException("Unknown query language: " + searchParameters.getLanguage());            }        }    }‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

I could do some more investigation and find out what is expected here:
PropertyDefinition propertyDef = getDictionaryService().getProperty(QName.createQName(field.substring(1)));

[mrogers]

If you come to the conclusion that there is a bug in Alfresco please raise it in JIRA so that it can be fixed.  
Even better would be to contribute your solution.

[lynnders]

I've setup an account in Alfresco Jira. Do I 'Create Issue' here - https://issues.alfresco.com/jira/browse/ETHREEOH/fixforversion/10572  ?

I'm using the Community Version 3.2r

[lynnders]

Raised in JIRA - https://issues.alfresco.com/jira/browse/ALF-3042

[dward]

The error indicates that the property you are using as a sort field with namespace http://www.alfresco.org/alfresco/blog and short name "created" does not exist in the data model. Have you checked that there is a *Model.xml file that registers this property? Perhaps you haven't re-installed one of the amps you previously had installed?

Whilst the NPE isn't very helpful here, it is indicative of another problem.