cancel
Showing results for 
Search instead for 
Did you mean: 

new rest2 webapp

tombaeyens
Champ in-the-making
Champ in-the-making
Huh, what is the rest2 webapp?

we're trying out a new rest style.  one that gives us better control.  we're not that happy with the indirect steps of the template rendering.  lot of things need to be right in order for it to work and we believe it could be done simpler.
so we're now trying out a simple servlet approach and building JSONObjects in the servlet that get streamed to the client.

the api should be fairly stable now.  rest api is not yet as there is already legacy in the original rest api.  so we want to combine 2 steps in one:
1) make the rest impl simpler
2) review the rest calls and sync them with the api as it is stable now.

but there is no final decision on this servlet strategy yet.   we're still in try-out mode.
23 REPLIES 23

falko_menge
Champ in-the-making
Champ in-the-making
I'd like to add, that I clearly see value in having a generic REST API for remote interactions with the engine. From what I heard, the current REST API seems to be a bit too tailored to the three web applications shipped with Activiti.

Can we collect a list of changes that need to be made in terms of URL patterns and representations in order to make the REST API more generic?

Based on this set of changes, we could decide on one the following options:
1. Do the changes in the existing REST module, which would in turn require to adapt all our webapps. However, we are already quite busy with implementing the last features for the GA release in the webapps. Hence, there is a high risk of breaking existing features and not completing planned features.
2. If there are too many changes, a second REST module may avoid those risks. The existing REST module could then be marked as deprecated. But, I don't think that we will be able to switch all webapps to use a different REST API until the GA release, as there is an even higher risk of breaking existing features and not completing planned features.

nils1
Champ in-the-making
Champ in-the-making
As I understood, the work on the new REST API (2) is planned for the next three to four months. As long as it doesn't break the existing web apps, I don't see a reason not to do it. I won't really be able to dedicate time to it myself, but I'm interested in the outcome. Once the API is there, we'll be able to judge whether we can easily switch the webapps to it.

We should just try to keep effort and actual benefits in balance. If it takes too much time to switch the webapps to the new API, the time might be better invested in making the existing API stable and consistent. But as long as you take the existing one as a starting point and make sure the new API provides everything the webapps need, the transition shouldn't be too difficult.

Cheers,
Nils

frederikherema1
Star Contributor
Star Contributor
@Bernd:
I think i'll ditch the "roll our own" solution based on the feedback received Smiley Wink
About the extra modules. The custom cycle rest services would be exposed in the activiti-webapp-cycle itself, so no extra jars/wars are needed. Since the UI (cycle, explorer, probe) will evolve much more than the engine API, it would be a bit harder to mark the whole REST (including custom UI stuff) as stable and guarentee the same level of stability and backwards-compatibility. A split will address this issue.

@Nils, Falko:
It's indeed a bit risky to do a whole architecture change right before a GA release. Anyway, first thing I will try is to create unit/integration tests for the existing REST-api, testing within webapp container to make it more stable.

tombaeyens
Champ in-the-making
Champ in-the-making
@Bernd: the motivation to split up the rest calls between UI specific and generic rest calls lies in our experience on how we create the rest calls now.  Most of the view related rest requests are just based on how the needs for the current UI.  Those calls group a number of data fetching api calls and build a data structure that contains all the data to render a particular screen.  Those type of UI related calls are not generic, but UI specific.    On the other hand, most of the action rest requests map 1-1 to api service calls.  Those requests are more generic and can be reused by multiple applications.  That's the motivation to split up the generic, API based rest requests from the specific UI based rest requests.  We think it's best to include the UI specific rest requests as part of the specific .war

frederikherema1
Star Contributor
Star Contributor
Hi,

We're sticking with the Spring-surf Webscript, using JSONObject instead of the *.ftl's for rendering JSON responses.

I've implemented a solution for this (which I'll checkin in a few minutes on trunk). It's already plugged-in into the webscript-context, but as long as the *.ftl template is available, the FreemarkerTemplateProcessor will be used. Once we are confident about the solution, the ftl's are removed and JSONObject stuff kicks in.

Curently, all rest-calls (identity, engine, process, repository, task) have new implementation on trunk, except for the api.cycle.* calls.

A quick overview of the implementation:

* Plugged in a TemplateProcessor into the webscript context, which is registered automatically when context starts/reloads:
<bean id="jsonObjectTemplateProcessor" class="org.activiti.rest.builder.JSONTemplateProcessor">
      <property name="templateProcessorRegistry" ref="webscripts.web.templateregistry" />
      <property name="defaultEncoding" value="UTF-8" />
      <property name="templateNamePrefix" value="org/activiti/rest/api/" />
      <property name="indentFactor" value="2" />
  </bean>

* For each webscript we want to have rendering for using JSONObject, we add a bean in spring-context, eg.
<bean id="org.activiti.rest.api.repository.deployments.get.json.builder"
        class="org.activiti.rest.builder.repository.DeploymentsObjectBuilder">
    <property name="templateName" value="repository/deployments.get.json" />
  </bean>

The bean implements interface org.activiti.rest.builder.JSONObjectBuilder, which is picked up automatically by our TemplateProcessor:
public interface JSONObjectBuilder {

  String getTemplateName();
  JSONObject createJsonObject(Object model) throws JSONException;
 
}

* When for example webscript "org/activiti/rest/api/repository/deployments.get" is called, in format JSON, out TemplateProcessor will receive a request to render template 'org/activiti/rest/api/repository/deployments.get'. Since we have a JSONObjectBuilder regsitered with templateName "repository/deployments.get.json" (also note the 'templateNamePrefix' is used when matchin templates) , this bean will be called to create a JSONObject, which is written to the response by the TemplateProcessor.

private DeploymentJSONConverter converter = new DeploymentJSONConverter();

  @Override
  @SuppressWarnings("unchecked")
  public JSONObject createJsonObject(Object modelObject) throws JSONException {
    JSONObject result = new JSONObject();
    Map<String, Object> model = getModelAsMap(modelObject);
    List<Deployment> deployments = (List<Deployment>) model.get("deployments");

    JSONUtil.putPagingInfo(result, model);
    JSONArray deploymentsArray = JSONUtil.putNewArray(result, "data");

    if (deployments != null) {
      for (Deployment deployment : deployments) {
        deploymentsArray.put(converter.getJSONObject(deployment));
      }
    }
    return result;
  }
public class DeploymentJSONConverter implements JSONConverter<Deployment> {


  public JSONObject getJSONObject(Deployment deployment) throws JSONException {
    JSONObject json = new JSONObject();
    JSONUtil.putRetainNull(json, "id", deployment.getId());
    JSONUtil.putRetainNull(json, "name", deployment.getName());
   
    String deploymentTime = JSONUtil.formatISO8601Date(deployment.getDeploymentTime());
    JSONUtil.putRetainNull(json, "deploymentTime", deploymentTime);
   
    return json;
  }



The result model created by the existing DeploymentsGet is used to create a JSONObject. THe DeploymentJSONConverter is reusable, this calls our model's getters to get the data into JSONObject. Adding paging info can be done in a generic way.

* Only the rendering has been altered, the web script implementations haven't changed a single letter, so all the UI-guys' hard work hasn't gone to waste Smiley Wink Other representations can still be plugged in using the normal webscript-way offcourse.
* JSONUtil has some convenience methods to add values/arrays and format dates using the spring-surf iso8601DateFormatter.

_____________________________

Currently, Exporer and Probe run fine with the new implementations. Although, we should have the REST covered by unit/integration tests. Looks like we have 2 options:

* Use embedded tomcat which enables us to sue the same ProcessEngine, so we can set up some test-data using the API and pull it out using Restlet (which provide one-liners to do a single rest-call)
* Use WebScriptTestServer which is present in /scr/test/java of spring-surf webscripts. I can't seem to find out how to get it running, but I hope Eric can give me a few pointers on this.

WDYT?

erikwinlof
Confirmed Champ
Confirmed Champ
Hi, here comes a bit of pro-freemarker feedback, since I i'm not 100% about the new solution 🙂

Please note that this response is only about response rendering, it has nothing to do with having a core api and additional ui rest apis.

As I've understood the complaints against templates are basically 2:

#1. An engine api change might break the rest api.
This is true and a good argument. It can be solved by the junit tests. Joram showed me some Alfresco tests that did it and seemed satisified with how it was done. Alfresco is using this for their rest api having thousands of lines of freemarkerjson responses and it works for them.


#2. Its hard for the engine team to maintain since you don't know freemarker.
Well I hope I'm not being hard on you know, but its like saying I don't want to use Activiti for processes since I already know BPM4.
And since we are planning to use freemarker in the forms, you will need to learn it anyway 🙂
Anyway Freemarker is extremely easy to learn and use, and in the bottom is a crash course containing all you need for Activiti development.


# Why I think programmatic rendering is a bad idea:

A. The overview of how the JSON response looks is totally lost.

B. The amount of code lines needed are much greater, just consider the condition examples below to be written in java.

C. The benefit of getting a toJSON method in all pojos and use them in the rest api is not realistic since the pojo model will never match the services methods in 100%, some examples:
- Task (resourceName isn't included anymore therefore needing a whole new class RestTask on some objects but not all, confusing an creates even more code to maintain)
- RestProcessDefinition (same story as line above)

D. More examples where the POJO doesn't match a response:
- In the /process-engine response ProcessEngineInfo doesn't cover the complete response and additional "version" attribute is required)

E. "modern" frameworks like rails, django use templates making it easier to use.

F. Freemarker is a robust framework with very good documentation: http://freemarker.sourceforge.net/docs/index.html

G. We don't use the webscripts framework in a way its not intended.

(H). Most of all alfrescans and alfresco users as well for that matter know how to use json in freemarker and the webscripts api making it easy for them to read and understand the code.

############################
# FREEMARKER CRASH COURSE: #
############################

# Examples
Hello ${user.firstName}!          // Prints firstName if it exists, if it doesn't it will crash.
Hello ${user.firstName!"Guest"}  // Prints firstName if it exists, if it doesn't exist the defatul value 'Guest' will be printed
A number ${count}                // Will print and format ok as long as it doesn't exceed the number 1000, then it will be printed as "1 000"
A number ${count?c}              // will be printed as expeceted (i.e. "1000")  for all values
A date ${iso8601Date(today)}     // This is how we print dates in the activiti rest so its formatted as iso8601 dates

To use macros string escaping and library files:

activiti.lib.ftl ( a file with a macro inside)
——————————————————

<#escape x as jsonUtils.encodeJSONString(x)>

<#macro printName name>
  Name: ${name}
</#macro>

</#escape>

… can be used and imported as a lib in another file as:
————————————————————
<#import "../activiti.lib.ftl" as restLib>
<@restLib.printName "Erik Winlöf"/>

All strings will be json-escaped but the int & dates will need special formatted as explained above.

Conditions example that checks if the name is defined or not:
"name": <#if processDefinition.name??>"${processDefinition.name}"<#else>null</#if>,

List example that uses the "freemarked built in" of "_has_next" to decide if a comma sign shall be printed:
[
  <#list processDefinitions as processDefinition> 
  {
    "id": "${processDefinition.id}",
    "key": "${processDefinition.key}",
    "name": <#if processDefinition.name??>"${processDefinition.name}"<#else>null</#if>
  }<#if processDefinition_has_next>,</#if>
  </#list>
]

erikwinlof
Confirmed Champ
Confirmed Champ
I did a quick count of the amount of code needed to support and write in the 2 implementations (and note that the numbers first numbers below include the whole cycle rest api implementation for the freemarker version but not in the java version).

# Number of lines of code in freemarker version
%> find resources/webscripts/org/activiti/rest/api/ -type f -name "*ftl" -exec wc -l {} \; | awk '{total += $1} END{print total}'
411  (actually 278 without the cycle implementations 133 rows)

# Number of lines of code in java version
%> find java/org/activiti/rest/builder/ -type f -name "*java" -exec wc -l {} \; | awk '{total += $1} END{print total}'
1475

# Number of characters of code in freemarker version
%> find resources/webscripts/org/activiti/rest/api/ -type f -name "*ftl" -exec wc -c {} \; | awk '{total += $1} END{print total}'
12613 (actually 7998 minus the cycle implementations 4615 characters)

# Number of lines of code in java version
%> find java/org/activiti/rest/builder/ -type f -name "*java" -exec wc -c {} \; | awk '{total += $1} END{print total}'
49819

So as long as I have typed these commands properly, apologies otherwise :-), it meant that when excluding the cycle rest api the java implementation requires:
5.3 times more lines
and
6.2 times more characters
than the freemarker implementation… at its current state!

davidcognite
Star Contributor
Star Contributor
Once we are confident about the solution, the ftl's are removed and JSONObject stuff kicks in.
[…]
WDYT?

I'm still not confident that this is the best approach. I think the above demonstrates how hard the new REST API would be for non Java developers to follow.

The templating layer (in addition to all the other benefits) gives visibility over the expected results,
e.g.: The FTL files generally contain something along the lines of this:
{
  "id": "${processDefinition.id}",
  "key": "${processDefinition.key}",
  "name": <#if processDefinition.name??>"${processDefinition.name}"<#else>null</#if>,
  "version": ${processDefinition.version},
  "deploymentId": "${processDefinition.deploymentId}",
  "resourceName": "${processDefinition.resourceName}",
  "startFormResourceKey": <#if processDefinition.startFormResourceKey??>"${processDefinition.startFormResourceKey}"<#else>null</#if>
}

Having the response spelled out like that gives developers an easily accessible reference as to what properties they can expect, where they come from and which ones might be null. If any properties were formatted (e.g. dates or numbers) it would show what format should be expected.

How can front end Activiti developers or developers of third party applications (who may not be working in Java and may not have a Java IDE) determine the same information with your JSONObjectBuilder approach?

erikwinlof
Confirmed Champ
Confirmed Champ
Any thoughts from the rest? ( sorry about that 🙂
Personally I think we shall stick with the freemarker, since I can't see the "win" of using the JSONObject approach (except for getting the possible refactor errors when running compile rather than running test).

Cheers,

:: Erik

jbarrez
Star Contributor
Star Contributor
I wasnt able to follow the discussion every line, but here's my opinion on some of the points:

B. The amount of code lines needed are much greater, just consider the condition examples below to be written in java.
* LOC's don't mean anything. With auto-completion, auto-import, etc that every IDE has, I dont think you claim that LOC = productivity. We're not the eighties anymore.

C. The benefit of getting a toJSON method in all pojos and use them in the rest api is not realistic since the pojo model will never match the services methods in 100%, some examples:
- Task (resourceName isn't included anymore therefore needing a whole new class RestTask on some objects but not all, confusing an creates even more code to maintain)
- RestProcessDefinition (same story as line above)
These were just hacks introduced as a last-minute fix in the last release. There are jira issues to replace these with proper handling.

E. "modern" frameworks like rails, django use templates making it easier to use.
* 'modern' frameworks such as (G)Rails *do not* have templating as their primary rendering method as you state.
They use a much more powerful concept called builders, which comes closer to the Java approach: http://www.grails.org/doc/latest/guide/6.%20The%20Web%20Layer.html#6.1.7%20XML%20and%20JSON%20Respon....
In fact, in Rails you could use templating for json. But the more accepted practice is to call .to_json() on whatever you want to expose.

F. Freemarker is a robust framework with very good documentation: http://freemarker.sourceforge.net/docs/index.html
So is Java.

G. We don't use the webscripts framework in a way its not intended.
Why not? Who decides on that? For me, it is a natural addition to webscripts.