new rest2 webapp

[tombaeyens]

Huh, what is the rest2 webapp?

we're trying out a new rest style.  one that gives us better control.  we're not that happy with the indirect steps of the template rendering.  lot of things need to be right in order for it to work and we believe it could be done simpler.
so we're now trying out a simple servlet approach and building JSONObjects in the servlet that get streamed to the client.

the api should be fairly stable now.  rest api is not yet as there is already legacy in the original rest api.  so we want to combine 2 steps in one:
1) make the rest impl simpler
2) review the rest calls and sync them with the api as it is stable now.

but there is no final decision on this servlet strategy yet.   we're still in try-out mode.

[frederikherema1]

There are 2 REST approaches we consider instead of using ftl-templates:

1) Using the existing spring surf webscripts, but replace the freemarker-templates with rendering done by a JSONObject, based on the model created in the existing ActivitiWebScript implementation which use the API.

Pros:
• Leveraging the rest-posibilities built in into spring surf to map url’s to java code
• Since we populate JSONObjects based on activiti domain model, it’s more easy to see where they are used (a simple Cmd+G will show where data is used), instead of templates which contain only text-references to these properties.
• Using existing WebScript implementations with some tweaks here and there.
• JSONObject generates valid JSON and escapes accordingly.

Cons:
• Dependency on spring-surf just to use rest-functionality


2) Create a custom REST servlet which handles the rest calls by delegating to the appropriate handlers, using JSONObject to create valid JSON output and stream it back to client. Large part of existing Webscript java code can be reused which calls activiti API.

Pros:
• No dependency at all, just plain old servlet spec
• Same advantages mentioned in 1) thanks to JSONObject (escaping + using domain model)

Cons:
• Creating parsing/delegating mechanism from scratch, which can takes a couple of days of development
•
________________________________________________________________________________________

Another thing we want to tackle is splitting up the REST-api in the folowing parts:

1) UI-independent REST, containing calls that map right onto the activiti java API, eg: Get list of deployments.

2) A single rest api for each of the UI’s we have (explorer, cycle, probe,…), since some rest methods are created for those. Examples:

• Get tasks summary. Returns number of tasks for all groups the user belongs to, assigned to him/her and unnasigned. This call is absolutely needed ion explorer to prevent calling a bunch of seperate rest-methods that contain too much data anyway. This is a valid usecase from a task/user oriented point of view, but isn’t really generic engine api stuff.

• Cycle-related rest calls, which also use cycle-related services/data which isn’t nessesairy always deployed alongside the activiti-rest-webapp.

The webapps (explorer, probe, cycle) will expose extra REST-services (if needed) in their own war. The spring surf services used in the components will use 2 rest-endpoints: the core api-rest and their own custom REST-services.

Only downside of this is that if other apps want to use the specific REST-services exposed this way by UI’s is that the UI webapp should be deployed to make use of the included rest-services. Offcourse, we can create seperate war for the UI-secific REST (eg. activiti-wabapp-cycle-rest), but we want to avoid a million war’s being deployed

When the custom REST is included in the cycle/explorer/probe the 'dependency on spring-surf' stated in the 'cons' of the first solution proposed in 'replace freemarker templates' are a non-issue.

___________________________________________________________

We want this change to be implemented in 3-4 months so we can provide the same backward-compatibility/stability guarantees on core rest api, as we do on the engine API.

To facillitate this, unit-tests should be added to test the rest api to signal that any engine refactorings may have broken rest API behaviour.

Major changes to rest-api after it’s marked “stable” (now it’s still marked as experimental) will use a deprecation-strategy. Depending whether or not the (deprecated) old rest service calls still work in future versions, they can be removed after being marked as deprecated after a certain amount of time.


Quite some material to read through above, some good lecture for a rainy sunday afternoon Please share any remarks, thoughts, suggestions or other solutions!!

[falko_menge]

Maybe, Restlet is worth a try.

It's API is much more suitable for REST than the Servlet API and we already have the dependency in Activiti, as we are using it as a REST client in Cycle. In a past project, I made some good experiences with it's server capabilities and it could save us the effort of building an own parsing/delegating mechanism on top of the old Servlet API.

[frederikherema1]

Falko,

Restlet is also indeed a valid option. Since it supports using the JAX-RS annotations it's also future-proof for JEE6.
Apache CXF also implements this standard, I'll have a look at that as well.

thanks for the feedback, keep it coming

[nils1]

Hi guys,
since I've been working with spring-surf and freemarker templates for a while now, implementing the activiti-cycle webapp, I thought I'd quickly share my thoughts on this discussion:

1) Data transformation calls for a template engine
I agree that it would be nice to have IDE support for refactorings, i.e. warnings when you rename a class or remove an attribute. However, transforming data is cumbersome in java. Annotations can be a solution, but aren't always flexible enough. So you need to define some kind of mappings, whether these are XML files, freemarker templates like in spring-surf, or custom java code like i.e. in Restlet 2.0. Also, JSON isn't the only format we need to support, we also stream binary content or process plain HTML forms that are posted to the server. Which ever framework you choose, it should support these things in an easy way.

2) "Don't re-invent the wheel"
Spring-surf currently takes care of a whole bunch of things you didn't mention in this discussion so far. Apart from doing data transformation with freeemarker templates, it also handles parameters, dispatches the content-type of HTTP requests and chooses the content representation accordingly, it takes care of session handling, authentication and authorization and provides the basic HTML scaffold for the web applications, which are mainly written in JavaScript/ YUI. Re-implementing all this with the "plain old servlet spec" would mean to be writing a whole new framework.

3) "Innovation is seldom hindered by platform"
I agree that a thorough review of the existing REST API is a good idea and it is essential to make it stable and consistent. However, the existing infrastructure is not the reason this isn't the case yet. The REST API simply grew according to UI requirements. Now that we have a fairly stable java API and know more about what we need in the UIs, we can concentrate on making the existing REST API stable and consistent. Spring-surf certainly won't get in the way!

Cheers,
Nils

[frederikherema1]

Nils, thanks for sharing your thoughts.

You are right, there is more that meets the eye in the spring-surf REST running right now (authentication, dispatching, parameter parsing, …). Can I conclude that you would suggest to keep the spring-surf setup and try to use JSONObjects instead of the *.ftl to have a bit more safety, leaving all the other stuff (like streaming bytes instead of JSON) as is?

A quick note about your thought nr.3, the issue about reorganizing rest API methods to separate parts is indeed not related to the technology we use. Maybe I wasn't so clear in first my comment, but they are two separate issues.

[davidcognite]

My thoughts on changing the technology the REST API is based upon:

1) Separation of logic and presentation
As Nils says, there needs to be some layer between the application logic and the view the user gets - the question should be: Is FTL the best templating/markup language for the job?

2) Target audience
Who are we expecting to be doing API modifications, customisations and extensions? Are they Java developers? UI developers? Will the majority of them come from an Alfresco background (with pre-existing knowledge of Spring Surf)?

3) If it ain't broke…
As I understand it, from previous discussions with Tom, one of the main issues is that when bits of the core change, it can be tricky getting that bubbled up to the REST layer and into the UI - however as we're reach a release candidate, are things not going to become more stable, removing a lot of the need to be able to refactor the code easily? Minor changes will become easier with time as developers on both side understand the other better. Having just had the code freeze for RC1, is this the time for a major re-architecture?

4) Consistency
There's something to be said for using the same framework as the other webapps and keeping things consistant.

David.

[nils1]

@Frederik:

JSON objects might seem like a nice solution if you only want to transform your data to JSON. However, a REST API should be able to respond with different content-types, according to the request.

I don't know what the best solution is, but I don't think JSON objects are the solution. As David said, we'll need some layer between the application logic and the view. I can add that I found it easy to work with spring-surf and freemarker.

Maybe a separate REST module to evaluate whether an alternative technology yields better results isn't a bad idea. I would suggest to have a look at how Restlet 2.0 deals with data transformation.

Cheers,
Nils

[frederikherema1]

Nils, indeed. Actually, I meant using a custom "renderer" on top of the existing WebScripts' returned model hashmap, using JSONObjects. This way, any other format is still possible, only the actual rendering of JSON is done in java-code instead of freemarker which gives us the desired "safety".

This week I'll digg a bit deeper into the alternatives like the JAX-RS api (using restlet).

[bernd_ruecker]

Just a very quick opinion on this from me.

I don't think it is a good idea to make a big change in that area now, where we already have a lot of stuff and somehow understood how the current technology works. We already have to provide own stuff around myBatis, what e.g. Hibernate could do out of the box. I don't want to do the same in the are of REST. And I think it is still a good idea, that the web apps use the same technology then REST, less to know to understand the code. I agree with David, that changes in that API gets less frequent every day. IT's a bit annoying, yes. But it's the same on the DB level, where I have to change one attribute in several mapping SQL's and SQL scripts. But these 5 minutes here and there aren't worth the effort we need for such a refactoring I think

About splitting the REST-webapps: I don't see a big advantage here (unless we really want to aim for seperate installations of Cycle and Engine, which was blocked when I brought it up for database. Why for REST then?). And I see more Maven projects and especially much more complicated class loading. Currently just putting some stuff in the Tomcat lib directory already is a bit hacky in terms of classloading (and causes some trouble as for Cycle we would have to add more jars there, which we avoid by putting cycle in the REST/WEB-INF/Lib folder, where it truly belongs). The more web-apps, the more complicated class loading.

Cheers
Bernd