topic Re: I've implemented an extensible BPMN parser for Activiti in Alfresco Archive

I've implemented an extensible BPMN parser for Activiti

iravanchi — Mon, 27 Sep 2010 15:39:08 GMT

As a part of my project, I needed to extend BPMN parser to support my own activities in the process definition.I imported a few of the source codes of Activiti into my project and implemented the following solution:BpmnParser supports multiple xmlns definitions, and each element / attribute is deleg

Re: I've implemented an extensible BPMN parser for Activiti

iravanchi — Tue, 12 Oct 2010 12:25:04 GMT

I'm not getting any feedback on this in the forum, or in the email.
Would you let me know if I should follow-up on this or not?
Or let me know what to do about this?

I think if I'm ever going to merge my code with the actual code, the sooner is the better.
Because as the trunk changes, there's more changes to be merged.

-Hamed

Re: I've implemented an extensible BPMN parser for Activiti

farrukh_najmi — Thu, 14 Oct 2010 13:39:53 GMT

Hi Hamed,

As a newbie user my opinion may not be of much significance but… FWIW your proposed extensibility features seem to be very desirable from a end-user perspective.

I suspect the dev team is very busy to comment on your proposed idea in the abstract but may be more helpful if you did further due diligence. Here is what I suggest based on experience with other OS projects:

1. Send a signed contributors agreement (Just saw that you already did that):

http://docs.codehaus.org/pages/viewpage.action?pageId=163872771

2. File an RFE issue in ACT JIRA requesting the features you propose:

http://jira.codehaus.org/browse/ACT

3. Attach your proposed changes as a patch diff file to the issue and then post an update on this thread with link to the issue

This will hopefully help the dev team be able to understand what your proposal is all about and respond to it.

Re: I've implemented an extensible BPMN parser for Activiti

iravanchi — Thu, 14 Oct 2010 14:00:45 GMT

Hi,

Thanks for your comments.

1. Yes, I've sent the signed agreement (scanned version, using email). I'm not sure if I need to send a hard-copy by mail too. And Tom didn't tell me what to do next (I asked him if I need to do anything else)

2. There is already a JIRA issue for this: http://jira.codehaus.org/browse/ACT-178
I will add a comment in the issue now.

3. I've done my changes in the classes in my own project, not in a check-out of the Activiti code.
It will take me some time to do the changes on the Activiti code (for check-in / creating a patch) which I'm totally willing to do.
But since there's no response on this from the team, I'm thinking that creating a branch or replying to a post is much easier that reviewing and pallying a patch (which affects several files). So if they don't have time (or intention) for that, they won't look at my patch either. So why spend the time? If i have heard as much as "Send us a patch" sentence from them, I would have prepared and sent it two weeks ago.

-Hamed

Re: I've implemented an extensible BPMN parser for Activiti

tombaeyens — Thu, 14 Oct 2010 14:12:35 GMT

i'm processing my backlog after having been on vacation. please bear with us for a moment. sorry for the delay.

from a first incomplete glance, i would be concerned with 1) i don't think your proposal is valid bpmn. it's not allowed to define custom activities like that. 2) changing and extending the parser would take us a long time to discuss. then we need to think and agree on how we actually want to build in this extensibility. that's not a priority for us atm. the better we understand what you want to do, how and why the easier it is for us to accept a contribution. and also we need to see how this is generic enough so that it serves the community at large. hopefully i'll be able to process my backlog soon so that i can look deeper into this proposal

Re: I've implemented an extensible BPMN parser for Activiti

iravanchi — Thu, 14 Oct 2010 14:32:05 GMT

I know it's not valid BPMN, but it's the same as what you've done with "activiti:" attributes and elements.
Actually it is possible to exactly separate the Activiti extensions from the pure BPMN in the parser that I've done with simple re-arrangements.

Anyway, let me know what to do, if you're interested.

Re: I've implemented an extensible BPMN parser for Activiti

frederikherema1 — Thu, 14 Oct 2010 14:41:00 GMT

The activiti-extentions are actually valid BPMN, since the serviceTask's attributes may be extended (with for example activiti:type).

Also, it allows custom elements under the tag '<extensionElements>', like:

<serviceTask id="sendMail" activiti:type="mail">
  <extensionElements>
    <activiti:field activiti:name="from" activiti:string-value="order-shipping@thecompany.com" />
    <activiti:field activiti:name="to" activiti:string-value="${recipient}" />
    <activiti:field activiti:name="subject" activiti:string-value="Your order ${orderId} has been shipped" />
    <activiti:field activiti:name="html">
      <activiti:string>
        <![CDATA[
          <html>
            <body>
              Hello ${male ? 'Mr.' : 'Mrs.' } ${recipientName},<br/><br/>
                 
              As of ${now}, your order has been <b>processed and shipped</b>.<br/><br/>
                  
              Kind regards,<br/>
                  
              TheCompany.
            </body>
          </html>
        ]]>
      </activiti:string>
    </activiti:field>      
  </extensionElements>
</serviceTask>

Re: I've implemented an extensible BPMN parser for Activiti

farrukh_najmi — Thu, 14 Oct 2010 15:54:07 GMT

On a related note, as previously discussed on other threads the actviti parser currently supports a small subset of BPMN 2.0:

http://forums.activiti.org/en/viewtopic.php?f=4&t=298

One issue I am encountering is that it does not provide warnings when it encounters unsupportred types leaving users to stumble upon limitations (since they are not documented explicitly anywhere yet).

Also, I am wondering what are the latest plans for BpmnParser to use a set of data binding classes generated from JAXB or Eclipse BPMN2 EMF project. This would be really a great improvement to the robustness and spec coverage within the parser.

Re: I've implemented an extensible BPMN parser for Activiti

iravanchi — Thu, 14 Oct 2010 16:59:01 GMT

Let me re-phrase my original message:
I've refactored the BPMN parser inside the activiti that:
* Makes it easier to extend the BPMN coverage in future developments
* Makes it possible to add valid extensions (like Activiti extensions) without the need to modify the original parser code, and in a clean way
* Makes it easy to intercept / override PVM code generation from BPMN
* Makes it possible to incorporate existing BPMN parser for creating new custom languages

What I originally wanted to do was the last one (create my own lang based on BPMN) but I wasn't aware of the explicit extensibility specification in the BPMN spec. I should review the specs to see what is exactly considered valid and what is not valid in BPMN. Maybe I can use the valid extensions and don't need to extend it the way I did.

And I wasn't proposing that my way was the right way. And I also know that my code can be improved. I am expecting discussions and feedback so that I can do the right thing, considering the core developers' experience.

Also, I'm willing to spend time and develop in Activiti, since I already know the engine code to some extent. If the core developers assign tasks to me, I'd be happy to contribute my time.

About warnings on unsupported BPMN elements, I think the current Activiti parser produces warnings on the log output, but I'm not sure. But in my refactored code, each unsupported element inside the scopes, are reported as a WARNING on the log output.

Re: I've implemented an extensible BPMN parser for Activiti

farrukh_najmi — Thu, 14 Oct 2010 17:22:39 GMT

About warnings on unsupported BPMN elements, I think the current Activiti parser produces warnings on the log output, but I'm not sure. But in my refactored code, each unsupported element inside the scopes, are reported as a WARNING on the log output.

I just switched to latest trunk bits (5.0.rc1-SNAPSHOT) and I see a lot more warnings now. Thanks dev team!

One missing warning I can spot still is when there is a sendTask in a process. Since sendTask is unsupported in parser AFAIK, I think there should be a warning on its usage.

Re: I've implemented an extensible BPMN parser for Activiti

farrukh_najmi — Thu, 14 Oct 2010 18:18:42 GMT

from a first incomplete glance, i would be concerned with 1) i don't think your proposal is valid bpmn. it's not allowed to define custom activities like that.

+1

Looking at Hamed's BPMN example more carefully I now see that it is based upon invalid BPMN and does not extend BPMN using extensionElement as pointed by another post on this thread.

I think Hamed has the right idea (though details are off) that there is a need to provide a framework in the impl for adding handlers for custom extension elements. The same extension framework should be used internally by actviti dev team for actviti-defined extensions and exposed to activiti developers in user community to implement support for their own custom extensions.

The details of Hamed's proposed solution is not correct since it is not valid BPMN.

Tom, does the dev team already have a framework for handling custom extensions in mind? If not, is there support and encouragement from dev team
for Hamed and I to work out an extensibility proposal that is based on valid BPMN?

On a related note what are the plans for using JAXB or BPMN2 EMF project within the parser?

Re: I've implemented an extensible BPMN parser for Activiti

tombaeyens — Fri, 15 Oct 2010 07:02:18 GMT

We're don't think its a good idea to switch our BpmnParser to JAXB.
We can consider improvements to the current parser, but then we need to be sure that the solution covers also the tooling.

We have an idea in mind that looks like this

<serviceTask activiti:type="someTypeOrAClassName">
  <extensionElements>
    <field name="cfgParam1">stringValue</field>
    <field name="cfgParam2"><integer value="293" /></field>

In case short type names are used, then the BpmnParser (as part of the ProcessEngineConfiguration) should have a list of full class names that implement ActivityBehavior. Those classes should indicate their short type name with an annotation.

This way, the modeler and designer tools can dynamically add support for ActivityBehavior's that are plugged in by doing introspection on the ActivityBehavior class.

thats our current line of thinking.

Re: I've implemented an extensible BPMN parser for Activiti

iravanchi — Fri, 15 Oct 2010 08:33:23 GMT

Hi,

I agree with not using JAXB. The current parser, in my opinion, is both more flexible and more efficient, than if you'd used JAXB. (except the long list of if .. else if … else if … comparisons in the parser, which can be made more efficient using a lookup map)

I have a few notes, that won't clash with your current idea. I want to know what you think about it:

1. I think if <serviceTask> can have custom attributes, such as activiti:type, it should be able to have other custom attributes too, like "myOwnNs:type", through some extensibility.

2. If you agree with 1, then I think the best design would be the one that handles "activiti:" extensions the same way that it handles other extensions. You can even distinguish between pure-BPMN parsing, and BPMN with activiti extensions parsing.

3. If you provide some features in the engine, it doesn't necessarily mean that you have to support it in the designer. For example, you allow a list of "deployers" in the engine, but the designer only creates BPMN files. This way, a user can extend the engine and use it in his own way, in addition to what Activiti provides already.

4. Being able to re-use any part of the class library is a good feature to have. For example, I might want to use the BpmnDeployer (or Parser) to validate an XML that I get from my user, and to see if the engine supports it already, and gather error messages independently, without deploying the XML into the Activiti tables.

I'd be happy to hear what you think about the above.
Meanwhile, I will take a look at the BPMN specs one more time, to see exactly what it allows for extensions. I'll update my design accordingly, and post a couple of class / sequence diagrams for you to see what I have.

-Hamed

Re: I've implemented an extensible BPMN parser for Activiti

falko_menge — Fri, 15 Oct 2010 09:13:10 GMT

As BPMN's XML schema doesn't block Element Substitution, I was just about to argue that the custom Flow Nodes suggested by iravanchi could be made valid with the following XML schema:


<?xml version="1.0" encoding="UTF-8"?>
<schema
    xmlns="http://www.w3.org/2001/XMLSchema"
    targetNamespace="http://my/custom/namespace"
    xmlns:tns="http://my/custom/namespace"
    xmlns:bpmn="http://www.omg.org/spec/BPMN/20100524/MODEL"
    elementFormDefault="qualified">

    <!–
        This is a schema-valid BPMN extension, which is invalid with respect
        to the meta model as described in section "8.2.3 Extensibility" of the
        BPMN specification.
    –>

    <import namespace="http://www.omg.org/spec/BPMN/20100524/MODEL" />

    <!– <io:readFile id="readFromSource" sourceFile="D:\new.txt" targetVariable="fileContent" /> –>

    <element name="readFile" type="tns:tReadFile" substitutionGroup="bpmn:flowElement" />

    <complexType name="tReadFile">
        <complexContent>
            <extension base="bpmn:tTask">
                <attribute name="sourceFile" type="string" />
                <attribute name="targetVariable" type="string" />
            </extension>
        </complexContent>
    </complexType>

</schema>

However, after reading section "8.2.3 Extensibility" of the BPMN specification again, I learned that the BPMN Meta Model only allows custom XML elements that are contained in extensionElements elements. This means that the direct children of the process element can only be elements from the official BPMN XML schema.

This still leaves the question of how to deal with the quite frequent request for custom elements open.
In general, I believe that it is good to have a compact set of task categories that gives users hint about what they have to do and what the machines do for them. I think that the seven Task Types of BPMN 2.0 are broad enough to put any custom task into one of those seven categories. So why don't we restrict us to, e.g., Service Tasks with Extension Elements and be sure that our processes stay compatible with the majority of BPMN tools? We could still extend our particular tools to display our extended Service Task, e.g., with another icon or even completely different.

Re: I've implemented an extensible BPMN parser for Activiti

tombaeyens — Fri, 15 Oct 2010 11:46:28 GMT

@iravanchi did you see the ParseListeners already in the code? with that (undocumented for now) feature you can parse custom attributes

Re: I've implemented an extensible BPMN parser for Activiti

iravanchi — Fri, 15 Oct 2010 20:58:42 GMT

@falko
I've planned to read BPMN extensibility again tomorrow, but as you already did and you're fresh, does BPMN allow such <extensionElements> in all of the process children? For example, can I have extensions on flows and user tasks too?
During the previous version of our project (which used jBPM 3) the requirement to add extra custom information to such elements in the XML raised a few times, and I think I'll probably need the same stuff in BPMN too.
For example, I'm sure that I'd be adding additional event attributes for allowing processes to be triggered by my application's custom events.

@tom
Yes, I saw the ParseListeners, and I've actually extended it to have two methods (before and after). I don't remember why I added the "after" method, I'd check my code tomorrow at work and write back.
What I did (in brief) was to create some kind of ElementParseHandler, each of which can declare a set of element names they can handle (plus xmlns).
The parser then creates a Map of the handlers, and delegates parsing of each element to its handler.

-Hamed

Re: I've implemented an extensible BPMN parser for Activiti

farrukh_najmi — Mon, 18 Oct 2010 12:21:30 GMT

We're don't think its a good idea to switch our BpmnParser to JAXB.

I agree with not using JAXB. The current parser, in my opinion, is both more flexible and more efficient, than if you'd used JAXB. (except the long list of if .. else if … else if … comparisons in the parser, which can be made more efficient using a lookup map)

I do not wish to beat a dead horse but only interested in learning the rationale behind not wanting to use jAXB.

I realize JAXB is not as efficient as raw XML processing. What other considerations make you both feel that its not a good fit for parsing BPMN2?
Again I only wish to understand your rationale and am not trying to make a case for JAXB. Thanks.

Re: I've implemented an extensible BPMN parser for Activiti

iravanchi — Mon, 18 Oct 2010 13:28:38 GMT

What other considerations make you both feel that its not a good fit for parsing BPMN2?

Maybe if I wanted to write the BPMN Parser from scratch, I might have used JAXB. Specially provided that we have an XSD ready for generating JAXB classes. So, I'm not saying that JAXB is not a good fit. Anyway, here's why I felt I'm not comfortable with JAXB in here:
(Note that none of these are concrete reasons, just generally the way I feel about the issue)

1. BpmnParser does two things. One is parsing the XML, and one is generating PVM objects based on what is read from XML. So if we use JAXB, most of the current code will remain, not replaced.

2. Using JAXB means strict conformance of the XML to the object model. In the current implementation, it's rather easier to handle errors in custom ways.

3. It's easier to change / extend the current parser, than if the parser have used JAXB. For example, in JAXB code you need to specify possible sub-classes of a super class user @XmlSeeAlso. So if you want the parser to handle a new element, you'll either have to change the library code, or re-implement the whole JAXB class hierarchy that is bound to BPMN. In the current parser, it's a matter of comparing strings.

All of the above items can be considered a positive point for JAXB too, depending on your point of view.
Even for the performance, since there is many string comparisons and string lookups in the current code, we can't be sure that using JAXB means lower performance.

I think the concrete reason for not changing it would be: there's no (special) gain for the user of Activiti to do so.
From your point of view, as a user of Activiti, what can be better if Activiti uses JAXB or not internally?

Re: I've implemented an extensible BPMN parser for Activiti

farrukh_najmi — Mon, 18 Oct 2010 14:05:50 GMT

What other considerations make you both feel that its not a good fit for parsing BPMN2?

Maybe if I wanted to write the BPMN Parser from scratch, I might have used JAXB. Specially provided that we have an XSD ready for generating JAXB classes. So, I'm not saying that JAXB is not a good fit. Anyway, here's why I felt I'm not comfortable with JAXB in here:
(Note that none of these are concrete reasons, just generally the way I feel about the issue)

1. BpmnParser does two things. One is parsing the XML, and one is generating PVM objects based on what is read from XML. So if we use JAXB, most of the current code will remain, not replaced.

2. Using JAXB means strict conformance of the XML to the object model. In the current implementation, it's rather easier to handle errors in custom ways.

3. It's easier to change / extend the current parser, than if the parser have used JAXB. For example, in JAXB code you need to specify possible sub-classes of a super class user @XmlSeeAlso. So if you want the parser to handle a new element, you'll either have to change the library code, or re-implement the whole JAXB class hierarchy that is bound to BPMN. In the current parser, it's a matter of comparing strings.

All of the above items can be considered a positive point for JAXB too, depending on your point of view.
Even for the performance, since there is many string comparisons and string lookups in the current code, we can't be sure that using JAXB means lower performance.

I think the concrete reason for not changing it would be: there's no (special) gain for the user of Activiti to do so.
From your point of view, as a user of Activiti, what can be better if Activiti uses JAXB or not internally?

Thanks for sharing your thoughts Ahmed.

The main benefit of using JAXB that I see is that it will likely encourage a tighter support for BPMN2 syntax in the parser and will be much more robust and easier to maintain in the long term. Perhaps that is not compelling enough benefit in the short term.

BTW, handling of extension elements can be done in JAXB RI via separate compilation of the extension schema and using the generated bindings of extension schema with the core schema:

http://weblogs.java.net/blog/2006/09/05/separate-compilation-jaxb-ri-21

Re: I've implemented an extensible BPMN parser for Activiti

iravanchi — Mon, 18 Oct 2010 18:36:47 GMT

Yes, I agree with you about the long-term maintainability, and cleaner code.
Maybe Tom has some more concrete reasons for why he didn't think JAXB is a good choice.

And, BTW, I'm NOT "Ahmed". I'm "Hamed"

-Hamed