Problem with job table

[houssam]

Hello,

Am facing a problem with activiti that prevents us to run properly the whole app.
I am having a process that is meant to send 4 emails to each user. We use a timer to set up the delay in between 2 emails.
The first shot ( process launching) is done properly ( the 400 users receive their emails)
While runing the second shot activiti stops and blocks the whole app.
I have tried to solve the issue by:
*Restarting the application (still with no success), unless i delete act_ru_job
*Canceling email sending (still with no success)

Ps: Please note that emails are sent by an asynchronous java service.

Any suggestion?

[trademak]

I'm a bit lost. You have 3 posts and in one of these posts you say you don't have asynchronous tasks. In this post you say you do have an asynchronous service task. What do you mean with blocks the whole app? Did an error occur when executing the service task? Could you share your process definition?

Best regards,

[houssam]

Sorry Tijs it may looks confusing but here are answears tothe questions:
*No asynchronous tasks managed by activiti
*It blocks because all my requests to server are blocked on pending status , which prevent the users to access.
*I don't see any error in logs:
*Here bellow process definition:
<?xml version="1.0" encoding="UTF-8"?>
<definitions xmlns="http://www.omg.org/spec/BPMN/20100524/MODEL" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:activiti="http://activiti.org/bpmn" xmlns:bpmndi="http://www.omg.org/spec/BPMN/20100524/DI" xmlnsmgdc="http://www.omg.org/spec/DD/20100524/DC" xmlnsmgdi="http://www.omg.org/spec/DD/20100524/DI" typeLanguage="http://www.w3.org/2001/XMLSchema" expressionLanguage="http://www.w3.org/1999/XPath" targetNamespace="http://www.activiti.org/test">
    <process id="phoning" name="My process" isExecutable="true">
        <startEvent id="startevent1" name="Start"></startEvent>
        <serviceTask id="servicetask1" name="email confirmation" activiti:expression="#{phoningProcessService.sendEmailConfirmation(token, userId)}"></serviceTask>
        <parallelGateway id="parallelgateway1" name="Parallel Gateway"></parallelGateway>
        <userTask id="confirmation" name="confirmation" activiti:assignee="${token}"></userTask>
        <serviceTask id="emailWelcome" name="email de bienvenue" activiti:expression="#{phoningProcessService.sendEmailWelcome(userId)}"></serviceTask>
        <serviceTask id="accountActivation" name="activation compte" activiti:expression="#{sharedAnnouncementService.validationAccountAndAnnouncement(userId)}"></serviceTask>
        <parallelGateway id="parallelgateway2" name="Parallel Gateway"></parallelGateway>
        <parallelGateway id="parallelgateway3" name="Parallel Gateway"></parallelGateway>
        <endEvent id="terminateendevent1" name="TerminateEndEvent"></endEvent>
        <exclusiveGateway id="exclusivegateway1" name="Exclusive Gateway"></exclusiveGateway>
        <serviceTask id="getNumber" name="nombre relance" activiti:expression="#{phoningProcessService.getNombreRelance(nombreRelance)}" activiti:resultVariableName="nombreRelance"></serviceTask>
        <exclusiveGateway id="exclusivegateway2" name="Exclusive Gateway"></exclusiveGateway>
        <intermediateCatchEvent id="timerintermediatecatchevent1" name="TimerCatchEvent">
            <timerEventDefinition>
                <timeDuration>PT5M</timeDuration>
            </timerEventDefinition>
        </intermediateCatchEvent>
        <intermediateCatchEvent id="timerintermediatecatchevent2" name="TimerCatchEvent">
            <timerEventDefinition>
                <timeDuration>PT5M</timeDuration>
            </timerEventDefinition>
        </intermediateCatchEvent>
        <exclusiveGateway id="exclusivegateway3" name="Exclusive Gateway"></exclusiveGateway>
        <serviceTask id="servicetask2" name="email relance" activiti:expression="#{phoningProcessService.sendEmailRelance(token, userId)}"></serviceTask>
        <exclusiveGateway id="exclusivegateway4" name="Exclusive Gateway"></exclusiveGateway>
        <serviceTask id="servicetask3" name="annulation du compte" activiti:expression="#{phoningProcessService.annulationAccount(userId)}"></serviceTask>
        <endEvent id="terminateendevent2" name="TerminateEndEvent"></endEvent>
        <sequenceFlow id="flow1" name="nombreRelance&gt;3" sourceRef="exclusivegateway2" targetRef="timerintermediatecatchevent2">
            <conditionExpression xsi:type="tFormalExpression"><![CDATA[${nombreRelance>3}]]></conditionExpression>
        </sequenceFlow>
        <sequenceFlow id="flow2" sourceRef="timerintermediatecatchevent2" targetRef="servicetask3"></sequenceFlow>
        <sequenceFlow id="flow3" sourceRef="servicetask3" targetRef="terminateendevent2"></sequenceFlow>
        <sequenceFlow id="flow4" name="nombreRelance&lt;=3" sourceRef="exclusivegateway2" targetRef="timerintermediatecatchevent1">
            <conditionExpression xsi:type="tFormalExpression"><![CDATA[${nombreRelance<=3}]]></conditionExpression>
        </sequenceFlow>
        <sequenceFlow id="flow5" sourceRef="timerintermediatecatchevent1" targetRef="exclusivegateway3"></sequenceFlow>
        <sequenceFlow id="flow6" name="nombreRelance&lt;3" sourceRef="exclusivegateway3" targetRef="servicetask2">
            <conditionExpression xsi:type="tFormalExpression"><![CDATA[${nombreRelance<3}]]></conditionExpression>
        </sequenceFlow>
        <userTask id="phoneUser" name="CC:Envoi 3eme mail" activiti:assignee="moderatorPhone"></userTask>
        <serviceTask id="servicetask4" name="relance annulation" activiti:expression="#{phoningProcessService.sendEmailAnnulation(token, userId)}"></serviceTask>
        <sequenceFlow id="flow7" name="nombreRelance==3" sourceRef="exclusivegateway3" targetRef="phoneUser">
            <conditionExpression xsi:type="tFormalExpression"><![CDATA[${nombreRelance==3}]]></conditionExpression>
        </sequenceFlow>
        <sequenceFlow id="flow8" sourceRef="phoneUser" targetRef="servicetask4"></sequenceFlow>
        <sequenceFlow id="flow9" sourceRef="servicetask2" targetRef="exclusivegateway4"></sequenceFlow>
        <sequenceFlow id="flow10" sourceRef="servicetask4" targetRef="exclusivegateway4"></sequenceFlow>
        <sequenceFlow id="flow16" sourceRef="getNumber" targetRef="exclusivegateway2"></sequenceFlow>
        <sequenceFlow id="flow20" sourceRef="parallelgateway2" targetRef="accountActivation"></sequenceFlow>
        <sequenceFlow id="flow21" sourceRef="accountActivation" targetRef="parallelgateway3"></sequenceFlow>
        <sequenceFlow id="flow22" sourceRef="parallelgateway2" targetRef="emailWelcome"></sequenceFlow>
        <sequenceFlow id="flow23" sourceRef="emailWelcome" targetRef="parallelgateway3"></sequenceFlow>
        <sequenceFlow id="flow24" sourceRef="parallelgateway3" targetRef="terminateendevent1"></sequenceFlow>
        <sequenceFlow id="flow25" sourceRef="startevent1" targetRef="servicetask1"></sequenceFlow>
        <sequenceFlow id="flow12" sourceRef="servicetask1" targetRef="parallelgateway1"></sequenceFlow>
        <sequenceFlow id="flow14" sourceRef="parallelgateway1" targetRef="confirmation"></sequenceFlow>
        <sequenceFlow id="flow13" sourceRef="parallelgateway1" targetRef="exclusivegateway1"></sequenceFlow>
        <sequenceFlow id="flow15" sourceRef="exclusivegateway1" targetRef="getNumber"></sequenceFlow>
        <sequenceFlow id="flow11" sourceRef="exclusivegateway4" targetRef="exclusivegateway1"></sequenceFlow>
        <sequenceFlow id="flow26" sourceRef="confirmation" targetRef="parallelgateway2"></sequenceFlow>
    </process>
    <bpmndi:BPMNDiagram id="BPMNDiagram_phoning">
        <bpmndi:BPMNPlane bpmnElement="phoning" id="BPMNPlane_phoning">
            <bpmndi:BPMNShape bpmnElement="startevent1" id="BPMNShape_startevent1">
                <omgdc:Bounds height="30.0" width="30.0" x="40.0" y="283.0"></omgdc:Bounds>
            </bpmndi:BPMNShape>
            <bpmndi:BPMNShape bpmnElement="servicetask1" id="BPMNShape_servicetask1">
                <omgdc:Bounds height="55.0" width="105.0" x="130.0" y="273.0"></omgdc:Bounds>
            </bpmndi:BPMNShape>
            <bpmndi:BPMNShape bpmnElement="parallelgateway1" id="BPMNShape_parallelgateway1">
                <omgdc:Bounds height="40.0" width="40.0" x="290.0" y="280.0"></omgdc:Bounds>
            </bpmndi:BPMNShape>
            <bpmndi:BPMNShape bpmnElement="confirmation" id="BPMNShape_confirmation">
                <omgdc:Bounds height="55.0" width="105.0" x="370.0" y="573.0"></omgdc:Bounds>
            </bpmndi:BPMNShape>
            <bpmndi:BPMNShape bpmnElement="emailWelcome" id="BPMNShape_emailWelcome">
                <omgdc:Bounds height="55.0" width="105.0" x="760.0" y="529.0"></omgdc:Bounds>
            </bpmndi:BPMNShape>
            <bpmndi:BPMNShape bpmnElement="accountActivation" id="BPMNShape_accountActivation">
                <omgdc:Bounds height="55.0" width="105.0" x="760.0" y="619.0"></omgdc:Bounds>
            </bpmndi:BPMNShape>
            <bpmndi:BPMNShape bpmnElement="parallelgateway2" id="BPMNShape_parallelgateway2">
                <omgdc:Bounds height="40.0" width="40.0" x="680.0" y="580.0"></omgdc:Bounds>
            </bpmndi:BPMNShape>
            <bpmndi:BPMNShape bpmnElement="parallelgateway3" id="BPMNShape_parallelgateway3">
                <omgdc:Bounds height="40.0" width="40.0" x="900.0" y="575.0"></omgdc:Bounds>
            </bpmndi:BPMNShape>
            <bpmndi:BPMNShape bpmnElement="terminateendevent1" id="BPMNShape_terminateendevent1">
                <omgdc:Bounds height="28.0" width="28.0" x="1002.0" y="578.0"></omgdc:Bounds>
            </bpmndi:BPMNShape>
            <bpmndi:BPMNShape bpmnElement="exclusivegateway1" id="BPMNShape_exclusivegateway1">
                <omgdc:Bounds height="40.0" width="40.0" x="350.0" y="190.0"></omgdc:Bounds>
            </bpmndi:BPMNShape>
            <bpmndi:BPMNShape bpmnElement="getNumber" id="BPMNShape_getNumber">
                <omgdc:Bounds height="55.0" width="105.0" x="460.0" y="183.0"></omgdc:Bounds>
            </bpmndi:BPMNShape>
            <bpmndi:BPMNShape bpmnElement="exclusivegateway2" id="BPMNShape_exclusivegateway2">
                <omgdc:Bounds height="40.0" width="40.0" x="600.0" y="190.0"></omgdc:Bounds>
            </bpmndi:BPMNShape>
            <bpmndi:BPMNShape bpmnElement="timerintermediatecatchevent1" id="BPMNShape_timerintermediatecatchevent1">
                <omgdc:Bounds height="30.0" width="30.0" x="685.0" y="153.0"></omgdc:Bounds>
            </bpmndi:BPMNShape>
            <bpmndi:BPMNShape bpmnElement="timerintermediatecatchevent2" id="BPMNShape_timerintermediatecatchevent2">
                <omgdc:Bounds height="30.0" width="30.0" x="605.0" y="400.0"></omgdc:Bounds>
            </bpmndi:BPMNShape>
            <bpmndi:BPMNShape bpmnElement="exclusivegateway3" id="BPMNShape_exclusivegateway3">
                <omgdc:Bounds height="40.0" width="40.0" x="760.0" y="150.0"></omgdc:Bounds>
            </bpmndi:BPMNShape>
            <bpmndi:BPMNShape bpmnElement="servicetask2" id="BPMNShape_servicetask2">
                <omgdc:Bounds height="55.0" width="105.0" x="913.0" y="90.0"></omgdc:Bounds>
            </bpmndi:BPMNShape>
            <bpmndi:BPMNShape bpmnElement="exclusivegateway4" id="BPMNShape_exclusivegateway4">
                <omgdc:Bounds height="40.0" width="40.0" x="945.5" y="15.0"></omgdc:Bounds>
            </bpmndi:BPMNShape>
            <bpmndi:BPMNShape bpmnElement="servicetask3" id="BPMNShape_servicetask3">
                <omgdc:Bounds height="55.0" width="105.0" x="707.0" y="387.5"></omgdc:Bounds>
            </bpmndi:BPMNShape>
            <bpmndi:BPMNShape bpmnElement="terminateendevent2" id="BPMNShape_terminateendevent2">
                <omgdc:Bounds height="28.0" width="28.0" x="871.0" y="397.5"></omgdc:Bounds>
            </bpmndi:BPMNShape>
            <bpmndi:BPMNShape bpmnElement="phoneUser" id="BPMNShape_phoneUser">
                <omgdc:Bounds height="55.0" width="105.0" x="885.0" y="183.0"></omgdc:Bounds>
            </bpmndi:BPMNShape>
            <bpmndi:BPMNShape bpmnElement="servicetask4" id="BPMNShape_servicetask4">
                <omgdc:Bounds height="55.0" width="105.0" x="1071.5" y="183.0"></omgdc:Bounds>
            </bpmndi:BPMNShape>
            <bpmndi:BPMNEdge bpmnElement="flow12" id="BPMNEdge_flow12">
                <omgdi:waypoint x="235.0" y="300.5"></omgdi:waypoint>
                <omgdi:waypoint x="290.5" y="300.5"></omgdi:waypoint>
            </bpmndi:BPMNEdge>
            <bpmndi:BPMNEdge bpmnElement="flow21" id="BPMNEdge_flow21">
                <omgdi:waypoint x="865.0" y="646.5"></omgdi:waypoint>
                <omgdi:waypoint x="919.0" y="646.5"></omgdi:waypoint>
                <omgdi:waypoint x="919.6190476190476" y="614.6190476190476"></omgdi:waypoint>
            </bpmndi:BPMNEdge>
            <bpmndi:BPMNEdge bpmnElement="flow11" id="BPMNEdge_flow11">
                <omgdi:waypoint x="945.6665278934222" y="34.83347210657785"></omgdi:waypoint>
                <omgdi:waypoint x="370.0" y="30.0"></omgdi:waypoint>
                <omgdi:waypoint x="370.0" y="190.0"></omgdi:waypoint>
            </bpmndi:BPMNEdge>
            <bpmndi:BPMNEdge bpmnElement="flow20" id="BPMNEdge_flow20">
                <omgdi:waypoint x="699.578947368421" y="619.578947368421"></omgdi:waypoint>
                <omgdi:waypoint x="699.0" y="646.5"></omgdi:waypoint>
                <omgdi:waypoint x="760.0" y="646.5"></omgdi:waypoint>
            </bpmndi:BPMNEdge>
            <bpmndi:BPMNEdge bpmnElement="flow23" id="BPMNEdge_flow23">
                <omgdi:waypoint x="865.0" y="556.2535211267606"></omgdi:waypoint>
                <omgdi:waypoint x="919.0" y="556.0"></omgdi:waypoint>
                <omgdi:waypoint x="919.5" y="575.5"></omgdi:waypoint>
            </bpmndi:BPMNEdge>
            <bpmndi:BPMNEdge bpmnElement="flow10" id="BPMNEdge_flow10">
                <omgdi:waypoint x="1124.0" y="183.0"></omgdi:waypoint>
                <omgdi:waypoint x="1124.0" y="35.0"></omgdi:waypoint>
                <omgdi:waypoint x="985.5" y="35.0"></omgdi:waypoint>
            </bpmndi:BPMNEdge>
            <bpmndi:BPMNEdge bpmnElement="flow22" id="BPMNEdge_flow22">
                <omgdi:waypoint x="700.0" y="580.0"></omgdi:waypoint>
                <omgdi:waypoint x="700.0" y="556.0"></omgdi:waypoint>
                <omgdi:waypoint x="760.0" y="556.2666666666667"></omgdi:waypoint>
            </bpmndi:BPMNEdge>
            <bpmndi:BPMNEdge bpmnElement="flow3" id="BPMNEdge_flow3">
                <omgdi:waypoint x="812.0" y="413.5358565737052"></omgdi:waypoint>
                <omgdi:waypoint x="871.005441184258" y="411.8902865008374"></omgdi:waypoint>
            </bpmndi:BPMNEdge>
            <bpmndi:BPMNEdge bpmnElement="flow2" id="BPMNEdge_flow2">
                <omgdi:waypoint x="635.0" y="415.0"></omgdi:waypoint>
                <omgdi:waypoint x="707.0" y="415.0"></omgdi:waypoint>
            </bpmndi:BPMNEdge>
            <bpmndi:BPMNEdge bpmnElement="flow5" id="BPMNEdge_flow5">
                <omgdi:waypoint x="714.9953146961218" y="168.37488286740304"></omgdi:waypoint>
                <omgdi:waypoint x="760.4878048780488" y="169.5121951219512"></omgdi:waypoint>
            </bpmndi:BPMNEdge>
            <bpmndi:BPMNEdge bpmnElement="flow4" id="BPMNEdge_flow4">
                <omgdi:waypoint x="620.0" y="190.0"></omgdi:waypoint>
                <omgdi:waypoint x="620.0" y="168.0"></omgdi:waypoint>
                <omgdi:waypoint x="685.0" y="168.0"></omgdi:waypoint>
            </bpmndi:BPMNEdge>
            <bpmndi:BPMNEdge bpmnElement="flow1" id="BPMNEdge_flow1">
                <omgdi:waypoint x="620.0" y="230.0"></omgdi:waypoint>
                <omgdi:waypoint x="620.0" y="400.0"></omgdi:waypoint>
            </bpmndi:BPMNEdge>
            <bpmndi:BPMNEdge bpmnElement="flow15" id="BPMNEdge_flow15">
                <omgdi:waypoint x="389.9300699300699" y="210.06993006993008"></omgdi:waypoint>
                <omgdi:waypoint x="460.0" y="210.31578947368422"></omgdi:waypoint>
            </bpmndi:BPMNEdge>
            <bpmndi:BPMNEdge bpmnElement="flow24" id="BPMNEdge_flow24">
                <omgdi:waypoint x="939.3939393939394" y="594.3939393939394"></omgdi:waypoint>
                <omgdi:waypoint x="1002.0068309347809" y="592.4372865332881"></omgdi:waypoint>
            </bpmndi:BPMNEdge>
            <bpmndi:BPMNEdge bpmnElement="flow7" id="BPMNEdge_flow7">
                <omgdi:waypoint x="780.0" y="190.0"></omgdi:waypoint>
                <omgdi:waypoint x="780.0" y="210.5"></omgdi:waypoint>
                <omgdi:waypoint x="885.0" y="210.5"></omgdi:waypoint>
            </bpmndi:BPMNEdge>
            <bpmndi:BPMNEdge bpmnElement="flow25" id="BPMNEdge_flow25">
                <omgdi:waypoint x="81.88242814331352" y="300.5"></omgdi:waypoint>
                <omgdi:waypoint x="130.0" y="300.5"></omgdi:waypoint>
            </bpmndi:BPMNEdge>
            <bpmndi:BPMNEdge bpmnElement="flow6" id="BPMNEdge_flow6">
                <omgdi:waypoint x="780.0" y="150.0"></omgdi:waypoint>
                <omgdi:waypoint x="780.0" y="117.0"></omgdi:waypoint>
                <omgdi:waypoint x="913.0" y="117.35849056603774"></omgdi:waypoint>
            </bpmndi:BPMNEdge>
            <bpmndi:BPMNEdge bpmnElement="flow16" id="BPMNEdge_flow16">
                <omgdi:waypoint x="565.0" y="210.25581395348837"></omgdi:waypoint>
                <omgdi:waypoint x="600.0925925925926" y="210.09259259259258"></omgdi:waypoint>
            </bpmndi:BPMNEdge>
            <bpmndi:BPMNEdge bpmnElement="flow26" id="BPMNEdge_flow26">
                <omgdi:waypoint x="475.0" y="600.5"></omgdi:waypoint>
                <omgdi:waypoint x="680.5" y="600.5"></omgdi:waypoint>
            </bpmndi:BPMNEdge>
            <bpmndi:BPMNEdge bpmnElement="flow13" id="BPMNEdge_flow13">
                <omgdi:waypoint x="310.0" y="280.0"></omgdi:waypoint>
                <omgdi:waypoint x="310.0" y="210.0"></omgdi:waypoint>
                <omgdi:waypoint x="350.0" y="210.0"></omgdi:waypoint>
            </bpmndi:BPMNEdge>
            <bpmndi:BPMNEdge bpmnElement="flow9" id="BPMNEdge_flow9">
                <omgdi:waypoint x="965.5" y="90.0"></omgdi:waypoint>
                <omgdi:waypoint x="965.5" y="55.0"></omgdi:waypoint>
            </bpmndi:BPMNEdge>
            <bpmndi:BPMNEdge bpmnElement="flow14" id="BPMNEdge_flow14">
                <omgdi:waypoint x="310.0" y="320.0"></omgdi:waypoint>
                <omgdi:waypoint x="310.0" y="600.5"></omgdi:waypoint>
                <omgdi:waypoint x="370.0" y="600.5"></omgdi:waypoint>
            </bpmndi:BPMNEdge>
            <bpmndi:BPMNEdge bpmnElement="flow8" id="BPMNEdge_flow8">
                <omgdi:waypoint x="990.0" y="210.5"></omgdi:waypoint>
                <omgdi:waypoint x="1071.5" y="210.5"></omgdi:waypoint>
            </bpmndi:BPMNEdge>
        </bpmndi:BPMNPlane>
    </bpmndi:BPMNDiagram>
</definitions>

Best regards,

Houssam

[frederikherema1]

In case you do calls to the engine, which cause an "asynchronous" task to be executed, the job-executor is used. In case the job-executor is quite busy, it can be that all threads in his/her pool are occupied. If this is the case, by default, the CALLING thread will be used to execute asynchronous activity. This will be the job-scheduling thread, effectively blocking any additional jobs from executing. This behaviour can be changed (see org.activiti.engine.impl.jobexecutor.RejectedJobsHandler and org.activiti.engine.impl.jobexecutor.CallerRunsRejectedJobsHandler).

I'm suspecting (judging from the numbers you're talking about) your job-executor pool is too small. Since the job-scheudling thread will be running a job, any new jobs added (by calls) will possibly be delayed (although not 100% certain about this, have been some changes in 5.14 in this area), explaining why the request are pending.

Consider sending emails in large batches (when you expect large quantities) instead of using one job for one email. In general, it's a good idea to use another system/service for sending emails and let the activities queue an email. So the external service can do the retry-ing and the async handling…