v23.1 - Notifications don't seem to work if they use properties where a value is not present

[MVannLeicester]

Problem

If a formatted notification uses a property for any part of it - recipient, subject, body, etc. - they don't get sent by the system and just error.  The file remains in the cache directory. 

Here's an example error that gets logged to Event Viewer when this happens:

[02/01/2024 15:23:27 -06:00 Error] An error occurred while loading the request with ID '60259'. Message: 'Unexpected character encountered while parsing value: Ā. Path '', line 0, position 0.'.

However, if I manually run the Unity Scheduler service using an EP3 application server, it sends.

I've eliminated everything except the one thing in common which is that the affected notifications depend on properties that should be getting set in Workflow.  In a couple of situations the value doesn't get set, and that's being addressed upstream.

Regular text notifications appear to be unaffected. It's only formatted ones.

EDIT: More context.  For some reason the system seems to think a recipient isn't there when it is.  Here's the Diagnostics:

Workflow Exception Report: Inner Exceptions 'True' Stack Trace 'True'Exception Level 0: Type 'Hyland.Workflow.TaskExecution.TaskExecutionException' Message 'Task list execution failed: Exception Type Hyland.Core.Distribution.NoRecipientsException: Cannot enqueue the email since the recipient list is empty.Action: [2100] Send notification: document received (Send Notification)'Exception Level 0: Stack Trace   at Hyland.Workflow.Core.TaskExecution.TaskExecutor.HandleExecuteOrResume(SuspendedTask resumeTask)   at Hyland.Workflow.Core.TaskExecution.TaskExecutor.Execute()   at Hyland.Workflow.Core.Scheduler.SchedulerTimer.ProcessItem(ISession session, IQueue queue, IWorkItem workitem, TaskExecutorSettings executorSettings, Int64 itemsprocessed, Boolean isfirst, Boolean islast)Exception Level 1: Type 'Hyland.Workflow.Tasks.TaskException' Message 'Exception Type Hyland.Core.Distribution.NoRecipientsException: Cannot enqueue the email since the recipient list is empty.Action: [2100] Send notification: document received (Send Notification)'Exception Level 1: Stack Trace   at Hyland.Workflow.Core.ActionAndRuleBase`1.HandleTaskExecutionException(IWorkflowTrace trace, Exception e)   at Hyland.Workflow.Core.ActionAndRuleBase`1.Execute(ISession session, IWorkItem item, ITaskExecutionArgs executionArgs)   at Hyland.Workflow.Core.TaskExecution.ActionExecutor.ExecuteOrResume(IAction action, ActionState actionState, SuspendedTask resumeTask, String resumeTaskState, ITaskExecutionArgs executionArgs)   at Hyland.Workflow.Core.TaskExecution.ActionExecutor.Execute(IAction action, TaskExecutorState executorState, ITaskExecutionArgs executionArgs)   at Hyland.Workflow.Core.TaskExecution.TaskListExecutor.ExecuteOrResumeImpl(ITaskList taskList, TaskListState taskListState, SuspendedTask resumeTask, ITaskExecutionArgs executionArgs)   at Hyland.Workflow.Core.TaskExecution.TaskListExecutor.InternalExecute(ITaskList taskList, TaskExecutorState executorState, ITaskExecutionArgs executionArgs)   at Hyland.Workflo…

However, in distributionqueue there are clearly recipients, and if I manually release the messages using EP3, they send fine (prefixes edited out for privacy):

The logic that eventually sends this notification is inefficient and will be rebuilt at some point here soon because it's doing a lot of unnecessary work.  But I'm still not sure why the EP3 application server can send it fine and the 23.1 application server cannot.

Question

Is there any known issue regarding Notifications requiring all properties to exist/have a value before sending?  I don't know if this was an issue in EP4, 5 or v22.

This is specifically 23.1.3.  I haven't yet tried 23.1.2.

[Edward_Hudson · Accepted solution]

Thanks for the reply, I have a feeling this might be related as well:

https://community.hyland.com/forum/threads/82474-parallel-upgrade-distribution-services-not-sending-...

[Edward_Hudson]

Did you ever find an resolution? Currently getting the same error with 23.1.3.

[MVannLeicester]

Negative.  Can only confirm 100% that 

- It's only Formatted messages

- Where a property didn't get a value for whatever reason and that property is part of the message body or subject

- EP3 application server sends fine (but then can't send other types of notifications)

Problem is the distribution queue is a single table - so running a different service won't work and there's no way to assign those notifications to a specific service/app server.  Which means I can only do it manually through Unity Management Console for now until that logic is rebuilt to either force the property or trap when it's missing.  Which is on deck anyway but there are a lot of notifications impacted. 

[Edward_Hudson · Accepted solution]

Thanks for the reply, I have a feeling this might be related as well:

https://community.hyland.com/forum/threads/82474-parallel-upgrade-distribution-services-not-sending-...

[MVannLeicester]

Definitely the same symptoms.  So when they "fixed" whatever from before, it seems like something got broken.

IN my case the Unity Scheduler version matches the application server.  The users are on that same application server.  I don't think it's the application server; I think it's something to do with the published version of the notification, because the Formatted notification gets published via the application server; and these notifications were published in EP3 and not republished since.

I'll test republishing one of the most heavily used notifications to see if it makes a difference.