Using Concurrency and Parallelism -- will variables within a For Each have a separate instance for each parallel thread?

[MelissaReed]

I understand that turning on concurrency in a For Each loop can greatly improve performance.

What I'm missing is how are variables kept scoped within each concurrent thread? 

Below I have an example of three variables that are set during a For Next loop

* ScanOpEmail -- whose value is set based on some values in the current item()

* NextStepName -- whose value is set based on the Item's Current Step and category value

* cntComplete -- which increments by 1 for each item whose Current Step is "Completed"

Here's what I'm experiencing when I turn on Concurrency and leave degree of parallelism at the default of 20:

           Record 1 hits Thread 1 sets variables (ScanOpEmail, Next Step, cntComplete).

           Meanwhile Thread 2 kicks off and populates those *same* variables based on Record 2's information. 

           And so forth until we have 20 threads going.  

           By the time Record 1 hits the Email command, the variables have been overwritten with data from other records,

           And this happens for every single record processed.

I've gotta be missing something here -- what good is concurrency if you can't scope variables within each instance and they're constantly overwriting each other?  

Is there a way to scope variables to a specific parallel thread so variables aren't overwritten when concurrency is turned on? 

So that while a  thread is processing Record 1 the variables variables (ScanOpEmail, Next Step, cntComplete) retain their value during the life of that loop.   (Then there's the issue of cntComplete.... but I could punt that if I could scope the variables on a per-thread basis.)

What am I missing here -- is anyone using For Each / Concurrency on successfully?

[eric-cheng · Accepted solution]

Hi @MelissaReed ,

Concurrency is useful in certain cases only.  As stated by yourself, when you have concurrency enabled or even parallel branches, you can run into issues when using variables.  A workaround is to use Compose instead of variables inside each branch but it is again, only applicable for certain use cases depending on your business logic.  Here are some blogs which talks about this (link, link)

--------------------------------------------------------------------------
If I have answered your question, please mark my post as a solution
If you have found my response helpful, please give it a thumbs up

Connect on LinkedIn

[MelissaReed]

@eric-cheng 

The Compose action -- BRILLIANT!  Never occurred to me but that would definitely solve the problem in my "real" flow.

And the second link was exactly the kind of article I had been looking for but hadn't been able to find.  

Thanks so much for your help!