Very slow performance looping over array

[kandiyohiv]

My flow is taking 10 minutes and it seems like it should take less then 1 minute. Hoping someone can provide some suggestions.  The time spent seems mostly around the array operations. 

1) Get SharePoint file properties based on Document ID (excel file)

2) Get tables from the excel file (there's only one table)

3) List rows in table (634 rows).  Only one column in the table. 

4) json parse the value from this one column to get the value stored in the column

5) Loop over the 634 items from the json parse output and add the loop value to an array until we accumulate 500 items in the array.

6) Convert the 500-item array to a comma-delim list using the join function

7) send an email with the string from step 6 in the body

8.) empty the array and continue the process for the remaining 134 items. 

This is a practice dataset.  My actual dataset will have around 5,000 item (so 10 full loops) and I can't have this thing running for 2 hours...

Any ideas?

[v-xiaochen-msft]

Hi @kandiyohiv ,

If each loop doesn't depend on each other, consider turning on the loop's concurrency setting.

Best Regards,

Wearsky

[ivan_apps]

Agree with this - increase concurrency. 

[takolota]

“4) json parse the value from this one column to get the value stored in the column

5) Loop over the 634 items from the json parse output and add the loop value to an array until we accumulate 500 items in the array.”

You should be able to use a Select action to get all values from a single column in like 1 second.

If you need to only get the 1st 500 values for some reason, just use the take( ) expression.

“6) Convert the 500-item array to a comma-delim list using the join function”

Should be fine, but why not input the JSON array into the Create HTML table action for better formatting of the list for the email?

[kandiyohiv]

I tried turning on concurrency,  but it didn't help at all. 

[kandiyohiv]

@takolota Thanks for the feedback.  I tried "chunking" my array into pieces using take() and skip() like you've suggested, but I couldn't figure out how to only get the single column of excel values into an array.  My excel value was always trapped in a json item, or part of a key-value pair, and I couldn't figure out how to grab just the value part for my array.  If I could chunk it into 500 rows (and have it only contain my value) then I could use join() to turn it into the comma string that I need without any looping.  Any ideas how to do this?

Regarding the html table,  I simplified my description for this post.  I'm actually doing something different with the comma string that wasn't necessary to mention here. 

[takolota · Accepted solution]

@kandiyohiv 

You can Select an array of any column values by replacing item()?['Email'] in the action/expression below with item()?['ColumnName']
You will need to select that right-side button on the Select action to get it to a single textbox input over the mapping table input.

[kandiyohiv]

@takolota Yes! This is what I needed.  I had been trying to do earlier, but couldn't figure out how to get my values solo in an array. 

I was able to totally eliminate the json parse and all looping over the array. I used a select() as you instructed to take the data directly from "list rows present in a table" and then used take() and skip() to chunk my array into batches inside of a "do until" loop with a counter that I increment 500 at a time. I then directly convert the 500 row array to a comma delim string using join(), increment the counter from 0 to to 500, loop runs again,  and I get my remaining 134 items.  My flow now takes about 5 seconds. 

Thank you very much for your help!