Append to array variable limitation

[Sear]

Seems like there are maximum items limitation that when using append to array variable, it can only have 500 items in total.  But my result may have 100 thousand+ items which comes from a do until loop (each loop only return 1 thousand items), thus I need set an array variable and append to the variable in each loop. The finial goal is to create a csv table to include all the items and then create a csv file in SharePoint list.

I think this is a very common case, do we have any solution or workaround?

[manuelstgomes]

Hi @Sear 

I have a proposal for a workaround. 

When adding to your array, check if you're reaching the max. If you're doing so, append to the CSV, all the values created as far, and reset the array. Repeat until the number of items is finished. 

There's no need to hold everything in an array before you can generate the CSV. You can keep on appending stuff.

Can you please check if and let me know if you have any questions?

If I have answered your question, please mark your post as Solved.
If you like my response, please give it a Thumbs Up.

Cheers
Manuel

[CFernandes]

Can you please have a look at https://powerusers.microsoft.com/t5/Building-Flows/Append-to-Array-Variable-limited-to-500-items/td-...

[Sear]

Hi @manuelstgomes 

Thanks for your quick reply, to append to the CSV, I think I need have a string variable which also has the maximum size of 100MB limitation when using a append to string function, but at least it can hold more items than append to array variable.

by the way, I need one csv file which contains all the results instead of split files.

[Sear]

Hi @CFernandes 

I read that post before I post my question, that workaround can't resolve my issue, as I'm using a custom connector and can't enable the built-in pagination, the maximum top count of my customer connector is 1000 and the total count may have 100 thousand.