Append to Array Variable limited to 500 items

[KurtEichler]

I have a SQL view with 1,101 rows.  I've setup a Flow to:

get all SQL rows (no issues)

get all items in a SharePoint list (no issues)

Initialize a variable of type array (no issues) then append my SharePoint get items list results to the array variable. 

Here you can see there are only 500 items instead of 1,101.

When I was testing, I filtered out data so I only received less than 100 rows.  This worked as expected.  Now, when I run my Flow and more than 500 SharePoint items are returned, the Flow does a comparison to see if my row in SQL exists in my SharePoint list.  After the 500th item, new items are getting created as a duplicate.

I checked the settings of the Apply to each step and Append to array variable.  There is not a setting that can be changed to pull in more than 500 items.  I suspect it's the Append to array variable step as I have another Apply to each that I'm not using a variable on and that shows all 1,101 items waiting to get updated.

Does anybody have a workaround for this?  I can't find any MS documentation stating there's a 500 item limit when working with an array.

[KurtEichler]

Update: Received a reply on the Facebook Flow page.  Here's my update on what fixed this.

Yesterday, I pulled my SharePoint get-items output and was certain I was getting all of my results. However, I pulled them again this morning and was only getting 500. I set my Top Count to 5000, re-ran my Flow and now pulling in all 1,101 list items. Not sure how Pagination differs in this case, but I also set that to 5000 to avoid any issues. Looking at the limits, https://docs.microsoft.com/en-us/flow/limits-and-config, It doesn't state anywhere in the documentation about a 500 item limit. Some articles I came across mentioned a 100 item limit while others ran in to a 5000 item limit. MS can and should improve upon their documentation.

[GabrielStJohn]

Hey, @KurtEichler!

Thank you for posting to the Flow Community Forum! It appears you have found a solution to your issue! If you feel as though your issue has been solved and you are satisfied with one of the previous replies, please click "Accept as Solution" on the reply that contains the solution so that this thread will be marked for other users to easily identify!

Also, please visit the documentation page where you were not able to find the info that you needed and scroll to the bottom of the page to sign-in and give your feedback on the specified documentation!

Thank you for being an active member of the Flow Community! 

-Gabriel
Microsoft Flow Community Manager

[Anonymous]

@KurtEichler thanks a lot for sharing the solution you found.

I had the same issue - when using Append to string variable, the outputs were limited to 100 results. I was using this action to stop duplicates getting inserted in my SharePoint list, so the default setting was not picking up all the duplicates, and flow would fail.

Setting Top counts to a high value in Get Items fixed it and my flow runs as expected.

I agree with you that Microsoft should improve its documentation on Flow / Power Automate. It's very difficult to find fixes that would be relatively easy with traditional code.