Existing Flow has stopped functioning, Get Files (properties only) can no longer see all files, Top Count no longer resolves issue

[tomatotech]

Good day,

I have developed a Flow (#1) for a SharePoint that needs to archive files weekly, from one location filled with user folders and files, to a different read-only "archive" location in the same SharePoint library. This flow saves hours a week, and is critical to a certain department's SharePoint library processes. At the start of the year, I resolved the issue of the Flow not seeing all the files within these folders via the Top Count option, setting it to 999999. 

Some time last month, this seems to have shifted. No changes were made to the Flow over the year, and it has worked flawlessly since I patched the Top Count issue. However, slowly over the weeks in September, more files were missed every run. At first it was a handful, then half the targeted directory... Now, I am back to not being able to get any files in this folder of the library. Has anything changed with Flow/Power Automate (that I am not seeing) to have caused this?

The "Get Files" output is seemingly limited to 160 total items, all folders and documents that don't match the following Apply to Each Condition, now. The Condition makes sure it is at the location PATH1/username/PATH2/, doing Contains Path on Path1/ and Path2/, is of a proper document type, and that the folder path does NOT contain OneDrive/ (as I had a user somehow drag-and-drop(?) their entire OneDrive folder into their SharePoint folder...).  See picture below.

I have a simpler Flow (#2) that does the same for another process (only .xlsx docs), however even that fails. Hence, my believing the Condition (that has worked fine for months) is not the issue at hand. (Still, pictured below.)

These are my first-ever Flows, so I am open to suggestions. Has anything changed since the start of September? I know the rebranding has been running, and there have been service disruptions, but this issue has been worsening over the past month consistently at every scheduled weekly run.

Condition for Flow #1:

Condition for Flow #2:

Get Files (properties only) screenshot:

[PatrickMSFT]

tomatotech,

We have had some service issues the past couple of weeks, interoperation with other services (like SharePoint Online) was impacted as part of those concerns. If this is still reproducing for you, though then it's unlikely these services issues are involved.

The Flows you describe, especially this simpler Flow, read like they're logically sound. If they're not reporting any failures then this leans in the direction of a performance issue.

Have you also noticed these Flows running more slowly than before? We updated our throttling around the time you state these Flows began to have issues, you might be losing items as a part of that. This is even more likely if you're processing thousands of items.

For more on these changes and guidance in response:
https://support.microsoft.com/en-us/help/4531688/troubleshooting-slow-running-flows

The short of it is that the license you have matters and the structure of the Flow does, as well. It's all covered in the above link.

The SharePoint connector itself also has limits/restrictions:
https://docs.microsoft.com/en-us/connectors/sharepointonline

If you go through this and can't seem to find a link, I'd suggest opening a support incident through the Power Platform Admin Center. Although we're not solution experts, we can certainly help verify the nature of your issue.

Let us know!

Patrick

[tomatotech]

Patrick,

Thank you for your response. It certainly is not a speed issue, as at the end of August the Flow was taking 17 minutes or more (Apply to Each had 250 items). The first run on September 7th was 7 minutes (224 items). September 14th only had 195 items, at under 2 minutes. The most recent run: 160 items at 17 seconds.

While troubleshooting, I noted there was a "Degree of Parallelism" option on the Apply to Each that I had messed with, to no avail. It is currently set to 50, but no matter what that is set to, the input from Get Files (properties only) remains unchanged.

I will read all these over tomorrow, and see if I can find out more.

( Waiting on next run on Monday [10/12] to see if it manages to find any items )

- TomatoTech

[tomatotech]

@PatrickMSFT,

The flow ran again last night, and "Succeeded" in 17 seconds. The core issue is the Get Files (properties only) block just doesn't seem to be getting all the files within the folders I've targeted. The few dozen files that are there are seemingly ignored, whereas others unrelated are detected in the output.

For example, there should be a few dozen excel files (.xlsx) being detected. All the Output file shows from the run last night are four .xlsx files in total, none are in the file paths I want (hence no actions from the For Each Condition to work on those).

Is there a way I can look deeper into the Flow's execution to see why this is occurring? Is it hitting one of the limits listed and just silently terminating that step? This all used to work perfectly without a hitch, for months, until something on the platform changed at the start of September to cause this. The Flow itself was not altered.

Thank you for hour help in this,

- TomatoTech