Is it normal that Power Automate is so slow in making simple Loops where it adds strings to variables or compares Elemts with each other?

[NiBes]

I created a Flow where it should get all files and Folders on a SharePoint Side and it will color all empty Folders red if they do not contain any files, otherwise, they will get yellow.

It adds them each in an array (Folders in "Folders Array" and Files in "Sorted Files array" / "Unique Files Path Array" where unique file Paths are stored)

I then used an  "apply to each" loop to get "all file Folders" Paths split and added all parts of the file path to each unique file Path. I then added each Path only once to the "All Unique File Folders Array"  via the union Function.

This part worked quite well. It may be optimized but it seems ok.

The next part a bigger Apply to each loop, takes a lot of time.

In it I compare each Filepath on SharePoint with the "All Unique File Folders Array" and if they match I change the Info in "Folders Array" to Is_Full = true. If they do not match they will receive Is_Full = false.

This takes a long time to work. But I'm not sure how I could implement this kind of logic in power automate style.

I know you could achieve better results by using Concurrency. But then I do not know how to implement breaks in loops. (Maybe I'm too spoilt from languages like C++ / Python and do not understand Power Automate structure well enough)

Start of the LoopThe Inner Loop (Part1)The Inner Loop (Part2)End of the Loop Block

The last part is a simple "apply to each" loop where it loops threw all Elemtens in the "Updated Folder Arry" where all Infos are saved for each folder. There I use concurrency enabled because I could use a simple if clause to make the color change for each folder on its own.

Is there a simple way to make this flow faster?

How could I use maybe concurrency in the Apply to Each Loop where they compare the All Unique File Folders Array with the "Folders Array"?

Timing with only 36 Folders and 26 populated:

Flow Timing part 1Flow Timing part 2Flow Timing part 3

[FLMike]

@NiBes 

Unfortunately its near impossible for us to give you concrete answers without us being able to run your flow. Its great you shared it, so thank you, but its still very hard.

I would go to the Details Page of your Flow

P.S. As for Concurrency, its possible sure and you could use it, but you cannot use Variables in them or it will mess the values in the variables up, as they are not thread safe.

You might be able to instead.

-have a parent flow

-have that parent flow in an Apply to each with concurrency on and have it pass to the Child the folder its supposed to compare and the path it compares it too

This is high level, but it would speed you up alot

If you like my answer, I would really appreciate if you please Mark it as Resolved, and give it a thumbs up, so it can help others

Cheers

Thank You
Michael Gernaey MCT | MCSE | MCP | Self-Contractor| Ex-Microsoft
https://gernaeysoftware.com
LinkedIn: https://www.linkedin.com/in/michaelgernaey

[takolota]

@NiBes 

You could probably drastically speed up your flow by learning to use Filter array actions, like instead of conditionals in a loop,

https://www.powertechtips.com/filter-array-power-automate/

Select actions, & the back-end SharePoint column names to use in filters to automatically select only folders & only not folders (files) directly on your data pull

https://sharepoint.stackexchange.com/questions/293905/how-to-query-only-documents-from-a-library-fil...

[NiBes]

@FLMike ,

thank you for your input/help.

I will dig deeper into that topic.

Could you recommend one source where this is taught step by step?

Hopefully, that would help.

[NiBes]

@takolota ,

OK never heard of them directly.

Maybe there is a fast way to find matching names / Paths in both lists then?

Because I think I kind of separated both File Folders and only Folders quite well. Because I also used a Filter Query in the first HTTP Get all Files / Folders request to SharePoint. (FSObjType eq 0 / ... 1) Right now the only problem I see is the Horendes Time my loop takes to compare the two lists for matches and then update the information about that Folder.

I tried to scale the Folderstructure a bit (Now 52 Folders with 25 Populated down the Path)and that took too long for a useful use case.

Flow with 52 Folders_part1Flow with 52 Folders_part2

This Flow would be only used once per day or once per week to update the folder's visual structure. For creating or uploading folders/files I created a separate one to update each upper folder each time a new file is created.

[takolota]

@NiBes 

If you have an array of all the folder items and an array of all the full paths of all the file items, then you should be able to use the Filter array on all the folder items with the condition checking if…

string(InsertFilePathsArray) contains item()?[‘FolderNameColumnName’]

That would get folders with any files for yellow.

And/Or you could use ‘does not contain’ to get folders without any files for red.

However, this assumes that any folder name isn’t potentially repeated elsewhere in the file paths, like for an earlier level of folders.

[NiBes]

@takolota

OK so you mean I should add a condition around the Filter Array? Or did you mean I should edit the Filter array in the advanced mode like "@equals("Folder("Full_Folder_Path"),File_Folder")"

In this case, I still have to use two apply each loop where one has to loop threw "Folders Array" => Folder and the other threw "All Unique File Folders Array" => File_Folder. Or am I completely off?

I'm still stuck in the Python-like thinking in my head where I have to make a loop in a loop when I do not want to use "list(set(A).intersection(B))". 😕

Something like: (Sorry bad pseudocode)

For Folder in "Folders Array":
  For File_Folder in "All Unique File Folders Array":
    if(Folder('Full_Folder_path') == File_Folder):
      Do something
      Break

Also here are some sample values saved in each Array to compare.

Folders Array:

[
  {
    "Name": "07 Projekte",
    "Ist_Folder": true,
    "Folder_Path": "Technology/",
    "Full_Folder_Path": "Technology/07 Projekte",
    "ID": 1
  },
  {
    "Name": "08 VT",
    "Ist_Folder": true,
    "Folder_Path": "Technology/",
    "Full_Folder_Path": "Technology/08 VT",
    "ID": 2
  },...
]

All Unique File Folders Array:

[
  "Technology/07 Projekte",
  "Technology/07 Projekte/04 PVB MOWITAL",
  "Technology/07 Projekte/04 PVB MOWITAL/C in progress",
  "Technology/07 Projekte/00 Archiv",
  "Technology/04 Instandhaltung",...
]

[takolota]

@NiBes 

The Filter array has something like a built in loop. It iterates over each item in the array you feed it in the From input & runs the condition. Then it only returns items where that condition results to true.

But I suspect it uses a GPU & it definitely avoids a lot of back & forth with API calls to the servers for each action in a loop that slow things down.

[NiBes]

@takolota

Ok, I tried to use your Filter Array setting. I overlooked that I could change the equal to contains.

Now the Body gets populated but it does not filter it at all.

It seems like it passes everything threw.

From: variables('Folders Array')

@contains(string(variables('All Unique File Folders Array')), string(item()?['Full_Folder_Path']))

Not Filtered Output

Any Ideas why? Or how i could filter it right?

[takolota]

@NiBes 

Isn’t folder path going to reference the parent folder of each item, not the folder name itself which is in the Name property?

[NiBes]

@takolota 

As far as I figured out, the folder path in the Folder array is as you said the parent folder / Path to the parent folder.

The Full_Folder_Path will normally select the direct file or item from the request.

But I filtered in the HTTP Request already only for folders so the Full Folder Path is the direct path to the Folder with the Name Property (The folder Item).

My problem with the filter array is I could not debug what and how it selects/compares the items. So it is still possible that it compares them wrong.

Every time the Full_Folder_Path of one Folders Array Items matches with one Item in All Unique File Folder Array I will change the Color information for that one.