Build an instant flow which iterate over a SharePoint library that have 2 million documents and folders

[johnjohn123]

We have a SharePoint document library which contain around 3,000 customers subfolders inside the root folder, and total of 2 million files and sub-folders inside the 3,000 customer sub-folders.

Now we added a custom column named "CustomerFilesLastModifed" , which should be equal to the latest date any file inside the related customer sub-folder was modified.. for example let say we have this case:-

1) Customer sub-folder named "CustomerA"

2) Inside the "CustomerA" sub-folder we have 2 files:-

 - Expenses.xlsx was modified on 12 Feb 2022

 - Profile.docx was modified on 10 March 2023

3) then the "CustomerFilesLastModifed" date for "CustomerA" sub-folder should be equal to 10 March 2023 (the latest modified date for the files inside the sub-folder).. and so on for the other customers sub-folders.

so for this to work i will need to create an instant flow which can loop through all the 2 million documents.. so does Power automate support this type of operations? or i need to look into creating an azure function or power shell script?

Thanks

[Matthy79 · Accepted solution]

Hello @johnjohn123 

I think Power Automate can do this but you have to be careful with the limits. You don't have to iterate all 2 mio files and don't test it with all the folders at first. But this could be a possible way:

1.) Get all customers (Get all files from main folder, here you can set Top Count to 3 for example)

2.) For each customer get files order by "Modified desc" Top Count 1 (apply to each)

3.) Update customer folder (first(outputs('Get_files_(properties_only)')?['body/value'])['Modified'])

You can turn on "Concurrency Control" to speed things up.

[Matthy79]

Hello @johnjohn123 

Is this problem solved? Have you tried it that way, did you use a different approach or have you changed the requirement of needing this functionality?