Collection from multiple libraries transferring to Sharepoint list

[jessHawley]

I have a Power App I created that collects from multiple libraries into search galleries that allow for a user to search documents and preview them. The issue is that it takes more than 15 minutes to load the collection. I would like to send the collection to a SharePoint list so that it is only pulling from one location and the data/file is saved and then just updates when the app is opened. I cannot get the thumbnail to transfer or create a new thumbnail from the URL that transfers. Is there anyway to transfer the thumbnail of the document to the list? This is a key Component of the Gallery search function. Or is there another way to save the Galleries and just have them update on load?

[RandyHayes]

@jessHawley 

This is the disadvantage of trying to pull all the data into your app.  It takes a long time, consumes more memory than you want and causes issues down the road.

Consider making your app more dynamic.  Meaning - sure, consider putting a list together that contains "pre-processed" information that makes your app more streamlined, but when it comes to getting to the real records, do so at that point...not pull it all in up front.

Apps should be slim and streamlined.  Think about writing a Google search app...sure you have a Search textinput...but to search, would you then want to have pulled down all of the data from the internet into your app so you can search for it?  Of course not.  Same with any other app...think thin.  Even with the above example, items on the internet are indexed and keyworded so that the search occurs on that data, not the actual internet page.

Same concept for your app...if you need to be able to search in a specific way, then consider a supporting list that would have the data indexed/recorded in such a way that your app can perform a streamlined search while *never* pulling in all the records.  Only when a record is needed should it then be brought up.

Also, thumbnails are generated on the SharePoint side, you cannot change them or transfer them.  But, with the above mentioned solution, you would not need to.  Your supporting list would indicate where the record is and your app would get it as needed (and thus the thumbnail).

I hope this is helpful for you.

[jessHawley]

yes that works but it does not allow for the document preview in the app.

This is the search Gallery and preview I setup, I can remove the collection from the gallery and just put a library but the thumbnails do not pull. Also is there another way to add multiple libraries to the search? 

Filter(Search(CernerReferencesMassCollection,inpText.Text,"{Name}"),'{IsFolder}'=false, !(".xlsx" in '{FilenameWithExtension}'),!(".xlsm" in '{FilenameWithExtension}'),!(".mp4" in '{FilenameWithExtension}'))

[RandyHayes]

@jessHawley 

Change your preview then to pull from the actual doc library when needed!

Yes, you can search multiple locations.  However, when dealing with document libraries, things get pretty fuzzy pretty quickly.

First, I will explain what I mean. 

A document library is just a list.  It looks like a file system, but it is not, it's just a list.  The document library creates lots of columns to support itself...like IsFolder.  It's not a real column, it is a calculated column.  And, it is really only there for the "concept" of folders...there are no folders in a list!  In other words, with that particular column (IsFolder) it is really only there to support SharePoint viewing of the list to simulate the "file/folder" concept.  But...still a list!

Why I mention this is because you have to have alternate means of effectively searching large libraries.  Many will think - "ok, my library has 10k files in it, and that's a problem, BUT, I am only going to search a folder of that library that has only 30 files in it"  Well...it will not work, because there really is no folder, just a list with 10k files in it.

So, things get pretty ugly when you try to do non-delegable actions on the library (such as Search, or filtering on calculated columns (i.e. IsFolder, {Name}, {FileNameWithExtension}, etc), or checking for a false condition - all non-delegable).

AND, that is all just one list/library!  You want multiple ones.

So now, with that all said.  In looking at your formula, you are only searching the {Name} column (which is not delegable).  If that is all you need to search, then there might be an intermediate step...keep your collection, but don't load it with all the data.  The ONLY thing you need in your collection is the {Name} value, the List reference, and the ID of the item.  That should load fairly quickly.

The key then is, for the document preview, is to LookUp the record from the selected list for the selected ID.  

Now, in looking at your design, you have a thumbnail in each record in the gallery, and then when selected, the thumbnail again...do you really need both?  The one in the gallery is going to be the more performance killing one.

That image control in the gallery can simply do a lookup to the corresponding record list and ID and get the Thumbnail, but that will cause pause in your app.  So best to eliminate if you can.  If not, you are at least cutting down tremendously on the 15 minute start up wait but you are spreading that timing out to places where it is really needed.

Hopefully this all makes sense.

[jessHawley]

Ok maybe I am not understanding. The preview function for the documents is dependent on the thumbnail of the selected item.

Substitute(Gallery4.Selected.'{Thumbnail}'.Large,"/thumbnail","/pdf")

Is there a way to view the pdf from '{Link}'?

If possible I would like to tailor the collection to Name, FilenameWithExtension, and Link. But when I tried to select just {Name} as the item in my collection it would not run. 

Here is the current collection -

ClearCollect(CernerReferencesMassCollection,'Cerner Reference Documents','Supplemental Training Materials','UIS Documents & Presentations', 'Provider Champion Series', Training, 'Content and Structure Reports', Documents_1,Documents_2)

[RandyHayes]

@jessHawley 

Yes, your gallery would not have that thumbnail property, it would have the name and the ID and the list.

So your image would be more like this:

Substitute(
    LookUp(
        Switch(Gallery4.Selected.Library,
            "CerenerReferencesMassCollection", CerenerReferencesMassCollection,
            "Cerener References Documents", 'Cerener References Documents',
            ...etc..
        ),
        ID = Gallery4.Selected.ID
    ).'{Thumbnail}'.Large,
    "/thumbnail",
    "/pdf"
)

Ideally (on second thought from my prior response), an intermediate list would still be highly beneficial in your case.  The reason is, looking at your formula to collect all those lists, you can tell PowerApps to only "show" certain columns, but SharePoint does not support one key feature - Explicit Column Selection.  Only SQL supports it that I am aware of.  What it does is cause the connector to ONLY return columns that are requested.  

So, for example, issuing a:  ShowColumns(list, "{Name}", "ID") on SharePoint will STILL return ALL columns in the "list" part of that formula.  It pulls everything into memory and THEN it strips all the other columns out in the results.

Whereas, that same formula on a SQL list would only ever have the two columns worth of data transmitted to the app.  A huge performance saver!  But...not available to SharePoint.

So what is an intermediate list??  This would be something that would be generated by the use of PowerAutomate.  The concept is that if any change is made in any of the libraries, you would have a flow on them that is triggered and it would take the Calculated Name, ID and List name and put them into a separate list.  It would be THAT list that you would use in the app.  You would have all the data you need then to be able to get everything from any library based on that list.

Even with the intermediate list, you can still run into record limits (and you are still running into record limits with your current formula!) But, with the intermediate list, you can work around it much better than you could with the library itself.

Again...hope this is all making sense.