Collection Build Time

[forbudt4u]

Just curious if someone could possibly explain Collection processing time guidelines base on experience and/or documentation?  There may not be an exact answer, but I'm running into an issue that makes it very difficult for end users when searching because of the loading time, and I'm trying to find a happy medium to work with.

My current conditions are as follows

• I have a gallery [SearchGallery_1] Item's property that's filtering from a SharePoint list based on the text entered into a text input box.  The gallery produces the results I need pretty quickly... usually around a second for every 50 records to completely load, so for example, it takes around 3 seconds if the search produces 150 records.
• For  a host of reasons (mainly multiple level filtering), [SearchGallery_1] is not visible as I am using the results of this gallery to build a collection [_SearchGalleryResults].  However, for reference, here is a link to a thread I started specifically about why I was doing this that I got absolutely no bites on, but was able to work through it (https://powerusers.microsoft.com/t5/Building-Power-Apps/Filtering-within-Pop-Up-Guidance/m-p/857070).
• My collection is initiated/built based on the Search Text Input box's OnChange property with the DelayOutput set to true.
• I have a visible gallery [SearchGallery] (which the user can see results in) that has it's items property to display the results of the built [_SearchGalleryResults] collection.

Issue and reason for question about Collection build times

• In my testing I will place the non-visible SearchGallery_1 in a visible status so I can compare the two galleries as I type search text into my text input box.  As always, the SearchGallery_1 pretty much instantaneously shows my results, but the visible SearchGallery takes roughly 5 seconds for every 10 records I have.  So, that 150 record result used as an example above can take 1 to 1 1/2 min's to show the results.  Note that before I changed the text input DelayLoad property to true, the process would take twice as long and even lock up the app (even in the app edit mode) and force me to close out if it was a larger pool of records.  
• I fully understand the Collection must build from the SearchGallery_1 results before my visible SearchGallery gallery can display the results, but this process seems to take a much longer time than it should.  This is strictly my opinion based on the fact that I'm doing something very similar with another screen's OnVisible property and the gallery results are produced fairly quickly.  The only difference being that the results of this gallery are static and not dependent on changing search text.
• I personally think the delay is tied to what I am using to initiate/build my collection (OnChange property of the text input), but to my knowledge, I don't think I can build a collection once a gallery fully loads. Seeing that my collection depended SearchGallery_1 is pretty much instant, this would be a better option over what I'm currently doing.    

With all that being said, I'm hoping I can get some experts opinions on what I'm experiencing here and see if there's something I could do a little different to help minimize the load time?  It's almost unbearable, but, I absolutely need to maintain my multi-level filtering functionality.  

[RandyHayes]

@forbudt4u 

So, I looked at your referenced post as well.  I think you got no hits on it because you hit on it yourself.  A lot of us that answer posts in here will scroll through the lists looking for posts with 0 responses.  If a post has a response or more, then we move on to the next.  There are just too many posts to read all.  So, in the future, you might just Edit your original post and add new information as needed.  Otherwise it looks like someone is responding.

Now....that said.

Collections are overhead for your app.  Consider that (especially looking at the other post) you are duplicating data over and over and over again.
You first have the main collection - your datasource.  Many don't realize, but this is already a "collection"/table.  It has some very special "powers" to it to be able to get data as needed and to sync with the main session table (the one that gets created in the cloud when you start your app).

So...if you then duplicate all of this into a collection, then you end up forcing the data on demand feature of the datasource which causes it to then get all the data and send it to your app collection (the datasource, not your other collection).  So, now that this datasource has the table, your ClearCollect is clearing an in memory table and then duplicating all the data in the datasource table into the collection table.  

THEN you are using that collection as the basis for the Gallery search results and THEN you are creating a collection on the Gallery results to use on another Gallery.  PLUS, each time the screen is made visible, you are clearing all the collections and doing it all over again!

Can you sense anything that might slow your app down yet??

PowerApps is a no-code solution platform.  It is build with the same concept as Excel with referential formulas and calculations.  It builds on that power tremendously with other functions, but that first point should never be lost.

Think about Excel...if you build a formula in a cell to do some calculation on, let's say, some costs and quantities, if you then need to calculate tax percentages or other additional calculations, you build on that cell by putting a formula in another cell to reference the result of the first.  And...you repeat this process building on results to get you what you want in all places.  You DON'T create a variable for each cell and then set other cells to build on the variable...that doesn't makes sense.  Yet, there are places for that in Excel.

Why mention this?  Because Collections are variables.  They are snapshots of tables at a given point of time.  They do NOT update unless you update them.  And...the answer usually is - ClearCollect.  Which means they are flushed and refilled - a time consuming process.

My suggestion and advice is this.  There are only a handful of places where a collection is REALLY needed.  Try to avoid using them unless you have those scenarios (offline ability, the inability to delegate, an in memory database).

Use the datasource directly.  Cut out the redundant Galleries.  Place well structured formulas in your Items properties.  Utilize dynamic data when possible. Don't try to "engineer" a solution to a performance problem that you created based on a problem you didn't have.  If your app is done and you experience performance issues, THEN resolve the issue.  Keep it Super Simple!! 

From the experience side of things, we design a lot of apps.  From fairly simple to very complex (speaking in terms of business logic).  Rarely do we use a collection.  Never do we use invisible Galleries to provide data for another.  Rarely do we use invisible controls for data for others.  Often do we use dynamic variables (I wish I could get my video done on this process...it is extremely valuable).  Always do we keep it simple.

So, that is my $0.02 on the subject.  Too often do people think the answer to performance is a collection.  More often do people find that once they have them in place, performance gets worse.  Now, my statements are "in general" for everything here.  There is a time and place for all the pieces available in the PowerApps suite.  But, you don't always need a heavy tractor trailer truck if all you need to haul is a table lamp...and you just might find that the tractor trailer creates some expenses and problems you would never have had before.

I hope this is helpful for you.

[forbudt4u]

@RandyHayes 

Thank you for your honesty.  I definitely see where me replying to my post could have caused confusion and that's why I didn't get any replies. 

From a logic standpoint, everything else you touched on ALL makes sense and I can follow it.  In my linked original post (and in this one), I tried to relay that I didn't think using a collection was the best route to take.  I also didn't understand why the original developer used that method for our main screen OnVisible property to formulate the original pop-up filter or why they used a pop-up for our admin portal over a separate screen.  However, that's what they did and that's where I'm at.  Unfortunately, the developer in question suffered the fate a lot of companies have due to Covid, so we're stuck trying to resolve a lot of issues not having anyone to ask questions to or development docs to rely on.

Your advice has pretty much convinced me that I need to totally go back to the drawing board on our main Asset Management Screen, Admin Portal, Galleries, and filters as a whole.  The sad part is that it's doing what I need it to do, it's just taking too long with one important aspect.  

Thanks again, Randy!

[RandyHayes]

@forbudt4u 

The unfortunate part is that a lot of designers reuse the "examples" found in the docs.  These weigh heavily on collections for "quick examples".   How many times have you sat through a presentation about development where the presenter said - "now, you wouldn't do it this way in your development, I'm just doing this to demonstrate the core topic."?  Unfortunately the docs don't say that.  Then compound that with people who start doing that method and then do blogs and videos with it...pretty soon everyone thinks that's just how you do it.  It's not!!  My guess, in looking at that formula in the other post, your designer got a lot of his info from those sources without really grasping the concepts.

Now, my best suggestion would be to re-do it and get rid of it and design it 1000% better.  But, it sounds like you are constrained from that.  So, perhaps some simple changes like, in the OnVisible, checking to see if you have ALREADY collected the data.  If not, then do it.  If so, then don't.  Also looking at the Gallery formulas and combining them in a way to reduce the redundant gallery.

Little things like that can help a lot. 

Also, make use of the monitoring tool to really focus on what IS and what IS NOT a performance issue.  If you want the most for any changes you make, use that tool to make sure you are really changing something that is causing a problem.  Nothing worse then putting an hour into redesigning something to solve a problem, only to find that the problem still exists.

And, as always...post on the forum if you get stuck or have questions...just be careful - you'll get a lot of "you need a collection for this" responses sometimes - because there is a "conditioning" that that's the way you do it.