Child Flow has much higher latency when launched in non-maker context ???

[HEATFreight]

Have a Power Apps canvas app with one maker (myself) and one user.

To ensure SharePoint data is private, the app launches a parent flow in app user context which launches a child flow in my context.

This means the Power Apps user is running the Child Flow as a non-maker user. This way the SharePoint lists can be locked down such that no user can read or write to items not created by them. Thus all the records are private (unless I cache them by having user patch records directly from the canvas app in their own context, which I do plan to implement eventually to minimize latency).

Yet, although the user has no permissions to create, view, or edit the records, they can trigger the Child Flow to create, view, and edit records on their behalf. This works fine. (Edit: I should clarify, I have assigned the users who are members of the SharePoint lists, which is myself and one other user currently, the ability to view and edit records that they create and to create records themselves but not view or edit records created by others. However, the canvas app is not yet querying SharePoint for most of these records directly in their own context. Only some lists containing non private material are open to viewing by any user in their own context, but they can't edit those, only view. Those are queried directly from SharePoint in user context on start, for reduction in latency and simplicity. Then there may be some other non-private lists which allow each user to create, edit, and view all records, or possible to create and view but not edit. There is definitely a class of records which only the maker should have view-all/edit-all access to, but there needs to be a higher-level synchronization of the records so that individual users can't create interfering entries —duplicate invoice numbers for instance. So, although I may have certain user permissions enabled for the private lists, I am not yet using those permissions to directly query and edit SharePoint lists in user context. So far I'm merely using the higher-level context to manage all the C.R.U.D. operations on the private data.)

I'm using this technique to collect all the data to initialize a shipping and billing app. It has a full CRM with various types of entities. I mean there is a lot of stuff going on in this app. There are at least half a dozen different types of forms, some of them quite extensive. It was a complex effort to figure this out, and no small feat to get it to launch efficiently. There is a ton of concurrency in the App.OnStart code, like dozens of variable/collection initializations, multiple non-private SharePoint list queries, and some Parent/Child Flow calls that gather API tokens and other data to populate the galleries in the app.

I'm using mostly SharePoint HTTP requests in the Power Automate flows to collect the data that gets returned to the app (as opposed to the basic SharePoint actions), and to reduce latency I have these SharePoint HTTP requests getting only 10 of each type of private record in the initialization of the app.

So launching the app in my (maker) context nominally takes about 10 seconds minimum and up to 30 seconds or so at the most, longer when loading up the newest published version. Even then probably no more than 45 seconds. Typically under 20 seconds even if updating to newest published version.

Launching the app in the other user's context nominally takes a minimum of 20 seconds and up to about 45 seconds at the most, maybe a minute if loading newest published version of the canvas app. But almost always at least 30 seconds or so.

So, we can conservatively say that the non-maker user takes as much as 3x longer to launch the app in their context than it takes me to do so in my maker context. I'm guessing it's around 1.5x to 2x slower on average, like sometimes it's almost good but most of the time it's slow enough to be almost unusable. I would say 1 minute is unusable, and that's kind where it's at if the user is loading a newly published version.

Besides caching records in user context and querying the user's cache on launch instead of using the Parent/Child Flows to query that same data in non-maker user context, what can I do to speed up the app launch?

Am already using the "preload app" setting. Can't increase concurrency any further. Only avenue I see to massively reducing latency is to relegate use of Parent/Child Flow logic to specific purposes, like first time logging in to the app or when external API calls run, but to otherwise cache all user SharePoint data in user's own context and fetch that directly from SharePoint on app launch.

From my experience, the direct-from-user's-context SharePoint queries will have no noticeable latency, whereas the Parent/Child Flow logic to accomplish the same query will take at least a few seconds or several seconds.

If the non-maker user's Parent/Child Flow latency were closer to that of the maker, I wouldn't need to cache the user's private data at all. Parent/Child Flow latency is totally manageable in maker's context.

Also, the app needs to scale, as there will hopefully be more users eventually. With that in mind, I think figuring out how to cache in user's context is on my horizon no matter what, but sure would be nice to speed up Parent/Child Flow run latency!

See cross-post here

[HEATFreight]

Wow, this has gotten substantially worse after I made some changes to try to get the main page of the app to load quicker.

I simply rearranged the concurrency logic to put the CRM.Run() functions at the end, because those consist of 4 or 5 tiers of running 6 separate SharePoint queries and then collecting and arranging that data.

Occasionally when I have a need for some new global variable, I place it at the start of the very first Concurrent() loop in App.OnStart. My guess is that placing low priority variables unnecessarily high in the concurrency order may cause bad performance. I'm thinking the highest latency functions need to be placed as high up in the concurrency as possible with low priority items coming later.

Anyway, so even after tidying up the CRM.Run() functions and placing them at the end of App.OnStart, somehow the user is now getting consistent app launch times of a minute or more, so much worse than before. Yet I can launch the app in under 10 seconds now. My improvements sped up the app launch for myself but slowed it down for the user!

It's roughly an order of magnitude slower for the user!

The trick I used to obtain the fast startup was that I set the variable which determines if we've loaded enough data to proceed to main page of app earlier, effectively, by placing all the CRM (which stands for customer resource manager by the way) queries and functions after the varShowMainPage=true variable.

Then I added a new global variable varCRMDataLoaded=false at the very start of the first App.OnStart concurrency loop. I set varCRMDataLoaded=true after the CRM data section of the App.OnStart completes.

So App.OnStart basically runs in stages by my logic, and I've given it another stage. The first stage checks if user is authenticated. Second stage proceeds to authentication page if not or it loads the main page data if user is already authenticated, then navigates to that screen. The last stage of App.OnStart continues running in the background after the main data has already finished loading and the main page is being rendered. Only when that last App.OnStart stage finishes and all the CRM data is loaded does the CRM button become visible.

Thus I can launch a complicated and sprawling canvas app in under 10 seconds to main functionality, with a few seconds more of CRM data loading in the background after that.

Yet as a result of these changes the non-maker user takes consistently over 60 seconds to load the same app with the same amount of data.

What I may try now is moving all the CRM data queries to the CRM button itself and have them run right when the user clicks the button for the first time. Otherwise I'll try to rearrange App.OnStart concurrency loops to place low priority logic further down the list (in case I'm hitting some kind of limit that makes items in a single loop not run entirely concurrently).

The one other change that happened since the last time I launched the app in the user's context is that I've added a PowerApps-triggered cloud flow that includes O365 GetMyProfile connector and Outlook SendAnEmail connector running in user's context (whereas typically I'm using Parent/ChildFlow logic to run most other connectors in my own context). Not sure if the new connectors are somehow slowing down user's launch time, but one thing I noticed is the background color of the app that I usually see while it's loading has now disappeared for both the user and myself, at least at first, and the screen is just blank white for awhile before the familiar yellow color appears, followed by my "Loading..." image. In my case it still launches in like 8 seconds. I'm seeing user's app launch take 65+ seconds.

Edit: Nope, moving the CRM queries out of the App.OnStart and placing them in a button to be run later does not reduce the launch latency noticeably. Still seeing 8 seconds to launch it myself, although previously that 8 seconds included loading up the CRM, but now I have another 3 or 4 seconds to load the CRM whenever I do click the button for it. The user launch time is still around a minute or more. Maybe on average slightly quicker now, but not enough to be usable. Need it consistently under 30 seconds. Something about user's context is slower to load the app compared to maker's context, and it seems to get exponentially worse the more complex the app is and the more connections there are and the more flows there are. The app must be loading up connector permissions for flows directly connected to the app, because it really seems that everything got worse for the user-context app launch latency after I added O365 GetMyProfile connector and Outlook SendAnEmail connector to one of the flows that the app uses.

[HEATFreight · Accepted solution]

Update: It was the "Office 365 Users" connector that was running super slow for the non-maker user!

I put a status label in the canvas app and updated the status in each Concurrent() function to see which steps were causing the most latency, and then compared the maker user app launch vs. the non-maker user app launch. I definitely needed to rearrange the concurrency logic, because it really helps to lump all the expensive API calls into one Concurrent() function and put anything that executes quickly wherever it needs to go to make the arrangement of the expensive API calls work, even if that means some things run by themselves. Better to have non-current initializers, if needed, to allow high latency functions to run concurrently than to have it all run concurrently but with the high latency functions split across different Concurrent() functions.

I'm going to take this to the extreme and try to put all my child flows into one parent flow, and I'll move as much of the logic as possible into that parent flow. This way all the highest latency operations can occur right alongside each other. The parent flow will ultimately take longer to run than any of the other parent flows otherwise would have, but hopefully less than the sum of the simpler parent flow runtimes.

All these rearrangements of the logic will have a big impact, I'm hoping, but definitely removing the "Office 365 Users" connector from the canvas app has already had a massive impact! I used the User() function to get user's name and email address, then hardcoded a collection of the O365 user info instead. Much quicker!

I also removed the Outlook Email connector and O365 Users connector from a parent flow as well. Any superfluous connectors in the canvas app itself or any flows it directly triggers will result in a huge impact on the startup time for the end user.

The only connectors I have in the canvas app now are "Bing Maps", "Spatial Services", "Office 365 Outlook", and "SharePoint. I use no other connectors in the canvas app and no connectors at all in any of the flows that the canvas app calls directly (except "Run a Child Flow"). The only flows that have any connectors whatsoever are all Child Flows, which the canvas app does not trigger directly. This makes a HUGE difference!

[HEATFreight]

Update: It was the "Office 365 Users" connector that was running super slow for the non-maker user!

I put a status label in the canvas app and updated the status in each Concurrent() function to see which steps were causing the most latency, and then compared the maker user app launch vs. the non-maker user app launch. I definitely needed to rearrange the concurrency logic, because it really helps to lump all the expensive API calls into one Concurrent() function and put anything that executes quickly wherever it needs to go to make the arrangement of the expensive API calls work, even if that means some things run by themselves. Better to have non-concurrent initializers, if needed, to allow high latency functions to run concurrently than to have it all run concurrently but with the high latency functions split across different Concurrent() functions.

I'm going to take this to the extreme and try to put all my child flows into one parent flow, and I'll move as much of the logic as possible into that parent flow. This way all the highest latency operations can occur right alongside each other. The parent flow will ultimately take longer to run than any of the other parent flows otherwise would have, but hopefully less than the sum of the simpler parent flow runtimes.

All these rearrangements of the logic will have a big impact, I'm hoping, but definitely removing the "Office 365 Users" connector from the canvas app has already had a massive impact! I also removed the Outlook Email connector and O365 Users connector from a parent flow as well. Any superfluous connectors in the canvas app itself or any flows it directly triggers will result in a huge impact on the startup time for the end user.

The only connectors I have in the canvas app now are "Bing Maps", "Spatial Services", "Office 365 Outlook", and "SharePoint. I use no other connectors in the canvas app and no connectors at all in any of the flows that the canvas app calls directly (except "Run a Child Flow"). The only flows that have any connectors whatsoever are all Child Flows, which the canvas app does not trigger directly. This makes a HUGE difference!