Cross environment deployments

[Pchan3]

Hi All,

We are in the process of developing some canvas apps in our organisation.

We are at the stage for one of our apps to deploy to a test environment, so that our users can do acceptance testing.

Trying to export the app and the related flows in a solution has not been as smooth as we would expect it to be.

Note: We are working exclusively with SharePoint lists for data connections

Solutions (Microsoft recommended):

• Power App data sources need to delete and reconnect each export
• Environment variables need to update each export
• All people picker fields in the power app need to be edited to turn searching on, every export.
• All cloud flows connections stop working, each action/function/trigger needs to be edited to have a new connection.
• Some flows or parts of flows need to be recreated for the flow to work again.
• Flows in solutions do not appear in the PowerApp’s “Actions” tab to be used. If we save as outside of the solution, it shows up.

PowerApp connections to SharePoint lists need to be recreated, then the App needs to be saved, published for this to work, this should be able to be achieved with environment variables, without publishing the app again.

If we try to export and import the app(and flows), not using a solution;

• We have to edit the powerapp and reconnect the trigger from powerapp flows.
• Environment variables are not supported and requires significant changes to each flow and the PowerApp to point to the correct sources.
• We have remove powerapp trigger flows from the app before export, then add them again after import.

We would like an easy way to export to another environment, without the flows & connections breaking and for us not to touch the PowerApp. We can foresee that any feature update or bugfix will break the production version.

Does anyone have any insight on how we can overcome these limitations/issues?

Thanks in advanced.

[cchannon]

Solutions are the correct way to move this content. What might appear to be connections "breaking" between environments is actually intentional behavior. It is assumed that when you promote between environments you will want to target other environments or use other credentials to create the connections, so the connections themselves do not move in a solution; only connection references move in a solution. Connections and Connection References are two different things and when you create a Flow or Canvas App you need to ensure you are using the right one.

Ensure that your Flows/apps were created in a solution originally. Otherwise they might be using just connections and not connection references. The distinction is important because a connection reference will move with the solution and prompt you to update the destination and credentials when you import it to the new environment; a connection will not move in a solution.

[Pchan3]

Thanks @cchannon ,

We were able to get into a support meeting with Microsoft to demonstrate all of the issues we are facing with deployments, I felt it would be best to document the issues here, and get help from the community and Microsoft, If any of the issues we face are solved, we will update this thread. 

We have confirmed that solutions are the way to go, this is the way we want to go as it allows for child flow triggering in addition to the other benefits. As per Microsoft's recommendations we have abandoned out of solution deployments. The following documented issues are all relating to solution deployments.

We are using connection references, All of our flows & app were recreated within a solution. We did clean up a number of them as there were many duplicates, so this is now better. However, several issues still remain. I hope I am explaining this well enough.

• DataSources have to be removed and re-added from within the PowerApp, we are using environment variables to store this information. They do not update within the PowerApp. Microsoft have said this is not expected behaviour.
• People picker fields are effectively being disabled is also unexpected behaviour, has anyone encountered this before? or any other field that has issues in a deployment.
• Buttons with Power Automate actions need to be removed and added again, the code against the OnSelect needs to be copied out into notepad and pasted back in for it to work.
• We can't use a SharePoint view from one list to another with the same name. It works based on GUIDs. This means we have to edit the flow to add this in to get it to work. Is there a way around this? We need to get these migrations as seamless as possible.
• We have a content conversion action in a flow, which when it gets migrated, the following error occurs

InvalidTemplate. Unable to process template language expressions in action 'Send_an_email_to_requestor_with_comments' inputs at line '1' and column '19802': 'The template language expression 'body('Convert_Approver_Comments_to_Text(Approve)')?['body']' cannot be evaluated because property 'body' cannot be selected. Property selection is not supported on values of type 'String'. Please see https://aka.ms/logicexpressions for usage details.'.​

This is fixed by removing the output of the previous action which is a "HTML to Text Content Conversion" action and adding them back.

Thanks once again

[PerOveSand]

I have some of the same issues.

"DataSources have to be removed and re-added from within the PowerApp, we are using environment variables to store this information. They do not update within the PowerApp. Microsoft have said this is not expected behaviour."

Take a look at you original App, and remove all connections. Then add them again using the Environment variables. Also check you galleries and formulas that reference the old connection names. Update them to use the names of the Environment Variables. (I have a prefix on my Environment variables, like SPList - <name of list> for SharePoint List and SPSite - <name of site>, for SP sites.) Makes it easy to add and check the Power app connections and actions.

"People picker fields are effectively being disabled is also unexpected behaviour, has anyone encountered this before? or any other field that has issues in a deployment."

I have not experienced this, yet...

"Buttons with Power Automate actions need to be removed and added again, the code against the OnSelect needs to be copied out into notepad and pasted back in for it to work."

This was irritating me as well. After I started to create flows in the Solution, this is not an issue anymore.

BUT, there is also a new way to fix it, if you get the problem again. 

In the Power App Settings - Upcoming features - Preview,  you can turn on Enable Power automate pane.

Now when you edit the App, you can remove the connection and re-add it without doing anything with buttons and formulas.

We can't use a SharePoint view from one list to another with the same name. It works based on GUIDs. This means we have to edit the flow to add this in to get it to work. Is there a way around this? We need to get these migrations as seamless as possible.

Not experienced, yet...

We have a content conversion action in a flow, which when it gets migrated, the following error occurs.

Not experienced, yet...

[dave-jorgensen]

My experience (which is a lot) doing the same thing:

• DataSources have to be removed and re-added from within the PowerApp, we are using environment variables to store this information. They do not update within the PowerApp. Microsoft have said this is not expected behaviour.

The solution will be (but not currently) to have the SharePoint lists as environment variables but they don't currently work in PowerApps (but they do in Flows). This is a known issue, that I've had a ticket open for a while on. Current resolution is to indeed delete the datasources and re-add them after deployment.

• People picker fields are effectively being disabled is also unexpected behavior, has anyone encountered this before? or any other field that has issues in a deployment.

I've seen similar behavior within components, where changing the data source sets searchable to false, and I'm pretty sure it's because the field specified in the search doesn't exist when the datasource is in error state. If you don't have thousands of users as your people picker choices, I'd suggest putting the users all in a local collection and then using the local collection as the datasource in the combobox field. Load the users off of the app start.  

• Buttons with Power Automate actions need to be removed and added again, the code against the OnSelect needs to be copied out into notepad and pasted back in for it to work.

If you fix your connection / connectors issue this likely goes away, following earlier suggestions to make sure you are using connection references and not connections <= that is VERY important. But that being said, try fixing the flows before even opening the app to change the datasources. If that still doesn't work, you don't need to delete the button. just delete the flow from the app, add a new button to the app, and re-add the flow (this sets the new button OnSelect to call the flow, instead of overwriting it in your existing button). You can (keeping that new button selected) add all your flows and it will just keep setting the OnSelect there. Then delete the new button you just added.

• We can't use a SharePoint view from one list to another with the same name. It works based on GUIDs. This means we have to edit the flow to add this in to get it to work. Is there a way around this? We need to get these migrations as seamless as possible.

Once the bug with using SharePoint lists and environment variables is fixed, I expect this will work as well. 

• We have a content conversion action in a flow, which when it gets migrated, the following error occurs

Haven't seen that myself.

[PerOveSand]

Update to my earlier post: I have now done a few more exports and imports, with the same solutions I have successfully done before.

NOW they loose their connections. 

So I have to open the imported app, remove and add the connections again! Even if they are Environment Variables AND I have set them during import. 

I even set the values for the Env Vars, in the Default Solution. But still the connections in the app pointed to the wrong SharePoint site (dev site).

I also see that Combo Boxes set the Search settings to false, as mentioned by @dave-jorgensen 

And go through all the Combo boxes and set search to true.

I have also seen that the App is NOT upgraded, when doing an Upgrade import. The version is the same, and import process is all ok. But when opening the App, the Screen that I updated, was like the last version. I had to fix it by deleting the Solution and import again as a new solution.

So I think the ALM is almost useless! Not trustworthy at all!

(Is this a conspiracy to get us all to use DataVerse instead? 😉 )

Have anyone tried using Azure DevOps pipelines instead?

[dave-jorgensen]

RE: I have also seen that the App is NOT upgraded, when doing an Upgrade import. The version is the same, and import process is all ok. But when opening the App, the Screen that I updated, was like the last version. I had to fix it by deleting the Solution and import again as a new solution.

You need to understand the concept of layering. When you deploy a managed solution to an environment, it is the base layer. You then make edits (ie changing the data sources), this creates an unmanaged solution on top of the base layer. NOTE that this unmanaged layer is EVERYTHING. It's not just a "i changed the datasource" but more of "I changed everything to customize the app here", and that top layer takes precedent. Now, when you deploy an upgrade, you are upgrading the BASE layer, but your unmanaged layer remains on top of your base and this is why it looks like the old version. Do a google on removing the unmanaged layers on top of your managed layers. This is what you need to do after an upgrade - you need to remove ALL unmanaged layers. Then, you'll have to redo the remove and add sharepoint datasources (which creates a new unmanaged layer) since you need to remove all unmanaged layers first so that your new base managed solution becomes "the boss" then you remove / re-add the sharepoint datasources for the customizations (ie unmanaged layer) on top of the base (managed) layer)

the key thing to keep in mind is:

1. deploy a managed solution
2. make your edits (such as changing sharepoint data source) which creates an unmanaged layer
3. deploy your changes in a new version of a managed solution
4. delete the unmanages layers in your apps and flows
5. edit the apps / flows as needed for changing data sources and so on.

This is where the environment variables come in, if you use them in your flows (sharepoint site urls, sharepoint list urls) then you don't need to make the edits, since the environment variables don't change per se after a deploy (though environment variables still have bugs), Environment variables for datasources like sharepoint don't work yet in powerapps so you won't get the proper effect using an environment variable as a datasource. Microsoft is working on this bug, but for flows they work great.

[PerOveSand]

Wow @dave-jorgensen! This is really a very good explanation and very helpful!

I did read the docs about the layers, but I certainly haven't perceived that an unmanaged solution layer is added on top of the base layer, if you edit the app in the Standard Solution.

Maybe it is better to just use unmanaged Solutions in both Dev and Prod? Then the upgrade and layering will not be a problem, since unmanaged has no layers?

Anyway. Thank you very much for your insights. 

Must now create a procedure document to fix all the annoying stuff 🙂

[dave-jorgensen]

the layers sitting on top is why you had success deleting the solution and re-installing, since that removes the base managed layers and all the unmanaged layers on top. One of the options when deploying using the classic CRM user interface is to remove all existing customizations (which are your unmanaged layers), you could give that a try.

Anyway, the IDEA with managed solutions is you put your site specific things like urls for data sources (DEV, TEST, PROD) into environment variables and that's what you change when you deploy the first time (and only the first time). You're supposed to NOT be editing apps and flows, so try to think of it that way. What needs to happen to avoid creating the unmanaged layers in the first place.

[Pchan3]

Hi All,

Just an update to, to where we currently stand.

• Recent Power Platform (Apps) Updates have resolved the People Picker field issue.
  • Our Authoring version is: 3.21113.12 We manually updated this for each Canvas App.
  • The App I was working on was about 5 versions behind.
• We believe we are now on the following Power Automate version or later (Oceania):
  • Power Automate Portal Version 2111.1 - Release Notes | Microsoft Docs
  • The Power Automate flow (with Content Conversion) was created before this version, we removed the affected action and added it again. The deployment to Test was successful.
  • We believe when this new version was brought out, it breaks the flow on import.

  • Body dynamic outputs are now generated using @body('actionName') instead of @outputs('actionName')?['body']

• We are using an Environment variable to store the GUID of the List View.
  • This is less than Ideal, but we don't have to edit the flow.
• Adding the Power Automate Pane has helped. this required changing the authoring version to the latest, in order to enable it. Thank you @PerOveSand 

It isn't over yet! we still have issues with Data sources, working with Microsoft to get this sorted. We will have to investigate using DataVerse (maybe it is a conspiracy!). The apps we are developing are some of the first outside of D365, so not too sure on mixing them. 

Thank you all for your suggestions & support!