Power App export in csv or excel a large dataset of 18000 elements

[dignus93reply]

Hello flowers,

The request is simple, the goal is hard. From power app we have to click on a button and download a big dataverse table of more than 18000 elements. This is a renewal of an older project made in sharepoint 2013 and some other technology i dont know. I know for sure that it was based on sql, and the download was taking 10 to 30 seconds max. Do you think it is possible to get this results in power apps, power automate, dataverse?

For now we try in the beginning with this code component:
https://github.com/yashag2255/exportT...
Nice and fast, but the max rows downloaded are 2000 (the delegation limit you set in powerapps.

So we try with a flow and an async responser. The user click download a flow start and after sometime a csv is loaded on sharepoint and given to the users. Pretty slow but effective, and the user dont have to wait until the finish of the download. He can go around the app and when it is finished a file is given.

Now we would like to go on with the flow approach, we are trying to make it faster. So we are trying to avoid every apply to each.
We have a list row action, a filter array, a parse json, a create csv table. 
It is good, but we have to filter this data a bit and we are trying to do with a fetch xml command. If you use fetch xml the list rows will give you only 5000 rows. So we are using this kind of approach to get all rows:
https://www.youtube.com/watch?v=TWJ7liMse34

There is a do until and inside a list rows, we paginate the data and get 5000 rows at a time. Now the problem is that at every run of do until we create a csv table... We should combine this csv table together... In a fast and optimized way.

It is a long post and will keep this update with screenshot and other stuff. It is a working in progress. If you have some suggestion feel free to teel me. The request is simple the goal is hard. Dataverse, powerapps, power automate are not amazing when it comes to big data handling, but a lot of new stuff are coming. I hope you can help me.

Ciao,
Alessandro

[leo85]

Hi Alessandro,

this is a long post and by the end it is not really clear what you actually want.

Could you please very briefly

- post a screenshot of your current flow

- describe what you currently have

- state what you want to make better.

We have a flow, which is manually triggered (1), gets 30.000 items from Dataverse (2), puts it in a CSV file (3) and saves it to SharePoint (4). This process takes about 30 seconds in total. It has got 4 actions.

[takolota]

Anyone looking to do something like this may find these two resources helpful...

Quickly export up to 100,000 SharePoint list records to a CSV action: https://tomriha.com/quick-way-to-export-sharepoint-list-to-excel-file-using-power-automate/

Quickly convert most CSV files to Excel: https://powerusers.microsoft.com/t5/Power-Automate-Cookbook/CSV-To-New-Excel-Table/m-p/1826096#M964