Generating test data to demonstrate performance and UX

[mhomol]

I have a request to show off performance and the user experience within the Dataverse database we've designed and the accompanying model-driven app with at least 50,000 records in at least one of the tables.  What approach would you guys recommend?  I can think of a way I could leverage the ChatGPT API and a Power Automate Cloud Flow to do this, but figured I'd check here before building something from scratch.

[a33ik]

Hello,

I would go with SSIS and KingsWay soft + Data Spawner datasource - https://www.kingswaysoft.com/products/ssis-productivity-pack/help-manual/data-generation-and-anonymi...

[gulshankhurana]

Hi @mhomol 

Mockaroo Data Munger tool in xrmtoolbox could be another option. Here is the link:

Mockaroo Data Munger  · XrmToolBox

For generating over 1000 contacts a day using Mockaroo, you'd need to go for a paid-plan. Here's the link for their pricing options:
Mockaroo - Random Data Generator and API Mocking Tool | JSON / CSV / SQL / Excel

I hope this information helps.

If you appreciate my efforts then please give my reply a Thumbs Up. If it helps resolve your query then please accept it as a solution.

Kind regards

Gulshan

[dpoggemann · Accepted solution]

Hi @mhomol ,

Are you looking for a way to load the 50K records or are you looking for a way to test and report on the performance of the executions?  You could leverage Application Insights to detail the telemetry data of your Power App (https://learn.microsoft.com/en-us/power-platform/admin/analyze-telemetry) 

If you are looking to load the data, I would go old school and use the data import template to load the records through Excel.  You can then randomize your data in Excel any way you want etc. and then run the import in batches of around 5K records at a time.  I would expect this would not take much time and provide a lot of flexibility in loading your data.  https://learn.microsoft.com/en-us/power-platform/admin/download-template-data-import 

[ChrisPiasecki]

My preferred approach is also what @a33ik suggests if you have some development experience. Kingswaysoft Integration Toolkit for SSIS is free if you're running it on your development machine. Great for on-demand test data generation.

Otherwise, generating the data yourself in Excel and importing it is the no-code approach.

---
Please click Accept as Solution if my post answered your question. This will help others find solutions to similar questions. If you like my post and/or find it helpful, please consider giving it a Thumbs Up.

[mhomol]

Really appreciate the ideas.  The performance aspect, in this case, is more from the customer's perspective of how the model-driven apps will perform when they know there's tons of data in it.  They are new to Dataverse so there's low trust.  But I do love the suggestion of making sure that telemetry is also being layered in for them, so I'm definitely going to add that to the backlog.  🙂

[mhomol]

Thank you.  I have never used this tool before, so will take a look and give it a whirl.  Seems pretty awesome though.

[mhomol]

I get an error trying to upload the file that Microsoft gives me - even for a file as simple as something that just had name and owner in it.  I used emails for the owners.  Is this a known issue or am I just doing something boneheaded?

[dpoggemann]

Hi @mhomol ,

Owner field needs to be the full name field of the user in the SystemUsers table in Dataverse, not an email address.  That will then automatically find the user and relate the owner to that value.  If you don't pass in Owner, it will default to you being the owner of the record.