500 item limit in CDM entity search filter(need to switch to asp.net app if this exists for long)

[AmitLoh-Powerap]

500 item limit in CDM entity search filter, this makes it very dofficult to use for any business scenario(export, data analysis)

because I have 50k records and search filter may return sometimes 5k or 20k and I need to analyze this data(so export)

Currently its only exporting 500 first items which does not meet any business criteria(imagine you are doing google search and it returns only 3 items), sadly if this is permanant issue like sp list 5k limit I will have to inform this to our sponsors of the project and most likely as it does not meet business need to filter and export we will have to do asp.net app which we did not want to do.

I will atleast need some good workaround. One thing I observed is there is export data link in CDM screen(can you give me some workaround based on that?)

[amitnaik]

@pranavnaidu24 This solution won't work for list more than 4000, items correct?

[pranavnaidu24]

@bmassmann 

ID column will not be available under Indexed Columns, please choose Title or any additional columns you may have.

Hope this helps.

[pranavnaidu24]

That's right @amitnaik .

The solution proposed works only up to 4000 records.

Need to tweak, like

ClearCollect(col1, Filter(DataSource, Column >= 2001 && Column <= 4000))

Or using starts with.

[upputuriyamini2]

Hi

I tried with above code but that temporary collection only get 500 records. Doesn't get more than 500. could you please suggest to get more than 500,  

I tried to pull more records to collection but every time collection shows First 500 records only. I doesn't get next 500 records. 

I just simply put RecordID>500 && RecordID<=1000 it pulls only from RecordID 1 to 500 only. i doesn't give next Records.

[pranavnaidu24]

@upputuriyamini2 : Hope you have set the Data row limit to Max of 2000 under Advanced settings

[upputuriyamini2]

Thanks for the information.

In my database i have 7000 records. I would like to search a record whose record id is in between 4500 to 5000.

So with this setting and with the above code, when i try to search records between 4500 to 5000 with a search keyword, it is actually searching in between 1 to 2000 only. 

Is there any way to retrieve records in the specific interval like (4500 to 5000 or 6000 to 6500) ?

[upputuriyamini2]

Hi @pranavnaidu24 

I read the above article which describe the "This solution works only up to 4000 records."

Ok, Fine  about that, at least given solution is work RecordID 500 to RecordID 2000.

my main problem is not getting more than 500 records using the given code(page4 msg 35). and i specify with in that to show like RecordID500 to RecordID 1000 but it doesn't give any record. 

Please give the solution of my problem or any modifications need to this approach I only get 500 records.

Thanks

[rizkyaditya]

Hello @mr-dang,

I have powerapps application and it use Sharepoint List as datasource.
I tried your solution for my need, and it went well except for the third part where the ForAll is used to pull the record.

I got runtime error in the powerapps App checker.

And the error statement is as follow

This is the code that i use, with minor modification for the variable or datasource name

//Collection fetching repetition
UpdateContext({dataPertama: First(DB_Karyawan)});
UpdateContext({dataTerakhir: First(Sort(DB_Karyawan,ID_DB_Karyawan,Descending))});
UpdateContext({maxRun: RoundUp((dataTerakhir.ID_DB_Karyawan - dataPertama.ID_DB_Karyawan)/100,0)});

//Collection ID reference
ClearCollect(Run,Filter(loop100,Number<=maxRun));

//Create collection
Clear(CollectTEST);
ForAll(Run,Collect(CollectTEST,Filter(DB_Karyawan,ID_DB_Karyawan >= dataPertama.ID_DB_Karyawan+nilaiMin && ID_DB_Karyawan < dataPertama.ID_DB_Karyawan+nilaiMax)))

I limit my app for only 100 items., so it repeat the collection 3 times.
Because i only have 229 data in sharepoint list.

Does this problem has anything to do with sharepoint as my datasource?
Please advice.

Thank you!
Rizky Aditya - ID

[m_jeppesen]

How is it possible to get "RecordId" from a CDS record, in PowerApps?

When I'm doing the second formula you've got in your example, I get an error on "RecordId" field

UpdateContext({lastrecord: First(Sort(datasource,RecordId,Descending))});  

[stravers]

I know I'm really late to the party on this but thank you for submitting this process.

This has allowed me to load up a document library from SP with nearly 7000 records. It took a while but worked perfectly.