Query based on record ID - Delegation Workaround

[RyanBarton]

There is a need to query based on record ID if you're creating a relational data model.

However, if you have thousands of records in a given table, your results are going to be incomplete due to the delegation limit. 

Is anyone interested in being able to query any number of records based on record ID to get around the delegation limit of 2000? I have a workaround. Its ridiculous, but it works. 

It will be a long post. Let me know if you're interested and ill write up the workaround using a simple example.

Microsoft,

Please allow us to have the option to turn this limitation off.

[RandyHayes]

@RyanBarton 

What kinds of issues are you facing with delegation in regard to ID?  That is typically one of THE fields that is delegable and safe to use.

[RyanBarton]

So lets say you set your delegation limit to 10 records. Your SharePoint List contains 20,000 records.

Then use Filter(SPListName, ID > 0), which will pertain to all 20,000 records.

This will return the entire SharePoint List?

[RandyHayes]

@RyanBarton 

No, but it will return the first 10 it finds greater than 0.

At the same time, if you set your Formula to Filter(SPListName, ID > 10000) it will return the first 10 it finds greater than 10,000.

ID check can be delegated (as can numeric columns)

We had a recent conversation about methods to pull in complete datasources to avoid delegation issues on non-delegable columns.

In my travels, I had taken the information about delegation on SharePoint sources that only the = operator was delegable on SharePoint.  This was direct from the Connector Documentation which STILL states that the = operator is the only delegable operator for ID.  I was told in message 13 that the other operators are delegable on numerics and the documentation has a mistake.  I have not put this to the test yet though.

So...your original post here mentioned a query based on record ID for a relational data model.  In many cases, you are going to be getting the related record by its ID - in which case looking for ID=13,200 would return that record as it is delegable.  However, a non-delegable operation would only return any match within your record limit settings. 

ex. Filter(SPsource, DateColumn=DateValue("3/6/2019")) would only return as many records as it could find within your limit you set - if you needed the full filter on that column, you would have to pull down the entire list into a collection and perform the filter on that.

[RandyHayes]

Oh, and that delegation warning on the > is apparently a mistake that can be ignored according to my cited other posting messages.

[RyanBarton]

"At the same time, if you set your Formula to Filter(SPListName, ID > 10000) it will return the first 10 it finds greater than 10,000."

I wish this was true.

Setting your delegation limit to 10 means only the first 10 records in the list will be evaluated. Assuming that our list IDs are 1-20,000 and are in order, Filter(SPListName, ID > 10000) would actually return an empty collection, as only records 1-10 are evaluated in this statement.

[RyanBarton]

Am I missing something here? I get the delegation warning on both "ID" and ">"

[RyanBarton]

Well... we are not comparing apples to apples here.

In SharePoint only the "=" operators works. SQL handles the ">" operator.

Thanks for taking the time to chat!

[RandyHayes]

@RyanBarton 

Like I say, I have not vetted that information given to me by @timl yet, but the reference article seems to indicate that vetting

But, I at least know you could get ID = 12000 if looking for it and the limit was 10

And again, going back to the original statement - looking for a relational record.  Wouldn't you be doing that by an ID value?  That *should* be doable.

But I believe we're confusing two different aspects here perhaps - delegation and record limits.  These are two different things.  They often go together because delegation becomes an issue after you exceed your record limit.  But, they, by themselves are not the same issue.

So - record limit set to 10.  Filter(... ID>0) will only bring 10 records (thats your limit).  Filter(... ID>10000) will bring 10 records, but it didn't "stop looking" after it hit record 10.  Instead the query was delegated to the DataSource and it did the search and returned the limit.

[RandyHayes · Accepted solution]

@RyanBarton 

Please take a look at the referenced posting.  The poster there had done this test on the delegation of < > and others and found that they DO delegate on ShaerPoint despite the documentation. 

ALTHOUGH - after re-reading, I seem to get the indiciation that the poster did not, in the end, find that these were delegable...not sure.  @timl you mentioned that the other operators on SharePoint numerics such as ID would delegate?  Did I misunderstand that?

At any rate @RyanBarton ...  I should have mentioned this in the very beginning...YES, very interested to hear your solution.  If you think it is more relevant to the other thread, maybe you would post it there instead.  We've got some pretty big messages postings in that one as well.

I'm ALWAYS interested in ways people are finding to overcome the limits.

Thanks for sharing.