Find only the latest unique item in a SharePoint List

[fnanfne]

Hi all

As per subject, I've been trying to accomplish this for quite a while now using collections in PowerApps but have yet to find a solution. I was hoping PowerAutomate would have an intuitive one but not having much luck. The below pic illustrates what I want to achieve.

The problem is that I can't use 'normal' methods like Distinct, as the duplicates I want to remove (or not get) are not distinct but unique. The list has dozens of columns with unique values but only the Item (text type) and Date (date type) columns are relevant.

Simply put, how can one remove the three oldest Belt entries in the example above?

Using the Get-Items action and a Filter Query, I'm able to get all the items I want within a specific date range, and also based on other parameters like the number column, but I can't figure out how to subsequently (or in the same step?) filter the Item column to only have one unique item in the resulting list.

Any help appreciated.

[Matthy79]

Hello @fnanfne 

you could sort the array and after that use „first“ or „last“ to get your item.

[SamLed]

Hi fnanfne,

I understood you want to get the latest record for each unique item, here how I would do it:

• My test input data:

•  Create an array variable to store output records, Get your rows, Add a Select action to fetch all "Item" and use union() expression to get unique value from Select:

• Add an Apply to each to loop over unique "Item", Filter Get rows outputs to fetch every record matching current "Item", Select "Date" converted to int by format 'yyyyMMdd' from filter output, return max number and convert it back to string:

NB: I saw date in format dd/MM/yyyy, I first parse that date using 'fr-FR' locale to get ISO format date. Then format it to yyyyMMdd and finally into int: 

int(formatDateTime(parseDateTime(item()?['Date'],'fr-FR','dd/MM/yyyy'), 'yyyyMMdd'))

• Add a compose to rebuild date from int to input "Date" format, filter Get rows outputs again to now fetch record matching "Item" and newer "Date", finally, append record to array variable. I use first() because filter array returns an array, I need only the item it contains:

NB:

Expression to rebuild date:  

replace(replace(replace('dd/MM/yyyy', 'yyyy', substring(outputs('Compose_-_get_max_date_int_value'),0,4)), 'MM', substring(outputs('Compose_-_get_max_date_int_value'),4,2)), 'dd', substring(outputs('Compose_-_get_max_date_int_value'),6,2))

Expression to filter array: @And(equals(item()?['Item'], items('Apply_to_each')), equals(item()?['Date'], outputs('Rebuild_date')))

Result :

______________________________________________________________

If I have answered your question, please Accept the post as solution.
If you like my response, please Thumbs Up.

[SamLed]

Hi,

Don't forget to mark as solution if it fulfilled your need in order to close the topic 😉

[Chriddle]

My idea is the following:

• Create an array from the Item col and make its items unique (in the Apply to each)
• Sort the rows by date reversed (newest on top, in the Filter's From)
• Filter the rows by Item
• In the nested Apply to each 2, skip each first row and delete all other rows.

Apply to each:

union(
	xpath(
		xml(json(concat('{"root":{"item":', outputs('Compose'),'}}'))),
		'//Item/text()'
	),
	json('[]')
)

Filter array:

From:

reverse(sort(outputs('Compose'),'Date'))

Filter:

@equals(item()['Item'], items('Apply_to_each'))

Apply to each 2:

skip(body('Filter_array'),1)

Compose 2:

This shows the row that must be deleted

This action has to be replaced with an according Delete row action 😉

items('Apply_to_each_2')

[Matthy79]

@fnanfne 

please tell us if you are happy with one of the solutions. If you provide sample data, I will also provide a solution that doesn’t need loops and can handle large data as well.

[SamLed]

Hi @fnanfne ,

I have to admit that the solution provided by @Chriddle  is really elegant. I especially like the use of XPath to get unique item and reverse sorting to get most recent one. That's saving so many actions. Here's what my solution becomes inspired by his work:

I used same input data and got same results, but look how it lost weight !

My solution returns an array of newest of each unique items, @Chriddle's one offers the ability to delete all items except newest. Whichever solution you prefer, believe me, @Chriddle's one deserves to be mark as solution.

Sam

[Matthy79 · Accepted solution]

Hello all,

We often have the problem to reach the Power Automate limits and runtime is also a pain point so I always try to avoid apply to each whenever it is possible.

This is how I would do it.

[SamLed]

@Matthy79 

Hehe the ultimate one I like to see, loop less, great job!

For Number and Date in select you're taking last of array sort by date, I guess?

You deserve your Kudo 😁

You both inspired me, nice topic, on my way to upgrade my flows!

[Matthy79]

The sort by date is during the xml conversion. After that it is just picking the last found item using xpath. You can see the formulas inside the note. The Number Select is the same es the Date Select. Just change Date to Number.

[SamLed]

Oh, of course the notes... get it, many thanks.

[Chriddle]

I like flow golf 😁

This saves the final Compose action in @Matthy79 's great suggestion:

From:

union(
	xpath(
		xml(json(concat('{"root":{"item":', outputs('Compose'),'}}'))),
		'//Item/text()'
	),
	json('[]')
)

Map Id:

last(
	xpath(
		xml(json(concat('{"root":{"item":', sort(outputs('Compose'), 'Date'),'}}'))),
		concat('/root/item[Item=''',item(),''']/Id/text()')
	)
)

Map Item:

item()

Map Date:

last(
	xpath(
		xml(json(concat('{"root":{"item":', sort(outputs('Compose'), 'Date'),'}}'))),
		concat('/root/item[Item=''',item(),''']/Date/text()')
	)
)

Map Number:

last(
	xpath(
		xml(json(concat('{"root":{"item":', sort(outputs('Compose'), 'Date'),'}}'))),
		concat('/root/item[Item=''',item(),''']/Number/text()')
	)
)

To defend my previous suggestion:
The external Apply to each can be avoided, but the internal one is due to the fact that a Delete row should take place anyway if you assume that the Excel file will be cleaned.

[takolota]

@fnanfne 

There’s also this template that will get distinct records for any given column criteria without any loops & without any xpath:

https://powerusers.microsoft.com/t5/Power-Automate-Cookbook/Get-Distinct-Records/td-p/2191533

If alternatively you want to remove/delete the duplicates, here is the companion Remove Duplicates template flow with a similar set-up: https://powerusers.microsoft.com/t5/Power-Automate-Cookbook/Find-and-Remove-Duplicates/td-p/2191403

[fnanfne]

@Matthy79 @takolota @Chriddle and @SamLed 

Apologies for my belated reply, I have been on annual leave. I'm still catching up on on everything at work and have not given this another look, will do soon. However, I'm appreciative of all your helpful and detailed replies, thank you very much!