Extract from web page does not extract all of the data

[Anonymous]

Hi

I'm posting this to the Desktop board. Hopefully someone on this board can assist.

I am trying to scrape data from LinkedIn's Talent Hub.

However, only the first 4 people out of a page of 25 are extracted.

The issue happens when the Live Web Helper tool comes up and I right click on the “elements” to extract them, some elements are "auto-detected" but then after the auto-detect fails, even though I right-click to select the similar element and where after it does continue to recognise the other similar elements.

However, those people after the fail are not extracted (see my annotated screenshots below).

Any assistance will be greatly appreciated.

Thanks, Mark.

[Koen5]

Dear Gemtech,

From what you describe I cannot figure out what the issue is/could be.

Maybe as an alternative, would it be an option to try and cycle through the various Talent records via URL? As such you would open a Talent profile per page and would possibly remove any 'confusion' the flow might have in a page with a complete listing of all Talent profiles.

Kind regards,

Koen

[Anonymous]

Hi @Koen5 

Thanks for your suggestion but as you can see below I still have a problem.

Am I missing something?

Your assistance is greatly appreciated. 

Thanks, Mark.

[Ankesh_49]

@Anonymous Could you please try it some other browser MS Edge or FF

[VJR]

Each of the names may not be under the same html structure.

Check the html of each of the name element, paste it in notepad one below the other and see how it is changing from one to another.

Apply a similar type of solution suggested by @Pavel_NaNoi in the last post here.

Also check how "contains" is used when some price items are coming out blank in this video.

[Anonymous]

Hi @Ankesh_49 Thanks for your suggestion. 

I tried identifying each persons tags rather than each person's URL.

Both Chrome and Edge extract just the first 3 people and stops after that (see screenshot below).

Thanks for your suggestion.

Regards, Mark.

[Anonymous]

Hi @VJR 

Many thanks for your suggestions.

I tried @Pavel_NaNoi 's suggestions but wasn't able to make progress.

Below is a series of annotated screen shots of my testing.

Your advice on this will be appreciated.
I'll also follow the video you kindly shared and will post my results once I have my mind around what that solution involves.

Your assistance is greatly appreciated.

Thanks, Mark.

[VJR]

Hi @Anonymous 

Not to do the same as mentioned in the other post, but make changes relevant to your website only.

Were you able to do this in Notepad and what did you come up with? That could tell you where to make changes and what.

Check the html of each of the name element, paste it in notepad one below the other and see how it is changing from one to another.

[Ankesh_49]

@Anonymous Could you please check this video: https://youtu.be/AuweGc379Vg for reference.

[Anonymous]

Hi @Ankesh_49 Many thanks for sharing that video. I now understand what you and @VJR have been trying to explain about creating a dynamic variable and why @VJR was telling me to paste to Notebook to see the differences.

I'll post an update when I've got my Flow working using my newly-gained knowledge 😁

You guys have been VERY helpful.
Many thanks 👍

[Ankesh_49]

@Anonymous Great news!! Looking forward to your update.

Cheers,

Ankesh

--------------------------------

If this post helps answer your question, please click on “Accept as Solution” to help other members find it more quickly. If you thought this post was helpful, please give it a Thumbs Up.

[Anonymous]

Hi @VJR and @Ankesh_49 

I followed the video and was able to identify the elements (see below).

I now need to create the dynamic variable for extracting all links but am stuck at the point explained in the screenshots below.

I look forward to your advice.

Many thanks, Mark.

[VJR]

Here is a post that can help you in making a selector dynamic and adding variables into the selectors.

https://powerusers.microsoft.com/t5/Power-Automate-Desktop/Dynamic-Selector-variable-in-selector-in-...

[Anonymous]

Many thanks for your prompt response @VJR 

That solution will let me modify the selector but how do I extract the initial 17886 value that I need to increment and then customise using the %myVar% ?

[VJR]

Option 1: if you are sure that it always starts from 17886 then you can set this value in a variable and increment it.

Option 2: If not then,

From the above screenshot it looks like you already have the UI element.

Get the "id" value of that element using the below action which will return you 17886.

[Anonymous · Accepted solution]

Hi @VJR and @Ankesh_49 

I gave up on the approach I was using and after watching the below video my other approach to creating my flow is working out.

Many thanks indeed for your kind support.

7 Tips to build better selector in Microsoft Power Automate Desktop - YouTube