Extract dynamic Data from WebPage

[Sumitra_Sai]

Hi, I'm currently extracting all the data from a website as some contents change on search of something.

For e.g., in a Food Recipe website, I search for Tomato Rice. There will be more contents, but Ingredients and Instructions will be common for all the recipes. So, I'm extracting all the data as the placeholder of Ingredients and Instructions will be different for each recipe. 

On extraction, it'll come as DataWebPage[0]['result'] and inside that the extracted will be stored as below. But it's failing sometimes as it couldn't find the exact text. But this should happen dynamically.

My task is I need to extract all the data and from that I need to find the "Ingredients" and extract the ingredients part alone. Similarly, I need to find "Instructions" and extract the instructions part alone. After extracting them separately, I need to store them in a variable.

Please help me to resolve this issue,

Thanks in advance for your help.

[Chetan_Ninghot]

Hi,Sumitra_Sai 

Can you share screenshot of you flow and link of this website , its better for more understanding

(Note:- if you got your solution you can mark as solution and gives kudos)

Thanks & Regards

Chetan Ninghot

[Sumitra_Sai]

Hi @Chetan_Ninghot , thanks for your reply

This is the only action I have used for extraction

Besan Ladoo Recipe | Besan Ke Ladoo | Besan Laddu (hebbarskitchen.com)

[Agnius]

Try using Get details of web page and extract the entire page source.

Then, when you have the page source text, you can use Parse text with regular expressions enabled to retrieve the specific values.

You can use the following regular expression to get the ingredients: 

(?<="recipeIngredient":\[)[^\]]*(?=\])

It would look like this:

It will return the ingredients into a variable called %Match%.

You can then use this one for the instructions:

(?<="recipeInstructions":\[.*"text":")[^"]*(?=","name")

The instructions will actually be returned as a list of values, so you need to make sure to set the Parse text action to return all matches, instead of the first occurrence only. The action will look like this:

It will return the instructions into a variable called %Matches% that will be a list of text strings.

So, you might want to use Join text afterwards to join the list into a single string.

-------------------------------------------------------------------------
If I have answered your question, please mark it as the preferred solution. If you like my response, please give it a Thumbs Up.

I also provide paid consultancy and development services using Power Automate. If you're interested, DM me and we can discuss it.

[Sumitra_Sai]

Hi @Agnius , thanks for your help.

I tried the method which you have responded. But the Match is taking as empty.

[Agnius]

Are you sure that you made your Get details of web page action extract the page source specifically, and not just the text?

I've tried it and it works fine for me.

Try copying and pasting the following code to your PAD flow designer to have the actions created for you:

WebAutomation.GetDetailsOfWebPage BrowserInstance: Browser DetailsType: WebAutomation.DetailsType.Source Details=> WebPageProperty
Text.ParseText.RegexParseForFirstOccurrence Text: WebPageProperty TextToFind: $'''(?<=\"recipeIngredient\":\\[)[^\\]]*(?=\\])''' StartingPosition: 0 IgnoreCase: False OccurrencePosition=> Position Match=> Match
Text.ParseText.RegexParse Text: WebPageProperty TextToFind: $'''(?<=\"recipeInstructions\":\\[.*\"text\":\")[^\"]*(?=\",\"name\")''' StartingPosition: 0 IgnoreCase: False OccurrencePositions=> Positions Matches=> Matches

-------------------------------------------------------------------------
If I have answered your question, please mark it as the preferred solution. If you like my response, please give it a Thumbs Up.

I also provide paid consultancy and development services using Power Automate. If you're interested, DM me and we can discuss it.