Loop through multiple links on web page and get details

[awadheshyona]

Hi All,

I am new to PowerAutomate Desktop and I want to know how can I achieve this.

1. Open web browser(chrome) and paste a URL.

2. This URL has multiple links which open in a different page if clicked.

3. Click link and extract some information from each clicked link.

I am able to do all of the above process with suggestions provided by @Highboy , however only for one of the first link(or any link) on that URL. I want to repeat this process for all links available on that URL and grab all information. I tried Web recorder also but it does not go back to the URL and opens second link and so on. I tried adding it one by one and it works however number of links on the URL(web page) will keep on changing daily.

Please suggest. Let me know if you need further details.

@Highboy  Can you please help.

Regards,

Awadhesh

[Highboy · Accepted solution]

I am not sure i understand what you are trying to do.
Could you attach a screenshot of the webpage you are working with?

But this could work

Use "Extract data from web page" to extract all the links, and loop throuh all of them - no need to go back to first page.

[awadheshyona]

That is really helpful @Highboy  This is what I was trying to achieve and it worked. Thanks once again.

[BzshBzsh]

Hi, @awadheshyona @Highboy 

I am trying to do the same thing or at least something very similar. I have a web page with 116 links on it. Each link takes me to a contact page where I need the Name, Phone, Fax and Address. I would like all the results to show up in 1 excel sheet with 116 rows. Using the example, I was able to save the links, have the webpage load and have the first link spit out the right data, then it loops, opens the next link but spits out a blank excel document. What am I missing? Also, I'm not sure if this matters but some of the links don't have a fax number.

[BzshBzsh]

I got it figured out!

I needed to save my extracted data as a variable and then use that variable to write it to an excel sheet.

Here is what my loop looks like.

[mhenderson95]

Would you be able to post a screenshot of the entire flow you put together? I am trying to do something very similar and keep hitting a wall!

[BzshBzsh]

You may not need all the excel steps and you could simply launch excel before your loop starts. I have mine set up to copy the current data from sheet 1, paste it into sheet 2, erase the data from sheet 1 and then paste the new information into sheet 1. I then have conditional formatting set in my excel to check for differences between the 2 sheets. This may not be the best way to have it set up but it seems to work for me. Hopefully you have luck too.

[Powerlearnerd]

Hi All,

I am new to PowerAutomate Desktop and I want to know how can I achieve this.

1. Open web browser(chrome) and paste a URL.

2. This URL has multiple links which open in a different page if clicked.

3. Click link and extract some information from each clicked link.

4. Then store all information in one excel file

I am able to do all of the three above processes, however with the fourth step I struggle as it is then opening all the results in different new excel files. I want the information to be stored in only one excel file. Does somebody know the answer to this problem.

Please suggest. Let me know if you need further details.

[Powerlearnerd]

Hi Highboy,

Could you help me out with a similar sturggle cause I'm desperately looking for an answer. I also would like to loop thourgh multiple links on a web page and get the details. I want to enter a product name in www.exporthub.com and then webscrape all information of indivual webpages from suppliers. Hope you could help me out.

[BzshBzsh]

I had a similar problem and solved it above. When you are extracting data in your loop you need to save it as a variable not excel. Then use "get first free row" then "Write to excel worksheet". At this step you will pick your variable that you saved. Then finally "Increase Variable" by 1. Check my screenshot and let me know if it helps you.

[Powerlearnerd]

Hi @BzshBzsh, 

I very much appreciate your reply. 

I noticed your prior contributions before but still I can't manage to get the right results. I included a screenshot of the whole flow I made so you can see what is lacking. 

I want to scrape from these engineered wood suppliers on Export Hub 

for each of them their - Business nature, - Country region etc. when you click on their link.

Could you indicate where in my flow there is a mistake. Much appreciation in advance.

[BzshBzsh]

Try changing your last step to your extracted links variable and add row1 to the end.

[BzshBzsh]

Try changing your last step to your extracted links variable and add row1 to the end.

[Powerlearnerd]

Thank you very much @bszetey  this helped me a lot. 

This is how I finally got the data in Excel. 

Instead of having the information summed up in one column, I'd like to transpose each item with information into a row. So you will have all manufactuers in one column and number of people in the next column etc. Is there an easy way to do this with PowerAutomate. Thanks again in advance. 

[foxngn84]

May i know when you say extracting data & save as variable. I assumed under your screenshot will be "DataFromWebPage" right? But when you Increase variable, I see the variable name is "ExtractedLinksrow1" which doesn't seems to be the same one?

May I know where does the ExtractedLinksrow 1 came from? 

I'm trying to do something similiar with dynamic link, but it keep failed to loop the following link & it's only get the first link. 

Each of the view po below is dynamic, and each time the amount of the link shown won't be the same amount. 

[BzshBzsh]

If you view my full flow screenshot from 02-11-2022 you will see that I extract data twice. The first time is outside of the loop where I pull all the links from the website and save it as extractedlinks. Then inside the loop I am pulling the data I want from each of those links. So it goes to the first link, takes the data, writes to excel then increasing the variable by one goes to the next link and repeats the process. I hope this helps!

[Highboy]

Sorry m8, have been busy, and haven't been here for a while.

But I am glad that you found a solution .

[Mohammed_Ishaq]

Hi All,

Is there any possibility to achieve this requirement which is discussed here in power automate cloud flow??
Any suggestion would be helpful at least until what point we can achieve.

[shivani2]

Hi @Highboy 

I want to download the files by clicking from the multiple links on webpage. 

I am hoping it is the same thing as mentioned in the post. But i am stuck after doing the step of extracting links from the webpage. how do i proceed further. from downloading the files from these links. 

[shivani2]

Would you be able to give a proper instruction for this. I am trying to do something similar to this, but i am struggling to do it. 

My problem statement is defined in steps below:

1. I am logging into the website.

2. Go to the proper section in website from where i have to download multiple excels. 

3. Click on links for downloading the excels. 

Implemented Solution:

1. I was able to login to the website. 

2. Was able to go the appropriate section in the website. 

3. Stored all the links on the website in the excel. 

Problem: When i apply the loop in the extracted links, it is downloading the excel from the same link again and again. 

I want to download different excels from the extracted links. I am attaching my sequence of the flow. Please let me know where i am going wrong.