Load More Button is Not Working

[CPee]

I've created a flow to allow continuous webscraping when the button "Load More" is available however, it is not working as intended and does not proceed to the next page. Is there a workaround for this? Thank you.

[Deenuji]

@CPee 

How does your 'Path' variable appear? Does it include a URL? If feasible, could you provide a visual representation of the web page where you intend to click the button?

Alternatively, if you have the URL of the button, you can utilize the following "Go to web page" action.

If your URL is dynamic, utilize the following action. Capture your UI element if it contains an Href URL and save it in a variable. Then, pass this variable to the "Go to web page" action as the URL parameter.

Thanks,
Deenuji Loganathan 👩‍💻
Automation Evangelist 🤖
Follow me on LinkedIn 👥

-------------------------------------------------------------------------------------------------------------
If I've helped solve your query, kindly mark my response as the solution ✔ and give it a thumbs up!👍 Your feedback supports future seekers 🚀. If you'd like to appreciate me, please write a LinkedIn recommendation 🙏

[CPee]

Hi @Deenuji ,

This is how the Path variable appears. 

Should I change the click the link to go to webpage instead?

[Deenuji]

@CPee 

Could you please share the screenshot like below as I want to see what are the attributes are available in your element :

Thanks,
Deenuji Loganathan 👩‍💻
Automation Evangelist 🤖
Follow me on LinkedIn 👥

-------------------------------------------------------------------------------------------------------------
If I've helped solve your query, kindly mark my response as the solution ✔ and give it a thumbs up!👍 Your feedback supports future seekers 🚀. If you'd like to appreciate me, please write a LinkedIn recommendation 🙏

[CPee]

@Deenuji this is the screenshot

[Deenuji]

@CPee 

It seems you're attempting to click on the "Load More" button on your webpage. I suggest unchecking the checkbox highlighted in red from (12 to 15) and see if it resolves your issue.

After removing those checkboxes, please test the element using the highlighted test option at the top.

If its not working then we will go with another approach. Pls test and let me know the results.

Thanks,
Deenuji Loganathan 👩‍💻
Automation Evangelist 🤖
Follow me on LinkedIn 👥

-------------------------------------------------------------------------------------------------------------
If I've helped solve your query, kindly mark my response as the solution ✔ and give it a thumbs up!👍 Your feedback supports future seekers 🚀. If you'd like to appreciate me, please write a LinkedIn recommendation 🙏

[CPee]

@Deenuji its not working 😫

[Deenuji]

@CPee 

If it's permissible, could you please provide me with the URL that you're using in your automation? This will allow me to try and assist you in resolving the issue.

Thanks,
Deenuji Loganathan 👩‍💻
Automation Evangelist 🤖
Follow me on LinkedIn 👥

-------------------------------------------------------------------------------------------------------------
If I've helped solve your query, kindly mark my response as the solution ✔ and give it a thumbs up!👍 Your feedback supports future seekers 🚀. If you'd like to appreciate me, please write a LinkedIn recommendation 🙏

[Agnius]

Can you please explain what it is exactly that does not work? The flow does not find the button and exits the loop? Does it click on the button at least once?

[CPee]

I tried extracting prices from this link 

I tried changing the flow see screenshot below and now I'm getting a runtime error found that the url is not found on the Go to web page action.

[CPee]

It's not clicking on the button though at least once. 

[Agnius]

If it's not clicking even once, it means the selector is off. Maybe the button does not actually appear before you scroll down to it, or perhaps it has changed. Have you tried recapturing the button?

[CPee]

@Agnius 

I have edited my flow and now I see that it clicks on the "Load more" button however, it does not capture all the items in the page. It only adds 8 out of 85. Do you know how I can ensure that all 85 items will be captured?

[Deenuji]

@CPee 

Please follow the below approach and see it resolves your issue[Extracted data I stored in variable so you can changed as per your requirement].

Code:

WebAutomation.LaunchChrome.LaunchChrome Url: $'''https://www.hp.com/us-en/shop/vwa/printers/prnttyp=Laser''' WindowState: WebAutomation.BrowserWindowState.Maximized ClearCache: False ClearCookies: False WaitForPageToLoadTimeout: 60 Timeout: 60 PiPUserDataFolderMode: WebAutomation.PiPUserDataFolderModeEnum.AutomaticProfile BrowserInstance=> Browser
LOOP LoopIndex FROM 1 TO 5 STEP 1
    MouseAndKeyboard.SendKeys.FocusAndSendKeysByTitleClass Title: $'''*HP*''' Class: $'''''' TextToSend: $'''{Next}{Next}{Next}{Next}{Next}{Next}''' DelayBetweenKeystrokes: 10 SendTextAsHardwareKeys: True
    WAIT 2
END
WAIT 4
WebAutomation.ExecuteJavascript BrowserInstance: Browser Javascript: $'''function ExecuteScript() { 
document.querySelector(\"#pageContent > div.vwa-page > div.content > div.vwaListContainer > div.vwaList > div.loadMore > button\").click();
}''' Result=> Result
WebAutomation.ExtractData.ExtractTable BrowserInstance: Browser Control: $'''html > body > div:eq(0) > div > main > div:eq(0) > div:eq(2) > div:eq(3) > div:eq(1) > div:eq(1) > div:eq(0) > div''' ExtractionParameters: {[$'''div > div > div > div:eq(1) > a > h3''', $'''Own Text''', $'''''', $'''Value #1'''], [$'''div > div > div > div:eq(3) > div:eq(0) > div > div:eq(1) > div:eq(0) > div''', $'''Own Text''', $'''''', $'''Value #2'''] } PostProcessData: False TimeoutInSeconds: 150 ExtractedData=> DataFromWebPage

How to copy/paste the above code into you PAD?

 
Thanks,
Deenuji Loganathan 👩‍💻
Automation Evangelist 🤖
Follow me on LinkedIn 👥

-------------------------------------------------------------------------------------------------------------
If I've helped solve your query, kindly mark my response as the solution ✔ and give it a thumbs up!👍 Your feedback supports future seekers 🚀

[Agnius]

Does the Extract data from web page include more than 8 items in preview when you have more of them loaded? The results might only load when you scroll through the list and may not be visible. Or it might also be that the results are split into sections that require some modifications to the selector for extraction.

Is the page publicly accessible? Can you share a link?

[CPee]

@Agnius it is showing more than 8 items in preview. Here's the link that I'm trying to extract.

[Deenuji]

@CPee 

Have you tried the above suggested code or approach? Its working fine for me and its extracting all the data and also click load more button too.

Thanks,
Deenuji Loganathan 👩‍💻
Automation Evangelist 🤖
Follow me on LinkedIn 👥

-------------------------------------------------------------------------------------------------------------
If I've helped solve your query, kindly mark my response as the solution ✔ and give it a thumbs up!👍 Your feedback supports future seekers 🚀

[CPee]

@Deenuji Update: I've manually added your flow but it only added all the items in the first page 48 out of 85 items.

I tried following your video in pasting the code but its not showing anything on my PAD, I'll try adding it manually.

[CPee]

@Deenuji I followed your flow and it is working however, it only returned all the details from page 1, a total of 48 out of 85 items.

[Deenuji · Accepted solution]

@CPee 

Challenging Task!! Here's the updated approach and workflow for it.

1. Set up DataTable: Initializes a DataTable named DataTable with columns "ProductName" and "Price".

2. Launch Chrome: Opens a Chrome browser window and navigates to the specified URL.

3. Get Total Pages: Executes JavaScript code to retrieve the total number of pages available for the printer listings.

4. Loop to Navigate Pages: Enters a loop to navigate through the pages of printer listings.

   a. Send Keyboard Inputs: Simulates keyboard inputs to scroll down the page to load additional printer listings.

   b. Extract Table Data: Extracts data from the printer listings table on the current web page.

   c. Merge Data: Merges the extracted data into the DataTable created earlier, adding rows for each printer's name and price.

   d. Check Pagination: Checks if there are more pages to navigate to. If yes, it proceeds to the next page; otherwise, it exits the loop.

5. Display Completion Message: Shows a message dialog indicating that the extraction process is complete.

I've uploaded the revised logic and updated workflow in a zip file. Please download and extract it. Inside, you'll find a text file containing the code. Copy all the code from the text file and paste it into your new Power Automate Desktop flow (Don't enable Power Fx while create your flow).

Thanks,
Deenuji Loganathan 👩‍💻
Automation Evangelist 🤖
Follow me on LinkedIn 👥

-------------------------------------------------------------------------------------------------------------
If I've helped solve your query, kindly mark my response as the solution ✔ and give it a thumbs up!👍 Your feedback supports future seekers 🚀