chromedp click doesn't work in my golang code. Can you identify the problem?

php editor strawberry, hello! Regarding the problem you mentioned, if chromedp click does not work in your golang code, I can help you find out the problem. chromedp is a library that uses Chrome DevTools Protocol for automation. The click method is used to simulate mouse click events. Possible problems include: 1. The page element is invisible or blocked by other elements, causing the click to fail; 2. The parameters of the click method are passed incorrectly; 3. The version of chromedp is incompatible with the Chrome browser version; 4. Other code logic issues . Please provide more details and I will give a solution as soon as possible.

Question content

I am using chromedp to develop scraper.

To get the content I want (the page html) I have to click on a specific button.

So I used chromedp.click and chromedp.outerhtml, but I only got the html of the page before the click, not the html of the page after the click was completed.

Can you see my code and suggest me how to fix it?

func runCrawler(URL string, lineNum string, stationNm string) {
    
        // settings for crawling
    opts := append(chromedp.DefaultExecAllocatorOptions[:],
        chromedp.Flag("headless", false))
    
        // create chrome instance
    contextVar, cancelFunc := chromedp.NewExecAllocator(context.Background(), opts...)
    defer cancelFunc()

    contextVar, cancelFunc = chromedp.NewContext(contextVar)
    defer cancelFunc()


    var htmlContent string

    err := chromedp.Run(contextVar,
        chromedp.Navigate(URL),
        chromedp.WaitVisible(".end_footer_area"),
        chromedp.Click(".end_section.station_info_section > div.at_end.sofzqce > div > div.c10jv2ep.wrap_btn_schedule.schedule_time > button"),
        chromedp.OuterHTML("html", &htmlContent, chromedp.ByQuery),
    )
    fmt.Println("html", htmlContent)
    checkErr(err)

I will give you the homepage and the buttons I need to click

Page URL: https://pts.map.naver.com/end-subway/ends/web/11321/home

The button area I need to click:

Thank you very much

Solution

The page you want to get has been opened in a new tab (target).

In this case we can use chromedp.waitnewtarget to create a chan from which we can receive the target id of the new tab. Then create a new context using the chromedp.withtargetid option so we can connect to the new tab. From here on, everything is familiar to you.

package main

import (
    "context"
    "fmt"
    "strings"

    "github.com/chromedp/cdproto/target"
    "github.com/chromedp/chromedp"
)

func main() {
    opts := append(chromedp.DefaultExecAllocatorOptions[:],
        chromedp.Flag("headless", false),
    )

    ctx, cancel := chromedp.NewExecAllocator(context.Background(), opts...)
    defer cancel()

    ctx, cancel = chromedp.NewContext(ctx)
    defer cancel()

    var htmlContent string

    ch := chromedp.WaitNewTarget(ctx, func(i *target.Info) bool {
        return strings.Contains(i.URL, "/timetable/web/")
    })

    err := chromedp.Run(ctx,
        chromedp.Navigate("https://pts.map.naver.com/end-subway/ends/web/11321/home"),
        chromedp.WaitVisible(".end_footer_area"),
        chromedp.Click(".end_section.station_info_section > div.at_end.sofzqce > div > div.c10jv2ep.wrap_btn_schedule.schedule_time > button"),
    )
    if err != nil {
        panic(err)
    }

    newCtx, cancel := chromedp.NewContext(ctx, chromedp.WithTargetID(<-ch))
    defer cancel()

    if err := chromedp.Run(newCtx,
        chromedp.WaitReady(".table_schedule", chromedp.ByQuery),
        chromedp.OuterHTML("html", &htmlContent, chromedp.ByQuery),
    ); err != nil {
        panic(err)
    }
    fmt.Println("html", htmlContent)
}

The above is the detailed content of chromedp click doesn't work in my golang code. Can you identify the problem?. For more information, please follow other related articles on the PHP Chinese website!