How to Use Headless Chrome in Golang with Godet

So many times it happens when we have to test a few cases at the time of development to ensure it is the same as provided on the website. It also happens when we have to fill up the lengthy, tedious, and monotonous form. I encountered this problem while filling up the contact us form at the time of enquiring about a car.

They ask too many questions which the customer had already filled up previously in the application for other dealers. If you have the same data in the database, it’s so easy and convenient to fill the form as I had that same data in my database. I was able to make the form fill up faster by opening up a golang headless chrome instance in my server. And then, I used Golang and godet to access the website, fill-up the way and get the contract printed as a pdf on my server saving the user an ample amount of time with the tedious process of filling up the form. So let me tell you how to do it.

The Procedure

Pre-installed:

• Golang
• chromium-browser

We will be going to navigate from Google.com to search some text and then screenshot and print the very first link of the search.

First, let us start by creating a primary go code that just prints out, “Hey All.”

So create a file that satisfies you GOPATH.

FileName: main.go

package main
 
import (
    "log"
)
 
func main() {
    log.Println("Hey All!")
}

Now let us open up chromium-browser by code and kill the process after waiting 2 minutes.

package main
 
import (
    "log"
    "os/exec"
    "time"
)
 
func main() {
    chromeapp := "chromium-browser"
    chromeappArg := []string{"--headless", "--hide-scrollbars", "--remote-debugging-port=9222", "--disable-gpu", "--allow-insecure-localhost"}
    cmd := exec.Command(chromeapp, chromeappArg...)
    err := cmd.Start()
    if err != nil {
        log.Println("cannot start browser", err)
    }
 
    time.Sleep(2 * time.Minute)
 
    killapp := "kill"
    killappArg := []string{"$(lsof -t -i:9222)"}
    cmd = exec.Command(killapp, killappArg...)
    err = cmd.Start()
    if err != nil {
        log.Println("cannot kill processes", err)
    }
}

Now in the 30-second sleep, if you open up another terminal and run lsof -i:9222, this will show you that the chromium-browser has now started and the dev-tools are running on port 9222.

You can see the tabs of the chromium-browser we created on localhost:9222

If you click on the about: blank, you’ll be able to see the process that we do on the opened tab in our chromium-browser.

Before connecting to chromium, we will need to install godet.

So on your terminal run, go get “github.com/raff/godet”.

Do you need assistance to solve your Golang error?
Hire Golang developer from us to fix the bugs and fine-tune your Golang app user experience.

This package creates a small layer between the google chrome dev tools and Golang.

Now we will navigate to www.google.com to do so add the following code.

package main
 
import (
    "log"
    "os/exec"
    "time"
 
    "github.com/raff/godet"
)
 
func main() {
    chromeapp := "chromium-browser"
    chromeappArg := []string{"--headless", "--hide-scrollbars", "--remote-debugging-port=9222", "--disable-gpu", "--allow-insecure-localhost"}
    cmd := exec.Command(chromeapp, chromeappArg...)
    err := cmd.Start()
    if err != nil {
        log.Println("cannot start browser", err)
    }
 
    // Will wait for chromium to start
    time.Sleep(5 * time.Second)
 
    // connect to Chromium instance
    remote, err := godet.Connect("localhost:9222", true)
    if err != nil {
        log.Println("cannot connect to Chrome instance:", err)
        return
    }
 
    // disconnect when done
    defer remote.Close()
 
    remote.PageEvents(true)
    remote.DOMEvents(true)
 
    _, err = remote.Navigate("https://www.google.com")
    if err != nil {
        log.Println("cannot connect to Chrome instance:", err)
        return
    }
 
    time.Sleep(30 * time.Second)
 
    killapp := "kill"
    killappArg := []string{"$(lsof -t -i:9222)"}
    cmd = exec.Command(killapp, killappArg...)
    err = cmd.Start()
 
    if err != nil {
        log.Println("cannot kill processes", err)
    }
}

The remote, err:= godet.Connect(“localhost:9222”, true) will connect to the chromium-browser and the line defer remote.Close() will close the connection on the full execution of our code. This is preferable to do so that we can prevent memory leaks.

The remote.PageEvents(true) remote.DOMEvents(true) will log all the PageEvents and DomEvents on our terminal.

_, err = remote.Navigate(“https://www.google.com“) this is the magic code that navigates the tab open in chromium-tab to www.google.com.

Now we are going to use the EvaluateWrap function to inject JS in the browser.

For more such functions, you can refer to https://github.com/raff/godet.

We will count the times that the page stops loading and inject js based on it.

package main
 
import (
    "log"
    "os/exec"
    "time"
 
    "github.com/raff/godet"
)
 
func main() {
    chromeapp := "chromium-browser"
    chromeappArg := []string{"--headless", "--hide-scrollbars", "--remote-debugging-port=9222", "--disable-gpu", "--allow-insecure-localhost"}
    cmd := exec.Command(chromeapp, chromeappArg...)
    err := cmd.Start()
    if err != nil {
        log.Println("cannot start browser", err)
    }
 
    // Will wait for chromium to start
    time.Sleep(5 * time.Second)
 
    // connect to Chromium instance
    remote, err := godet.Connect("localhost:9222", true)
    if err != nil {
        log.Println("cannot connect to Chrome instance:", err)
        return
    }
 
    // disconnect when done
    defer remote.Close()
 
    remote.PageEvents(true)
    remote.DOMEvents(true)
 
    _, err = remote.Navigate("https://www.google.com")
    if err != nil {
        log.Println("cannot connect to Chrome instance:", err)
        return
    }
 
    count := 0
 
    remote.CallbackEvent("Page.frameStoppedLoading", func(params godet.Params) {
        count = count + 1
        log.Println("TCL: remote.CallbackEvent -> count", count)
        switch count {
        case 1:
            _, err = remote.EvaluateWrap(`
                document.getElementsByClassName('gLFyf gsfi')[0].value = "Bacancy Technology"
                document.getElementsByClassName('gNO89b')[1].click()
            `)
            if err != nil {
                log.Println("Error executing js block 1:", err)
                return
            }
            time.Sleep(5 * time.Second)
        case 2:
            _, err = remote.EvaluateWrap(`
                document.getElementsByClassName('LC20lb')[0].click()
            `)
            if err != nil {
                log.Println("Error executing js block 1:", err)
                return
            }
            time.Sleep(5 * time.Second)
            // take a screenshot
            _ = remote.SaveScreenshot("SearchResultScreenshot.png", 0644, 0, true)
                // print a pdf
                err = remote.SavePDF(`./SearchResultPdf.pdf`, 0644)
                if err != nil {
                    log.Println("Error saving pdf.")
                }
            }
    })

    time.Sleep(30 * time.Second)
 
    killapp := "kill"
    killappArg := []string{"$(lsof -t -i:9222)"}
    cmd = exec.Command(killapp, killappArg...)
    err = cmd.Start()
    if err != nil {
        log.Println("cannot kill processes", err)
    }
}

Explaining what we have done based on cases

Case 1: Fill up the Google search box with “Bacancy Technology and clicking the search button.
Case 2: Clicking the first link and screenshotting and printing the link opened

When you run go run main.go after navigating to the file in your terminal, after 30 seconds, you’ll be able to see 2 files generated in the same directory.

I’ll attach the screenshot for the look of the browser(the one in localhost:9222), the screenshot created, and the pdf generated.

Browser:

Screenshot: