After a busy week at college and internship, finally I get free time at weekend to write my first article for August 2015. We discuss about some common methods of Selenium module in python today. Selenium is a library used for automated browser testing. However, in this post we discuss about using selenium module in python to make a google Search. The post breaks down into various blocks explaining on how to open a url in the browser via selenium python, search presence of a url in a page, click links present in a page and also open a new tab. These are the necessities to get started with selenium. You may also like to read my article on how to login to a website using selenium python. Starting quickly with no further delay.

Necessities to begin

1. python installed

2. selenium module installed

For Linux:

sudo pip install selenium

For Windows:

pip install selenium

Google search using selenium python

from selenium import webdriver
from selenium.webdriver.comon.keys import Keys
q = raw_input("Enter the search query")
q = q.replace(' ', '')
browser = webdriver.Firefox()
body = browser.find_element_by_tag_name("body")
body.send_keys(Keys.CONTROL + 't')
counter = 0
for i in range(0,20):
    browser.get("https://www.google.com/search?q=" + q + "&start=" + str(counter))
    body = browser.find_element_by_tag_name("body")
    if "thetaranights" in body.text:
        browser.find_element_by_xpath('//a[starts-with(@href,"http://www.thetaranights.com")]').click()
        break
    counter += 10

1. Import statements (Line 1 and 2)

These are the import statements that are required for initiating a browser later in our program and passing url parameters to the address bar in the browser.

2. Get query for google search (Line 3 and 4)

Here, we are taking a query for the google search via raw_input. Here is an example url for a google search which requires the spaces between the words to be replaced by “+” , an additional parameter start=0 is seen which specifies the search result of page 1. Similarly start=10 gives the search result of page 2.

https://www.google.com/search?q=bhishan+bhandari&start=0“

Hence, we after taking the input from the user, we replaces the spaces with +.

3. Instantiate a browser (Line 5)

The statement browser = webdriver.Firefox() opens up a new browser window.

4. Opening a new tab (Line 6 and 7)

These statement opens a new tab. The statement body = browser.find_element_by_tag_name(“body”) is to make sure we actually inside current tab’s body so that we can open a new tab with the combination of keyboard. body.send_keys(Keys.CONTROL + ‘t’) will open a new tab. For Mac replaceing CONTROL with COMMAND should work.

5. Opening a url in the browser (Line 10)

For opening a url in the browser, all you need to do is pass the url as an argument to the browser.get method. Remember I’ve given browser.get because we instantiated the browser earlier with browser = webdriver.Firefox().

6. Searching for a presence of certain url/text in the search result (Line 11 to 15)

Now again we assign the body of the current tab to the variable body. Then we check if “thetaranights.com” is present in the search result. If present, we run the statement browser.find_element_by_xpath(‘//a[starts-with(@href, “http://www.thetaranights.com”]’).click to search for the url in the search result which starts with “http://www.thetaranights.com” at the beginning and anything after it. We then use .click() over it to open the url. Since the result we are looking for is found and clicked. We break the loop. If the earlier statement if “thetaranights.com” in body.text was false meaning not found we would iterate and search for another page of google results and so on until 20 pages.

 Note: You can close the webbrowser with browser.quit()

So, now we know how to open a browser, open a new tab in the browser, go to certain website/url, search for link in the body of the page and click the link. If you have any questions regarding the codes/article, please mention below in the comment section. You may also be interested in my article on How to login to a website using selenium. Happy Coding