Practical Examples of Using Selenium

Selenium is commonly used for automating repetitive web tasks and testing web applications.

In this lesson, we will explore several use cases where Selenium can be practically applied in real-world scenarios.

Using Selenium, you can automate the monitoring of a specific web page's content and get notifications when updates occur.

For example, you might want to monitor the stock status of a product or check if new posts have been added to a forum.

Copy
# This is sample code and may not run as-is
from selenium import webdriver
from selenium.webdriver.common.by import By
import time

# Create a Chrome WebDriver instance
driver = webdriver.Chrome()

# Navigate to the webpage you want to monitor (e.g., product page)
driver.get("https://www.example.com/product-page")

# Identify the initial element to check (e.g., stock status)
initial_status = driver.find_element(By.ID, "stock-status").text

while True:
    # Refresh the page
    driver.refresh()
    
    # Check the current status
    current_status = driver.find_element(By.ID, "stock-status").text
    
    # Verify if the status has changed
    if current_status != initial_status:
        print(f"Status has changed: {current_status}")
        break
    
    # Wait for a specified period (e.g., 60 seconds) before checking again
    time.sleep(60)

# Close the browser
driver.quit()

The above example periodically checks a specific element on a webpage (e.g., product stock status) and outputs a notification if a change occurs.

Sometimes, you may need to gather data spread across multiple pages.

With Selenium, you can easily navigate through pages and collect the data you need.

Copy
# This is sample code and may not run as-is
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.common.keys import Keys

# Create a Chrome WebDriver instance
driver = webdriver.Chrome()

# Initial page setting (e.g., first page)
url = "https://www.example.com/products?page=1"

# Navigate to the page
driver.get(url)

while True:
    # Extract the desired data from the page
    products = driver.find_elements(By.CSS_SELECTOR, ".product-title")

    # Print the data
    for product in products:
        print(product.text)
    
    # Look for the "next page" button
    try:
        # Next page button
        next_button = driver.find_element(By.CSS_SELECTOR, ".next-page")

        # Navigate to the next page
        next_button.click()

        # Wait for the page to load
        driver.implicitly_wait(10)
    except:
        # If there is no next page, exit the loop
        print("Finished crawling all pages.")
        break

# Close the browser
driver.quit()

This code uses a while loop to navigate through pages and collect data.

It is very useful in situations where you need to fetch information spread across multiple pages in one go.

By using Selenium in this way, you can automate repetitive web tasks and efficiently gather large-scale data.

Make the most of Selenium in your practical work to boost your productivity :)