Wait until page is loaded with Selenium WebDriver for Python

python

selenium

webdriver

explicit-waits

byAlex Kataev·Mar 10, 2025

WebDriverWait and expected_conditions (EC) provide an effective strategy to synchronize with page load in Selenium WebDriver for Python. Here's the core method condensed into a small script:

from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By

driver = webdriver.Chrome()
driver.get("http://example.com")

# An element on the page acting as the beacon indicating page load
element_present = EC.presence_of_element_located((By.ID, "beacon"))
WebDriverWait(driver, 10).until(element_present)

Swap out "beacon" with the identifier for a component unique to your web page, ensuring you have arrived.

Strategies for dynamic content and AJAX

Dynamic web content and AJAX requests are pesky creatures that can result in incomplete scraping. Your WebDriver doesn't naturally wait for them. But don't fret. We've got your back with manual synchronization steps for script stability.

Tackle dynamic content

Pages sometimes continue to load content after the initial call. WebDriverWait and expected_conditions become our trusty tools:

from selenium.webdriver.common.by import By

# Waiting for Santa... Oops! I meant the new content triggered by scrolling:
new_content_locator = (By.CLASS_NAME, "santa_claus")
WebDriverWait(driver, 15).until(EC.presence_of_element_located(new_content_locator))

Master AJAX requests

While AJAX-based content may test your patience, like waiting for the next GoT book, it's worth the wait. Anticipate the execution and the population of DOM elements:

import time

# Waiting for George R.R. Martin to finish GoT... Just kidding, for AJAX state:
ajax_loaded = WebDriverWait(driver, 10).until(lambda d: d.execute_script("return window.ajaxFinished === true;"))
if ajax_loaded:
    # Congratulations, finally AJAX content is here!

Advanced expectations for explicit waits

Using staleness to detect page reloads

When a page reload is triggered by navigation or clicks, the trusty staleness_of can determine if the old page elements are now ghost elements, disconnected from the DOM:

old_element = driver.find_element(By.ID, "element_id")
driver.find_element(By.LINK_TEXT, "Redirect Link").click()
WebDriverWait(driver, 10).until(EC.staleness_of(old_element))

ReadyState property for load completion

To know when the page has loaded completely as relatable as waiting for your pizza delivery, check the readyState property:

WebDriverWait(driver, 10).until(lambda d: d.execute_script("return document.readyState") == "complete")

Advanced selectors with XPATH

Sometimes we face the hard task of finding Waldo, or complex to select elements. XPATH helps with these advanced selection capabilities:

WebDriverWait(driver, 10).until(
    EC.presence_of_element_located((By.XPATH, "//div[@class='content' and contains(text(), 'Loaded Text')]"))
)

Dealing exceptions and optimizing efficiency

Graceful handling of timeouts with context managers

Context managers employing yield facilitate a smooth exit during timeouts, like a ninja slipping out unnoticed:

from contextlib import contextmanager

@contextmanager
def wait_for_page_load(driver, timeout=10):
    old_page = driver.find_element_by_tag_name('html')
    yield WebDriverWait(driver, timeout).until(
        EC.staleness_of(old_page)
    )

Tailoring timeouts to observed load times

For effective wait management, adjust the WebDriverWait timeout value according to the average webpage load time:

# Wait time tailored after observing page load times
WebDriverWait(driver, 20).until(...)

Lambda magic with WebDriverWait

When the conditions get more intricate than a heist plan, turn to lambda functions:

WebDriverWait(driver, 10).until(lambda d: d.find_element(By.TAG_NAME, "p").get_attribute("class") == "loaded")

Visibility vs Presence of elements

Sometimes the element needs to be visible, at other times present is enough, like reading a book versus displaying it to show off:

# Wait for visibility
WebDriverWait(driver, 10).until(EC.visibility_of_element_located((By.NAME, "username")))

# Wait for presence
WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.NAME, "username")))