Python, Selenium : 'Element is no longer attached to the DOM'
python, selenium, selenium-webdriver, web-crawler
Solution
The problem is that the element is being detached by some javascript. So you should make the driver wait for the element: This is done by setting `implicitly_wait`, see:
from selenium import webdriver
ff = webdriver.Firefox()
ff.implicitly_wait(10) # seconds
...
myDynamicElement = ff.find_element_by_id("myDynamicElement")
from http://docs.seleniumhq.org/docs/04_webdriver_advanced.jsp#implicit-waits
Problem
I am scraping a website, www.lipperleaders.com. I want to extract the funds detail of Singapore. I have successfully implemented drop-down selection and extracted the content of the first page appearing after submission of the options. But when I try to go to next pages (by making the code to click next button) I am getting error `'Element is no longer attached to the DOM'`. My code is about 100 lines but I can give a general idea of the flow of execution of my code: ``` ... # creating driver object and all the imports def main(): ... result = find_elements_by_tag_name('span') ... driver.find_element_by_id("ctl00_ContentPlaceHolder1_ucDataPager_btnNext").click() main() main() ``` This code works fine for the 1st page but when `main()` is called again after clicking of the next button. Before this recursive method, I also tried putting this inside a loop, then also same error. And if I write the same code like: ``` # some code result = find_elements_by_tag_name('span') driver.find_element_by_id("ctl00_ContentPlaceHolder1_ucDataPager_btnNext").click() # some code driver.find_element_by_id("ctl00_ContentPlaceHolder1_ucDataPager_btnNext").click() . . ``` This code works fine w/o any error the next page loads and executes the code written after that. But I cannot write the same `driver.find_element_by_id().click()` for 500 pages, even I will have to repeat the rest of the code associated with each page. That's why I am trying for loop or recursion, but its not working for me. Please let me know what is the problem with my approach.