Python 3 script for logging into a website using the Requests module
cookies, python, python-3.x, python-requests
Solution
You are loading the wrong URL. The form has an `action` attribute:
<form method="post" action="https://www.ibvpn.com/billing/dologin.php" name="frmlogin">
so you must post your login information to:
https://www.ibvpn.com/billing/dologin.php
instead of posting back to the login page. POST to `soup.form['action']` instead:
r = s.post(soup.form['action'], data=body)
Your code is handling cookies just fine; I can see that `s.cookies` holds a cookie after requesting the login form, for example.
If this still doesn't work (a 404 is returned), then the server is using additional techniques to detect scripts vs. real browsers. Usually this is done by parsing the request headers. Look at your browser headers and replicate those. It may just be the `User-Agent` header that they parse, but `Accept-*` headers and `Referrer` can also play a role.
Problem
I'm trying to write some Python (3.3.2) code to log in to a website using the Requests module. Here is the form section of the login page: ``` <form method="post" action="https://www.ibvpn.com/billing/dologin.php" name="frmlogin"> <input type="hidden" name="token" value="236647d2da7c8408ceb78178ba03876ea1f2b687" /> <div class="logincontainer"> <fieldset> <div class="clearfix"> <label for="username">Email Address:</label> <div class="input"> <input class="xlarge" name="username" id="username" type="text" /> </div> </div> <div class="clearfix"> <label for="password">Password:</label> <div class="input"> <input class="xlarge" name="password" id="password" type="password"/> </div> </div> <div align="center"> <p> <input type="checkbox" name="rememberme" /> Remember Me </p> <p><a href="pwreset.php">Request a Password Reset</a></p> </div> </fieldset> </div> <div class="actions"> <input type="submit" class="btn primary" value="Login" /> </div> </form> ``` Here is my code, trying to deal with hidden input: ``` import requests from bs4 import BeautifulSoup url = 'https://www.ibvpn.com/billing/clientarea.php' body = {'username':'my email address','password':'my password'} s = requests.Session() loginPage = s.get(url) soup = BeautifulSoup(loginPage.text) hiddenInputs = soup.findAll(name = 'input', type = 'hidden') for hidden in hiddenInputs: name = hidden['name'] value = hidden['value'] body[name] = value r = s.post(url, data = body) ``` This just returns the login page. If I post my login data to the URL in the 'action' field, I get a 404 error. I've seen other posts on StackExchange where automatic cookie handling doesn't seem to work, so I've also tried dealing with the cookies manually using: ``` cookies = dict(loginPage.cookies) r = s.post(url, data = body, cookies = cookies) ``` But this also just returns the login page. I don't know if this is related to the problem, but after I've run either variant of the code above, entering `r.cookies` returns `<<class 'requests.cookies.RequestsCookieJar'>[]>` If anyone has any suggestions, I'd love to hear them.