Finding next input element using Mechanize?
mechanize, ruby
Solution
Mechanize uses Nokogiri internally to handle its DOM parsing, which is the basis of its ability to locate different elements in a page.
It's possible to access the parsed DOM, and, through it use Nokogiri to locate elements Mechanize doesn't normally let us find. For instance:
require 'mechanize'
agent = Mechanize.new
page = agent.get('http://www.example.com')
# Use Nokogiri to find the content of the <h1> tag...
puts page.at('h1').content # => "Example Domain"
For your search you'd want to use an XPath accessor to locate where "email" is in the page. Once you've done that you can locate the next `<input>` tag.
Starting from a simple HTML fragment, we'll pretend this comes from Mechanize:
page = Nokogiri::HTML('<div><form><p>email</p><input name="email"></form></div>')
puts page.to_html
Which looks like:
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN" "http://www.w3.org/TR/REC-html40/loose.dtd">
<html><body><div><form>
<p>email</p>
<input name="email">
</form></div></body></html>
Searching for "email":
page.at("//*[contains(text(),'email')]")
#<Nokogiri::XML::Element:0x3ff50d0c4bc0 name="p" children=[#<Nokogiri::XML::Text:0x3ff50d0c497c "email">]>
Building upon that, this gets the `<input>` tag:
input_tag = page.at("//*[contains(text(),'email')]/following-sibling::input")
#<Nokogiri::XML::Element:0x3ff50d09b75c name="input" attributes=[#<Nokogiri::XML::Attr:0x3ff50d09b5f4 name="name" value="email">]>
Once you've found that input tag, you can get the "name" from the tag using Nokogiri, and then tell Mechanize to locate and fill in that particular input field:
input_tag['name']
=> "email"
For a web form to function correctly, it has to have names for the elements. Those get passed to the server when the form is submitted. Without the names it'd take a lot of work to determine which input sent a particular piece of data, and, programmers being lazy, we don't want to work hard, so you can count on having a name to work with.
See "Ruby Mechanize, Nokogiri and Net::HTTP" for more information, plus a search of Stack Overflow, and reading the Nokogiri documenation and tutorials will give you lots of needed information for figuring out how to do the rest.
Problem
Using Mechanize, is it possible to find a phrase in the HTML of a page, for example, "email", and find the next `<input*` after that, and fill in that input field, and only that field?