Browsing in Python with Mechanize

Browsing with Mechanize

The mechanize module in Python is similar to perl WWW:Mechanize.
It gives you a browser like object to interact with web pages.
Here is an example on how to use it in a program.
import mechanize
br = mechanize.Browser()"")
# Follow second link with element text matching regular expression
response1 = br.follow_link(text_regex=r"cheeses*shop", nr=1)
assert br.viewing_html()
print br.title()
print response1.geturl()
print  # headers
print  # body
To get the response code from a website, you can the response.code
from mechanize import Browser
browser = Browser()
response ='')
print response.code
Get all forms from a website
import mechanize
br = mechanize.Browser()"")
for f in br.forms():
    print f
I found this post at that very accurate describes how
to emulate a browser in Python using mechanize.
Browsing with Python (written of Drew Stephens)
import re
from mechanize import Browser
br = Browser()
# Ignore robots.txt
br.set_handle_robots( False )
# Google demands a user-agent that isn't a robot
br.addheaders = [('User-agent', 'Firefox')]
# Retrieve the Google home page, saving the response "" )
# Select the search box and search for 'foo'
br.select_form( 'f' )
br.form[ 'q' ] = 'foo'
# Get the search results
# Find the link to; why did we run a search?
resp = None
for link in br.links():
    siteMatch = re.compile( '' ).search( link.url )
    if siteMatch:
        resp = br.follow_link( link )
Print the site
content = resp.get_data()
print content
The script above is split up to make it easier to read


Popular posts from this blog

How To Make IoT Messaging with BashShell

How To Make IoT Tweet with Python