unicode - Encoding problem downloading HTML using mechanize and Python 2.6 -

unicode - Encoding problem downloading HTML using mechanize and Python 2.6 -

browser = mechanize.browser() page = browser.open(url) html = page.get_data() print html

it shows unusual characters. suppose utf-8 string python doesn't know , cannot show properly.

how can convert string unicode string like

u = u'test'

it gzipped

def ungzipresponse(r,b): headers = r.info() if headers['content-encoding']=='gzip': import gzip gz = gzip.gzipfile(fileobj=r, mode='rb') html = gz.read() gz.close() headers["content-type"] = "text/html; charset=utf-8" r.set_data( html ) b.set_response(r) response = browser.open(url) ungzipresponse(response, browser) html = response.read()

python unicode encoding utf-8 mechanize