Saved web pages when opened shows nothing

Question

I'm using python to crawl a webpage and save it. And the code works properly. But when I open the web page it just shows the website name i.e., http://www.indiabix.com and not the actual content.

You can just go the website and save one of it's pages NOT the homepage but other pages like http://www.indiabix.com/database/questions-and-answers/. And when you open it, the page just shows this

enter image description here

and not this

enter image description here

The code I've written is simple

def writeToFile(link, name, title):
    response = urllib2.urlopen(link)
    webContent = response.read()
    f = open(name + '/' + title, 'w')
    f.write(webContent)
    f.close

You just pass the link, directory name and title of file.

I have checked in Chrome, Firefox and Safari and all show the same output. How can I resolve this issue to display the entire saved page fully.

Thank you.


Show source
| python   | html   | save   | web-crawler   2016-07-06 10:07 2 Answers

Answers ( 2 )

  1. 2016-07-06 10:07

    You can simplify something like this. Install requests.

    import os
    import requests
    
    def write_to_file(link, name, title):
        with open(os.path.join(name, title), "w") as fp: 
            fp.write(requests.get(link).content)
    
  2. 2016-07-06 10:07

    It's not possible. You can do an experiment by right-clicking the webpage and save the complete web files. And then use your browser to open the .htm file, you will get the same result. And when you inspect the page you can see there are some errors(files not found). So I think there are some errors within this webpage.

    enter image description here

◀ Go back