Thursday, April 28, 2011

Parsing HTML in Python

What's my best bet for parsing HTML if I can't use BeautifulSoup or lxml? I've got some code that uses SGMLlib but it's a bit low-level and it's now deprecated.

I would prefer if it could stomache a bit of malformed HTML although I'm pretty sure most of the input will be pretty clean.

From stackoverflow
  • Perhaps µTidylib will meet your needs?

  • http://www.xmlhack.com/read.php?item=1392 http://sourceforge.net/projects/pirxx/

    http://pyxml.sourceforge.net/topics/

    I don't have much experience with python, but I have used Xerces (from the Apache foundation) in the past and found it to be very useful. The learning curve isn't bad either, though I'm not coming from a python perspective. I suggest you consider it though. (The first two links I've included discuss python interfaces to Xerces and the last one is the first google hit on "python xml").

    Joe : I know you want an HTML parser, but these will be good starting places.
  • Why can't you use BeautifulSoup? It's a simple one-file python-module, so if you don't get it "installed", just copy-and-paste it's contents into your actual script.

    Robert Elwell : Beautiful Soup has a lot of problems that haven't been fixed yet for Python 3.
    Surya : also it doesnt stomach malformed html very well
    andybak : I thought dealing with malformed HTML was largely the point of BeautifulSoup?
  • Python has a native HTML parser, however the Tidy wrapper Nick suggested would probably be a solid choice as well. Tidy is a very common library, (written in C is it?)

  • html5lib is good:
    http://code.google.com/p/html5lib/

0 comments:

Post a Comment