Comments on: Web Scraping, HTML/XML Parsing, and Firebug’s Copy XPath Feature http://www.chinhdo.com/20080829/web-scraping-htmlxml-parsing-and-firebugs-copy-xpath-feature/ Chinh's not quite random thoughts on software development, .NET, gadgets, and other things. Tue, 09 Mar 2010 13:15:12 +0000 http://wordpress.org/?v=2.9.1 hourly 1 By: Sara http://www.chinhdo.com/20080829/web-scraping-htmlxml-parsing-and-firebugs-copy-xpath-feature/comment-page-1/#comment-62072 Sara Tue, 27 Oct 2009 19:54:09 +0000 http://www.chinhdo.com/20080829/web-scraping-htmlxml-parsing-and-firebugs-copy-xpath-feature/#comment-62072 Nice post on web scrappers, simple and too the point :), I use python for simple html web scrappers, but for larger projects i have used extractingdata.com <a href="http://www.extractingdata.com/web%20scraper.htm" rel="nofollow">web scrapper</a> which builds custom web scrappers and data extracting programs simple and fast Nice post on web scrappers, simple and too the point :) , I use python for simple html web scrappers, but for larger projects i have used extractingdata.com web scrapper which builds custom web scrappers and data extracting programs simple and fast

]]> By: Sen Hu http://www.chinhdo.com/20080829/web-scraping-htmlxml-parsing-and-firebugs-copy-xpath-feature/comment-page-1/#comment-6599 Sen Hu Sat, 27 Dec 2008 19:49:16 +0000 http://www.chinhdo.com/20080829/web-scraping-htmlxml-parsing-and-firebugs-copy-xpath-feature/#comment-6599 I use biterscripting for web scraping or harvesting. There is a good script example at http://biterscripting.com/Download/FS_WebPageToText.txt . Sen I use biterscripting for web scraping or harvesting. There is a good script example at http://biterscripting.com/Download/FS_WebPageToText.txt .

Sen

]]> By: Chinh Do http://www.chinhdo.com/20080829/web-scraping-htmlxml-parsing-and-firebugs-copy-xpath-feature/comment-page-1/#comment-4819 Chinh Do Wed, 03 Sep 2008 22:58:17 +0000 http://www.chinhdo.com/20080829/web-scraping-htmlxml-parsing-and-firebugs-copy-xpath-feature/#comment-4819 Philippe: Thanks for the info on XPather! I gave it a quick run over and it looks nice. One caveat: it doesn't work with XML documents (the generated XPath expression is always "/html"). You can get around this by changing your file extension to "htm" to get Firefox to treat your document as HTML. The other issue, not a major one in my opinion, is that the generated XPath expressions always contain "TBODY" elements underneath each "TABLE" element, whether the TBODY tags are actually there or not in the source. It's easy enough to manually edit out these extract TBODY tags, but it would be nice if you don't have to do that. I'll send a bug report to the author to see if this can be fixed in the next version. Philippe: Thanks for the info on XPather! I gave it a quick run over and it looks nice.

One caveat: it doesn’t work with XML documents (the generated XPath expression is always “/html”). You can get around this by changing your file extension to “htm” to get Firefox to treat your document as HTML. The other issue, not a major one in my opinion, is that the generated XPath expressions always contain “TBODY” elements underneath each “TABLE” element, whether the TBODY tags are actually there or not in the source. It’s easy enough to manually edit out these extract TBODY tags, but it would be nice if you don’t have to do that. I’ll send a bug report to the author to see if this can be fixed in the next version.

]]> By: Philippe Lhoste http://www.chinhdo.com/20080829/web-scraping-htmlxml-parsing-and-firebugs-copy-xpath-feature/comment-page-1/#comment-4793 Philippe Lhoste Wed, 03 Sep 2008 11:40:45 +0000 http://www.chinhdo.com/20080829/web-scraping-htmlxml-parsing-and-firebugs-copy-xpath-feature/#comment-4793 Interesting, I overlooked this feature. Now, I use another extension, XPather, which precisely do what you wish to have: inclusion of IDs (and classes) references in the XPath. Plus some other handy functionalities. Interesting, I overlooked this feature.
Now, I use another extension, XPather, which precisely do what you wish to have: inclusion of IDs (and classes) references in the XPath. Plus some other handy functionalities.

]]> By: Dew Drop - August 30, 2008 | Alvin Ashcraft's Morning Dew http://www.chinhdo.com/20080829/web-scraping-htmlxml-parsing-and-firebugs-copy-xpath-feature/comment-page-1/#comment-4624 Dew Drop - August 30, 2008 | Alvin Ashcraft's Morning Dew Sat, 30 Aug 2008 23:24:10 +0000 http://www.chinhdo.com/20080829/web-scraping-htmlxml-parsing-and-firebugs-copy-xpath-feature/#comment-4624 [...] Web Scraping, HTML/XML Parsing, and Firebug’s Copy XPath Feature (Chinh Do) [...] [...] Web Scraping, HTML/XML Parsing, and Firebug’s Copy XPath Feature (Chinh Do) [...]

]]>