utils.py

Revision 159, 272 bytes (checked in by dsandler, 2 years ago)
The refactoring monster was here. Now Digg's link-extraction code is factored out, so other sites can be added.

Line
1	import urllib2
2
3	USER_AGENT = 'Calcium/1.0 +http://feedtree.net/ (FeedTree + CoralCDN)'
4
5	def urlfetch(url, length=-1):
6	req = urllib2.Request(url)
7	req.add_header('User-Agent', USER_AGENT)
8	opener = urllib2.build_opener()
9	return opener.open(req).read(length)
10

Note: See TracBrowser for help on using the browser.