How to turn any RSS or Atom feed into JSON and CSV with Python

https3A2F2Fdev-to-uploads.s3.us-east-2.amazonaws.com2Fuploads2Farticles2F6l7mkngboxfaqub9e9js

RSS and Atom still power podcasts, changelogs, and “what’s new” pages. The formats differ, dates are inconsistent, and descriptions are often HTML. Here’s a small Python script (stdlib + requests) that discovers a feed, normalizes items, and writes a CSV. 1. Discover the feed from a website Many pages advertise their feed in <head>: import re, requests from urllib.parse import urljoin, urlparse UA = {“User-Agent”: “rss-tutorial/1.0″} def discover_feeds(page_url): html = requests.get(page_url, headers=UA, timeout=20).text found = [] for m in re.finditer(r”'<link[^>]+rel=[“‘]alternate[“‘][^>]*>”’, html, re.I): tag = m.group(0) if not re.search(r’type=[“‘]application/(rss|atom)+xml[“‘]’, tag, re.I): continue href = re.search(r’href=[“‘]([^”‘]+)[“‘]’, tag,…

Continue reading at DEV Community dev.to
Advertisement