Need Help With Python Webscraping!!!

Doc of urllib:

urllib.request module uses HTTP/1.1 and includes Connection:close header in its HTTP requests.

I tried to add this to your header and got two 200 response.

from urllib.request import Request, urlopen
import requests

headers = {"User-Agent": "Mozilla/6.0", "Connection": "close"}

# First request using urllib.request -> Success 200
test = Request("https://www.albumoftheyear.org", headers=headers)
res = urlopen(test)
print(res.status)

# Second request using requests -> Forbidden 403
req = requests.get("https://www.albumoftheyear.org", headers=headers)
print(req.status_code)

I am not expert at all in http request but apparently it tells the server to close the connection once it has sent a response. Instead of keeping it alive.

/r/learnpython Thread