TIL Fix missing certificates in Python 3.6 and Up on MacOS

Imagine my surprise when writing a simple Python script on my Mac, I suddenly got SSL errors on every urllib request over HTTPS. I checked the site certificate, looked good. I even confirmed on the Apple help documentation that they included the CAs for this certificate (in this case the Amazon certificates). I was really baffled on what to do until I stumbled across this.

\f0\b0 \ulnone \
This package includes its own private copy of OpenSSL 1.1.1.   The trust certificates in system and user keychains managed by the 
\f2\i Keychain Access 
\f0\i0 application and the 
\f2\i security
\f0\i0  command line utility are not used as defaults by the Python 
\f3 ssl
\f0  module.  A sample command script is included in 
\f3 /Applications/Python 3.11
\f0  to install a curated bundle of default root certificates from the third-party 
\f3 certifi
\f0  package ({\field{\*\fldinst{HYPERLINK "https://pypi.org/project/certifi/"}}{\fldrslt https://pypi.org/project/certifi/}}).  Double-click on 
\f3 Install Certificates
\f0  to run it.\
Link

Apparently starting in Python 3.6, Python stopped relying on the Apple OpenSSL and started bundling their own without certificates. The way this manifests is:

URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:581)>

The fix is to run the following from the terminal(replace 3.10 with whatever version you are running):

/Applications/Python\ 3.10/Install\ Certificates.command

This will install the certifi package, which has all the Mozilla certificates. This solved the problem and hopefully will help you in the future. Really weird choice by the Mac Python team here since it basically breaks Python.