How to web-scrape content that's password-protected, on social media, or is paywalled?

Tools

• WebRecorder.net — Company that develops open-source web archiving tools.
  • BrowserTrix
  • 'Archive Webpage' Chrome extension — Interactive archiving in your browser. [extension]
  • https://github.com/webrecorder/
  • background info: https://en.wikipedia.org/wiki/Webrecorder#Products
  • TODO — The site has a user forum. One of the forums there is the discuss stuff outside of WebRecorder.net forum. Explore this because it probably refers to other organization's tools that are similar to this.
• Conifer, by Rhizome.org
  • https://github.com/Rhizome-Conifer
  • background info: https://en.wikipedia.org/wiki/Rhizome_(organization)#Conifer

TODO: other stuff to explore

• tool(s): https://archoral.com/tools/
  • Videos that try to explain what they do: https://www.youtube.com/@archoral1920/search?query=archiving
• tool(s): https://archivebox.io/
• tool: https://github.com/internetarchive/brozzler (What is Brozzler?)
• list of possible tools: Ctrl-F for "Acquisition"
  • From its "hosted, closed source" list:
    • https://hanzo.co/chronicle/
    • https://www.mirrorweb.com/platform/website-archiving/ — Looks like they want to really monetize it.
    • https://www.pagefreezer.com/ — Looks like they want to really monetize it.
    • https://www.smarsh.com/smb/web-archive/ — Looks like they want to really monetize it.

TODO: other tools that might be closely-related

• tool: Archive-It [What is Archive-It?]
  • It says the list of pages to be archived are "curator-directed". So this seems to be a kind of "private archive". But is it capable of archiving social media sites, or things like YouTube?
  • Oh wow. Cool.