This story started in December, 2017 with Joy-Ann Reid (of MSNBC) apologizing for "insensitive LGBT blog posts" that she wrote on her blog many years ago when she was a morning radio talk show host in Florida. This apology was, at least in some quarters, (begrudgingly) accepted. Today's update was news that Reid and her lawyers had in December claimed that either her blog, and/or the Internet Archive's record of the blog had been hacked (Mediaite, The Intercept). Later today, the Internet Archive issued a blog post deny the claim that it was hacked, stating:
This past December, Reid’s lawyers contacted us, asking to have archives of the blog (blog.reidreport.com) taken down, stating that “fraudulent” posts were “inserted into legitimate content” in our archives of the blog. Her attorneys stated that they didn’t know if the alleged insertion happened on the original site or with our archives (Reid’s claim regarding the point of manipulation is still unclear to us).
...
At some point after our correspondence, a robots.txt exclusion request specific to the Wayback Machine was placed on the live blog. That request was automatically recognized and processed by the Wayback Machine and the blog archives were excluded, unbeknownst to us (the process is fully automated). The robots.txt exclusion from the web archive remains automatically in effect due to the presence of the request on the live blog.
Checking the Internet Archive for robots.txt, we can see that on 2018-02-16 blog.reidreport.com had a standard robots.txt page that blocked the admin section of WordPress, but by 2018-02-21 they had a version that blocked all robots, and as of today (2018-04-24) they had a version that specifically blocked only the Internet Archive's crawler ("ia_archiver"). As of about 5pm EDT, the robots.txt file had been removed (probably because of the Internet Archive's blog post calling out the presence of the robots.txt; cf. a similar situation in 2013 with the Conservative Party in the UK), but it may take a while for the Internet Archive to register its absence.
2018-04-25 update: Thanks to Peter Sterne for pointing out that www.blog.reidreport.com/robots.txt still exists, even though blog.reidreport.com/robots.txt does not. They technically can be two different URLs though the convention is for them to canonicalize to the same URL (which is what the Wayback Machine does). HTTP session info provided below, but the summary is that robots.txt is still in effect and the need for other web archives is still paramount.
Until the Internet Archive begins serving blog.reidreport.com again, this is a good time to remind everyone that there are web archives other than the Internet Archive. The screen shot above shows the Memento Time Travel service, which searches about 26 public web archives. In this case, it found mementos (i.e., captures of web pages) in five different web archives: Archive-It (a subsidiary of the Internet Archive), Bibliotheca Alexandrina (the Egyptian Web Archive), the National Library of Ireland, the archive.is on-demand archiving service, and the Library of Congress. For a machine readable service, below I list the TimeMap (list of mementos) generated by our MemGator service; the details aren't important but it is the source of the URLs that will appear next.
Beginning with the original tweets by @Jamie_Maz (2017-11-30 thread, 2018-04-18 thread), I scanned through the screen shots (no URLs were given) and looked for screen shots that had definitive datetimes (most images did not have them). The datetimes are (with ones for which we have evidence in bold, and the ones that we inferred by matching text are maked with "(inferred)"):
2005-04-25
2005-07-16
2005-07-21
2006-01-20 (inferred)
2006-06-05
2006-06-13 (inferred)
2006-10-03
2006-12-23
2007-02-21
2008-07-04
2008-10-16
2009-01-15
(update: because of canonicalization errors, some of the URLs are not being excluded; see below)
Most of those dates are pretty early in web archiving times, when the Internet Archive was the only archive commonly available, and many (all?) of the mementos in other web archives were surely originally crawled by the Internet Archive, even if on a contract basis (e.g., for the Library of Congress). Nonetheless, with multiple copies geographically and administratively dispersed throughout the globe, an adversary would have had to hack multiple web archives and alter their contents (cf. lockss.org), or have hacked the original site (blog.reidreport.com) approximately 12 years ago for adulterated pages to have been hosted at all the different web archives. While both scenarios are technically possible, they are extraordinarily unlikely.
While we don't know the totality of the hacking claims, we can offer three archived web pages, hosted at the Library of Congress web archive (webarchive.loc.gov), that corroborate at least some of the claims @Jamie_Maz.
2006-01-20
30/x Joy seemed very interested in Brokeback Mountain, but wouldn't watch it bc it featured two men hooking up.— Not a bot (@Jamie_Maz) April 18, 2018
She can't understand who is going to see it since she imagines everyone would be turned off by it. pic.twitter.com/ogBnPyDSUF
Evidence for this tweet can be found at (approximately midway): http://webarchive.loc.gov/all/20060125004941/http://blog.reidreport.com/
2006-06-05
19/x Joy - I am not a gay marriage supporter pic.twitter.com/wbf2QSkx7n— Not a bot (@Jamie_Maz) April 18, 2018
Evidence for this tweet can be found at (approximately 2/3 down): http://webarchive.loc.gov/all/20060608144033/http://blog.reidreport.com/
2006-06-13
I'm not sure this evidence maps directly to one of tweets, but it fits the general theme of anti-Charlie Crist: http://webarchive.loc.gov/all/20060615134635/http://blog.reidreport.com/
This memento also exists at archive.is; it is a copy of the Internet Archive's copy but it is not blocked by robots.txt because it is in another archive: http://archive.is/20060615134635/http://blog.reidreport.com/
2006-10-03
10/x Of course there are even more posts about Charlie Crist.— Not a bot (@Jamie_Maz) April 18, 2018
For some strange reason Joy posts a link to an article that claims Crist was involved in gay sex parties with Mark Foley. pic.twitter.com/qNazjxYQ7K
Evidence for this tweet can be found at (approximately midway): http://webarchive.loc.gov/all/20061010125903/http://blog.reidreport.com/
2008-10-16
7/x Joy calls Crist "Miss Charlie" and again declares his potential wedding to a women is a fraud and a "veep marketing strategy". pic.twitter.com/ZMCbEfURfn— Not a bot (@Jamie_Maz) November 30, 2017
Evidence for this tweet can be found at (approximately 1/3 down): http://webarchive.loc.gov/all/20081018020856/http://blog.reidreport.com/
In summary, of the many examples that @Jamie_Maz provides, I can find five copies in the Library of Congress's web archive. These crawls were probably performed on behalf of the Library of Congress by the Internet Archive (for election-based coverage); even though there are many different (and independent) web archives now, in 2006 the Internet Archive was pretty much the only game in town. Even though these mementos are not independent observations, there is no plausible scenario for these copies to have been hacked in multiple web archives or at the original blog 10+ years ago. There may be additional evidence in the other web archives, but I haven't exhaustively searched them.
We don't know the full details of what Reid's lawyers alleged, so perhaps there are details that we don't know. But the analysis from the Internet Archive crawl engineers, plus evidence in separate web archives suggest that the claim has no merit.
The case of blog.reidreport.com is another example of why we need multiple web archives.
We don't know the full details of what Reid's lawyers alleged, so perhaps there are details that we don't know. But the analysis from the Internet Archive crawl engineers, plus evidence in separate web archives suggest that the claim has no merit.
The case of blog.reidreport.com is another example of why we need multiple web archives.
--Michael
Thanks to Prof. Michele Weigle and John Berlin for bringing this issue to my attention and uncovering some of the examples.
Memento TimeMap for blog.reidreport.com:
2018-04-25 update: As noted above, Peter Sterne brought to my attention that the non-standard URL of www.blog.reidreport.com/robots.txt still exists (and is blocking "ia_archiver") even though the more standard blog.reidreport.com/robots.txt is 404.
Another 2018-04-25 update: The NYT has covered the story ("MSNBC Host Joy Reid Blames Hackers for Anti-Gay Blog Posts, but Questions Mount"), and there was an interview with Reid's computer security expert ("Should We Believe Joy Reid’s Blog Was Hacked? This Security Consultant Says We Should"), Jonathon Nichols.
I embed a statement from Nichols (released by Erik Wemple), and a tweet from Nichols clarifying that they were not suggesting that Wayback Machine's mementos were hacked, but rather the hacked blog was crawled by the Internet Archive.
This is where it's important to note that there maybe a discrepancy between the posts that Nichols is concerned with and those that @Jamie_Maz surfaced. There is (semi-)independent evidence of @Jamie_Maz's pages, with the ultimate implication that for those pages to have been the result of a hack, blog.reidreport.com would have had to been hacked as many as 12 years ago -- and for nobody to have noticed at the time.
Reid (& Nichols) could always unblock the Internet Archive and share the evidence of the hack.
Here's the statement of security consultant Jonathan Nichols regarding the claims of blog-hacking by MSNBC's Joy Reid. pic.twitter.com/wGAui8Mfa5— ErikWemple (@ErikWemple) April 25, 2018
1) WayBack was hacked— Jonathan Nichols (@wvualphasoldier) April 25, 2018
2) Joy was hacked
3) we said "Yo! Does your hack look like our hack!?"
4) PS: That's literally the industry standard (I've personally done it plenty of times)
5) We THEN got new data that showed it wasn't a hack of any archive.
Yet another 2018-04-25 update: Apparently there are some holes in the http vs. https canonicalization wrt robots.txt blockage, allowing some of posts to surface. Here's an example (via @YanceyMc):
https://web.archive.org/web/20060225041734/https://blog.reidreport.com/2005/10/harriet-miers-and-lesbian-hair-check.html
that page was captured in 2012; here's a version captured in 2006 (page originally authored in 2005; according to blogger):https://t.co/hrXwyC9wGH— Michael L. Nelson (@phonedude_mln) April 25, 2018
here's a copy of that copy in @archiveis:https://t.co/kRgDN7nY4fhttps://t.co/103n66HKe1
Also, @wvualphasoldier deleted his tweets then protected his account, so that's the reason the above embed no longer formats correctly.
Yet, Yet Another 2018-04-25 update:
Thanks to Prof. Weigle and Mat Kelly for providing examples of some of the URLs that are slipping through the robots.txt exclusion.
Here's one: https://web.archive.org/web/20060805055643/https://blog.reidreport.com
and another: https://web.archive.org/web/20050728132003/https://blog.reidreport.com:443/
Which has the following information that I thought I saw in the original @Jamie_Maz tweets but now I can't find it, so perhaps I'm misremembering. It certainly fits the overall theme.
