comment / found

Future Historians Probably Won't Understand Our Internet, and That's Okay

Archivists are working to document our chaotic, opaque, algorithmically complex world—and in many cases, they simply can’t.
What’s happening?

This has always been an easier question to pose—as Twitter does to all its users—than to answer. And how well we answer the question of what is happening in our present moment has implications for how this current period will be remembered. Historians, economists, and regular old people at the corner store all have their methods and heuristics for figuring out how the world around them came to be. The best theories require humility; nearly everything that has happened to anyone produced no documentation, no artifacts, nothing to study.

The rise of social media in the ’00s seemed to offer a new avenue for exploring what was happening with unprecedented breadth. After all, people were committing ever larger amounts of information about themselves, their friends, and the world to the servers of social-networking companies. Optimism about this development peaked in 2010, when Twitter gave its archive and ongoing access to public tweets to the Library of Congress. Tweets in the record of America! “It boggles my mind to think what we might be able to learn about ourselves and the world around us from this wealth of data,” a library spokesperson exclaimed in a blog post. “And I’m certain we’ll learn things that none of us now can even possibly conceive.”

Unfortunately, one of the things the library learned was that the Twitter data overwhelmed the technical resources and capacities of the institution. By 2013, the library had to admit that a single search of just the Twitter data from 2006 to 2010 could take 24 hours. Four years later, the archive still is not available to researchers. Across the board, the reality began to sink in that these proprietary services hold volumes of data that no public institution can process.

Across the board, the reality began to sink in that these proprietary services hold volumes of data that no public institution can process. And that’s just the data itself.

What about the actual functioning of the application: What tweets are displayed to whom in what order? Every major social-networking service uses opaque algorithms to shape what data people see. Why does Facebook show you this story and not that one? No one knows, possibly not even the company’s engineers.
  …
View source