Archiving a community's conversations
Volunteer networks produce a great deal of writing: mailing-list threads, chat logs, wiki pages, reports. Much of it disappears when a server is switched off, a domain lapses or a free service closes. Archiving is how a community remembers what it tried and learned, and how researchers and successors can understand it later. This article sets out the essentials.
What is at risk
Studies of "link rot" show how quickly the web forgets. A 2024 Pew Research Center analysis found that about a quarter of web pages that existed at some point between 2013 and 2023 were no longer accessible by late 2023, and that many links cited in news articles and on Wikipedia pointed to pages that had gone. Community sites are particularly fragile: they often depend on one volunteer's hosting account, or on a platform that can change its terms or close.
What to keep
- Mailing lists and forums: the full message history, with dates and threading.
- Chat: logs of public channels, if members agreed to logging, and summaries of meetings.
- Wikis and websites: page text and, where possible, edit history.
- Documents and media: reports, presentations, photos and recordings, with information on who made them and under what licence.
- Context: a short description of the community, its members' roles and its main projects, so that future readers understand what they are looking at.
Formats that last
The safest formats are open, documented and simple. Plain text and Markdown for writing; mbox for email archives, a format readable by most mail software; PDF/A for finished documents, an ISO standard designed for long-term preservation; CSV for tables; and WARC, standardised as ISO 28500 in 2009, for captures of websites. Avoid relying solely on a platform's internal database or a proprietary export that only that platform can read.
Web archives
The Internet Archive, founded in 1996, has captured hundreds of billions of web pages through its Wayback Machine. National libraries run their own web archives, often with a legal mandate to collect sites in their country's domain. A community can help by requesting captures of its key pages through the "Save Page Now" feature, and by keeping a list of its important addresses. Web archives are a safety net, not a plan: their captures are incomplete, may omit content behind logins or scripts, and can be removed on request.
Consent and privacy
Archiving conversations raises ethical questions. Members who wrote informally to a small group may not expect their words to be preserved indefinitely or made public. In the European Union, the General Data Protection Regulation (Regulation 2016/679) gives people a right to request erasure of personal data in certain circumstances (Article 17), with exceptions that include archiving in the public interest and historical research. Good practice, whatever the law:
- tell members, at the start, what will be archived and whether the archive will be public;
- keep private channels out of public archives;
- offer a way to remove or anonymise personal information on request;
- take extra care with material involving people at risk, such as participants in conflict zones or minors.
Who can help
Small networks rarely have archivists, but they can find allies. University libraries and national libraries sometimes collect the records of social movements and online communities, especially when the material documents events of public interest. Community archives, local history societies and subject repositories may accept deposits with agreed access conditions. Even a simple conversation with a librarian can clarify which formats to use and what metadata to record.
A simple archiving plan
- Name a keeper, and a deputy, responsible for the archive.
- Export regularly, at least every few months, in open formats.
- Store copies in more than one place: a member's computer, a cloud service and, for valuable material, an institutional repository or library.
- Write a readme that explains what the archive contains, how it is organised and what may be shared.
- Plan for the end: if the network closes, decide where the archive goes and who can grant access.
When a domain lapses
A frequent cause of loss is an expired domain name. Once it lapses, anyone can register it, and old links lead to unrelated content. Communities can protect themselves by registering domains for several years, keeping renewal contacts current, and depositing a copy of their site with an archive before letting a domain go. This publication itself occupies a domain that once served a different project; we explain this on our about page.
Archives as a resource
A well-kept archive is not only for historians. It lets a network answer "have we tried this before?", lets newcomers learn from earlier endeavors, and gives credit to people whose contributions would otherwise be forgotten. Related reading: turning chat into a knowledge base.
Common mistakes
- Keeping only the platform's copy. If the service closes or suspends the account, everything goes with it.
- Exporting without context. A folder of files with cryptic names is nearly useless ten years later.
- Making everything public by default. Some conversations were never meant for strangers, and publishing them can harm people.
- Waiting for the end. Archiving is easiest while the people who know what the material means are still around.
- Forgetting attachments and links. Email archives often lose attached files, and messages full of links to vanished pages lose their meaning. Save important attachments separately and request web-archive captures of key links while they still work.
Sources
- Pew Research Center, "When Online Content Disappears", May 2024.
- ISO 28500:2009, Information and documentation — WARC file format.
- ISO 19005-1:2005, Document management — Electronic document file format for long-term preservation — Part 1: Use of PDF 1.4 (PDF/A-1).
- Regulation (EU) 2016/679 (General Data Protection Regulation), Articles 17 and 89.
- Internet Archive, about the Wayback Machine: archive.org/about.
- Digital Preservation Coalition, Digital Preservation Handbook, 2nd edition: dpconline.org.