Wikipedia is a free, collaborative online encyclopedia written and edited through the volunteer efforts of literally thousands of people. The unique feature of Wikipedia (or any wiki) is that anyone can contribute or edit the content. Software features make it easy to identify gross acts of vandalism and revert articles to earlier, known good, states. The result is an amazing resource, in the depth and breadth of the content.

(UPDATE 12/1/05: Character assassination via Wikipedia)

But how authoritative is it? In other words, how sure can you be that information in the Wikipedia is true and complete?

Traditional encyclopedias employ armies of fact-checkers to go over every entry and make sure the encyclopedia is as accurate as it can be. This makes a traditional encyclopedia an authoritative source, and when you look something up in Britannica, you are relying on the reputation of the publisher that everything has been verified.

Wikipedia has no such resource, in any literal sense. However, the "anyone can edit" philosophy means that every reader has the ability to become a fact-checker, too, giving Wikipedia an army of potentially millions of fact-checkers. At least that's the theory.

Wikipedia != Authoritative?
Recently, this article caused a bit of a stir from its casual dismissal of the wiki model, and even the implication that Wikipedia was perhaps being deceptive by appearing too authoritative. Some suggested an experiment: insert some mistakes into Wikipedia, and see how long it lasts. Alex Halavais actually performed the experiment, and found that all his errors were removed within hours.

Actually, Alex wasn't the only one to think of performing this experiment. Roughly the same idea occurred to me at roughly the same time, but my experiment only ended today.

The Wiki Experiment
The goal was to try to get some insight into this question:

To what extent can you trust the information in Wikipedia?

I probed the question by inserting mistakes into Wikipedia, and seeing how long they took to be corrected. But before doing that, I wanted to make sure I inserted the right kind of mistakes: mistakes similar to ones which might sneak into Wikipedia either accidentally or on purpose under realistic conditions.

Wikipedia has already proven itself to be good at recovering from intentional vandalism and edit wars, the two most blatant forms of intentional degradation of Wikipedia. Wikipedia makes it easy to track all changes made by a particular person and undo them all, so if someone is out destroying pages, the damage doesn't last long [Note: I suspect this is why Alex's changes were all undone so quickly. Someone probably noticed that he had made a bunch of changes, and at least one of them was obviously wrong, so that person simply went back and reverted all of Alex's mistakes].

Procedure
Rather than looking like a vandal, I wanted to be more subtle. It isn't practical for me to change my IP address without seeking out a public access point (too much work!), so I decided to simply make one change a day for several days. That way, it wouldn't look like a mad editor, but a casual Wikipedia browser making occasional contributions.

In addition, I decided that adding details--rather than deleting or changing facts already in Wikipedia--would look more innocent. Finally, everything had to sound reasonable, but be factually wrong upon further research.

So, all my changes would take the form of plausible-sounding but wrong facts inserted into existing Wikipedia pages.

Results
I made five changes between August 30 and September 3. Not one of the changes was removed by September 4th, when I reverted them myself. Every change was in Wikipedia for at least 20 hours, and the longest was in for five days.

The changes were:

Layzie Bone (biographical page). I inserted "born 1973", but a quick Google search reveals that he was born in 1977.

Magni, from norse mythology. I said that he was commonly depicted wielding an axe or a spear. In fact, Magni was the only person other than Thor himself who could lift Thor's hammer, and Magni is commonly associated with that weapon. Interestingly, the fact about Thor's Hammer is in the Wikipedia entry (though they call it by the proper name, Mjollnir), yet nobody seemed to notice the incongruity that a god whose special power is lifting a hammer would be depicted with an axe or a spear.

Empuries, a Mediterranean town, I made the site of sadly lost Greek ruins. The Greek ruins are true enough, but they aren't lost, sadly or otherwise. This travel site helpfully informs us that Empuries has "lots of free parking close to the ruins" as well as a cafe and a museum at the archeological site.

Philipsburg, PA, became located at the junction of U.S. highway 233 and state route 503. Not U.S. highway 322 and state route 504, as most maps show.

Bernice Johnson Reagon, while apparently a prolific author, never wrote Georgia in Song. In fact, Amazon lists no such book by any author.

(Note: because I undid all my changes, you will have to back to the "history" tab for each page to see the pieces of misinformation. All my changes were from the IP address 65.27.75.56.)

Conclusion and Suggestions
I was disappointed that all my changes in Wikipedia went unchallenged. Surely a week was plenty of time, especially since fresh changes tend to get more scrutiny than old ones. I have to conclude that it would be very easy for subtle mistakes to sneak into Wikipedia, and go a very long time without being corrected.

Unfortunately, it is in getting these details right that an authoritative source is the most valuable.

One way to solve this weakness is to create a formal fact-checking mechanism. In Wikipedia, contributions of new material are certainly valuable, but fact checking is even more important. Perhaps each edit could be somehow marked as "unverified" until a second contributor is willing to vouch for its accuracy. While that wouldn't be a perfect solution, it would at least make the hurdle for misinformation or mistakes higher.

Other Experiments
Wikipedia isn't really a fact-checking mechanism so much as a voting mechanism. If someone reads an entry, unless something sounds blatantly false, he or she will likely accept what it says. If there is disagreement about the facts, an edit war could break out until a consensus view develops.

Given that, there are some other experiments which would be interesting to perform:

Believe It Or Not: The opposite of inserting plausible mistakes, try inserting a series of true facts which sound implausible. See if they get edited out.

Whole Cloth: There are a number of missing pages and stubs in Wikipedia, pages which have placeholders but no entry. Try creating an elaborate hoax out of whole cloth (but not too implausible). See if it gets corrected or deleted.

Good Guy, Bad Edit: Some edits appear safer than others. For example, someone who signs up with an account (rather than anonymous), and carefully notates the reasons for edits, may have greater social currency and thus be able to insert bigger whoppers into articles.