I am forwarding, with permission of Susan Chun, a conversation from the steve.museum list about incorrect/invalid tagging. I think it addresses well the issue of how social tagging might intersect with curatorial intervention -- whether that is catalogers doing LCSH or curators saying that Mrs Arnolfini isn't pregnant. Some have argued that "bad" tags will fall away on their own, with time.

------------------------

Hi Belinda,

Rob has done a really good job of articulating the process of describing the ways in which the steve team will review terms submitted in the research project and mark them as "appropriate" (i.e. "correct") and "invalid" (i.e. "incorrect"). We're doing this in order to make some generalizations about factors that we are testing and how they contribute to tagging behaviors and the tags that are collected. So, for example, we're interested in knowing whether showing captions to users increases or decreases the amount of tagging that users do, but also whether showing captions affects the likelihood of users submitting "appropriate" tags. To do this, we of course need to know exactly which tags are appropriate or not. We can do this in the research project itself, because its scope is limited. Most museums cannot expect to do this kind of term-by-term validation in a "live" tag collection setting, and probably also don’t want to for reasons articulated by Ron Murray.

We imagine that every participating museum will have to make their own decision about what to do about "incorrect" tags. Some organizations will find that collecting these incorrect tags may significantly improve searching and will simply use the tags in the background to support Web site searches without doing much else. Here, "incorrect" tags simply serve to lead users to more authoritative information, and even serve as an opportunity to gently re-educate users when their tag is incorrect as to substance (an abstract work representing nothing is tagged as a portrait). Another common kind of "incorrect" tag is misspelling, and here again knowing about common misspellings of, say, artists' names, could be very useful to museums who are seeking to improve Web access, possibly allowing museums to provide a kind of "did you mean?" service to searchers. Others (Seb? Effie?) who have live tagging applications running on their Web sites will have already decided what to do with i
ncorrect tags, and they should chime in here.

A few museums may feel that they want to "keep" only "correct" tags. There are a couple of ways in which the work that steve is doing can help here. Should institutions decide that they want to moderate tags, they may find that the tool that we are building to review tags for the research project will be useful to them. Alternatively, our research may begin to show patterns of "likelihood of correctness" that can be applied automatically. At the Metropolitan Museum, in early (and admittedly limited) proof-of-concept tests, we found that if a term had been supplied by a certain percentage of users -- I think it was 40% -- it was always accurate. And though this kind of certainty about accuracy will not hold up in future testing, it's possible that we can predict, based on numbers, that a term is more likely to be accurate than not. This may be especially likely if we add in other factors, for example, whether the work is representational, or two dimensional. We're hoping that
 the research will help us to make some broad generalizations about these factors and how they relate to accuracy.

Finally, to answer your question about bi-lingual tagging of indigenous artifacts. Shelly Mannion, a graduate student at the University of Lugano, in Switzerland, is using the steve tagger to support some very interesting research that she is doing about Tibetan art and the perceptions of it by young Tibetans. I'll ask Shelly to elaborate a bit more for the list, especially since I know that others here will be interested in hearing about the research project. Shelly has built a special steve "skin" for the project and is collecting tags in both English and German (not Tibetan, yet).

It’s good to hear that Te Papa is considering doing some work in tagging. These sorts of questions from staff are always the first step. Good luck!

Susan
[[log in to unmask]]

--------------------

From: Rob Stein <[log in to unmask]>
Reply-To: "steve.museum general discussions" <[log in to unmask]>
Date: Thu, 14 Jun 2007 09:29:53 -0400
To: "steve.museum general discussions" <[log in to unmask]>
Conversation: [steve.discuss] Issue of 'incorrect' tagging
Subject: RE: [steve.discuss] Issue of 'incorrect' tagging

Hi Belinda, 

 The approach we're taking with the steve project is one where we are building tools that will allow individual institutions to 'review' terms submitted by users against their contributed works of art.  The termReview tool as it stands in its early stages shows an image and all the unique terms that have been submitted by the public...  The termReviewer is then able to mark a term as appropriate, invalid, a "near missing" or mark it as a possible foreign language term...  All of the term review data is stored in the same database as the rest of the tagging information and therefore would be available for use in presenting these terms back to the public, if that is infact the desire.

 A current hot topic of conversation amounst steve-ers is how to determining which terms are appropriate and how this information should be used...  perhaps some of the more eloquent steve members can elaborate more on this topic...  however there seems to be a distinction between using submitted terms to facilitate search by the public... (even if the term is incorrect)  versus re-presenting invalid terms with the work online...

 I'll see if I can dig up a screenshot or two of this tool in action... so that we can give you a better idea of where things are going on our end...

Rob

----------------------------------

From: [log in to unmask]  [mailto:[log in to unmask]] On Behalf Of Belinda  Chu
Sent: Wednesday, June 13, 2007 10:54 PM
To:  [log in to unmask]
Subject: [steve.discuss] Issue of 'incorrect'  tagging

Hi, 
We are  looking into a scoping exercise with regard to implementing social tagging on  our online collections and Picture Library.  The issue of users  incorrectly tagging an object was brought up by our Collections Information  Manager (eg: tagging an item as a landscape when in fact it isn't).  I  would be interested in hearing from any institutions out there that are  currently utilising social tagging that may have come across this issue and  how it was addressed. For instance do you moderate the tags before they go  live on your site.

Also, is anyone out there currently allowing user generated tagging to be applied in a bilingual environment on indigenous  artefacts (For example in our case we have Maori and Pacific Island atefacts  in which members of those various communities may want to tag in their own  languages...?)

I look forward to hearing back from you. 
 
Cheers Belinda 

---------------

forwarded by Sherman Clarke, NYU Libraries - [log in to unmask]

__________________________________________________________________
Mail submissions to [log in to unmask]
For information about joining ARLIS/NA see:
        http://www.arlisna.org/join.html
Send administrative matters (file requests, subscription requests, etc)
        to [log in to unmask]
ARLIS-L Archives and subscription maintenance:
       http://lsv.uky.edu/archives/arlis-l.html
Questions may be addressed to list owner (Judy Dyki) at: [log in to unmask]