Error during command authentication.

Wonderful response, Janine, as usual—I can hear our thought processes running in tandem throughout your comments, until the journals assessment project info suddenly thrust you ‘way out in front! Clemson would do well to do catch-up. We’ve weathered several major journals/subscription analysis projects but increasingly look to collaborate on these with our consortial partners: ASERL, Lyrasis, PASCAL, and the Carolina Consortium. That’s where cooperation yields real influence with vendors.

 

UC has had to make extra efforts in journals assessment, I understand, to buttress their challenge to Elsevier, no?

 

Nuance, and disciplinary eccentricities, totally. Plus, we are so staff-poor in subject areas that our head of acquisitions, who’s also chairing this committee I’m on, is loath to require deep, subject-specific collection assessments by our liaisons; those in the main library are spread thin and don’t enjoy the level of collection knowledge & control that I have cultivated here in the Gunnin—the manageable size of the Gunnin is my superpower, vs. the size and age of their legacy collections. All of this to say we’re keen on tools that would enable us to benchmark against our list of R1 comparison institutions, at least. Whether GreenGlass (for what, $20K a pop?) or WorldShare CE—OCLC holds most of the cards.

 

We use LibQual every few years to keep up with how faculty and students perceive and use the libraries, but find it most useful for evaluating our services/performance rather than collections. Without added incentives (and depending on the timing of the survey in any semester), it has proven unreasonable to expect a significant number of faculty/grad students to offer up either praise or constructive criticism re: collections. They love US, great, but the input re: resources doesn’t cohere into useful guidance on collections decisions.

 

And bibliographies? Increasingly a lost art. Suddenly I feel old—perhaps because, like you, I’ve poured energy into ‘core’ lists, those precious artifacts of the next ten minutes.

 

Using the serials lists from gold-standard subject indexes for journals subscription analysis is the best I have come up with, as well, for that format, but such a project is fraught for any active academic library. Time, personnel! Danged if we do and danged doubly if we don’t, if we mean to climb.

 

I absolutely agree on the artifact level—the Gunnin has its own rare books collection, some 770 items. No, determinations of quality cannot depend upon circulation, or even usage—an opinion that puts me at odds with library staff--often. Utility and ‘relevance’ TODAY are not the be-all and end-all.

 

At this point, our R1 benchmarking committees are engaged in reconnaissance of the field of endeavor. Strategies for engagement are next up. I very much appreciate your advice and perspective.

 

 

 

Hi Kathy,

 

Thanks for asking the question: an interesting topic to ‘ponder!’

 

My first reaction is that it’s all about “La Nuance” (as my spouse frequently likes to quote Gustav Ecke saying in his days teaching at the East West Center; and Ecke was apparently somewhat of a Francophile, so maybe he was quoting Verlaine?).

 

Second reaction, in the context of our current info. lit. language (“Authority Is Constructed and Contextual”) is that collection quality is contextual. Presumably the collection was developed over time according to the various collecting policies that would have been in place? (Which presumably changed when curricular and research needs changed, and in response to fiscal constraints?) So is the question whether the collection is still serving current curricular and research support needs? Or that fiscal constraints are such that the collection is not able to support these needs? Because I would not want to compare the collections of institution X with Z and say one is higher quality than the other if they are serving very different needs and are being built according to completely different collecting policies… And the ILL net lending or net borrowing issue is interesting in the context of consortia, space constraints, just in time vs just in case collecting. My view on ILL is that a one-time borrowing of something needed by a researcher does not necessarily mean that you need to acquire it for your own library. But if students need something on an ongoing basis (or at least more frequently than a one-time researcher’s need), then yes, at some point it moves into the realm of something that the library needs to collect. So, if I decide not to collect something because my constituents can easily go see it at the Getty Research Library or borrow it from UC Berkeley, does that mean that my collections are inadequate? (I do this so that I can afford to buy the content that is needed at UCLA and I’m pretty convinced that UCLA has quality collections!).

 

Third reaction: perhaps it’s much easier to compare collections that support specific professional practice degrees that are accredited by professional agencies, when these accredited degrees have specific curricular requirements. But that might only serve to compare the minimum baseline collection needed to support these programs (rather than helping identify quality). In a sense, I’d be more interested to know whether collections in particular disciplines have all the titles in that discipline’s ‘standard bibliography/bibliographies.’ But that would only help to evaluate older collections: not so good for newer content that has not yet made it into such bibliographies. And I’m not aware of an easy, automated, way to do this. (I’ve always wished we could add that kind of information to our catalog records—somewhat akin to adding a repertory number as was done with Lugt numbers, but it would be a ‘standard bibliography’ numbering system). But maybe WorldShare can do this? They can compare your collections with a selection of lists (see: https://www.oclc.org/en/collection-evaluation/features.html.). And another point about the ‘subject authority’ of the resource producer: does that mean that items acquired because of their artefactual value (the typography, the illustrations, the book design/graphic design, etc., and artists’ books) rather than for the subject content would be excluded from being considered quality? And how does that work for works of fiction, poetry, plays, film scripts, children’s books, etc.?

 

Additional thoughts: would comparing your serial subscription lists to the lists of titles indexed in core article databases in various relevant disciplines be an indicator of ‘quality?’ Or has that become maddeningly difficult in this day & age of aggregators? (We’ve used GoldRush, https://www.coalliance.org/software/gold-rush, to do some content comparison).

 

And in all seriousness, the UCs have done a lot looking at evaluating scholarly value of our licensed scholarly resources. (Mostly to help make non-renewal decisions or to help us negotiate better deals).

“The strategy involves using objective metrics to calculate the value of scholarly journals and identify titles and publishers that make a greater or lesser contribution to the University’s mission of teaching, research, and public service. The value-based process is objective and quantifiable and is based on measures of utility, quality, and cost effectiveness, with a goal of alignment to UC’s user communities and programmatic needs.

 

A key aspect of this strategy is the use of a Weighted Value Algorithm by Subject Category to assess multiple vectors of value for each journal title under review. This methodology compares each UC e-journal title licensed for systemwide use at the University of California against other UC-licensed titles within the same subject category according to a variety of objective value indicators, in order to arrive at a comparative value for each journal within the UC shared licensing portfolio.”

The original analysis covered over 8,600 journals in 36 UC licensed e-journal packages. Since then the California Digital Library researched ways to provide improved benchmarks. “Staff interviewed a variety of academic experts in the fields of statistics, economics, and library and information science and conducted a written survey of librarians from the U.S. and Canada known for their expertise and experience with journal value assessment. Advice was also collected from UC librarians across a variety of subject specialties.”  As a result, some changes to the Algorithm were implemented to better facilitate our journal collection assessment processes. (Sorry, I can’t share the details).

 

We’ve discussed using GreenGlass for some specific projects at UCLA (See:  https://www.oclc.org/en/sustainable-collections.html) but have not yet done so.

 

Would a special LibQual+ survey of users’ perceptions of the collections help assess your users’ perspective on the issue? (see: https://www.libqual.org/home)

 

You might like to look at these reports:

https://www.rluk.ac.uk/wp-content/uploads/2014/12/RLUK-UDC-Report.pdf

https://www.oclc.org/content/dam/research/publications/2017/oclcresearch-hazen-symposium-2017.pdf

 

I’d be curious to have others share their report recommendations. And to hear everyone’s thoughts about ‘quality’ collection assessment as well!

 

Good luck and I hope you get to share your results!

 

Best wishes,

 

 

 

Dear Collective Wisdom,

 

I can run all varieties of collection reports in our library management system, but this latest challenge is LARGE: how to assess the quality of our entire university library’s holdings? What does quality mean when it comes to resources, and is that an upfront value or one tested by time and application? (I know—the answer is ‘yes.’)

 

First thoughts: some algorithm involving currency (but not necessarily--varies wildly by subject) and usage (the higher the quality, the more in demand—unless it’s just the best we have on hand to offer, which then points to…), held up against interlibrary loan requests (i.e., what should we own but don’t) and informed (this is the real bugger) by ‘best of’/’core’ standards per subject area, to the extent these exist (which involves much input by expert liaisons, and should also be demonstrated by usage, otherwise ‘core’ is just a vanity class).

 

Yet another facet: the subject authority of the resource producer…the point at which we really get into the weeds…

 

How would you go about assessing the quality of your library’s holdings, or is that assessment a ghost ship?

 

[Yes, I’m on a dean’s committee to determine what it means to be the library of an R1 Carnegie-rated university, and assess our libraries against that standard. Clemson is already R1 but we mean to do even better. And this is for ALL holdings, all disciplines.]

 

Kathy Edwards

Associate Librarian

Research, Instruction & Collection Development

Emery A. Gunnin Architecture Library

2-112 Lee Hall / 864.656.4289

 

~~~~~~~~~~~~~~~~~~~~~~
Mail submissions to [log in to unmask] For information about joining ARLIS/NA see: http://www.arlisna.org/membership/join-arlisna Send administrative matters (file requests, subscription requests, etc) to [log in to unmask] ARLIS-L Archives and subscription maintenance: http://lsv.arlisna.org Questions may be addressed to list owner (Judy Dyki) at: [log in to unmask]
~~~~~~~~~~~~~~~~~~~~~~