Libraries and librarians connect people to information. That's what we do. So there's the information part and there's the connecting part. Librarians gather, collect, license, and purchase information in the form of books, scrolls, artifacts, journals, web pages. And there's a lot to selecting what to gather and keep and balancing competing demands to get ready for the connecting. The connecting part has everything in it from organizing the information and listing it in a catalog, to negotiating the information need, to training users..
Libraries in the modern era have balanced spending on discovery tools and spending on the information itself. (there's facilities stuff, too, but we'll leave that off). We've cataloged our books in books, on cards,in databases.... with subject headings. We've placed them on shelves by topic. That takes staff time (not free) and there are transaction costs if we do it cooperatively, and then there are the costs for the system to allow users to find things in the catalog. The system might be a knitting needle type thing that goes through aligned holes or it might be a big wooden box or a system of servers, web access, and desktop computers. All of that allows you to find books and journals and government reports by subject, title, author, as appropriate.
To find chemical properties or to find an article or conference paper on a topic, there are abstracting and indexing services. I guess chem abs goes back to 1908. Science abstracts (now INSPEC) something like 1898. Beilstein and Gmelin are older - like 1700s - right? You get the point. They're expensive, too. They always have been, as far as I know.
Ok. Now. Buying content. Tons of money. Of course. The serials crisis brought this to a head, but you can't have 30-40 years of increases greater than inflation and thousands of new journals and not have to pay the piper. So we rob the book budget to pay for more journals. We lay off staff. We close branches. We rob the humanities to pay for the sciences - it's true, because the sciences and engineering bring money in. Sort of like football. But anyway. And then we pay for everything again - to get it in another format. We buy another copy of a book, to have it as an ebook, and the backfile of a journal to get electronic access. We pay tens of thousands for things we already have, to have them electronically. And we pay to create institutional repositories to collect our own stuff.
On top of all of that - and we' don't really ever stop anything - we have to buy new tools to connect the above. We license (paying annual fees) open url resolvers, federated search engines, recommender systems. I can't imagine going without an open url resolver now. Some of our vendors spend a lot of money on their interfaces for books or journals - so you'll go and stay instead of just grabbing a full text article and going.
I hear a lot about libraries dropping research databases now - picking messy, multipurpose crappy ones because they come with full text or because they cover every topic under the sun, and dropping the high quality specialty ones. And this is killing some high quality specialty discovery tools. Libraries are also putting more money into things like federated search - which are still really only for the undergrad. The grad student and professor will probably either use the high-powered specialty tool (like chem abs) or google.
After years of robbing discovery will we now have to rob content? Some libraries are walking away from discovery, abdicating that to google. The problem now isn't information scarcity, it's having the right information at the right time. If we allow any piece of this puzzle to go out of control and take over the whole, then we will not be carrying out our mission.
Sort of like football
Oi! Not at all like football, thank you very much. :-) (For one thing, considerably less lucrative I should imagine.)
From my admittedly modern and forward-looking perspective, this sounds things happened the wrong way around. Perhaps it's because discovery just wasn't all that good in the past, but couldn't discovery now help you use the content you have more efficiently? Is it too late to turn things around, now that discovery works well?
Now that discovery works well? Does it? It seems really broken and it seems to me that there's a long way to go until it's close to where it needs to be. Discovery tools are absolutely necessary in using the content we have, but it's hard and expensive in programmer time and money. Nothing's too late - we just need to evolve discovery and content together. Actually, we still need to do some experimentation because i don't think anyone has a good recipe for where we need to go.
Christina, I agree with you that the discovery tools aren't where they should be. I guess a better way of putting it would have been "now that the price/performance is becoming reasonable". It's really a matter of perspective. From the point of view of someone who knows what's possible, things aren't great, but from a historical perspective, things are getting much better, and I think that in a few cases they've gotten to the point where it's truly useful. I should also have noted that the majority of the improvement in that ratio has come from the price side of things.
In other words, now that it's not prohibitively expensive and nearly worthless, because there are startups that have already invested that programmer time and money and have some products you can pretty much just plug in to your site.
Things have had to improve on the data availability side, the format/standards side, and the methodology side, all in parallel. As you might have gathered, I'm optimistic about where things are heading.