March 30th, 2007
Ah, to be young. I just finished 39 straight hours of coding and after a quick 3 hour nap, ready to tackle the last remaining tickets on the 0.8 trac list. One small niceity that will make it into 0.8 is a diagnostic tool that will allow you to test a configured database before using it in real-time in LibraryFind. This will let you make sure that LF understands your OAI server, can read your OpenSearch resource or can talk to your Z39.50 client. While the 0.8 release won’t have a lot of updates in the admin functionality (that’s on the 0.9 cycle) — there will be some improvements that I think will make life a little easier for folks. Of course one of those improvements is the yet to be written ruby component for loading coverage data into LF. We have a PHP script that currently performs the feat, so I guess I ought to get coding again.
Oh, and before I forget. I’d like to set LF up so that you can use SFX as your OpenURL resolver — however, we are not an SFX shop. I understand that SFX provides a method for querying and getting data back in XML. If someone with SFX (hey Ross) has some documentation or some sample urls so I can see what the response format looks like — I’ll make sure 0.8 supports it.
Posted by reeset
No comments »
March 27th, 2007
Happily, we just about have 0.8 baked for release. I’m spending my week closing small issues and adding a few odds and ends to the tool. Lots of changes will be in version 0.8, including but not limited to:
On the API-side
- New thread pump: All plugins now will be placed into the thread pump allowing the process to run faster. In 0.7, not all items were threaded because of some issues that I had working with ferret in a threaded environment. Well, the new plug-in architecture for protocols seems to have addressed this problem — so all items are now threaded meaning that results should return faster
- OpenSearch Query support: LF will support the query of OpenSearch resources using RSS1.0 & RSS2.0. It will give me a chance to show off the new plug-in architecture for protocols. Also, I’ll try to include an example of an OpenSearch item in the sample database.
- Lots of code refactoring. Ranking, spell checking, etc. have all been placed into their own classes to allow for easier customization.
- EbscoHost Webservice connectors: I’ll be including a webservices connector plugin and a sample of how this works with EbscoHost. This will improve search speed a lot when dealing with EbscoHost, and will allow you to query using fuzzy searching
- Lots of fixes to 0.7, like removing changes to the harvester, some OSU specific information in the application.rb file, updating the Rails version in the vendor directory (1.2.3)
- New api for returning specific items within the cache
- Updated caching code
- LF can now act as an OpenURL server and client
- COinS support on the results screen (I hope)
- more
On the UI side
- Customizable tabs
- Adding a cart feature
- Replacing the Find in library link with an ILL link if the URL isn’t resolved.
- New UI Templates — this is a biggy. Tami has spent a lot of time redoing the public UI and the admin UI making it much cleaner and much easier to work with. I believe that this will make customization much easier for others.
- lots more
In addition to the 0.8 release, I’m hoping that we will get an opportunity to write up how we are currently putting LF into production. At current, our Unix admin is going over a number of different options available for running Rails applications, so once we finish stress testing (doing 500,000 random queries against the system and watching for it to fall down), we’ll post the setup information.
There are a few things in this release that didn’t get finished that will get moved to the top of 0.9. Most of these are administrative. I want to bring back the distributed knowledge-base to the application to make it easier to install the application and share config information. Also, the OpenURL coverage loading. This should be easier, but still, the documentation will be light and the admin support is still lacking. That will be high on the todo list. Also, if someone out there using SFX would be willing to provide some documentation on the SFX query/return format, I’ll make sure LF supports SFX so that you can use it, rather than the LF OpenURL tools if you like.
Anyway, that’s the quick update from our end. As I say, we are hoping to have this fully baked and out sometime next week, but it may slip later due to testing and writing new documentation — but we’ve been keeping busy and hopefully there will be something new for everyone. :)
–TR
Posted by reeset
No comments »
March 27th, 2007
I probably should have posted this on this page for those watching the blog for LibraryFind development. Please see http://oregonstate.edu/~reeset/blog/archives/428 for information about some of the current libraryfind code refactoring.
–TR
Posted by reeset
No comments »
March 14th, 2007
Ok — so we’ve had a few folks (our development group included) that have been running into problems running the harvester.rb file in LibraryFind after installing the 0.5 version of the ruby_oai gem. After updating ruby_oai 0.5, the following error will likely occur when the harvester is first run:
Error: superclass mismatch for class OrderedHash
I’ll admit, this type of error is a little disheartening. No code had changed (on our end), but a dependency update breaks something. So what did it? Well, it appears that this error showed up a lot when rails moved to the 1.2 branch. Rails apparently has its own OrderedHash class that was conflicting with classes found in dependencies (apparently, this is a common class name). It looks like one of the dependencies to the ruby_oai gem causes the conflict in our case (gem chronic (0.1.6)) A natural language date parser. So, Rails 1.2.3 modifies how this class is called. See this link: http://dev.rubyonrails.org/changeset/4318 to note the bug ticket. It’s been fixed in 1.2.3 (maybe 1.2.2, though I was seeing the problem while using 1.2.2).
So, if you are running into this issue, the following should correct this problem (at least, I’ve verified that it does in my 4 development instances). I’m assuming that you have already updated your ruby_oai gem to 0.5.
- Update rails to 1.2.3, including dependency updates (since the problem is really in the active support gem, you need version 1.4.2)
- Remove the vendor directory (or rename it). We (the development group) will be spending some time this week discussing how we deal with the vendor directory moving forward, because in this case, the fact that its a bit out of date is biting us in the back-side. We’ll have to consider what we are really getting by keeping it here (at least some of the gems currently stored in the directory)
- In the environment.rb file — you need to change the RAILS_GEM_VERSION to 1.2.3.
- Restart your server
That should correct the incompatibility. As I said, I’ve used this process on my 4 development branches and in each case, this solved the issue, and I believe that Jeremy can verify that it solved the issue for him as well. So, if you are seeing this problem with the harvester.rb file, this should help get you rolling again.
–TR
Posted by reeset
No comments »
March 12th, 2007
So since Code4Lib, one of the things that I’ve been toying with is taking ferret out of LibraryFind and replacing it with Solr. On the one hand, I like the faceting that Solr allows (though this is only somewhat useful in a LibraryFind federated context) but I don’t like adding the additional dependency. However, I’ve been spending some time this weekend finally getting a chance to play with the tool. I extracted my entire catalog (~2 million records), ran them through MarcEdit to get me a set of XML files for loading and created myself a project. For this first run, I went fairly simple. I store most of the MARC data (not all) and do some creative indexing (which would likely change in a production environment) and didn’t do anything with field boosting (which I most certainly would) and gave Solr a spin.
Well, I’m pretty impressed. One of the things I like about it most is how easy a nut with a little bit of time on their hands can implement a very cool replacement to a public opac. Given the time change this weekend, I decided that this would be a good weekend to work a lot — so I keep myself up for ~46 straight hours. I spend 6 of those hours (between 11 pm - 5 am Saturday to Sunday) working on this project and this is what I’ve quickly come up with:http://grok.library.oregonstate.edu/solr. This example is running on an underpowered dev machine — and still — performance isn’t that bad. I imagine that if I spend some time really thinking about the data I’m storing and the indexing, it would work even better. What I like even more — without using any frameworks, this represents ~500 lines of PHP code including the display elements. And my PHP code is a bit more verbose that it needs to be, since I don’t ues many of the PHP simple xml parsing tools. I prefer to work with XML in php as a structure. But the ~500 lines includes amazon interaction, solr interaction, search/display. Now that’s nifty.
So how does this fit into LibraryFind? Well, this actually could serve as a template for develping an “OPAC” view in LibraryFind that would allow users that want to keep the OPAC front an center in their library, an easier way to make that happen. In essence, allowing users to build “next generation” OPACs through the libraryfind interface. It could also work ancillary to libraryfind. Since LibraryFind works as a webserver, you could also build a tool like this example in Solr and then call LibraryFind through the webservice to integrate it into this view — so lots of options. I guess well see what the LibraryFind group wants to end up supporting.
But all in all, a nice weekend.
–TR
Posted by reeset
No comments »
