Leeds Met Repository Open Search Version 2.0

This is a bit of a trailer for our shiny new interface that I hope will go live in the next week or so and a run down of some of the new features.

It’s far from perfect and should still be seen as a beta – we very much need real users to start using it and I’m feeling a little nervous about how it will be received as I know how much work Mike, in particular, has put into it.

The interface has evolved from an SRU client developed for by IRISS – http://www.iriss.org.uk/learnx – which is available under GNU General Public Licence v.3 at http://code.google.com/p/sruopensearch/ (N.B.  We still intend to release our modified code under a similar licence.)  Learning Exchange Open Search is a great front end for searching intraLibrary but with just a simple search box lacked advanced search functionality that was essential for us.  We also wanted to use intraLibrary to manage resources for teaching & learning aswell as facilitating Open Access to our research collection in accordance with the EPrints model.

The tabbed interface incorporates an “Advanced search” form that allows users to cross reference multiple fields specifying AND/OR and they are also able to search for either “Research” or “Open Educational Resources” which uses authentication tokens to return results from the appropriate collections in intraLibrary:

There are also big changes in the way that results are returned; Mike has been able to use a unique identifier to build individual pages for each record so that a search will return a set of results that indicates whether or not each individual record has the full text available:

These titles then link through to a static HTML page comprising all of the metadata associated with that record including a published URL and, where the full text is available, a link to the PDF in intraLibrary:

This static page should be indexed more effectively than was the case before though there is one small fly left in the ointment in that the public URL generated by intraLibrary that is used to download the full text is dynamic which means it cannot be indexed by Google; I’m not sure if it will be possible for Intrallect to do anything about this though they are aware of the need for full text indexing and are looking into the problem.

Separate HTML pages for individual records

I’m returning here to an old theme that is still nagging away at the back of my mind and that I think still needs exploring further as the functionality of the SRU interface develops; both by Mike and I and by Intrallect in the context of their ongoing development of the research repository aspect of intraLibrary.

Can we generate individual HTML pages for records such that a search query could generate a list of hyperlinks that point to those individual pages rather than to the location URL stored in intraLibrary which is currently the case?  This would more closely approximate the way that EPrints and DSpace work and potentially solve the Google problem by providing an easily indexable page of static HTML for search engine spiders to crawl.  Could these pages also have nice, short, human readable URLs instead of convoluted search strings / machine-generated public URLs from intraLibrary.  Again more like EPrints/DSpace.  Currently the only way I can give a link to an item is:

http://repository.leedsmet.ac.uk/main/search.php?q=promoting+open+access+to+research&x=22&y=26&exacttext=1

(The SRU search string that will provide the metadata)

Or

http://repository-intralibrary.leedsmet.ac.uk/IntraLibrary?command=open-preview&learning_object_key=i05n27905t

(The machine generated public URL for the actual PDF)

I’ve recently been adding RSS feeds to http://repos-dev.leedsmet.ac.uk/main/browse.php and another issue (aside from the fact that the wrong field is exposed by RSS) is that these also point to the location URL stored in intraLibrary – the PDF in the case of full text but the published URL in instances where there is a citation only.  It would be much better if these feeds could point at a Leeds Met repository metadata record.

I simply do not have the technical insight to know whether any of this is achievable at all and, if it is, how big a job it will be.

Using the search interface (or interfaces) to search the repository for different types of content.

The search interface – http://repository.leedsmet.ac.uk/main/index.php – is now returning results in a manner appropriate to published, peer reviewed research papers which is fine as far as it goes but what about other types of content that we may want to expose externally?  I was already dimly aware of the issue with respect to conference papers, for example, which will require slightly different metadata within intraLibrary and which will then need to be formatted appropriately when returned by the search interface.  However, the issue was thrown into relief at a preliminary meeting yesterday to discuss using the repository for heritage photographs and other digitised content from the library archives and special collections which will require entirely different metadata.

After the meeting I promptly defined a new collection and uploaded a photo with minimal metadata – just title and description, not even bothering with classification at this stage, I’ve also exposed the collection to external search so it is being returned by the search interface which is obviously trying to format according to Mike’s research metadata template which is frankly gobbledegook.

So…I need to establish whether we can identify different types of content (probably by collection or by an appropriate metadata field) such that, say, the search interface ONLY returns research papers, then if photos are in their own collection can we filter out that collection so that it will not be returned by http://repository.leedsmet.ac.uk/main/index.php at all i.e they won’t be returned by any keywords in the metadata etc? Then we could, say, implement a second interface configured to search the ‘Heritage photos’ collection only and return results in an appropriate format i.e it would not return any research results as those collections would be filtered out?

OR might we want to search the whole repository from a single interface and could we perhaps apply different templates to different types of resource from a common interface? (My initial reaction is that separate interfaces would be better.)

So I’m hoping Mike will tell me this is dead easy and not impossible!

Organisational structure, metadata and resource discovery

In terms of the organisational structure and metadata for our repository we must consider that:

• Resources must be discoverable both internally (i.e. within intraLibrary itself) and, where appropriate, by external systems e.g. search engines

• Visitors to the repository must be able to browse repository content in an intuitive manner.

These requirements are related and can be fully realised by integrating a number of features of the software:

1. Groups/Processes
2. Collections
3. Application profiles (Metadata schema)
4. Taxonomy (or taxonomies)

Groups/Processes

intraLibrary uses IEEE LOM (Learning Object Metadata); the administrator can define multiple application profiles (metadata schema) and templates which can incorporate subsets of LOM that may be differentially applied to groups and collections based on content type.

We currently have two groups/processes defined (we may well need more):

• ‘Default – Upload and Catalogue’ is for research materials and uses an appropriate application profile  that is based on Dublin Core – though strictly speaking it is a subset of LOM mapped as closely as possible to unqualified DC (plus some bibliographical citation info that Intrallect have included in v3.0) (see below for further info).

• ‘Learning Object Contributors – Upload’ uses a different application profile appropriate to the respective types of resource.

Collections

Collections in intraLibrary serve several purposes:

• It is the means by which the administrator defines whether or not resources are discoverable by external systems or whether they can only be discovered by an authenticated user of the Leeds Met Repository.  The first scenario will be necessary for Open Access research material; the second scenario is likely to be necessary for some Learning Objects (though we may also want some LOs to be searchable and discoverable externally.)

• It was agreed at the last consultancy group meeting that the main organisational structure should not be based on Leeds Met faculties; however, it is still useful to be able to group (and browse) content in this way which can be achieved by the use of collections which can easily be renamed in the event of faculty name change. Moreover, resources can be stored in multiple collections and easily be moved between collections.

A user decides which group/process and which collection (or collections) they want to upload to as part of the workflow:

Collections are currently configured so that resources in the “Learning Object” collection will not be searchable by external systems whereas resources in the faculty collections will (to facilitate Open Acccess)

Metadata for Research material

Based on current practice, the most appropriate metadata schema for Open Access research material is Dublin Core which can be mapped with IEEE LOM.

Dublin Core	IEEE LOM equivalent
Title	Title
Description	Description
Subject	Keyword/Classification
Creator	Contributor
Contributor	Contributor
Publisher	Contributor
Format	Technical Format
Identifier	LOM Identifier/Location of Resource
Type	Type of Resource
Rights	Statement of Copyright and Restrictions
Coverage	Geographic / Time Period Coverage
Relation	Related Resource Description
Date	Date of Contribution
Language	Language of Resource

IEEE LOM permits several fields to be duplicated; this is necessary, for example, to accommodate the Dublin Core fields ‘contributor’; ’publisher’; ’format’ in multiple instances of the LOM field ‘contributor’.

intraLibrary V3.0 also incorporates Bibliographic citation metadata:

Metadata Field

bibliographicCitation

genre

title

volume

issue

date

spage

epage

NB. No field for ISSN – can be accommodated in a duplicate ‘description’ field.

Metadata and the search interface

The search interface is currently configured to return results according to a default template and returns only a subset of available metadata: Title; Description; Creator; Contributor; Publisher; Type; Subject; Copyright:

The metadata that is returned can be extended and we will wish to modify the way in which the information is presented and formatted, as standard Harvard style references for example. Mike T is currently working on this and there are one or two issues with respect to standardising the meta-data template and identifying and formatting the correct fields from the returned XML – as I understand it the SRU is currently configured to pick up dc and Mike is investigating whether it could pick up the lom instead which is more detailed – more on this soon!

NB.  Bib cite metadata is not yet searchable by SRU – Intrallect aim to incorporate this, though not before Summer 2009.

Organisational structure

The organisational structure, of course, is in many ways auxiliary to more powerful methods of resource discovery which utilise the full metadata record. Nevertheless it is still important to establish an intuitive system that enables users to browse and that can be used to easily present repository content for demonstration and promotional purposes.

• Review of other repositories of research content suggests that Library of Congress Classification (LCC) might be appropriate for our needs. The full system is complex with many more levels of specificity than we require and it is proposed that we use the top two levels of the classification only; we will also have the flexibility to extend the classification if necessary.

• The Joint Academic Coding System (JACS) subject hierarchy is generally used by UK institutions to identify the subject matter of programmes and modules which potentially makes it more suitable for Learning Objects – it is the system used by the national Learning Object repository JORUM for example.

• Resources in intraLibrary may be categorised against multiple classifications and we may wish to add other classification systems in the future e.g. Medical Subject Headings (MESH)

Incorporating browse functionality into the search interface

Users will not generally browse the research collection from within intraLibrary itself but from the external search interface. We will want to incorporate browse functionality into the interface to browse by subject or by faculty (the use of faculty ‘collections’ as discussed above easily facilitates this).

This functionality is partially integrated but will require further technical development:

Staff Development Festival

Leeds Met is a University of Festivals, we are often told, and the next fortnight will be the Staff Development Festival which will provide a unique opportunity to promote The Repository.  And I’m going Scuba diving, albeit in a swimming pool.

All the repository pieces are now in place and I intend to demo the search interface and the repository proper, presenting it as a semi-blank canvas that now needs to be painted upon by the University community.  I’ve already had some feedback from Jonathan Long, Director of the Carnegie Research Institute and a member of the consultancy group who would like a search to return formal Harvard references for each item emphasising that one of the reasons for setting up the repository is to increase the number of citations – the interface is just ‘out of the box’ at the moment and returning results in the manner of the IRISS interface – I’ll gather input over the next fortnight and Mike should be able to do some customisation when we have a clearer idea of what people want.  I might even have a go myself although, with no knowledge of php, I can’t make head nor tail of the site files, my web skills having stalled at basic HTML, CSS and (very basic) Java Script.  Mike has already added browse functionality which is, for the moment, based on faculty structure – I’ve set up a collection within intraLibrary for each faculty and it is these that Mike is using to generate the results although it should also be possible to use metadata fields – I think we will map DC ’subject’ onto LOM ‘keyword’.  It might be tricky to incorporate numbers of records after browse links and I’m waiting to see what Mike has to say on this.

Anyway, for now, I have 5 citations per faculty which is adequate for initial demonstrations and I’m working on some full text content – several of the citations I’ve uploaded are RoMEO green/yellow so it’s just a matter of getting hold of author versions.

As for promotional material, I’ve ordered a big purple recoil stand similar to that for the Library and I’ve three info sheets to print up in quantity:

The Repository is an introduction to the project and to IRs specifying our dual remit for the Leeds Met repository.

Open Access: What’s in it for you? emphasises the evidence that OA increases citation (using a graph from Steve Lawrence’s seminal article Online or Invisible? (2001) which is a bit out of date but by far the clearest visual representation I have been able to find.)

And

Copyright presents a very simple flowchart of the (self)-archiving process.

I shall also try to put together a narrated presentation to run when I’m not there.  A couple of lap-tops and we’re away!

Incidentally, here is a link to the search interface:

http://repos-dev.leedsmet.ac.uk/main/index.php

(Currently only accessible from a Leeds Met IP)

Getting there, slowly but surely

The Repository is really starting to take shape; the search interface has now been installed on a development server (as discussed previously, we are using the IRISS SRU client) and is returning very satisfying results on my test content. Now we can start adding the extra functionality (browse, advanced search) – well Mike T can at any rate, and my more technically inclined colleagues – and then to customise the look and feel, though Mike has already added an enormous Leeds Met Rose!

Ongoing development of the interface will also feed into PERSoNA – in a meeting today with John and Mike, Wendy and I discussed one initial approach being to embed the search box/additional search functionality from the interface into a google app (feeding into Leeds Met’s developing partnership with Google) or some kind of generic plug-in or widget. I’ll try to expand on this at some point on PERSoNA News and ask for some pertinent blog input from John and Mike.

And I’ve uploaded my first research paper! A colleague in the library has a paper published in the Reference Services Review – which is a subsidiary of Emerald – and RoMEO green; Do Academic Enquiry Services Scare Students? (This link to the Emerald full text, not the author’s version in The Repository.)

At the moment I am very much focussed on the Staff Development Festival in September and have also been uploading citation information for demonstration purposes – I hope to use the Festival to encourage folk to supply full text copies of their research papers which can then be uploaded in line with publishers’ copyright transfer agreements and we can finally start building that representative body of content. I’ve set up a basic taxonomy within intraLibrary based on Leeds Met faculties and intend to upload 5-10 citations per faculty which I’m linking through to publishers’ abstract pages where possible. This should give us the opportunity to review metadata and get a preliminary idea of the workflow as well as illustrating to people why they might want to release copies of their work from behind subscription barriers (look, there can be links to your work all over the web but you can’t get any further than the abstract without a subscription fee.) The final choice of taxonomy should also be informed by demonstrations to academic staff – we already know that the steering group does not want to base it on faculties as the major organisational structure.

Mike has said that he can do some very preliminary customisation of the search interface before the festival to illustrate how the external browse functionality might work – this will be based on the taxonomies as they currently appear within intraLibrary and, given the short amount of time, will be for demonstration purposes only and probably won’t return dynamic results but should give people the opportunity to visualise the interface and comment on its development.

Repository Steering Group meeting: 22nd July 2008

The staff development festival in September is a unique opportunity to promote the repository and our agenda for yesterday’s meeting aimed to get some much needed input from the steering group before the quiet month of August.

Item 1. Recap of previous meetings:

Documentation approved.

Item 2. Update on progress with intraLibrary

2a. Configuration:

Search interface (SRU):

Getting the search interface on line is the first priority – my request for the server is still pending with IMTS but I hope we can install the IRISS interface as is within the next few weeks (JohnG is installing it on a local server as we speak which can then be tranferred to our Leeds Met domain when it is available) and I think it will be straightforward to switch the CSS to get a very rough Leeds Met branding.

Content structure:

This is also crucial and needs to be put in place ASAP. Several members of the group expressed the opinion that it should not be based on faculties which tend not to be fixed entities within the university; it was also thought that such a schema would not reflect institutional emphasis upon cross-disciplinary research. There was consensus that organisation at the top level should be by content type (i.e. Research/Learning Objects) but exactly what hierarchy should be employed beneath is still not clear (library of congress subject headings?). We also need to make a decision on what other material types will be accomodated in the prototype (e.g. Dissertations and Theses)

Landing screen:

Technical challenges aside, the current conception of the landing screen is that it will essentially use the same template as the search interface i.e. it will be branded the same and share the same look and feel; it will also share some of the same functionality and link back ‘home’ to the search interface.

Given the close relationship between these configuration issues, a sub-group was identified that will liaise as necessary to develop the content structure; branding; look and feel; usability and will also inform the technical development of the additional functionality.

2b. Policies:

The group was briefed on the types of policies that need to be developed (see last post) with emphasis on the fact that the ’standard’ institutional repository policies may be insufficient for our requirements given our wider remit (i.e. not just research outputs). A sub-group was identified that will liaise as necessary to develop suitable policies.

2c. URL:

The suggestion mooted – repository.leedsmet.ac.uk – was deemed suitable by the group

Item 3. Content for the repository:

To discuss method of contacting researchers / research active staff and soliciting content

Review of draft correspondence for research active staff and discussion of when this would most usefully be disseminated; consensus that it would have the greatest impact some time after the staff development festival. Content was broadly approved though it was suggested that greater emphasis be placed on the benefits of OA to citation and the increased importance of citation under proposals for REF (to replace RAE).

Emphasis was placed on the need to identify and recruit interested parties within specific faculties/research groups to help drive the advocacy process to the wider community; liaison with University Research Office for appropriate contact lists.

(NB. This is an ongoing process that is already underway but will increase in profile with the implementation of the prototype system.)

The Staff development festival confirmed as a key opportunity.

There was discussion whether content would be full text only or would also comprise citation of material that we do not have copyright permission to make available as full text (i.e. bibliographic reference only). Given that including such material will enable us to ‘hit the ground running’ and considering the increasing importance of citation data/bibliometrics for the RAE / REF the consensus was that citations should be included at the outset.

Item 4. Authentication

It was emphasised to the group that we can be fully functional as a mediated repository without the need for authentication in the first instance.

A representative from IMTS was able to inform the discussion in the light of recent feedback from Intrallect and will continue to liaise as necessary.

Item 5. Integration with other Leeds Met systems

In light of the decision to include citations as well as full text, an important early integration will be with SFX such that citations in the repository can incorporate a link to Leeds Met holdings of subscribed material; hardly Open Access as it will only be available to authenticated staff and students but will offer another local route to that material and can also be used to generate data on OA friendly publishers and perhaps to raise awareness of OA.

The PowerLink to X-stream should also be a priority such that it is operational at the earliest opportunity.

NB. Precise functionality of the PowerLink still needs to be determined.

Other systems flagged up for integration were iTunesU and the streaming server; pending investigation!

The next meeting of the steering group will take place after the staff development festival, probably late September/early October.