4

Are there examples of metadata — lists of what a site offers, that users search to help them decide what to download — for non-uniform collections like UCI ml or openml.org ?
Metadata is possible for collections of files with uniform attributes, e.g. metadata-standards-and-best-practices-for-data-dictionaries-for-csv-files-data, but tough for kitchen sinks, or libraries of books .
(I use the term "TOC", table of contents, interchangably with "metadata"; experts please suggest a better term.)

There's quite a range of TOCs for non-uniform collections, depending on

  1. users: window shoppers / experts who know exactly what they're looking for
  2. how users want to search the TOC: click through web pages / download a list of files with attributes / ...
  3. static / extensible: who can add comments, add new files ?
  4. human readable / machine readable / both.

With this huge range, standards, even guidelines, are difficult; no single kind of TOC can be good for everybody. But examples of TOCs good for particular corners of this range, with detailed descriptions, could help opendata providers to improve their TOCs — which way is up ?

(Should this be a community wiki ?)

Community
  • 1
denis
  • 456
  • 2
  • 11
  • I've heard it described as 'collection-level metadata' (to differentiate from 'granule-level metadata' when you're trying to describe individual items). – Joe Jul 18 '13 at 20:06
  • I feel like I'm missing something. Are you asking for examples of sites powered by tools like Socrata and CKAN, both of which users can search to help decide what to download? – Joe Germuska Jul 20 '13 at 13:39
  • @Joe Germuska, I'm looking for simple TOCs for machine-learning collections like UCIml and openml: simple to get started, both for users and site people. How simple, how big are Socrata and CKAN ? – denis Jul 21 '13 at 15:13
  • Socrata and CKAN are systems used to publish data. For example, the city of Chicago and the state of Illinois are customers of Socrata (see data.cityofchicago.org or data.illinois.gov) The US Federal government is moving to a CKAN deployment (next.data.gov). I have never tried to administer a repository using either of them. Are you looking to migrate data in UCIml or openml to a new tool? Or for to make something like those with other data? – Joe Germuska Jul 21 '13 at 16:48
  • @Joe Germuska, like those -- simplest possible for a few hundred files. (Improving existing sites would be great, but people naturally stick to what they have.) – denis Jul 23 '13 at 17:44

2 Answers2

3

I don't know of a single all-encompasing standard. You basically need to look at what attributes are common to all the collections you're tracking, even if they don't necessarily seem similar at first glance.

If you're dealing with data, then DataCite is generic enough to describe collections of data without getting into specifics for each scientific discipline. (There are other more specific but still collection-level descriptions available; eg, SPASE for space physics data).

For physical objects, you may need to look into why you're interested in the items. For a museum, they might track collections by who donated them, or where they came from (archaeological dig, etc.), while someone who's operating a store would track that as a supplier and might have information such as lead time needed for orders, what type of things they manufacture, etc.

You also run into strange issues of what exactly is a collection. I deal with cataloging data for the most part, and there are issues with what the proper aggregation should be for cataloging. See Laura Wynholds's 'Linking to Scientific Data: Identity Problems of Unruly and Poorly Bounded Digital Objects' for a bit of a background. (and see Renear, Sacchi & Wickett's 'Definitions of Dataset in the Scientific and Technical Literature' to see we can't even agree on what the collective noun actually means)

To explain this with physical objects -- a car is a collection of parts; to the factory that needs to assemble the car and the dealership that's selling it, they might look at the car as one thing, or an aggregation of multiple components. Even something as simple as a movie can be broken into multiple scenes, thousands of individual images plus audio tracks, etc.

Joe
  • 4,445
  • 1
  • 18
  • 40
3

http://schema.org defines a collection of schemas that developers and content owners may use to add additional markup to online resources. The schemas are flexible and extensible, which may help with your non-uniform collections.

From their introduction:

Many sites are generated from structured data, which is often stored in databases. When this data is formatted into HTML, it becomes very difficult to recover the original structured data. Many applications, especially search engines, can benefit greatly from direct access to this structured data. On-page markup enables search engines to understand the information on web pages and provide richer search results in order to make it easier for users to find relevant information on the web. Markup can also enable new tools and applications that make use of the structure.

The folks at http://microformats.org/ offer similar formats, listed here: http://microformats.org/wiki/Main_Page

Chris Gansen
  • 176
  • 2