Network Working Group                                            C. Adie
Request for Comments: 1614        Edinburgh University Computing Service
RARE Technical Report: 8                                        May 1994
Category: Informational


                Network Access to Multimedia Information

Status of this Memo

   This memo provides information for the Internet community.  This memo
   does not specify an Internet standard of any kind.  Distribution of
   this memo is unlimited.

Abstract

   This report summarises the requirements of research and academic
   network users for network access to multimedia information.  It does
   this by investigating some of the projects planned or currently
   underway in the community.  Existing information systems such as
   Gopher, WAIS and World-Wide Web are examined from the point of view
   of multimedia support, and some interesting hypermedia systems
   emerging from the research community are also studied.  Relevant
   existing and developing standards in this area are discussed.  The
   report identifies the gaps between the capabilities of
   currentlydeployed systems and the user requirements, and proposes
   further work centred on the World-Wide Web system to rectify this.

   The report is in some places very detailed, so it is preceded by an
   extended summary, which outlines the findings of the report.

Publication History

   The first edition was released on 29 June 1993.  This second edition
   contains minor changes, corrections and updates.

Table of Contents

    Acknowledgements                                                2
    Disclaimer                                                      2
    Availability                                                    3
    0. Extended Summary                                             3
    1. Introduction                                                10
      1.1. Background                                              10
      1.2. Terminology                                             11
    2. User Requirements                                           13
      2.1. Applications                                            13
      2.2. Data Characteristics                                    18



Adie                                                            [Page 1]


RFC 1614        Network Access to Multimedia Information        May 1994


      2.3. Requirements Definition                                 19
    3. Existing Systems                                            24
      3.1. Gopher                                                  24
      3.2. Wide Area Information Server                            30
      3.3. World-Wide Web                                          34
      3.4. Evaluating Existing Tools                               42
    4. Research                                                    47
      4.1. Hyper-G                                                 47
      4.2. Microcosm                                               48
      4.3. AthenaMuse 2                                            50
      4.4. CEC Research Programmes                                 51
      4.5. Other                                                   53
    5. Standards                                                   55
      5.1. Structuring Standards                                   55
      5.2. Access Mechanisms                                       62
      5.3. Other Standards                                         63
      5.4. Trade Associations                                      66
    6. Future Directions                                           68
      6.1. General Comments on the State-of-the-Art                68
      6.2. Quality of Service                                      70
      6.3. Recommended Further Work                                71
    7. References                                                  76
    8. Security Considerations                                     79
    9. Author's Address                                            79

Acknowledgements

   The following people have (knowingly or unknowingly) helped in the
   preparation of this report: Tim Berners-Lee, John Dyer, Aydin Edguer,
   Anton Eliens, Tony Gibbons, Stewart Granger, Wendy Hall, Gary Hill,
   Brian Marquardt, Gunnar Moan, Michael Neuman, Ari Ollikainen, David
   Pullinger, John Smith, Edward Vielmetti, and Jane Williams.  The
   useful role which NCSA's XMosaic information browser tool played in
   assembling the information on which this report was based should also
   be acknowledged - many thanks to its developers.

   All trademarks are hereby acknowledged as being the property of their
   respective owners.

Disclaimer

   This report is based on information supplied to or obtained by
   Edinburgh University Computing Service (EUCS) in good faith.  Neither
   EUCS nor RARE nor any of their staff may be held liable for any
   inaccuracies or omissions, or any loss or damage arising from or out
   of the use of this report.





Adie                                                            [Page 2]


RFC 1614        Network Access to Multimedia Information        May 1994


   The opinions expressed in this report are personal opinions of the
   author.  They do not necessarily represent the policy either of RARE
   or of ECUS.

   Mention of a product in this report does not constitute endorsement
   either by EUCS or by RARE.

Availability

   This document is available in various forms (PostScript, text,
   Microsoft Word for Windows 2) by anonymous FTP through the following
   URL:

    ftp://ftp.edinburgh.ac.uk/pub/mmaccess/

    ftp://ftp.rare.nl/rare/pub/rtr/rtr8-rfc.../

    Paper copies are available from the RARE Secretariat.

0. Extended Summary

   Introduction

   This report is concerned with issues in the intersection of
   networked information retrieval, database and multimedia
   technologies.  It aims to establish research and academic user
   requirements for network access to multimedia data, to look at
   existing systems which offer partial solutions, and to identify
   what needs to be done to satisfy the most pressing requirements.

   User Requirements

   There are a number of reasons why multimedia data may need to be
   accessed remotely (as opposed to physically distributing the data,
   e.g., on CD-ROM).  These reasons centre on the cost of physical
   distribution, versus the timeliness of network distribution.  Of
   course, there is a cost associated with network distribution, but
   this tends to be hidden from the end user.

   User requirements have been determined by studying existing and
   proposed projects involving networked multimedia data.  It has
   proved convenient to divide the applications into four classes
   according to their requirements: multimedia database applications,
   academic (particularly scientific) publishing applications, cal
   (computeraided learning), and general multimedia information
   services.





Adie                                                            [Page 3]


RFC 1614        Network Access to Multimedia Information        May 1994


   Database applications typically involve large collections of
   monomedia (non-text) data with associated textual and numeric
   fields. They require a range of search and retrieval techniques.

   Publishing applications require a range of media types,
   hyperlinking, and the capability to access the same data using
   different access paradigms (search, browse, hierarchical, links).
   Authentication and charging facilities are required.

   Cal applications require sophisticated presentation and
   synchronisation capabilities, of the type found in existing
   multimedia authoring tools.  Authentication and monitoring
   facilities are required.

   General multimedia information services include on-line
   documentation, campus-wide information systems, and other systems
   which don't conveniently fall into the preceding categories.
   Hyperlinking is perhaps the most common requirement in this area.

   The analysis of these application areas allows a number of
   important user requirements to be identified:

      o    Support for the Apple Macintosh, UNIX and PC/MS Windows
           environments.

      o    Support for a wide range of media types - text, image,
           graphics and application-specific media being most
           important, followed by video and sound.

      o    Support for hyperlinking, and for multiple access structures
           to be built on the same underlying data.

      o    Support for sophisticated synchronisation and presentation
           facilities.

      o    Support for a range of database searching techniques.

      o    Support for user annotation of information, and for user-
           controlled display of sequenced media.

      o    Adequate responsiveness - the maximum time taken to retrieve
           a node should not exceed 20s.

      o    Support for user authentication, a charging mechanism, and
           monitoring facilities.

      o    The ability to execute scripts.




Adie                                                            [Page 4]


RFC 1614        Network Access to Multimedia Information        May 1994


      o    Support for mail-based access to multimedia documents, and
           (where appropriate) for printing multimedia documents.

      o    Powerful, easy-to-use authoring tools.

   Existing Systems

   The main information retrieval systems in use on the Internet are
   Gopher, Wais, and the World-Wide Web.  All work on a client-server
   paradigm, and all provide some degree of support for multimedia data.

   Gopher presents the user with a hierarchical arrangement of nodes
   which are either directories (menus), leaf nodes (documents
   containing text or other media types), or search nodes (allowing some
   set of documents to be searched using keywords, possibly using WAIS).
   A range of media types is supported.  Extensions currently being
   developed for Gopher (Gopher+) provide better support for multimedia
   data.  Gopher has a very high penetration (there are over 1000 Gopher
   servers on the Internet), but it does not provide hyperlinks and is
   inflexibly hierarchical.

   Wais (Wide Area Information Server) allows users to search for
   documents in remote databases.  Full-text indexing of the databases
   allows all documents containing particular (combinations of) words to
   be identified and retrieved.  Non-text data (principally image data)
   can be handled, but indexing such documents is only performed on the
   document file name, severely limiting its usefulness.  However, WAIS
   is ideally suited to text search applications.

   World-Wide Web (WWW) is a large-scale distributed hypermedia system.
   The Web consists of nodes (also called documents) and links.  Links
   are connections between documents: to follow a link, the user clicks
   on a highlighted word in the source document, which causes the
   linkedto document to be retrieved and displayed.  A document can be
   one of a variety of media types, or it can be a search node in a
   similar sense to Gopher.  The WWW addressing method means that WAIS
   and Gopher servers may also be accessed from (indeed, form part of)
   the Web.  WWW has a smaller penetration than Gopher, but is growing
   faster.  The Web technology is currently being revised to take better
   account of the needs of multimedia information.

   These systems all go some way to meet the user requirements.

      o    Support for multiple platforms and for a wide range of media
           types (through "viewer" software external to the client
           program) is good.

      o    Only WWW has hyperlinks.



Adie                                                            [Page 5]


RFC 1614        Network Access to Multimedia Information        May 1994



      o    There is little or no support for sophisticated presentation
           and synchronisation requirements.

      o    Support for database querying tends to be limited to
           "keyword" searches, but current developments in Gopher and
           WWW should make more sophisticated queries possible.

      o    Some clients support user annotation of documents.

      o    Response times for all three systems vary substantially
           depending on the network distance between client and server,
           and there is no support for isochronous data transfer.

      o    There is little in the way of authentication, charging and
           monitoring facilities, although these are planned for WWW.

      o    Scripting is not supported because of security issues

      o    WWW supports a mail responder.

      o    The only system sufficiently complex to warrant an authoring
           tool is WWW, which has editors to support its hypertext
           markup language.

   Research

   There are a number of research projects which are of significant
   interest.

   Hyper-G is an ambitious distributed hypermedia research project at
   the University of Graz.  It combines concepts of hypermedia,
   information retrieval systems and documentation systems with aspects
   of communication and collaboration, and computer-supported teaching
   and learning.  Automatic generation of hyperlinks is supported, and
   there is a concept of generic structures which can exist in parallel
   with the hyperlink structure.  Hyper-G is based on UNIX, and is in
   use as a CWIS at Graz.  Gateways between Hyper-G and WWW exist.

   Microcosm is a PC-based hypermedia system developed at the University
   of Southampton.  It can be viewed as an integrating hypermedia
   framework - a layer on top of a range of existing applications which
   enables relationships between different documents to be established.
   Hyperlinks are maintained separately from the data.  Networking
   support for Microcosm is currently under development, as are versions
   of Microcosm for the Apple Macintosh and for UNIX.  Microcosm is
   currently being "commercialised".




Adie                                                            [Page 6]


RFC 1614        Network Access to Multimedia Information        May 1994


   AthenaMuse 2 is an ambitious distributed hypermedia authoring and
   presentation system under development by a university/industry
   consortium based at MIT.  It will have good facilities for
   presentation and synchronisation of multimedia data, strong authoring
   support, and will include support for networking isochronous data. It
   will be a commercial product.  Initial versions will support UNIX and
   X windows, with a PC/MS Windows version following.  Apple Macintosh
   support has lower priority.

   The "Xanadu" project is designing and building an "open, social
   hypermedia" distributed environment, but shows no sign of delivering
   anything after several years of work.

   The European Commission sponsors a number of peripherally relevant
   projects through its Esprit and RACE research programmes.  These
   programmes tend to be oriented towards commercial markets, and are
   thus not directly relevant.  An exception is the Esprit IDOMENEUS
   project, which brings together workers in the database, information
   retrieval and multimedia fields.  It is recommended that RARE
   establish a liaison with this project.

   There are a variety of other academic and commercial research
   projects which are also of interest.  None of them are as directly
   relevant as those outlined above.

   Standards

   There are a number of existing and emerging standards for structuring
   hypermedia applications.  Of these, the most important are SGML,
   HyTime, MHEG, ODA, PREMO and Acrobat.  All bar the last are de jure
   standards, while Acrobat is a commercial product which is being
   proposed as a de facto standard.

   SGML (Standard Generalized Markup Language) is a markup language for
   delimiting the logical and semantic content of text documents.
   Because of its flexibility, it has become an important tool in
   hypermedia systems.  HyTime is an ISO standardised infrastructure for
   representing integrated, open hypermedia documents, and is based on
   SGML.  HyTime has great expressive power, but is not optimised for
   run-time efficiency.  It is recommended that future RARE work on
   networked hypermedia should take account of the importance of SGML
   and HyTime.

   MHEG (Multimedia and Hypermedia information coding Experts Group) is
   a draft ISO standard for representing hypermedia applications in a
   platform-independent form.  It uses an object-oriented approach, and
   is optimised for run-time efficiency.  Full IS status for MHEG is
   expected in 1994.  It is recommended that RARE keep a watching brief



Adie                                                            [Page 7]


RFC 1614        Network Access to Multimedia Information        May 1994


   on MHEG.

   The ODA (Open Document Architecture) standard is being enhanced to
   incorporate multimedia and hypermedia features.  However, interest in
   ODA is perceived to be decreasing, and it is recommended that ODA
   should not form a basis for further RARE work in networked
   hypermedia.

   PREMO is a new work item in the ISO graphics standardisation
   community, which appears to overlap with MHEG and HyTime.  It is not
   clear that the PREMO work, which is at a very early stage, is
   worthwhile in view of the existence of those standards.

   Acrobat PDF is a format for representing multimedia (printable)
   documents in a portable, revisable form.  It is based on Postscript,
   and is being proposed by Adobe Inc (originators of Postscript) as an
   industry standard.  RARE should maintain awareness of this technology
   in view of its potential impact on multimedia information systems.

   There are various standards which have relevance to the way
   multimedia data is accessed across the network.  Many of these have
   been described in a previous report [1].  Two further access
   protocols are the proposed multimedia extensions to SQL, and the
   Document Filing and Retrieval protocol.  Neither of these are likely
   to have major significance for networked multimedia information
   systems.

   Other standards of importance include:

      o    MIME, a multimedia email standard which defines a range of
           media types and encoding methods for those types which are
           useful in a wider context.

      o    AVIs (Audio-Visual Interactive services) and the associated
           multimedia scripting language SMSL, which form a
           standardisation initiative within CCITT (now ITU-TSS) to
           specify interactive multimedia services which can be
           provided across telephone/ISDN networks.

   There are two important trade associations which are involved in
   standardisation work.  The Interactive Multimedia Association (IMA)
   has a Compatibility Project which is developing a specification for
   platform-independent interactive multimedia systems, including
   networking aspects.  A newly-formed group, the Multimedia
   Communications Forum (MMCF), plans to provide input to the standards
   bodies.  It is recommended that RARE become an Observing Member of
   the MMCF.  A third trade association - the Multimedia Communications
   Community of Interest - has also just been formed.



Adie                                                            [Page 8]


RFC 1614        Network Access to Multimedia Information        May 1994


   Future Directions

   Three common design approaches emerge from the variety of systems and
   standards analysed in this report.  They can be described in terms of
   distinctions between different aspects of the system:

      o    content is distinct from hyperstructure

      o    media type is distinct from media encoding

      o    data is distinct from protocol

   Distributed hypermedia systems are emerging from the
   research/development phase into the experimental deployment phase.
   However, the existing global information systems (Gopher, WAIS and
   WWW) are still largely limited to the use of external viewers for
   nontextual data.  The most significant mismatches between the
   capabilities of currently-deployed systems and user requirements are
   in the areas of presentation and quality of service (i.e.,
   responsiveness).

   Improving QOS is significantly more difficult than improving
   presentation capabilities, but there are a number of possible ways in
   which this could be addressed.  Improving feedback to the user,
   greater multi-threading of applications, pre-fetching, caching, the
   use of alternative "views" of a node, and the use of isochronous data
   streams are all avenues which are worth exploring.

   In order to address these problems, it is recommended that RARE seek
   to adapt and enhance existing tools, rather than develop new ones.

   In particular, it is recommended that RARE select the World-Wide Web
   to concentrate its efforts on.  The reasons for this choice revolve
   around the flexibility of the WWW design, the availability of
   hyperlinks, the existing effort which is already going into
   multimedia support in WWW, the fact that it is an integrating
   solution incorporating both WAIS and Gopher support, and its high
   rate of growth compared to Gopher (despite Gopher's wider
   deployment).  Gopher is the main competitor to WWW, but its
   inflexibly hierarchical structure and the absence of hyperlinks make
   it difficult to use for highly-interactive multimedia applications.

   It is recommended that RARE should invite proposals for and
   subsequently commission work to:

      o    Develop conversion tools from commercial multimedia
           authoring packages to WWW, and accompanying authoring
           guidelines.



Adie                                                            [Page 9]


RFC 1614        Network Access to Multimedia Information        May 1994


      o    Implement and evaluate the most promising ways of overcoming
           the QOS problem.

      o    Implement a specific user project using these tools, to
           validate that the facilities being developed are truly
           relevant to real applications.

      o    Use the experience gained to inform and influence the
           development of the WWW technology.

      o    Contribute to the development of PC/MS Windows and Apple
           Macintosh WWW clients, particularly in the multimedia data
           handling area.

   It is noted that the rapid growth of WWW may in the future lead to
   problems through the implementation of multiple, uncoordinated and
   mutually incompatible add-on features.  To guard against this trend,
   it may be appropriate for RARE, in coordination with CERN and other
   interested parties such as NCSA, to:

      o    Encourage the formation of a consortium to coordinate WWW
           technical development.

1. Introduction

1.1. Background

   This study was inspired by the realisation that while some aspects of
   distributed multimedia technology are being actively introduced into
   the European research community (for instance, audiovisual
   conferencing, through the MICE project), other aspects are receiving
   less attention.  In particular, one category in which there seems to
   be relatively little activity is providing solutions to ease remote
   access to multimedia resources (for instance, accessing stored
   audio/video clips or images, or indeed entire multimedia
   applications, across the network).  Few commercial products address
   this, and the relevance of existing standards in this area is
   unclear.

   Of the 50 or so research projects documented in the recent RARE
   distributed multimedia survey [1], only about six have a direct
   relevance to this application area.  Where stated in the survey, the
   main research effort in these projects is often directed towards the
   "difficult" problems, such as the transfer of isochronous data and
   the design and implementation of object-oriented multimedia
   databases, rather than towards user-oriented issues.





Adie                                                           [Page 10]


RFC 1614        Network Access to Multimedia Information        May 1994


   This report is concerned with practical issues in the intersection of
   networked information retrieval, database and multimedia
   technologies.  It aims to establish actual user requirements in this
   area, to look at existing systems which offer partial solutions, and
   to identify what additional work needs to be done to satisfy the most
   pressing requirements.

1.2. Terminology

   In order to discuss multimedia information systems, we need a
   consistent terminology.  The vocabulary defined below embodies some
   of the concepts of the Dexter hypertext reference model [2].  This
   model is sufficiently general to be useful for describing most of the
   facilities and requirements of the multimedia information systems
   described in this report.  (However, the Dexter model does not
   describe searchable index objects - it is not a database reference
   model.)

    anchor             An identified portion of a node.  E.g., in a text
                       node, an anchor might be a string of one or more
                       adjacent characters, while in an image node it
                       might be a rectangular area of the image.

    composite node     A node containing data of multiple media types.

    document           Often used loosely as a synonym for node.

    hyperdocument      We refer to a collection of related nodes,
                       linked internally with hyperlinks, as a
                       "hyperdocument".  Examples are a database of
                       medical images and associated text; a module
                       from a suite of teaching material; or an article
                       in a scientific journal.  A hyperdocument may
                       contain hyperlinks to other data which exists in
                       internally with hyperlinks, as a
                       "hyperdocument". Examples are a other
                       hyperdocuments, but can be viewed as largely
                       self-contained.  It is a highlevel "unit of
                       authoring", but is not necessarily perceived as
                       a distinct unit by a reader (although it may be
                       so perceived, particularly if it contains few
                       hyperlinks to outside entities).

    hyperlink          Set of one or more source anchors and one or
                       more target anchors.  Also known simply as a
                       "link".





Adie                                                           [Page 11]


RFC 1614        Network Access to Multimedia Information        May 1994


    isochronous (adjective) Describes a continuous flow of data which
                       is required to be delivered by the network under
                       critical time constraints.

    leaf node          A node which contains no source anchors.

    media type         An attribute of data which describes the general
                       nature of its expected presentation.  The value
                       of this attribute could be one of the following
                       (not exhaustive) list:

                       o Text

                       o Sound

                       o Image (e.g., a "photograph")

                       o Graphics (e.g., a "drawing")

                       o Animation (i.e., moving graphics)

                       o Movie (i.e., moving image)

    monomedia (adjective)   Said of data which is all of the same media
                       type.

    multimedia (adjective)  Said of data which contains different media
                       types.  This definition is stricter than general
                       usage, where "multimedia" is often  used as a
                       generic term for non-textual data, and where it
                       may even be used as a noun.

    physical media     Magnetic or optical storage.  Not to be confused
                       with media type!

    [simple] node      A monomedia object which may be retrieved and
                       displayed as a single unit.

    source anchor      An anchor which may be "actioned" by the user,
                       causing the node(s) containing the target
                       anchor(s) in the same hyperlink to be retrieved
                       and displayed.  This process is called
                       "traversing the link".

    target anchor      an anchor forming part of a hyperlink, whose
                       containing node is retrieved and displayed when
                       the hyperlink is traversed.




Adie                                                           [Page 12]


RFC 1614        Network Access to Multimedia Information        May 1994


2. User Requirements

   User requirements in an area such as networking, which is subject to
   rapid technological change, are sometimes difficult to identify.  To
   an extent, technology leads applications, and users will exploit what
   is possible.

2.1. Applications

   Awareness of the range of networked multimedia applications which are
   currently being envisaged by computer users in the academic and
   research community leads to a better understanding of the technical
   requirements.  This section outlines some projects wh