Network Working Group C. Adie Request for Comments: 1614 Edinburgh University Computing Service RARE Technical Report: 8 May 1994 Category: Informational Network Access to Multimedia Information Status of this Memo This memo provides information for the Internet community. This memo does not specify an Internet standard of any kind. Distribution of this memo is unlimited. Abstract This report summarises the requirements of research and academic network users for network access to multimedia information. It does this by investigating some of the projects planned or currently underway in the community. Existing information systems such as Gopher, WAIS and World-Wide Web are examined from the point of view of multimedia support, and some interesting hypermedia systems emerging from the research community are also studied. Relevant existing and developing standards in this area are discussed. The report identifies the gaps between the capabilities of currentlydeployed systems and the user requirements, and proposes further work centred on the World-Wide Web system to rectify this. The report is in some places very detailed, so it is preceded by an extended summary, which outlines the findings of the report. Publication History The first edition was released on 29 June 1993. This second edition contains minor changes, corrections and updates. Table of Contents Acknowledgements 2 Disclaimer 2 Availability 3 0. Extended Summary 3 1. Introduction 10 1.1. Background 10 1.2. Terminology 11 2. User Requirements 13 2.1. Applications 13 2.2. Data Characteristics 18 Adie [Page 1]
RFC 1614 Network Access to Multimedia Information May 1994 2.3. Requirements Definition 19 3. Existing Systems 24 3.1. Gopher 24 3.2. Wide Area Information Server 30 3.3. World-Wide Web 34 3.4. Evaluating Existing Tools 42 4. Research 47 4.1. Hyper-G 47 4.2. Microcosm 48 4.3. AthenaMuse 2 50 4.4. CEC Research Programmes 51 4.5. Other 53 5. Standards 55 5.1. Structuring Standards 55 5.2. Access Mechanisms 62 5.3. Other Standards 63 5.4. Trade Associations 66 6. Future Directions 68 6.1. General Comments on the State-of-the-Art 68 6.2. Quality of Service 70 6.3. Recommended Further Work 71 7. References 76 8. Security Considerations 79 9. Author's Address 79 Acknowledgements The following people have (knowingly or unknowingly) helped in the preparation of this report: Tim Berners-Lee, John Dyer, Aydin Edguer, Anton Eliens, Tony Gibbons, Stewart Granger, Wendy Hall, Gary Hill, Brian Marquardt, Gunnar Moan, Michael Neuman, Ari Ollikainen, David Pullinger, John Smith, Edward Vielmetti, and Jane Williams. The useful role which NCSA's XMosaic information browser tool played in assembling the information on which this report was based should also be acknowledged - many thanks to its developers. All trademarks are hereby acknowledged as being the property of their respective owners. Disclaimer This report is based on information supplied to or obtained by Edinburgh University Computing Service (EUCS) in good faith. Neither EUCS nor RARE nor any of their staff may be held liable for any inaccuracies or omissions, or any loss or damage arising from or out of the use of this report. Adie [Page 2]
RFC 1614 Network Access to Multimedia Information May 1994 The opinions expressed in this report are personal opinions of the author. They do not necessarily represent the policy either of RARE or of ECUS. Mention of a product in this report does not constitute endorsement either by EUCS or by RARE. Availability This document is available in various forms (PostScript, text, Microsoft Word for Windows 2) by anonymous FTP through the following URL: ftp://ftp.edinburgh.ac.uk/pub/mmaccess/ ftp://ftp.rare.nl/rare/pub/rtr/rtr8-rfc.../ Paper copies are available from the RARE Secretariat. 0. Extended Summary Introduction This report is concerned with issues in the intersection of networked information retrieval, database and multimedia technologies. It aims to establish research and academic user requirements for network access to multimedia data, to look at existing systems which offer partial solutions, and to identify what needs to be done to satisfy the most pressing requirements. User Requirements There are a number of reasons why multimedia data may need to be accessed remotely (as opposed to physically distributing the data, e.g., on CD-ROM). These reasons centre on the cost of physical distribution, versus the timeliness of network distribution. Of course, there is a cost associated with network distribution, but this tends to be hidden from the end user. User requirements have been determined by studying existing and proposed projects involving networked multimedia data. It has proved convenient to divide the applications into four classes according to their requirements: multimedia database applications, academic (particularly scientific) publishing applications, cal (computeraided learning), and general multimedia information services. Adie [Page 3]
RFC 1614 Network Access to Multimedia Information May 1994 Database applications typically involve large collections of monomedia (non-text) data with associated textual and numeric fields. They require a range of search and retrieval techniques. Publishing applications require a range of media types, hyperlinking, and the capability to access the same data using different access paradigms (search, browse, hierarchical, links). Authentication and charging facilities are required. Cal applications require sophisticated presentation and synchronisation capabilities, of the type found in existing multimedia authoring tools. Authentication and monitoring facilities are required. General multimedia information services include on-line documentation, campus-wide information systems, and other systems which don't conveniently fall into the preceding categories. Hyperlinking is perhaps the most common requirement in this area. The analysis of these application areas allows a number of important user requirements to be identified: o Support for the Apple Macintosh, UNIX and PC/MS Windows environments. o Support for a wide range of media types - text, image, graphics and application-specific media being most important, followed by video and sound. o Support for hyperlinking, and for multiple access structures to be built on the same underlying data. o Support for sophisticated synchronisation and presentation facilities. o Support for a range of database searching techniques. o Support for user annotation of information, and for user- controlled display of sequenced media. o Adequate responsiveness - the maximum time taken to retrieve a node should not exceed 20s. o Support for user authentication, a charging mechanism, and monitoring facilities. o The ability to execute scripts. Adie [Page 4]
RFC 1614 Network Access to Multimedia Information May 1994 o Support for mail-based access to multimedia documents, and (where appropriate) for printing multimedia documents. o Powerful, easy-to-use authoring tools. Existing Systems The main information retrieval systems in use on the Internet are Gopher, Wais, and the World-Wide Web. All work on a client-server paradigm, and all provide some degree of support for multimedia data. Gopher presents the user with a hierarchical arrangement of nodes which are either directories (menus), leaf nodes (documents containing text or other media types), or search nodes (allowing some set of documents to be searched using keywords, possibly using WAIS). A range of media types is supported. Extensions currently being developed for Gopher (Gopher+) provide better support for multimedia data. Gopher has a very high penetration (there are over 1000 Gopher servers on the Internet), but it does not provide hyperlinks and is inflexibly hierarchical. Wais (Wide Area Information Server) allows users to search for documents in remote databases. Full-text indexing of the databases allows all documents containing particular (combinations of) words to be identified and retrieved. Non-text data (principally image data) can be handled, but indexing such documents is only performed on the document file name, severely limiting its usefulness. However, WAIS is ideally suited to text search applications. World-Wide Web (WWW) is a large-scale distributed hypermedia system. The Web consists of nodes (also called documents) and links. Links are connections between documents: to follow a link, the user clicks on a highlighted word in the source document, which causes the linkedto document to be retrieved and displayed. A document can be one of a variety of media types, or it can be a search node in a similar sense to Gopher. The WWW addressing method means that WAIS and Gopher servers may also be accessed from (indeed, form part of) the Web. WWW has a smaller penetration than Gopher, but is growing faster. The Web technology is currently being revised to take better account of the needs of multimedia information. These systems all go some way to meet the user requirements. o Support for multiple platforms and for a wide range of media types (through "viewer" software external to the client program) is good. o Only WWW has hyperlinks. Adie [Page 5]
RFC 1614 Network Access to Multimedia Information May 1994 o There is little or no support for sophisticated presentation and synchronisation requirements. o Support for database querying tends to be limited to "keyword" searches, but current developments in Gopher and WWW should make more sophisticated queries possible. o Some clients support user annotation of documents. o Response times for all three systems vary substantially depending on the network distance between client and server, and there is no support for isochronous data transfer. o There is little in the way of authentication, charging and monitoring facilities, although these are planned for WWW. o Scripting is not supported because of security issues o WWW supports a mail responder. o The only system sufficiently complex to warrant an authoring tool is WWW, which has editors to support its hypertext markup language. Research There are a number of research projects which are of significant interest. Hyper-G is an ambitious distributed hypermedia research project at the University of Graz. It combines concepts of hypermedia, information retrieval systems and documentation systems with aspects of communication and collaboration, and computer-supported teaching and learning. Automatic generation of hyperlinks is supported, and there is a concept of generic structures which can exist in parallel with the hyperlink structure. Hyper-G is based on UNIX, and is in use as a CWIS at Graz. Gateways between Hyper-G and WWW exist. Microcosm is a PC-based hypermedia system developed at the University of Southampton. It can be viewed as an integrating hypermedia framework - a layer on top of a range of existing applications which enables relationships between different documents to be established. Hyperlinks are maintained separately from the data. Networking support for Microcosm is currently under development, as are versions of Microcosm for the Apple Macintosh and for UNIX. Microcosm is currently being "commercialised". Adie [Page 6]
RFC 1614 Network Access to Multimedia Information May 1994 AthenaMuse 2 is an ambitious distributed hypermedia authoring and presentation system under development by a university/industry consortium based at MIT. It will have good facilities for presentation and synchronisation of multimedia data, strong authoring support, and will include support for networking isochronous data. It will be a commercial product. Initial versions will support UNIX and X windows, with a PC/MS Windows version following. Apple Macintosh support has lower priority. The "Xanadu" project is designing and building an "open, social hypermedia" distributed environment, but shows no sign of delivering anything after several years of work. The European Commission sponsors a number of peripherally relevant projects through its Esprit and RACE research programmes. These programmes tend to be oriented towards commercial markets, and are thus not directly relevant. An exception is the Esprit IDOMENEUS project, which brings together workers in the database, information retrieval and multimedia fields. It is recommended that RARE establish a liaison with this project. There are a variety of other academic and commercial research projects which are also of interest. None of them are as directly relevant as those outlined above. Standards There are a number of existing and emerging standards for structuring hypermedia applications. Of these, the most important are SGML, HyTime, MHEG, ODA, PREMO and Acrobat. All bar the last are de jure standards, while Acrobat is a commercial product which is being proposed as a de facto standard. SGML (Standard Generalized Markup Language) is a markup language for delimiting the logical and semantic content of text documents. Because of its flexibility, it has become an important tool in hypermedia systems. HyTime is an ISO standardised infrastructure for representing integrated, open hypermedia documents, and is based on SGML. HyTime has great expressive power, but is not optimised for run-time efficiency. It is recommended that future RARE work on networked hypermedia should take account of the importance of SGML and HyTime. MHEG (Multimedia and Hypermedia information coding Experts Group) is a draft ISO standard for representing hypermedia applications in a platform-independent form. It uses an object-oriented approach, and is optimised for run-time efficiency. Full IS status for MHEG is expected in 1994. It is recommended that RARE keep a watching brief Adie [Page 7]
RFC 1614 Network Access to Multimedia Information May 1994 on MHEG. The ODA (Open Document Architecture) standard is being enhanced to incorporate multimedia and hypermedia features. However, interest in ODA is perceived to be decreasing, and it is recommended that ODA should not form a basis for further RARE work in networked hypermedia. PREMO is a new work item in the ISO graphics standardisation community, which appears to overlap with MHEG and HyTime. It is not clear that the PREMO work, which is at a very early stage, is worthwhile in view of the existence of those standards. Acrobat PDF is a format for representing multimedia (printable) documents in a portable, revisable form. It is based on Postscript, and is being proposed by Adobe Inc (originators of Postscript) as an industry standard. RARE should maintain awareness of this technology in view of its potential impact on multimedia information systems. There are various standards which have relevance to the way multimedia data is accessed across the network. Many of these have been described in a previous report [1]. Two further access protocols are the proposed multimedia extensions to SQL, and the Document Filing and Retrieval protocol. Neither of these are likely to have major significance for networked multimedia information systems. Other standards of importance include: o MIME, a multimedia email standard which defines a range of media types and encoding methods for those types which are useful in a wider context. o AVIs (Audio-Visual Interactive services) and the associated multimedia scripting language SMSL, which form a standardisation initiative within CCITT (now ITU-TSS) to specify interactive multimedia services which can be provided across telephone/ISDN networks. There are two important trade associations which are involved in standardisation work. The Interactive Multimedia Association (IMA) has a Compatibility Project which is developing a specification for platform-independent interactive multimedia systems, including networking aspects. A newly-formed group, the Multimedia Communications Forum (MMCF), plans to provide input to the standards bodies. It is recommended that RARE become an Observing Member of the MMCF. A third trade association - the Multimedia Communications Community of Interest - has also just been formed. Adie [Page 8]
RFC 1614 Network Access to Multimedia Information May 1994 Future Directions Three common design approaches emerge from the variety of systems and standards analysed in this report. They can be described in terms of distinctions between different aspects of the system: o content is distinct from hyperstructure o media type is distinct from media encoding o data is distinct from protocol Distributed hypermedia systems are emerging from the research/development phase into the experimental deployment phase. However, the existing global information systems (Gopher, WAIS and WWW) are still largely limited to the use of external viewers for nontextual data. The most significant mismatches between the capabilities of currently-deployed systems and user requirements are in the areas of presentation and quality of service (i.e., responsiveness). Improving QOS is significantly more difficult than improving presentation capabilities, but there are a number of possible ways in which this could be addressed. Improving feedback to the user, greater multi-threading of applications, pre-fetching, caching, the use of alternative "views" of a node, and the use of isochronous data streams are all avenues which are worth exploring. In order to address these problems, it is recommended that RARE seek to adapt and enhance existing tools, rather than develop new ones. In particular, it is recommended that RARE select the World-Wide Web to concentrate its efforts on. The reasons for this choice revolve around the flexibility of the WWW design, the availability of hyperlinks, the existing effort which is already going into multimedia support in WWW, the fact that it is an integrating solution incorporating both WAIS and Gopher support, and its high rate of growth compared to Gopher (despite Gopher's wider deployment). Gopher is the main competitor to WWW, but its inflexibly hierarchical structure and the absence of hyperlinks make it difficult to use for highly-interactive multimedia applications. It is recommended that RARE should invite proposals for and subsequently commission work to: o Develop conversion tools from commercial multimedia authoring packages to WWW, and accompanying authoring guidelines. Adie [Page 9]
RFC 1614 Network Access to Multimedia Information May 1994 o Implement and evaluate the most promising ways of overcoming the QOS problem. o Implement a specific user project using these tools, to validate that the facilities being developed are truly relevant to real applications. o Use the experience gained to inform and influence the development of the WWW technology. o Contribute to the development of PC/MS Windows and Apple Macintosh WWW clients, particularly in the multimedia data handling area. It is noted that the rapid growth of WWW may in the future lead to problems through the implementation of multiple, uncoordinated and mutually incompatible add-on features. To guard against this trend, it may be appropriate for RARE, in coordination with CERN and other interested parties such as NCSA, to: o Encourage the formation of a consortium to coordinate WWW technical development. 1. Introduction 1.1. Background This study was inspired by the realisation that while some aspects of distributed multimedia technology are being actively introduced into the European research community (for instance, audiovisual conferencing, through the MICE project), other aspects are receiving less attention. In particular, one category in which there seems to be relatively little activity is providing solutions to ease remote access to multimedia resources (for instance, accessing stored audio/video clips or images, or indeed entire multimedia applications, across the network). Few commercial products address this, and the relevance of existing standards in this area is unclear. Of the 50 or so research projects documented in the recent RARE distributed multimedia survey [1], only about six have a direct relevance to this application area. Where stated in the survey, the main research effort in these projects is often directed towards the "difficult" problems, such as the transfer of isochronous data and the design and implementation of object-oriented multimedia databases, rather than towards user-oriented issues. Adie [Page 10]
RFC 1614 Network Access to Multimedia Information May 1994 This report is concerned with practical issues in the intersection of networked information retrieval, database and multimedia technologies. It aims to establish actual user requirements in this area, to look at existing systems which offer partial solutions, and to identify what additional work needs to be done to satisfy the most pressing requirements. 1.2. Terminology In order to discuss multimedia information systems, we need a consistent terminology. The vocabulary defined below embodies some of the concepts of the Dexter hypertext reference model [2]. This model is sufficiently general to be useful for describing most of the facilities and requirements of the multimedia information systems described in this report. (However, the Dexter model does not describe searchable index objects - it is not a database reference model.) anchor An identified portion of a node. E.g., in a text node, an anchor might be a string of one or more adjacent characters, while in an image node it might be a rectangular area of the image. composite node A node containing data of multiple media types. document Often used loosely as a synonym for node. hyperdocument We refer to a collection of related nodes, linked internally with hyperlinks, as a "hyperdocument". Examples are a database of medical images and associated text; a module from a suite of teaching material; or an article in a scientific journal. A hyperdocument may contain hyperlinks to other data which exists in internally with hyperlinks, as a "hyperdocument". Examples are a other hyperdocuments, but can be viewed as largely self-contained. It is a highlevel "unit of authoring", but is not necessarily perceived as a distinct unit by a reader (although it may be so perceived, particularly if it contains few hyperlinks to outside entities). hyperlink Set of one or more source anchors and one or more target anchors. Also known simply as a "link". Adie [Page 11]
RFC 1614 Network Access to Multimedia Information May 1994 isochronous (adjective) Describes a continuous flow of data which is required to be delivered by the network under critical time constraints. leaf node A node which contains no source anchors. media type An attribute of data which describes the general nature of its expected presentation. The value of this attribute could be one of the following (not exhaustive) list: o Text o Sound o Image (e.g., a "photograph") o Graphics (e.g., a "drawing") o Animation (i.e., moving graphics) o Movie (i.e., moving image) monomedia (adjective) Said of data which is all of the same media type. multimedia (adjective) Said of data which contains different media types. This definition is stricter than general usage, where "multimedia" is often used as a generic term for non-textual data, and where it may even be used as a noun. physical media Magnetic or optical storage. Not to be confused with media type! [simple] node A monomedia object which may be retrieved and displayed as a single unit. source anchor An anchor which may be "actioned" by the user, causing the node(s) containing the target anchor(s) in the same hyperlink to be retrieved and displayed. This process is called "traversing the link". target anchor an anchor forming part of a hyperlink, whose containing node is retrieved and displayed when the hyperlink is traversed. Adie [Page 12]
RFC 1614 Network Access to Multimedia Information May 1994 2. User Requirements User requirements in an area such as networking, which is subject to rapid technological change, are sometimes difficult to identify. To an extent, technology leads applications, and users will exploit what is possible. 2.1. Applications Awareness of the range of networked multimedia applications which are currently being envisaged by computer users in the academic and research community leads to a better understanding of the technical requirements. This section outlines some projects wh