http://www.ltg.ed.ac.uk/mpiro

M-PIRO Project Annual Report 2002

The M-PIRO project started in February 2000 with the aim of generating tailored descriptions, in Greek, Italian, and English, of museum objects. The perceived problem is that often labels in museums are quite hard to understand. Even when professionally edited, they can still contain obscure terms or be too densely packed with information.

Another problem is that labels are generally designed to be accessed in any order, so each must represent all the relevant information about their object. That means that small differences between two objects may be submerged in a sea of similar details. Finally, there is no guarantee that the visitor will actually find what they need. Unless they have unlimited time, or are especially systematic, they will not study every object and label.

M-PIRO assumes that all these problems can be addressed using natural language generation technology. And with personalisation technology at the core, lots of different delivery platforms can be exploited. In M-PIRO we have experimented with text, web-based presentations, speech synthesis, and virtual reality. Many more are possible.


Summary of 2002 Activities

The project made some significant advances in its final year, including:

  • The design and installation of the web and virtual reality presentations: a dedicated server which will host the final system has been purchased and installed at the Foundation of the Hellenic World, in Athens, and the Foundation has also designed and implemented the virtual reality presentation.
  • The production of an extended domain: the Foundation has provided all the necessary material for the final extended collection of 50 exhibits, drawn from various categories and chronological periods. The structure of these exhibits resembles the collections in a number of archaeological museums, in Greece and elsewhere.
  • The specification of prosody information to improve speech synthesis: the Exprimo generation engine produces XML structures which include information to be used by speech synthesisers, such as given-new distinctions and syntactic structures. Where synthesisers have to process text and largely guess their structures, this can greatly help the prosody, and hence the hearer's understanding.
  • The addition of a user-sensitive sentence aggregation component: this module breaks new ground by producing different syntax structures depending on the user type. For adults and experts, we allow a relatively large number of facts to be aggregated into a single sentence, producing fairly long, multi-clause statements. For children a sentence must contain fewer facts, and this results in syntactically simpler, and shorter, clauses.
  • The further development and refinement of the domain authoring tool: considerable effort has been spent on making the authoring tool as user-friendly as possible, and we can now preview generated texts in the tool itself. This allows authors to see immediately the results of their additions.
  • The generation of texts which vary lexically depending on the user type: the generation engine can now choose between a number of options when deciding which words to use to express a particular concept. As these choices can be specified according to user type, the output will once again vary depending on who is accessing the data - so less technical expressions can be used for children, for example.
  • The production of automatically hyperlinked descriptions: for web presentations, the XML output can contain links which allow users to get further information by directly clicking on a part of the text. This information is produced using the generation engine.
  • Improvements in text quality: a lot of effort went into improving various aspects of the generated texts. In addition to extending and refining the linguistic resources for all three languages, we implemented theoretically-based aggregation and comparison modules which greatly enhance the variety and quality of the output.

We expect to have achieved all of the project's major objectives by the time of the final review, in January 2003, and to have surpassed our original expectations in a number of areas.

Language Generation

Market Prospects

The prospects for the technology that M-PIRO has developed are excellent, and this is emphasised by the fact that the University of Edinburgh spun out a company during the life of the project (FourthPerson) which is run by researchers who previously worked on M-PIRO. The company's goals are to produce exactly the kind of tailored descriptions that Exprimo does in a variety of on-line marketing applications.

Demonstrator

Some of the recent additions to the generation capabilities are mentioned in the summary of 2002 activities above. In addition, it is useful to restate that we created the first Italian female voice for the Festival speech synthesiser and experimented with new voices for the MBROLA synthesiser. We also produced extensive syntactic resources for Italian and Greek, to match the pre-existing English resources, and completed the design and implementation of the DEMOSTHeNES Speech Composer for Greek.

To illustrate some of the capabilities of the present system, here is an example Exprimo-generated text laid out as it would appear in the web presentation, followed by some forward pointers to other exhibits:

This exhibit is a black kantharos; it was created during the hellenistic period and it dates from the late 4th century B.C. It was decorated with the West slope technique and it originates from Amphipolis. Currently it is in the Archaeological Museum of Kavala.

  • Other exhibits created during the hellenistic period:
    • A portrait made from marble
    • A panathenaic amphora originally from Attica
  • Other exhibits that originate from Amphipolis:
    • A figurine created during the hellenistic period


The system includes a sophisticated aggregation module which in this case has decided to combine a number of facts into a longish introductory sentence. For child users, this would not happen - understanding of the rhetorical relationship signalled by punctuation like the semicolon following `kantharos' is assumed to be an adult capacity. For children, then, the sentences are shorter and the punctuation simpler.

In the process of generating the text, the system has also made decisions about which other objects are related to the object in question in relevant ways, and has produced prioritised lists, which in the web presentation are clickable links. Certain parts of the text are also highlighted in the web version - such as `the hellenistic period' in the example above - and clicking on these elicits further information using the normal generation engine.

In different circumstances, having seen other exhibits before the kantharos, the text accompanying the graphic could be as shown below:

This exhibit is a black kantharos. Unlike the previous vessels, which were created during the classical period, it was created during the hellenistic period. The hellenistic period covers the chaotic period from the Death of Alexander the Great (323 B.C.), and the subsequent dissolution of his empire to the victory of the Romans over the Greeks at the Battle of Actium (31 B.C.), comprising a truly cosmopolitan or international range of artistic trends.

In this case the text includes a comparison (unlike the previous vessels .. ), and some generic information about the hellenistic period. Generic information like this is currently stored in the form of canned text.

A very important aspect of the final year's activity has been the design and implementation of the immersive virtual reality prototype at the Foundation of the Hellenic World. The virtual reality application includes five 3D reconstructions of pottery objects in the M-PIRO collection. The M-PIRO software has been adapted so that the system can be experienced on the Foundation's ImmersaDesk stereoscopic display. Visitors to the virtual reality collection will be able to select objects and hear, rather than read, personalized descriptions of these digitally reconstructed objects while interacting with them.

Resources

The system has produced quite extensive language and database resources. Using the authoring tool, museum experts have input the required information on 50 objects, and this has been accompanied by the necessary language resources in English, Greek, and Italian. Details on most aspects of the system are available from the public deliverables which are available on the M-PIRO web pages (see the link to `further information' below).


Promotion and Exploitation

The project partly sponsored the Edilog workshop, held over three days in September 2002 in Edinburgh, and ran a number of demos during the period. This was very successful, and we made a number of useful contacts with other projects. M-PIRO will also run a public event for media and related researchers during the final review period in Athens in January 2003. This will be hosted by the Foundation of the Hellenic World, who are extending invitations to the scientific and museums communities, and to the press, as well as disseminating information to the wider public.

Further Information

For more information, please see the project web page: http://www.ltg.ed.ac.uk/mpiro.