Workshop on Synthetic Worlds
University of Aizu and French Japan  
December 1993, Aizu Japan

by D.K.Kahaner,
( mirrored )

This week-long workshop was proposed by Kunii, and then endorsed and supported by the French government through the French Embassy. About 15 French scientists participated along with a somewhat larger number of Japanese; some sessions had as many as 50 in attendance. (French participants were a cross section of high level scientists from universities, government labs and industry). The Japanese were mostly from Aizu, but some from industry. The format was 30-45 minute lectures with ample time between for discussion, and then working discussion periods during which the participants remained in the conference room to have informal question and answer sessions, which became quite spirited at times. The French Science Councelor in Japan (Jean-Paul Smets-Solanes) was the General Co-chair, and he participated in all of the first day's program. Dr Francios-Xavier Testard-Vaillant, a computer scientist, who is an attache for S&T at the French Embassy, was the Program Co-Chair and participated the entire week. Testard-Vaillant emphasized the Workshop was organized to encourage better collaboration between France and Japan (Email: FXTV@ALTERN.COM).

The term Synthetic World was created for this Workshop, and is a general term that is intended to include topics such as VR, artificial life, graphics, as well as certain aspects of multimedia, speech, modelling, robots, and others. For many years Kunii has been interested in various aspects of this, and stated that he chose France as the country to co-host this Workshop because he somehow felt that related theoretical French science was at a very high level, and that the French seemed to have a good intuitive sense of what he was grasping at. In any case Kunii already has a specific cooperative agreement for a "Visual Computing Institute" involving scientists in Geneva and Lausanne, among others (see "pg.93" 12 Oct 1993). In what follows I summarize some of the talks, with primary emphasis on those from Japan. A complete program is attached to this report along with email addresses of those speakers who have them. Associated with the Workshop was an informal Proceedings. In the Proceedings, some talks are represented only by a title or abstract, others include copies of overheads, while others contain complete manuscripts. The program listing below distinguishes these. 

Kunii gave two talks during the first day, both describing in general terms his philosophy, some experiences, and projections for the needs and opportunities in future research. His view is that sophisticated and higher order analytic tools are required, including algebraic topology, homotopy, semiotics (study of signs), singularity modeling etc., to deal deal with large nonlinear and high dimensional problems. He feels that computer hardware improvements into the sub-micron range and parallel computing will not capture the combinatorial explosion associated with new problems such as study of the beginning of the universe, the human body and its internal organs, economics, topographical guidance systems, simulation of rain forest growth, etc). (In concrete terms in commented that polyhedral objects that are typically used in 3D models are much too simple and give away too much of the intrinsic smoothness and content of the physical entities being studied, and mentioned garment wrinkling models and models of the stock market Black Monday. He feels that surfaces of shapes have to be at least second order smooth to identify characteristic points, such as peaks, pits, saddles, etc.) He also touched upon the unbalance of computation which ignores emotional information (facial expression, gestural information, muscle control, etc) which is essential and plays a key role in the real world. He traced the evolution of computer architecture from numerical computation via sequential models to virtual storage architectures for file processing to a media architecture which includes a visual computer architecture. He believes that 4D models and an associated computer architecture will enable direct computation of 4D objects including direct input, processing and output. He gave as examples of the need for automatic capturing of expert skills (which are inherently 4D) in fields such as designing, assembling, crafting, performing R&D, teaching, learning, medical practicing, sporting, dancing, cooking, etc. In the areas of synthetic worlds Kunii noted that he has been developing a ski-instructor expert system and a martial arts expert system that use folds, cusps, singularity signs, Morse theory, etc. 

This was an exceptionally high level talk, designed to get the Workshop off in the right direction. I think that it was difficult for many participants to relate these ideas to the topics of their everyday work, but was also was seen as a challenge to them to expand their horizons. 

J.Sakarovitch, an expert in Automata Theory first provided an overview of the institute he heads, Inst Blaise-Pascal a French public sector research institute in computer science, with about 200 permanent staff and 200 PhD students and a US$2M budget (excluding salaries). He also explained that many other French scientists have joint or non permanent appointments. Concerning future challenges, he emphasized three. 

A. Meeting needs 
Increasing computing power needed as theoretical science moves away from analytical models. Better programming facilities needed, included OO, actor & agent languages, logic programming, and non-lingual programming. B. Escaping pitfalls Relating to recognizing the model, fearing buzzwords, and being wary of the metaphors. 

C. The Unix Syndrome
Emphasizing that we are the builders and experimentators of the systems to come, the social power of what is computed, and the overlooked factor--the consequences of dissemination of IT. Reffye (CIRAD) gave two talks, each showing the fantastic progress his group has made in simulating growth of trees, forests, etc. These were exceptionally realistic and utilize a tremendous amount of detailed information about characteristics of specific plants, trees, etc. A concrete application has been to landscape design showing how specific landscapes will age and change over years; the group has many clients. De Reffye also mentioned that some of his software is in use at Tokai Univ in Japan.
Meyer (Ecole Normale Superieure) discussed the design of simulated animals and robots capable of preprogrammed, learned, and evolved (adapted) behavior, entities that are called animats. This is a relatively new field with a healthy basic science community producing interesting fundamental results but without many theories or generalizations, and currently mostly empirically based. Potential applications appear to be in the areas of collective robots (ex: space of marine exploration), collective problem solving and planning, etc. Meyer mentioned that significant work in evolving hardware was occurring in Japan, and also mentioned his hope that work on nano-robots (as typified by Prof Nakajima's research at U-Tokyo) would help with animats. 
Goodwin (Aizu) gave a detailed overview of the differences between life and non-life and listed a set of criteria by which one might judge candidates for life forms. Among those listed, only humans and chain letters got a perfect score, indicating just how difficult it really is to establish "aliveness". He then went on to describe some of the general directions in artificial life, and concluded with a plea that work is needed to understand the mathematical structures that are implied by genetic operators and genetic search, e.g., more theory and less empiricism. 
Myszkowski (Aizu) discussed work in modeling a designed but unbuilt synagogue, and the lighting models needed to simulate this accurately, provide walkthrough animation, etc. When this project is completed, the system will be exhibited at the Museum of Modern Art in NY as an example of a tool for virtual revisiting of architectural masterpieces. He commented on the heavy computational demands and the usefulness of parallel computing for lighting simulation, meshing and image display algorithms. When I asked about obtaining better performing parallel processing equipment, Kunii commented that had tried to obtain a specialized Fujtisu VG1600 accelerator, claimed to be much more powerful than comparable SGI equipment, but Fujitsu told him they could not sell it to Aizu because of COCOM restrictions. (Why?) 
Limantour (Medialab, France -- no relation to MIT activity of same name), representing a small French research lab (35 people) focusing on computer graphics, character animation, virtual reality, and image processing, showed some examples of their work. One interesting project was a demonstration of the French ISDN (Numeris) shown last year at Monte Carlo. (This project was also discussed in detail by Queau in his presentation on Televirtuality.) In this experiment, two users in different places were connected via ISDN, and then moved and interacted within the same virtual environment, the Cluny Abby. Computer support was by two SGI 440VGXT machines. (During Imagina'94 16-18 Feb 1994, a related experiment will be conducted between Keio University in Japan and Monaco.) 
Vaillant, described VR work at his company, the huge (100K person) Thompson CSF. They see VR markets in the following areas.  Simulation and training for defense Complex man-machine-interface and ergonomy Teleoperation of robots Multi-media and telecommunications Medical applications CAD/CAM Entertainment and games The company's objectives are to keep track of state of the art VR technologies and applications for current and future markets, keep at the forefront with basic research, and to develop VR for dedicated applications. He gave one example of a 3 year Esprit funded project (94-96) involving 8 companies focused on the reconstruction of reality from image sequences (the project is budgeted at 215person-months), with subtopics of vision (recovering scenes from sequences of images), photometry (evaluating material properties from optical point of view), and VR (create a data-base usable in a VR system). 
There were two papers by Japanese researchers on synthetic factories or manufacturing. Noma (Kyushu Inst of Tech) looked at the requirements and components for developing factories in virtual environments in order to pre-examine activities in factories and then to find problems before the operations are conducted in real factories. Noma pointed out that there are essential differences between existing VR research and work on synthetic factories. A key difference is that in a traditional VR environment, users interact with the environment so that they can feel as if they exist in it, what he calls a single or local view. In a factory, however, users cannot understand the activities in that way. A useful system should have multiple levels (or aspects) of views, and these should be integrated so that users can combine global and local views. He also feels that a synthetic factory system needs to be furnished with a collection of tools to help users make evaluations and decisions concerning activities in the factory. He introduced an engineering animation system (ANIMENGINE), first described almost 10 years ago which provides a local view of design, and an assemblability testing system (SYDEM) based on Noma's claim that in factories, solid representation (design) and assembly are equally important. He also described process modeling and scheduling in mathematical terms and defined an optimal schedule. He concluded that "integration" will be the key issue for future research in synthetic factories. 

Miura (Aizu) described his activities at Aizu's Shape Modeling Lab in virtual manufacturing. His lab is engaged in parallel mesh subdivision based on vertex label assignment for FEM generation, and simulation of dynamic interaction. He sees applications in the micromachine area which has new, or at least unexpected or unfamiliar physics (space craft operation). 

Cohen (Aizu) also discussed points of view and frames of reference, and emphasized that there are many applications were these need to be changed. He gave one specific example of a racing car game where the user can choose to be in the cockpit, just behind the vehicle, or in two different positions above the car. He also explained the opportunities for superimposing multiple displays, not only video displays but audio signals as well, and also the use of clustered or split perspectives, etc. He illustrated this with a project involving several virtual rooms displayed on a workstation where sound from loudspeakers is adjusted according to the location of the user's icon. This rapid-fire presentation left most participants breathless but also intrigued by the many new ideas presented. Cohen is also working on a summary of activities in the Tokyo region in arcades and game parlors illustrating or using advanced technology for capture, transmission, and reproduction of experiences. He has prepared a short document "Cybertokyo" on the topic, and readers can contact him directly for copies.
Peroche (Ecole des Mines de Saint-Etienne) described the use of geological models for erosion. He began by stating his opinion that fractals, the wonderchild of computer imaging, is unsuitable because nature is not self similar. His approach is via bedrock models of terrain, including a hexagonal height field with geological structures in each node (including rock stiffness, permeability, ability to chemical dissolution, vegetal cover, etc), and the use of simulation laws (gravity creep, mechanical erosion, chemical erosion, alluvial deposition, etc), but that erosion was limited to that due to water. His computations are done on a 100x100 grid with 10m between grid points. Typically he computes 100 steps, although he admitted that he has no way to relate these to units of physical time. Also, the influence of soil, ice cracking, etc., is not considered. This paper clearly indicated to me just how much remains to be done to model such phenomena accurately.
Bellec (Bull) discussed the activites at Bull in VR and how these might influence the development of an artificial-world workstation. Bull's main interest seems to be in flight simulation applications, including aerodynamics, landscape simulation, air traffic control interaction, multi-plane interaction, weather simulation, etc., leading to systems that include real weather and real plane injection as well as actual cockpit presentation. They are also interested in applications to other transportation media such as cars. Concerning an experimental workstation, Bellec stated that the goal should be to allow humans to exercise the same senses in a VR system as they usually recognize in the real world (images, touch, hearing, smell, taste), but admitted that some of these were much further away than others. However, looking to the year 2000 he made the following predictions about a potential workstation.

A. Memory. Up to 200GB main memory with 16GB writable and 100GB ROM as secondary. This was based on the assumption that slide quality photos need about 100MB before compression and that 200CDs of classical music require about 100GB.

B. Processor. 250MIPS from a single RISC chip, using super scalar architecture and cache bandwidth (however, he sees problems with heat dissipation) and forecasts that one cpu will not be powerful enough. Thus parallel processing is the likely direction, with 4-16cpus at current workstation prices. Software will be OO, but remains the major challenge.

C. Input devices. Mice, etc., with no major problems anticipated. Sensors, available now for process control can be included in such a workstation if costs can be driven down. Display cameras are already available, but one current problem is control of the camera field (e.g., remote control by remote users?). Perhaps a type of electronic zoom into
HDTV images. Some kind of 3D viewing is needed. (See also remarks below about 3D TV.)

D. Sound. Major problems are with software, e.g., sound recognition, synthesis of voice and location, translation, etc.

E. Output. 1200x800 is okay for 20" CRT, but acceptable quality does not yet exist for flat panels. Problems with field of view could be solved by using goggles if public accepted. Bellec feels that digital TV technology will play a role here.

F. Telecommunications. Feels that requirements for communicating to/from such a workstation are less than for HDTV, however, multiple lines will be needed for teleconferencing.

He concluded that by extrapolating technological trends for ten years one sees that no large infrastructures are needed (such as HDTV requires), but that for practical sales, the cost of such a workstation needs to be in the range of today's PC.

Naritomi (SEGA) described his company's new product SATURN, which will be shipped Dec 1994. He emphasized that SEGA is focusing on a low price and easy to use system. SATURN is built using a 32 bit RISC chip from Hitachi 28.5MIPS (each system will have two, allowing 5 visual planes for scrolling and using a double video display processor). The system supports 24 bit color (16M colors); 3D graphics utilizes 3D polygons, and motion at 60FPS is claimed to be very smooth. Various resolutions are supported, up to 740x480. A 44KHz 16 bit sampled sound processor supports up to 32 voices, and a DSP produces special 3D sound effects including reverberation, delay, distortion, etc. The sound processor is controlled by a separate M68000. An attached CD-ROM will have double or quad speed (up to 600Kb/sec) with a variety of formats (CD-audio, CD karaoke, photo-CD, electronic book, video-CD). This can store about 100 photos, and using MPEG-1 compression, 74 minutes of 30FPS video can be shown. The CD-ROM will also have some write ability so that users who are playing games can resume from their current point. (SEGA anticipates it will price this system at under US$1K -- amazing how the price of this much hardware has dropped.) Naritomi also discussed future activities. The company is clearly anxious to avoid having to reinvent everything and wants to take advantage of the huge number of PC platforms now available, while at the same time not make their products too portable. He emphasized that they intend to develop common operating system compatibility while still retaining speed and ease of operation. Another keyword is alliances, with Yamaha for multimedia applications, with Hitachi, Victor, and other for consumer electronics, with Time-Warner for CATV applications, etc. Another plan is for electronic distribution of game software using CATV lines. Their plan to is have about 50 games on a cyclic transmission system, which they feel will allow no more than about a 90sec wait time (the maximum SEGA feels consumers will tolerate).
Nomura (Matsushita) discussed applications of VR to kitchen design system. I have described this in an earlier report (see "3d-2-92.1", 21 Feb 1992), and so will not go into detail here. By using head mounted display and data glove, users can examine the details of various kitchens and determine if a potential design is suitable. This project has been in operation for several years and uses standard technology. But it interested me as an example of how actively Japanese companies try to integrate advanced techniques into business activities. Admitting the existence of all the obvious problems with current VR systems (slow, poor resolution, uncomfortable, etc) Matsushita still elected to go forward with this system for commercial applications as an step toward what they call kansai (emotion) engineering. This involves trying to develop a translation system between a consumer's feeling and real design components. The idea is that when a customer expresses their feelings toward an object using adjectives (smoother, lighter, more modern, etc), then detail design items (such as object style, color, material, size, etc) are selected. The process uses a well known multivariate regression approach designed for qualitative data. (Thus the kitchen system is just one instance of this idea.) Nomura claims that Matsushita has sold over 300 kitchens using this process. This seems a very small number to me, but Nomura states that their goal of a 30 minute design has been met and that the company is satisfied with the results. (The company has designed more than 30,000 kitchens and maintains this data on a database called Matsushita Amenity Total Interior System, and it is from this that designs are selected based on the kansai approach mentioned above.)  There is also work to integrate the customer's VR design into Matsushita's CAD system and from there to their manufacturing operations. Another step is to move from kitchen to entire house design and we were told that there is already movement in this direction. MITI has been funding 36 companies, since 1989, on a seven year project on the technical developments for new houses. In addition to usual construction issues, questions of space, light, temperature, sound, airflow, and safety are said to be under study. Nomura concluded by stating that computer power is limited, and that he would like 5,000 times as much performance within five years. 
Sugimoto (Pioneer) gave an overview of VR and his projections on its development. As Sugimoto was a former head of R&D at NHK, it isn't too surprising that he feels that developments in display technologies are key to progress in VR, with workstation performance and (head or hand) tracking having a lesser role. He compared CRT and LCD technologies, although most of this was well known to the participants. He also emphasized the work on HDTV meshes very well with VR because its aspect ratio was more correct. He also showed a typical plasma display (PDP) with built in laser disk player, two embedded speakers and two remote rear speakers. Sugimoto was also very enthusiastic about 3D displays. He showed a new Pioneer 2-eye system with a 720x480 TFT display and also mentioned several experiments with 3D autostereoscopic (lenticular, glassless) displays, especially the 8-camera one from NHK (see "nhk-oh.93" 10 Jun 1993 for a description of this system). He stated that Pioneer and Sanyo are developing, jointly with NHK a commercializable 50" autostereoscopic system, and that "this is becoming a common technology in Japan."
Fellous (INA) described work allowing computer graphics images to mix with images from real scenes or to allow film-makers to include CG elements into their live shoots. One interesting aspect of this work is that of geometrical coherency, e.g., correct positioning of virtual elements wrt real elements, correct modelization, on many frames of the virtual camera. For example, one may want to see a 3D computer graphic of a building in its future (real) environment. Geometrical coherency implies that in the resulting combined sequence, the building will be at the right place and will keep stable as the physical camera moves around the physical environment. Of course, other aspects of visual coherency need to be considered too (lights, shadows, reflections, textures, etc). 
Troccaz (Grenoble) gave a very interesting summary of more than 500 neurosurgery interventions that her group has supported with VR techniques, and the severe demands for reliability, etc. In progress work is on radiotherapy, spine surgery, dental, wrist, and cranio-facial surgery, intro-utero surgery, node exploration, and simulation of anesthesia. VR is an exciting technique for medical applications that is being examined closely in many places, but for most participants Troccaz's talk was their first exposure to the operating room details and constraints.
Some research reported at this workshop represented updates of work that has also been described in earlier reports. 

Thalmann (Geneva) discussed work on virtual actors, overviewing work of her lab for more than a decade (see also another description of this in the report "pg.93" 12 Oct 1993).

Also Takemura (ATR) discussed teleconferencing research with realistic sensations, see "al.93" 20 Oct 1993, and "atr.92" 7 Feb 1992. This system creates the image of a conference room using computer graphics in real time. It also constructs images of the remotely located conference participants. Participants can engage in virtual cooperative manipulation of objects, for example a model of a car for a cooperative design project. There are still a great many problems to be dealt with, but ATR scientists have developed a prototype system that Takemura describes. There is a close parallel between this work and that described by Queau above.


DAY I: Synthetic Worlds

Philosophy: Models to Create Synthetic Worlds inside Computers -- Methods and Cultural, Scientific, Industrial and Medical Implications beyond Virtual and Artificial Reality. (Slides)
Tosiyasu L. Kunii

Prospective: Some Challenges of the (Synthetic) Worlds to Come -- A Computer Scientist View (Nothing)
Jacques Sakarovitch, Director Institute Blaise-Pascal
4 Place Jussieu, 75252 Paris CEDEX 05. Email: SAKAROVITCH@LITP.IBP.FR

Architecture: Romance of Synthetic Worlds -- Dream and Reality of
Workstations. (Slides)
Tosiyasu L. Kunii

Artificial Life as Synthetic Biology
Paul Bourgine, AL & AI Lab, CEMAGREF (Abstract only)

Artificial Life

Tree Growth: Modeling of Metamorphosis and Spatial Interactions in the
Architecture and Development of Plants (Paper)
Philippe de Reffye, CIRAD

The Animate Approach to Artificial Life (Nothing)
Jean-Arcady Meyer, Ecole Normale Superieure

The Meaning of Life: Real and/or Artificial (Paper)
James M. Goodwin, U-Aizu, Tel: +81 242-37-2619; Fax: +81 242-37-2731

Synthetic Art

Sound Synthesis in Computer Music (Paper)
Naotoshi Osaka, Information Science Research Lab, NTT Basic Research
Lab, 9-11 Midori-cho 3-Chome, Musashino-shi, Tokyo 180 Japan.
Tel: +81 422-59-4824; Fax: +81 244-59-3393; Email: OSAKA@SIVA.NTT.JP

Virtual Revisiting of Architectural Masterpieces and the Problem of
Lighting Simulation (Paper)
Karol Myszkowski, U-Aizu (see address above)

Performance Animation and Virtual Reality (Text and picture examples)
Philippe Limantour, Medialab France (Email: LIMANTOU@DMI.ENS.FR)

Synthetic Factories

Towards the Synthetic Factories: Requirements and Components (Paper)
Tsukasa Noma, Dept of AI, Faculty of Computer Science and Systems
Engineering, Kyushu Inst of Technology, 680-4 Kawazu, Iizuka, Fukuoka
820, Japan. Tel: +81 948-29-7500; Fax: +81 948-29-7601

Virtual Reality in Thomson CSF (Nothing)
Regis Vaillant, Thomson-CSF, Orsay France. Email: REGIS@THOMSON-LCR.FR

Virtual Manufacturing -- Concepts and Effects on Manufacturing in the
Future (Paper)
Kenjiro T. Miura, U-Aizu

DAY III Synthetic Communication

Landscapes synthesis through erosion simulation (Nothing)
Bernard Peroche, Ecole des Mines de Saint-Etienne

Simulating Autonomous Life for Virtual Actors (Paper)
Nadia Magnenat-Thalmann, University of Geneva

Influence of Emerging Synthetic Worlds on System Architectures (Nothing)
Jean Bellec, Bull

Virtual Space Teleconferencing System (Paper)
Dr Haruo Takemura, Senior Researcher, AI Dept, ATR Communication
Systems Research Labs, 2-2 Hikaridai, Seika-cho, Soraku-gun, Kyoto
619-02 Japan. Tel: +81 7749-5-1272; Fax: +81 7749-5-1208

DAY IV Synthetic Society for Everyone

Besides Immersion: Frames of Reference and Points of View (Paper)
Michael Cohen, U-Aizu, Tel: +81 242-37-2537; Fax: +81 242-37-2549

Telvirtuality: Merging of Telecommunication and Virtual Reality (Paper)
Philippe Queau, INA (Inst National de l'Audiovisuel), Email: QUEAU@INA.FR

SEGA's Multimedia (Slides)
Yuzo Naritomi, SEGA Enterprises Ltd

Synthetic Perception

Towards a Complete Representation by Means of Computer -- the
Instrumental Communication Interface (Paper)

Virtual Space Support System using Kansai Engineering (Paper)
Junji Nomura, Senior Staff Researcher, VR R&D Group, Info Sys Center,
Matsushita Electric Works, 1048, Kadoma, Osaka 571, Japan
Tel: +81 6 908-6835; Fax: +91 6 900-2766

Present Status of Synthetic Perception and its Future Technology
Tendency (Slides)
Masao Sugimoto, Pioneer Electronic Corporation

Synthetic Being

Towards Autonomous Synthetic Actors (Paper)
Daniel Thalmann, Swiss Federal Inst of Technology

The Synthetic T.V. System (Paper)
Armand Fellous, INA, Email: ARMAND@INA.FR

Teaching English Pronunciation to Japanese Through Sound and Image
Understanding (Slides)
Hisako Murakawa, U-Aizu


Landscape Creation (See paper of same author above)
Philippe de Reffye, CIRAD

Virtual Medicine (Nothing)
Jocelyne Troccaz, Inst Nat Polytech de Grenoble,

The remaining sections of this report are available to ATIP subscribers

Tokyo Office: Asian Technology Information Program (ATIP)
Harks Roppongi Building 1F
6-15-21 Roppongi, Minato-ku, Tokyo 106
Tel: +81 3 5411-6670; Fax: +81 3 5411-6671

U.S. Office: Asian Technology Information Program (ATIP)
P.O. Box 9678,
Albuquerque, New Mexico 87119-9678
Tel: (505)-842-9020; Fax: (505)-766-5166

For further information
Send email to :
Access WorldWideWeb Site :

Complete ATIP reports on Asian Science and Technology go to subscribers
and collaborating organizations by direct distribution, or via electronic
access. These contain text and often, charts, graphs and pictures. Reports
for unrestricted distribution often contain summarized, or abstracted
information. Sponsors can also obtain specific follow up information -
including copies of proceedings, selected papers, exhibition particulars,
updates, translations, query searches, etc. Contact ATIP at