Event report

August 4, 2022

Reproducing Food Textures Using Onomatopoeia

Experimental meetings of the 3D Structure Oden Society. Melt. Extra Research / X Project Report

Hello everyone, this is Saito from the 3D Structure Oden Club Executive Committee (we belong to the Metamaterial Design Technology Research Team and go by name of “Melt.”).

Oden is a traditional Japanese stew consisting of several ingredients such as boiled eggs, daikon, konjac, and processed fishcake. It is considered a winter soul-food and you will find thousands of variations across the country!

From January to March 2022, we conducted a prototyping workshop titled “Experimental Meeting of the 3D Structure Oden Club” to experiment with 3D-printed fishcake to reproduce different manga onomatopoeia as food textures. Our goal was to use computational design to give a physical shape to food-related sound effects inspired by Japanese comics, such as “mekyomochi” or “funyapori”. Read on to learn more about our Innovation Procedural Intuition Driven Development process!

Turns out, it’s possible to fill a 3D printer with fish paste…

The inspiration from this project came from our Melt. Open Meeting #2 “3D printed chocolate: texture as a material” where our guest speakers Ryosuke Wakasugi of Byte Bites Inc. and Hiroshi Mitachi of Shinkogeisha mentioned that fish paste could also be used in 3D printing.

…So we thought, let’s try it together!

The 3D Structure Oden meeting series has been conducted openly throughout the entire process. That enterprise developed into an opportunity for the general public to explore the infinite possibilities of food printing technology and a chance for users to join us in the process of seeking out new possibilities for food printer technology.

If the goal is to give shape to a texture, we need to start from impressions

The goal of our 3D Structure Oden was to create a new texture that no one has ever felt before. We adopted an open innovation development process to explore the possibilities of texture design inspired by the onomatopoeia of manga.

With the help of an online whiteboard service called Miro, which allows multiple users to simultaneously view and make edits, we began our work by fusioning different onomatopoeia to create “chimera onomatopoeia”. We eventually selected twelve of them.

After that, we established a system that classified each of our new chimera onomatopoeia in terms of six aspects based on subjective impressions: “hardness,” “uniformity” “size,” “crunch,” “viscosity,” and “smoothness”. We used these parameters as a reference to convert each of them chimera onomatopoeia into a model that can be printed by a 3D printer.


Creating a modeling system that anyone can use

We used a procedural CG modeling process to generate 3D models based on the chimera onomatopoeia as conceptual entities (metamaterials). By using a computational approach that created models based on uniform algorithms, we hoped to democratize the process of creating the 3D construct oden, making it into something that didn’t require the use of any special skills. dditionally, in order to facilitate open debate on the 3D models for the 3D construct oden we created, we used the CG modeling web service, Nodi, which allows for open source node base 3D modeling, and implemented a prototype of a 3D Construct Oden Generator in full cooperation with Mr. Wakasugi of Byte Bites inc.

For this process itself, we wanted to focus on getting something that worked, something that could create new textures based on onomatopoeia, whether those textures were ‘correct’ or not. With that in mind, we implemented the development of a quick and dirty system that disregarded the degree of consistency with the onomatopoeia input from the generator and model evaluation items.

If you’d like to learn more about it, the system is available on Nodi.

3D Oden Generator: https://nodi3d.com/editor/vhRjMYgbBGdU9QYtWJV0

Printing 3D Structure Oden

First of all, we took the tentative onomatopoeia-textured oden created with Nodi and printed them on a regular 3D printer using resin. Our hopes were pretty high after these first tests.

By the way, the model in the picture is a chimera onomatopoeia born of the fusion of “’bari” (used to represent the sound of crunching), and “metaa” (a battle sound effect). With their powers combined, they become the “Barimetaa” model.

I realize it might be a bit hard to tell just by looking at the photo, but what do you think?

The actual output of our edible onomatopeia oden ingredient test. We printed three kinds of test shapes (cube, triangular pyramid, and sphere).

Time to test the actual food texture

On March 18th, we organized a tasting session for our 3D Structure Oden meetup. We printed 3D food models of three chimera onomatopoeia, and collected the impressions of ten participants regarding their textures and so on.

We compared the actual tasting impressions to our initial parameters and earned feedback like “this used more consonants than I’d thought,” “there are a lot of sounds that seem to be connected with softness,” or “there’s a lot of variation in hardness, which seemed like it’d be the easiest to assign a numerical value.”

We analyzed the onomatopoeias from the perspective of linguistic processing to see if and how the impressions produced by language can influence the act of eating and feeling the food’s texture. As a result of analyzing our re-imagined onomatopoeia, or “re-onomatopoeia”, we saw the possibility that a correlation, however slight, could indeed be found.

Natural language processing analysis of the data gathered

With help from Kei Ichikawa from the International Arts Complex Information Life Studies group (IACILS), we analyzed and created heatmaps of the language-texture correlations using the Levenshtein distance, the Gestalt pattern patching, the Jaro-Winkler distance, and the Levenshtein method metrics.

As an example, on the heatmaps we can see that the onomatopoeia “nyuiin” shows a positive correlation between hardness and viscosity, and includes textures both firm and sticky. See the results of each analysis, as well as Ichikawa’s comments, below.

Levenshtein distance

Let’s take a deeper look at the Levenshtein distance between “nyuiin” original onomatopoeia and the new “nyuiin” re-onomatopoeia. The Levenshtein distance is a string metric for measuring the number of changes (such as swaps, deletions, and additions of characters) needed to change one word into another. In this case, we examined our data using an index where 1 represents total similarity and 0 represents no similarity.

You can see the results below, where the Levenshtein distance method did not detect much similarity. However, II think we might get an entirely different picture if we measured the degree of similarity using another metric.

Chuku-Chuku – 0.000000
Nyuuu – 0.000000
Nyoki-Nyoki – 0.333333
Myooh – 0.400000
Funyai – 0.200000
Kyumofu – 0.000000
Powaan – 0.200000
Monyopo – 0.000000

Correlation heatmap

These heatmaps graph out the degree of correlation among different attribute groups such as “ofyuchi”, “barimetaa”, and “nyuiin”. To understand how to look at the data, take for example the topmost row, which allows you to see hardness and its correlation to other attributes. Using the topmost row as our example, then within a Nyuuin environment, hardness and viscosity are positively correlated, so we might imagine that a Nyuuin texture is both firm and sticky.

Gestalt pattern matching

This is a method for determining the degree of matching between two strings by taking the longest common substring and then recursively using the same process on the strings leftover before and after the substring.

If these don’t have the same repetitions as the original, they won’t be considered a match, so this method doesn’t usually result in complete matches, but “nyuiin” had a relatively high number of matches.

Nyuiin
——–
0.0
0.0
0.36363636363636365
0.5
0.4444444444444444
0.0
0.2222222222222222
——–
Ofyuchi
——–
0.0
0.0
0.0
0.0
0.0
0.0
0.0
——–
Barimetaa
——–
0.0
0.0
0.0
0.0
0.2222222222222222
0.0

Jaro-Winkler distance method

This method is a little bit similar to the Levenshtein distance method, but calculates the distance based on the total number of characters in common as well as characters which don’t need to be transposed. Because of this, we can say that the partial matches for our example strings are significant.

Nyuiin
——–
0.0
0.0
0.6622222222222223
0.6888888888888888
0.6333333333333333
0.0
0.48333333333333334
——–
Ofyuchi
——–
0.0
0.0
0.0
0.0
0.0
0.0
0.0
——–
Barimetaa
——–
0.0
0.0
0.0
0.0
0.48333333333333334
0.0

Levenshtein method

To be honest, we probably shouldn’t get too caught up with the Levenshtein method. It takes into account the number of swaps and deletions, so it is probably the best for capturing nuance, but I also feel it may be a bit too complicated to explain and use.

Ofyuchi
——
Kyuwan – 0.0
Moyon-Moyon – 0.0
Mochi-Mochi – 0.0
Fuwapu – 0.0
Nuchofunya – 0.0
Noko-Noko – 0.0
Kyui – 0.0
BuchuReʔʔ – 0.0
Barimetaa – 0.0
——
Pyuyon – 0.0
Gyu-Gyu – 0.0
Boro-Boro – 0.0
Nyomo-Nyomo – 0.0
Goririn – 0.2
Mochohonii – 0.0

These three methods of parsing strings have all developed alongside search engines, so they have a much stronger application at correcting subtle omissions and misspellings. For this reason, it can be argued that it might be a bit of a tall order to use them in a case like ours, where we want to find matches between words with similar nuances.

Despite this, our efforts here showed surprising results in the case of “nyuiin” and its high matchrate.

In particular, when using the Jaro-Winkler method, we derived an approximately 0.69 rate of similarity with the “myooh” onomatopoeia. The nuance isn’t really conveyed by language, but isn’t it interesting to get a similarity rate as high as this?

Using the numerical data for the six texture attributes of oden, there’s a possibility we could present some new onomatopoeia based on the correlation heatmap.

Let’s take once more the “nyuiin” onomatopoeia as an example. First, we have discovered that the “nyuiin” oden shape features a positive correlation between hardness and viscosity.

This means that we can predict its 3D structure will be both firm and sticky. Therefore, we should be able to use this data to reverse-engineer onomatopoeia that will correlate to firm, sticky textures. By separating the attributes numerically, it is possible in theory to create onomatopoeia which are even more suited to the printed structures.

This system may be applied within companies for copyright and product name development (although I can’t say if they would be willing to invest so much effort into it).

Anyway, let’s call our reverse-engineered onomatopoeia “re-re-onomatopoeia”, shall we?
All right then, so it might be a bit more interesting to look at some examples that leave out the values from our initial onomatopoeia.

Using the language processing logic, we went ahead and used three methods to measure the correlation between reverse-engineered onomatopoeia and the human-generated onomatopoeia.

Unfortunately, we didn’t find any high rate of similarity between them, but we did get interesting results for some of the onomatopoeia. When using the Jaro-Winkler method for “nyoiin,” there were four onomatopoeia out of seven that showed slightly high similarity rates. One of the indicators seemed to point towards the existence of a “buubakiki” sort of onomatopoeia that might possibly have a common correlation to food texture sound effects in general. The use of prolonged vowels and “na, ni, nu, ne, no” sounds, or even the shape of the written word itself might also be relevant factors to consider.

Future possibilities: toward computational food texture design

As our next step, we will be organizing a Melt. open meeting with Byte Bites. IACILS organizer Ichikawa will be joining as a guest to provide an expert perspective on language processing, and we will continue investigating the possibilities of implementing computational food texture design using the newly discovered onomatopoeia as metamaterials.

We hope we can create a world in which supermarket shelves stock onomatopoeia. At dinner tables all over the world over, parents will announce that they will be having teriyaki “poni” for dinner and the children will cheer: ‘Yay! I love teriyaki “poni”!’ Which onomatopoeia is your favorite?

To be continued in the Toward the Dawn of Computational Food Texture Design!

Born from the Experimental meetings of the 3D Structure Oden Society held between January and March of 2022, the Dawn of Computational Food Texture Design is a development collective that combines digital fabrication technology and computational design to create new textural and food creation processes.

We’re trying to create a world in which edible onomatopoeia are commonplace, by facilitating interdisciplinary discussion and prototyping among individuals well-versed in different areas of expertise such as haptics, computational design, natural language processing, machine learning and digital fabrication.

Now that it’s over, our overall impression about this project is…

Hurray!!! It makes me really proud that the tasting tickets for the 3D Structure Oden sold out in the blink of an eye. Based on this evidence, I am starting to think that people in Nagoya might love 3D Structure Oden. Someday, a great writer might write a space fantasy novel featuring sound-eating monsters… But through this project, we ourselves have already transformed into these strange sound-eating creatures. (Saito)

We just went ahead and launched this project, and sure enough, it flew in the opposite direction. I felt a bit confused. What is the next step for us now? ​​ I, for one, can’t wait for the day when I can swing over to the store and find a stack of “monyuu” arranged on a supermarket display. Though, if I saw that I think I’d probably just take a picture and leave. (Asai)

There’s something really powerful about taking something like this from just being an idea, to a plan, to shaping it into an actual event. Thanks to these prototyping sessions, I feel like I’ve become a bit of an expert at 3D printing with fish paste. The workshop structure of these events allowed us to gather a lot of suggestions for food texture design, and I’m looking forward to building even more innovative processes in the future. (Wakasugi)

Want to get in touch with the Metamaterial Design Technology Research Team, Melt.?

Project participation inquiries:

You can apply to participate in our project through our Facebook Group.
Please feel free to browse around event announcement and activity report posts

For inquiries relating to our various projects and to collaborate with Melt:

At Melt. we call “X Projects” all the initiatives and events we organize with the cooperation of outside groups. This is an intuition-driven framework which includes a wide variety of activities in many different formats of metamaterial design, and everything from research to prototyping. If you want to get out there and search for new possibilities without being tied to old patterns of thought, if you’re looking get away from desk-bound theory and get your hands on some actual practice with real projects that close the loop between planning and implementation… By all means, please go ahead and contact us via the form below.

Melt. is also promoting design with materials that capture physical phenomena, so we are always on the lookout for measurement equipment, databases, and other resources we can make use of.

We would love to hear from companies willing to lend us point-cloud data, 3D scanners, microphones for collecting sounds, processing software and more. If you happen to work for such a company, and think that Melt.’s activities look like fun, or that you’d like to join in our discussions, not just provide equipment, please reach out to us at the email address below.

What Melt. can do for you…

  • We use technology to create undiscovered design methods that can be applied to exploring new sales channels.
  • Through our conferences and workshops, we help grow and nurture user communities in a wide range of topics.
  • “I didn’t know you could use it like that!” Through our projects, we generate bursts of insight that lead to all kinds of unsuspected methods and applications.

Contact

info.nagoya@fabcafe.com(Saito)

Author

  • FabCafe Global Editorial Team

    This articles is edited by FabCafe Global.

    Please feel free to share your thoughts and opinions on this article with us.
    Contact us

    This articles is edited by FabCafe Global.

    Please feel free to share your thoughts and opinions on this article with us.
    Contact us

Get in touch

Subscribe to FabCafe Global monthly newsletter for more stories in innovation and design.

Our Business Services

Building products and services that push innovation for companies