Recently, six undergraduates from the Southern University of Science and Technology (SUSTech) used a series of different methods to achieve automatic recognition, generation, and retrieval of oracle bone inscriptions. This included image and text processing, deep learning algorithms, the development of innovative neural network algorithms and Generative Adversarial Network (GAN), combined with knowledge of oracle morphology, semantics, and context.

The teaching achievement paper “Multimedia Meets Archaeology: A Novel Interdisciplinary Teaching Approach” has been accepted by the 2021 Frontiers in Education (FIE) Conference.

The undergraduate students who took part in this research were Ming ZENG, Muzhen YANG, Haotian LU, Wei WANG, Ruiling XI, and Shuyu ZHANG. Professor Jiang LIU guided them from the Department of Computer Science and Engineering (CSE), along with Professor Jigen TANG from the Center for Social Sciences, and the research team of Xiaoqing ZHANG, Jingfan HU, and Wen ZHONG.

The first step of the project team was to establish an oracle database and store the recognized oracle images and their corresponding Chinese inscriptions into the database. So far, 558 single inscriptions and more than 18,000 images have been entered.

The next step was to use a residual neural network (ResNet) model to recognize the handwritten oracle images. The third step was to select the pix2pix Generative Adversarial Network (GAN) as an automatic method to generate oracle, increasing the sample diversity. Lastly, it used the retrieval algorithm to compare the unknown oracle images with the existing ones and retrieve the unknown oracle images from the database.

The project team has begun to develop a WeChat application that can be used for display and interaction to make oracle’s reading accessible to the public and increase their understanding of oracle.

This project fully integrates the scientific strength of SUSTech’s archaeology research and the innovation of multimedia artificial intelligence algorithms. At the same time, it provides a model for exploring the interdisciplinary cooperation and teaching and learning practice of SUSTech.