Plans Underway To Develop Android Apps From Texts Descriptions

Researchers at Bangladesh University of Engineering and Technology (BUET) and University of California- Los Angeles (UCLA) have announced that they have developed a framework that would make it possible to create Android applications from text descriptions. This news was presented in a paper pre-published on arXiv.

“Our research group at the Department of CSE, BUET works on bridging the gap between human language and programming language to make computer programming more accessible,” Masum Hasan, one of the researchers who carried out the study, told TechXplore. “We have previously worked on natural language-based program repair, language-based code search, code document generation, etc. In continuation, we wondered if fully functioning software could be built from natural language specification.”

It should be noted however that, most existing models for creating software based on text descriptions rely on end-to-end neural machine translation (NMT) models, similar to the one underpinning Google Translate. While these NMT-based techniques for text to app translation have achieved promising results, most of them have been unable to generate large programs with hundreds of lines of communication. These limitations led Hasan and his colleagues to create a new formal language that captures the complexity of app source code, representing it in a highly compact form.

“We designed an intermediate formal language that can represent a user specified app in a fraction of the tokens—as much as 20 times less,” Hasan said. “We then developed a compiler that can convert the compact representation into actual source code and trained a neural machine translation model to translate natural language to this intermediate format, which can then be compiled into an app.”

This means that the researchers trained a translation model to generate representations in the intermediate language they created, rather than creating raw source code, as most existing text to app techniques would. By transforming text descriptions into this formal language, their framework allowed them to create fully fledged mobile applications, which proved unfeasible using previously proposed methods.

“Generating long sequences is a known limitation of current sequence generation models,” Hasan explained. “Source code is usually much longer than natural language and the current best models still struggle to achieve 20% accuracy in generating short code snippets. This makes it impractical to create large software, such as mobile applications consisting of hundreds of lines of codes, with current sequence generation networks.”

The framework created by Hasan and his colleagues, aptly named Text2App, can create Android applications using instructions given in English language. To use the system, users simply need to describe a mobile application with a specific scope and the framework automatically creates it for them. A video demo of Text2App is now available on GitHub, at: https://text2app.github.io/

Hasan said. “Imagine you want to memorize your grandmother’s special recipe for pasta. Instead of remembering the entire recipe step by step, you can only learn the steps that are unique, and look up the rest of the steps from a cookbook. This will significantly reduce the amount of effort you would have to spend. Similarly, our intermediate app representation only captures the specific requirements provided by the user through the natural language instruction and the compiler fills in the rest.”

This work by Hasan and his colleagues could allow researchers to re-think program and app generation, as it has introduced a new way through which programs can be summarized or represented that is easier for AI technology to understand. Notably, the AI models trained by the researchers exhibited a far higher capacity than other techniques to convert natural language into apps.

“If we design programming languages or frameworks with AI in mind, our current AI technologies can perform remarkably better in code generation from human language,” Hasan said. “Additionally, programming languages are made to be understood by compilers, and not entirely by AIs. We demonstrate that relying on AI models alone for complex tasks like code generation is not adequate, and additional support from components like compilers can help a great deal.”

The most notable achievement of the study carried out by this team of researchers is the development of the intermediate formal language that enables the conversion of text into an app, as well as a compiler that ultimately creates the app. So far, Hasan and his colleagues explored a mere fraction of its capabilities, but they are now also asking the open-source community to contribute to its further evaluation and development, which could make the idea even bigger. In a nutshell, this could become one of the biggest innovation made in a very long time.

“With more development efforts, Text2App can become a full-fledged natural language- based app development platform,” Hasan said. “In our work, we also show that it is possible to create applications from surprisingly complex instructions with our system with the help of a large language model, GPT-3. However, more thoughts in this direction are necessary to make it fully reliable.”

The ease this provides would make android applications more accessible and easier to create, thereby bringing more businesses to the outside world. It will also reduce the price of creating apps to meet your spec without compromising quality.