Researchers at Bangladesh University of Engineering and Technology (BUET) and University of California- Los Angeles (UCLA) recently developed a framework that can create Android applications from text descriptions. This new app was presented in a paper pre-published on arXiv.
“Our research group at the Department of CSE, BUET works on bridging the gap between human language and programming language to make computer programming more accessible,” Masum Hasan, one of the researchers who carried out the study, told TechXplore. “We have previously worked on natural language-based program repair, language-based code search, code document generation, etc. In continuation, we wondered if fully functioning software could be built from natural language specification.”
Most existing models for creating software based on text descriptions are based on end-to-end neural machine translation (NMT) models, similar to the one underpinning Google Translate. These models utilize NMT frameworks to translate human language into source code.
While some NMT-based techniques for text to app translation have achieved promising results, most of them are unable to generate large programs with hundreds of lines of code. To overcome this limitation, Hasan and his colleagues created a new formal language that captures the complexity of app source code, representing it in a highly compact form.
“We designed an intermediate formal language that can represent a user specified app in a fraction of the tokens—as much as 20 times less,” Hasan said. “We then developed a compiler that can convert the compact representation into actual source code and trained a neural machine translation model to translate natural language to this intermediate format, which can then be compiled into an app.”
Essentially, the researchers trained a translation model to generate representations in the intermediate language they created, rather than creating raw source code, as most existing text to app techniques would. By transforming text descriptions into this formal language, their framework allowed them to create fully fledged mobile applications, which proved unfeasible using previously proposed methods.
“Generating long sequences is a known limitation of current sequence generation models,” Hasan explained. “Source code is usually much longer than natural language and the current best models still struggle to achieve 20% accuracy in generating short code snippets. This makes it impractical to create large software, such as mobile applications consisting of hundreds of lines of codes, with current sequence generation networks.”
The framework created by Hasan and his colleagues, dubbed Text2App, can create Android applications using instructions in English. To use the system, users simply need to describe a mobile application with a specific scope and the framework automatically creates it for them. A video demo of Text2App is now available on GitHub, at: https://text2app.github.io/
Hasan said. “Imagine you want to memorize your grandmother’s special recipe for pasta. Instead of remembering the entire recipe step by step, you can only learn the steps that are unique, and look up the rest of the steps from a cookbook. This will significantly reduce the amount of effort you would have to spend. Similarly, our intermediate app representation only captures the specific requirements provided by the user through the natural language instruction and the compiler fills in the rest.”
The recent work by Hasan and his colleagues could allow researchers to re-think program and app generation, by introducing a new way through which programs can be summarized or represented that is easier for AI technology to understand. Notably, the AI models trained by the researchers exhibited a far higher capacity than other techniques to convert natural language into apps.
“If we design programming languages or frameworks with AI in mind, our current AI technologies can perform remarkably better in code generation from human language,” Hasan said. “Additionally, programming languages are made to be understood by compilers, and not entirely by AIs. We demonstrate that relying on AI models alone for complex tasks like code generation is not adequate, and additional support from components like compilers can help a great deal.”
The most notable achievement of the study carried out by this team of researchers is the development of the intermediate formal language that enables the conversion of text into an app, as well as a compiler that ultimately creates the app. So far, Hasan and his colleagues explored a mere fraction of its capabilities, but they are now also asking the open-source community to contribute to its further evaluation and development.
“With more development efforts, Text2App can become a full-fledged natural language- based app development platform,” Hasan said. “In our work, we also show that it is possible to create applications from surprisingly complex instructions with our system with the help of a large language model, GPT-3. However, more thoughts in this direction are necessary to make it fully reliable.”
An artificial neural network to acquire grounded representations of robot actions and language
More information:
Text2App: a framework for creating android apps from text descriptions. arXiv:2104.08301 [cs.CL]. arxiv.org/abs/2104.08301
Review4Repair: Code review aided automatic program repairing. arXiv:2010.01544 [cs.SE]. arxiv.org/abs/2010.01544
Language models are few-shot learners. arXiv:2005.14165 [cs.CL]. arxiv.org/abs/2005.14165
CoDesc: A large code-description parallel dataset. arXiv:2105.14220 [cs.CL]. arxiv.org/abs/2105.14220
2021 Science X Network
Citation:
Text2App: A framework that creates Android apps from text descriptions (2021, June 2)
retrieved 2 June 2021
from https://techxplore.com/news/2021-06-text2app-framework-android-apps-text.html
This document is subject to copyright. Apart from any fair dealing for the purpose of private study or research, no
part may be reproduced without the written permission. The content is provided for information purposes only.
