A Lightweight Approach to Uncover Technical Information in Unstructured Data

Nicolas Bettenburg bio photo By Nicolas Bettenburg


Developer communication through email, chat, or issue report comments consists mostly of largely unstructured data, i.e., natural language text, mixed with technical artifacts such as project-specific jargon, abbreviations, source code patches, stack traces and identifiers. These technical artifacts represent a valuable source of knowledge on the technical part of the system, with a wide range of applications from establishing traceability links to creating project-specific vocabularies. However, the lack of well-defined boundaries between natural language and technical content make the automated mining of technical artifacts challenging. As a first step towards a general-purpose technique to extracting technical artifacts from unstructured data, we present a lightweight approach to untangle technical artifacts and natural language text. Our approach is based on existing spell checking tools, which are well-understood, fast, readily available across platforms and impartial to different kinds of textual data. Through a handcrafted benchmark, we demonstrate that our approach is able to successfully uncover a wide range of technical artifacts in unstructured data.

Download the Full Paper

The full paper is available for download, if you want to learn more about Lightweight Techniques for Mining Unstructured Data.


If you would like to cite the research in your own work, please use the following citation:

   author    = "Nicolas Bettenburg and
                Bram Adams and
                Ahmed E. Hassan and
                Michel Smidt",
   title     = "A Lightweight Approach to Uncover Technical Artifacts in
                Unstructured Data",
   booktitle = "ICPC",
   year      = "2011",
   pages     = "185-188",
   ee        = "http://doi.ieeecomputersociety.org/10.1109/ICPC.2011.36",
   crossref  = "DBLP:conf/iwpc/2011",
   bibsource = "DBLP, http://dblp.uni-trier.de"

   title     = "The 19th IEEE International Conference on Program Comprehension,
                ICPC 2011, Kingston, ON, Canada, June 22-24, 2011",
   booktitle = "ICPC",
   publisher = "IEEE Computer Society",
   year      = "2011",
   isbn      = "978-1-61284-308-7",
   ee        = "http://ieeexplore.ieee.org/xpl/mostRecentIssue.jsp?punumber=5968117",
   bibsource = "DBLP, http://dblp.uni-trier.de"

Legal Disclaimer

This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author’s copyright. In most cases, these works may not be reposted without the explicit permission of the copyright holder.