Domain-Adapted Data Science

Ole Magnus Holter and Basil Ell develop approaches that make use of domain knowledge in the context of semantic parsing of textual requirements. Their goal is to formally represent (parts of) the meaning of textual requirements, so that the meaning of requirements, such as ”Shell boilers with a shell diameter of 1400 mm or greater shall be designed to permit entry of a person and shall be provided with a manhole for this purpose”, becomes more accessible to machines and the management of requirements can be improved [3]. Therefore, they make use of rule-based approaches, pre-trained large language models, declarative domain knowledge, and linguistic knowledge.

Egor V. Kostylev and colleagues study theoretical and practical connections between graph neural networks (GNNs), a modern structure-aware machine learning architecture, and classic logic-based knowledge representation formalisms. In particular, they designed a family of monotonic GNNs that allow for an efficient translation to Datalog logic-based language, and developed an efficient INDIGO system for knowledge graph completion [4,5]. These connections may be further developed to more general and tight connections between logical reasoning and statistical inference, as well as used in specific use cases.

In the context of a task relevant for the oil and gas industry, namely reservoir analogue identification, Summaya Mumtaz and Martin Giese have shown that a similarity measure based on the combination of domain knowledge (in the form of a taxonomy) with classical frequency-based features leads to significantly better results [6]. The disputation of Summaya’s PhD thesis «Hierarchy-based Similarity Measures and Embeddings – Supporting Machine Learning by Knowledge» took place in November 2021. Click here to read more about this project.

Jiaoyan Chen, Ernesto Jimenez-Ruiz, Ole Magnus Holter and colleagues have developed an ontology embedding framework named OWL2Vec* that can embed symbolic knowledge in an OWL ontology into a vector space, so that the information can be consumed by machine learning algorithms. OWL2Vec* can be directly applied to ontology completion tasks such as subsumption prediction as well as to help address machine learning challenges, such as sample shortage, by injecting symbolic knowledge [8,9].

Actionable recourse (AR) techniques are a popular class of post-hoc interpretability approaches that help the users of ML models to obtain their desired decision from a machine learning model. Given an individual’s preferences, an AR recommends feasible changes to their corresponding input that lead to the desired outcome by the model. To generate realistic ARs, it is important to capture and exploit the domain’s information and the preferences of the users in the explanation process. Peyman Rasouli and Ingrid Chieh Yu are working on a model-agnostic framework that combines user/domain-level knowledge with model/data-level information to create plausible ARs that can guide individuals to obtain their desired decision from any ML classification and regression model in a simple and efficient manner [10].

Current explainable artificial intelligence (XAI) techniques only rely on the observational data to analyze and explain the behavior of machine learning models. To increase the comprehensibility and faithfulness of explanations of ML models, hence, it is essential to exploit domain knowledge that bridges between the models and human concepts. Peyman Rasouli and Ingrid Chieh Yu aim to integrate domain knowledge (in the form of knowledge graphs and taxonomies) with structured/tabular data to provide more feasible, comprehensible, and faithful explanations. Peyman will submit his PhD thesis in November 2022.

Roxana Pop is investigating neural methods for temporal prediction on structured data, with an emphasis on graph neural networks extensions. Her overarching objective is to combine neural methods with symbolic AI approaches, and extract logical rules from the trained neural networks for making the predictions. In this way, the predictions can be understood by human experts, but the extraction is data-driven, making the best of the two AI paradigms.

In 2018, Farhad Nooralahzadeh and colleagues demonstrated a system for extracting relations from scientific text [18]. This tool was built with convolutional neural networks using semi-supervised learning and structured domain knowledge. The tool was a top performer in the SemEval international shared task: Task 7 on Semantic Relation Extraction and Classification in Scientific Papers. The system ranked third out of 28 participants. The disputation of Farhad’s PhD thesis «Low-Resource Adaptation of Neural NLP Models» took place in October 2020. Click here to read more about this project.

Basil Ell develops approaches to align symbolic data (i. e., ontologies) with sub-symbolic data (e. g., texts or tables). The alignment enables labeled training data to be generated via distant supervision for approaches such as information extraction (IE) for ontology population or natural language generation (NLG). Having symbolic and sub-symbolic data aligned means obtaining hybrid data that can be processed by hybrid approaches. He received a best paper award at LDK 2021 – 3rd Conference on Language, Data and Knowledge, for his work on mining association rules that help to bridge between text and data [16]. Recently, he investigated the benefits of taking into account literals (i.e., unstructured data) for the prediction of links in knowledge graphs [17]. Furthermore, he develops statistical approaches that are applied to symbolic data (KGs) for the purposes of identifying regularities and anomalies, for the prediction of missing facts, for the evaluation of the structural plausibility of facts, for bridging between structured and unstructured data (as in IE, question answering, NLG), and the structural classification of regions within graphs (which is similar to sequence labeling, but on graphs).