Abstract
In recent years, availability of massive amounts of data, the global nature of collaborative research, the emergence of web services platforms, and interconnected resources on the Internet lead to the heavy uses of data integration, data exploration and data analysis tools for domain researchers in their everyday work.
In this new model, modern scientific applications rely heavily upon accessing, integrating and analyzing online data utilizing computational tools from numerous sources in highly ad hoc and dynamic ways.
Meanwhile, open online data repositories are often heterogeneous and outside users' control. What's more, their services are offered as is - without any coordination with the site.
Domain researchers (domain experts) have extensive domain knowledge but not necessarily skills and knowledge on web protocols, databases, and formal query languages.
Although new technologies such as microservices, semantic web, and cloud computing have helped, a system in which domain researchers are comfortable designing new applications and developing them without technical support from computer scientists is yet to be realized.
It turns out that many available artifacts are not suitable for such a computing paradigm without substantial adaptation or customization, and many well-regarded systems insist on substantial manual intervention.
A friendly graphical interface and a web services based system that is clunky but covers plenty of resources has not been created yet.
The question that still needs to be answered is if users can state the query needs in an easy way that computers understand it and execute it without further help from experts. When queries involve a large set of complex, heterogeneous and distributed sciences databases and tools, the answer is truly nontrivial.
In this dissertation, we present a visual querying language, named VisFlow, to frame arbitrary workflow queries over a distributed set of online databases.
We discuss the query execution model by illustrating how it is translated to a script querying language called BioFlow and how BioFlow is mapped to executable code.
We also discuss its query mapping strategy that aims to improve efficiency using parallelization whenever possible.
We highlight its novelty by evaluating VisFlow workflow composition and execution, showing workflow composition and comparing the time of execution in a geoscience pipeline. Five examples are present to show that VisFlow is capable of seamlessly dealing with traditional, web service based and semantic web databases and online analytics.
Finally, we conclude our discussion with VisFlow's planned future research.