Connecting scientific data to scientific experiments with provenance

S Miles; E Deelman; P Groth; K Vahi; G Mehta; L Moreau; K Chiu; R Buyya

doi:10.1109/E-SCIENCE.2007.22

Connecting scientific data to scientific experiments with provenance

S Miles, E Deelman, P Groth, K Vahi, G Mehta, L Moreau, K Chiu (Editor), R Buyya (Editor)

Research output: Chapter in Book/Report/Conference proceeding › Conference paper

27 Citations (Scopus)

195 Downloads (Pure)

Abstract

As scientific workflows and the data they operate on, grow in size and complexity, the task of defining how those workflows should execute (which resources to use, where the resources must be in readiness for processing etc.) becomes proportionally more difficult. While "workflow compilers", such as Pegasus, reduce this burden, a further problem arises: since specifying details of execution is now automatic, a workflow's results are harder to interpret, as they are partly due to specifics of execution. By automating steps between the experiment design and its results, we lose the connection between them, hindering interpretation of results. To reconnect the scientific data with the original experiment, we argue that scientists should have access to the full provenance of their data, including not only parameters, inputs and intermediary data, but also the abstract experiment, refined into a concrete execution by the "workflow compiler". In this paper, we describe preliminary work on adapting Pegasus to capture the process of workflow refinement in the PASOA provenance system

Original language	English
Title of host publication	Third IEEE International Conference on e-Science and Grid Computing
Subtitle of host publication	Bangalore, India. 10-13 December 2007
Editors	Geoffry Fox, Kenneth Chiu, Rajkumar Buyya
Place of Publication	Los Alamitos, CA
Publisher	IEEE Computer Society
Pages	179-186
Number of pages	8
ISBN (Print)	9780769530642, 0769530648
DOIs	https://doi.org/10.1109/E-SCIENCE.2007.22
Publication status	Published - 2007
Event	3rd IEEE International Conference on e-Science and Grid Computing - Bangalore, India Duration: 1 Jan 2007 → …

Conference

Conference	3rd IEEE International Conference on e-Science and Grid Computing
Country/Territory	India
City	Bangalore
Period	1/01/2007 → …

Access to Document

10.1109/E-SCIENCE.2007.22

Connecting scientific data to scientific experiments with provenanceAccepted author manuscript, 285 KB

Cite this

Miles, S., Deelman, E., Groth, P., Vahi, K., Mehta, G., Moreau, L., Chiu, K. (Ed.), & Buyya, R. (Ed.) (2007). Connecting scientific data to scientific experiments with provenance. In G. Fox, K. Chiu, & R. Buyya (Eds.), Third IEEE International Conference on e-Science and Grid Computing: Bangalore, India. 10-13 December 2007 (pp. 179-186). IEEE Computer Society. https://doi.org/10.1109/E-SCIENCE.2007.22

@inbook{65a47270e57543ca8ea11f657acc4e17,

title = "Connecting scientific data to scientific experiments with provenance",

abstract = "As scientific workflows and the data they operate on, grow in size and complexity, the task of defining how those workflows should execute (which resources to use, where the resources must be in readiness for processing etc.) becomes proportionally more difficult. While {"}workflow compilers{"}, such as Pegasus, reduce this burden, a further problem arises: since specifying details of execution is now automatic, a workflow's results are harder to interpret, as they are partly due to specifics of execution. By automating steps between the experiment design and its results, we lose the connection between them, hindering interpretation of results. To reconnect the scientific data with the original experiment, we argue that scientists should have access to the full provenance of their data, including not only parameters, inputs and intermediary data, but also the abstract experiment, refined into a concrete execution by the {"}workflow compiler{"}. In this paper, we describe preliminary work on adapting Pegasus to capture the process of workflow refinement in the PASOA provenance system",

author = "S Miles and E Deelman and P Groth and K Vahi and G Mehta and L Moreau and K Chiu and R Buyya",

year = "2007",

doi = "10.1109/E-SCIENCE.2007.22",

language = "English",

isbn = "9780769530642",

pages = "179--186",

editor = "Geoffry Fox and Kenneth Chiu and Rajkumar Buyya",

booktitle = "Third IEEE International Conference on e-Science and Grid Computing",

publisher = "IEEE Computer Society",

note = "3rd IEEE International Conference on e-Science and Grid Computing ; Conference date: 01-01-2007",

}

Miles, S, Deelman, E, Groth, P, Vahi, K, Mehta, G, Moreau, L, Chiu, K (ed.) & Buyya, R (ed.) 2007, Connecting scientific data to scientific experiments with provenance. in G Fox, K Chiu & R Buyya (eds), Third IEEE International Conference on e-Science and Grid Computing: Bangalore, India. 10-13 December 2007. IEEE Computer Society, Los Alamitos, CA, pp. 179-186, 3rd IEEE International Conference on e-Science and Grid Computing, Bangalore, India, 1/01/2007. https://doi.org/10.1109/E-SCIENCE.2007.22

Connecting scientific data to scientific experiments with provenance. / Miles, S; Deelman, E; Groth, P et al.
Third IEEE International Conference on e-Science and Grid Computing: Bangalore, India. 10-13 December 2007. ed. / Geoffry Fox; Kenneth Chiu; Rajkumar Buyya. Los Alamitos, CA: IEEE Computer Society, 2007. p. 179-186.

Research output: Chapter in Book/Report/Conference proceeding › Conference paper

TY - CHAP

T1 - Connecting scientific data to scientific experiments with provenance

AU - Miles, S

AU - Deelman, E

AU - Groth, P

AU - Vahi, K

AU - Mehta, G

AU - Moreau, L

A2 - Chiu, K

A2 - Buyya, R

A2 - Fox, Geoffry

A2 - Chiu, Kenneth

A2 - Buyya, Rajkumar

PY - 2007

Y1 - 2007

N2 - As scientific workflows and the data they operate on, grow in size and complexity, the task of defining how those workflows should execute (which resources to use, where the resources must be in readiness for processing etc.) becomes proportionally more difficult. While "workflow compilers", such as Pegasus, reduce this burden, a further problem arises: since specifying details of execution is now automatic, a workflow's results are harder to interpret, as they are partly due to specifics of execution. By automating steps between the experiment design and its results, we lose the connection between them, hindering interpretation of results. To reconnect the scientific data with the original experiment, we argue that scientists should have access to the full provenance of their data, including not only parameters, inputs and intermediary data, but also the abstract experiment, refined into a concrete execution by the "workflow compiler". In this paper, we describe preliminary work on adapting Pegasus to capture the process of workflow refinement in the PASOA provenance system

AB - As scientific workflows and the data they operate on, grow in size and complexity, the task of defining how those workflows should execute (which resources to use, where the resources must be in readiness for processing etc.) becomes proportionally more difficult. While "workflow compilers", such as Pegasus, reduce this burden, a further problem arises: since specifying details of execution is now automatic, a workflow's results are harder to interpret, as they are partly due to specifics of execution. By automating steps between the experiment design and its results, we lose the connection between them, hindering interpretation of results. To reconnect the scientific data with the original experiment, we argue that scientists should have access to the full provenance of their data, including not only parameters, inputs and intermediary data, but also the abstract experiment, refined into a concrete execution by the "workflow compiler". In this paper, we describe preliminary work on adapting Pegasus to capture the process of workflow refinement in the PASOA provenance system

UR - http://www.scopus.com/inward/record.url?scp=42449095840&partnerID=8YFLogxK

U2 - 10.1109/E-SCIENCE.2007.22

DO - 10.1109/E-SCIENCE.2007.22

M3 - Conference paper

SN - 9780769530642

SN - 0769530648

SP - 179

EP - 186

BT - Third IEEE International Conference on e-Science and Grid Computing

PB - IEEE Computer Society

CY - Los Alamitos, CA

T2 - 3rd IEEE International Conference on e-Science and Grid Computing

Y2 - 1 January 2007

ER -

Connecting scientific data to scientific experiments with provenance

Abstract

Conference

Access to Document

Other files and links

Fingerprint

Cite this