3 March 2025 1800 UTC

[ktoddbrown]

Agenda

These calls are lead by @ktoddbrown and are generally open calls. Please contact them for call in information. See #72 for overall strategic direction of meetings.

(5 | 1800) Opening

• Who is here https://github.com/ktoddbrown/SoilDRaH/wiki/2025-Meeting-Roll-call
• Note taking?
• Review and consent to agenda

(5 | 1805) Check ins

• How are you showing up? https://feelingswheel.com/

(20 | 1820) Scoping: Data rescue

Context: Data rescue is the transformation of information from a non-machine readable format to a machine readable one. This can include scanning paper copies of reports or records, transcribing scanned records into flat text documents, or extracting x-y values from a scanned graph. Generally this stage should be as close to the original information as practical for specific purpose, and should be extendable for future purpose. This stage must maintain provenance of the data including links to the original source, how the data was made machine readable, what data was not rescued, and who did this work.

Status: Currently this stage is being prototyped and documentation is under development in the wiki.

• Solo: Take 3-5 minutes and review the wiki.
• Round: Is the context complete and good enough to build documentation off of? Is there additional scope that is not captured in this context?
• Round: What task is coming out of this item? Who can take this on?

(20 | 1840) Scoping: Data harmonization

Context: Data harmonization is the process of transforming data from one data model to a second. In this context, a data model is a digital set of relational data tables and/or graphical (list-based) data including both primary data and metadata objects. In this project, data harmonization focuses on ensuring complete annotations that link the data objects and fill in any missing gaps needed for the data curation phase including units, control vocabulary, methodology details, and definitions. Any text extractions from non-machine readable sources should build off of a Data Rescue.

Status: Currently this stage is documented in the wiki but does not include references to data rescue.

• Solo: Take 3-5 minutes and review the wiki.
• Round: Is the context complete and good enough to build documentation off of? Is there additional scope that is not captured in this context?
• Round: What task is coming out of this item? Who can take this on?

(20 | 1900) Stuck to unstuck and ToDo's

ToDo:

• @ktoddbrown write up recap from 17 February 2025 #71, draft 2025 priorities in 2025 Recaps #72, and drafted today's agenda

• Round: How can the group help you move forward in this project?

(5 | 1920) Follow up on

• Round: What ToDo items are you taking away from today?

(5 | 1925) Check outs [appreciations, discoveries, and ToDo]

[ktoddbrown]

Notes

(5 | 1800) Opening

• Who is here https://github.com/ktoddbrown/SoilDRaH/wiki/2025-Meeting-Roll-call
• Note taking? Brandon and Vaasuki
• Review and consent to agenda

(5 | 1805) Check ins

Name and affiliation. Why you are here? How are you showing up? https://feelingswheel.com/

(20 | 1820) Scoping: Data rescue

Context: Data rescue is the transformation of information from a non-machine readable format to a machine readable one. This can include scanning paper copies of reports or records, transcribing scanned records into flat text documents, or extracting x-y values from a scanned graph. Generally this stage should be as close to the original information as practical for specific purpose, and should be extendable for future purpose. This stage must maintain provenance of the data including links to the original source, how the data was made machine readable, what data was not rescued, and who did this work.
Status: Currently this stage is being prototyped and documentation is under development in the wiki.

• Solo: Take 3-5 minutes and review the wiki https://github.com/ktoddbrown/SoilDRaH/wiki/Workflow-for-data-rescue.

• Round: Is the context complete and good enough to build documentation off of? Is there additional scope that is not captured in this context?

  • Good for now
  • Is this specific to manuscripts specifically? what about other types of data?
    • what about surveys?
    • There is as broader scope, but not yet captured in example(s)
  • Entire sub-field for finding relevant resources; how do we make identifying resources repeatable?
    • Appears context is complete, questions around how we identify and perhaps what is ‘assessed’ as being in scope
  • +2 for increased specificity
  • Perhaps more structure to the source material would be useful, particularly paying attention to ways to automate the process (or parts of the process)
    • Layla and Kathe will present work for CURE class
  • ROI is implicit at present; could we make that explicit?
    • New text required to address it
    • Not feasible to rescue all papers; what are the “metrics for decision making” regarding which papers to include or not (is there a better way to frame this?)

• Round: What task is coming out of this item? Who can take this on? (How do we identify a paper? How do we document references?)

  • Kathe will move the context to the wiki

• What else is coming up for you?

  • Tutorial for general audience
    • Screenshots
    • Zoom recording on ingest example(?)
    • Video for the CURE class...
  • How do we identify data sources that are worth our time?
    • Is there guidance for decision making?
    • Methods are crucial
  • How do we document sources and citations?
    • Web of science leverage
  • Can we create custom forms for Issues on GH which help guide users through the process?
  • How to incorporate or contact original authors?
    • Sometimes authors provide data not in original paper — how might this be handled? Lots of questions there—e.g. equivalent methods, updates/versions. Etc.
    • Should probably NOT accept correspondence data for this task
  • Are we filtering “open access” papers and/or data?
    • are we targeting a domain, or specific type of data or
    • are we focusing on licensing and focusing more on availability/reuse?
    • Will need to be mindful of copyright and how licensing affects reuse

(20 | 1840) Scoping: Data harmonization

Context: Data harmonization is the process of transforming data from one data model to a second. In this context, a data model is a digital set of relational data tables and/or graphical (list-based) data including both primary data and metadata objects. In this project, data harmonization focuses on ensuring complete annotations that link the data objects and fill in any missing gaps needed for the data curation phase including units, control vocabulary, methodology details, and definitions. Any text extractions from non-machine readable sources should build off of a Data Rescue.
Status: Currently this stage is documented in the wiki but does not include references to data rescue.

• Solo: Take 3-5 minutes and review the wiki https://github.com/ktoddbrown/SoilDRaH/wiki/Workflow-for-new-data-addition.

• Round: Is the context complete and good enough to build documentation off of? Is there additional scope that is not captured in this context?

  • Once you start the process, DO NOT make any other plans :)
  • Extend description of ’transformation’ right now it sounds like two databases are being ‘joined’ and it’s more complex to centralize. Reference level 0-2 data
  • Methodology will be an issue to address; parts of the methodology will be left off due to the nature of our interpretation
  • suggest pulling the entire methods section even ifi it’s only for reference later
  • How do you decide what is transformed and how?

• Round: What task is coming out of this item? Who can take this on?

  • Kathe Revisit to clarify ’transformation’ a bit more.
  • Kathe Move this over to the start of the wiki page

• Round: What else is coming up for you?

  • How will we document provenance?
  • how will we ensure consistency?
  • Are there ways to automate any of these tasks?
  • Currently duplication with data rescue
  • How do we guide methods merges between studies
  • is matching methods in harmonization or curation?

[ktoddbrown]

Recap

We had a scoping discussion on data rescue and harmonization today and established the context for each of these phases. Revisions of this context based on the conversation today are below. Our next call will be March 31 and focus on scoping the data curation phase as well as brainstorming next steps for the data rescue phase.

Data rescue

Data rescue is the transformation of information from a non-machine readable format to a machine readable one. This can include scanning paper copies of reports or records, transcribing scanned records into flat text documents, or extracting x-y values from a scanned graph. Generally this stage should be as close to the original information as practical for specific purpose, and should be extendable for future purpose. Information here may be primary or meta data. This stage must maintain provenance of the data including links to the original source, how the data was made machine readable, what data was not rescued, and who did this work.

Data harmonization

Data harmonization is the process of transforming data from the original collection of machine readable information to a standardize format. This original data collection may be composed of a machine-readable set of relational data tables and/or graphical (list-based) data including both primary data and metadata objects. The transformation is done via a set of scripts and manually created annotation file. This final format is a single data table with the following columns: id(s) - of variable - is type - with entry. Provenance in this stage is maintained by a read script, an annotation file documented any manual data extractions or assignments, and documentation of who did this work.