Data Management Plan
GUID: gov.noaa.nmfs.inport:71550 | Published / External
Data Management Plan
DMP Template v2.0.1 (2015-01-01)
Please provide the following information, and submit to the NOAA DM Plan Repository.Reference to Master DM Plan (if applicable)
As stated in Section IV, Requirement 1.3, DM Plans may be hierarchical. If this DM Plan inherits provisions from a higher-level DM Plan already submitted to the Repository, then this more-specific Plan only needs to provide information that differs from what was provided in the Master DM Plan.
1. General Description of Data to be Managed
The data described here result from benthic coral demographic surveys within belt transects of specified length and width for two life stages (juveniles and adults) across the US Pacific since 2013. This data is collected as part of the NOAA Pacific Islands Fisheries Science Center (PIFSC) and Ecosystem Sciences Division (ESD; formerly the Coral Reef Ecosystem Division) led National Coral Reef Monitoring Program (NCRMP) missions around the Mariana Archipelago, American Samoa, Pacific Remote Island Areas, and the Hawaiian Archipelago since 2013.
The in-situ diver coral demographic surveys are part of the Rapid Ecological Assessment (REA) surveys for corals and fish conducted during ESD-led NCRMP missions. From 2013 to 2017, a two-stage stratified random sampling (StRS) design was used to survey the coral reef ecosystems across sub-island sectors and three depth strata (shallow: 0-6m, mid: 6-18m and deep: >18m). Starting in 2018 a one-stage StRS design was employed. These sites represent a broad range of depths (1-25 m), habitat types (aggregate reef, patch reef, pavement, rock and boulder and rubble), coral cover, and diving conditions. Allocation of sampling effort was proportional to strata area and variance in coral density. The StRS design effectively reduces estimate variance through stratification using environmental covariates and by sampling more sites rather than sampling more transects at a site. Therefore, site-level estimates and site to site comparisons should be used with caution.
Starting in 2023 coral demographic data was also derived by the GIS analysis of benthic Structure from Motion (SfM) imagery. The source imagery was collected during in-water surveys conducted by divers, and is documented separately here: https://www.fisheries.noaa.gov/inport/item/63095.During each SfM survey, an 18m transect line was deployed along the isobath and photographed using underwater cameras while swimming in a back and forth swim pattern for later processing. At depths of 0-18m, SfM surveys were conducted over a 3 x 20m area and at depths >18m, a 3 x 13m area was surveyed, both with the transect running down the middle of the survey area. The photographs were processed using Agisoft Metashape software to generate orthomosaic images that were analyzed in ArcGIS for juvenile and adult coral colony demographic metrics.
The data provide information on adult coral colony counts, morphology, size, partial mortality (old and recent dead), presence and causation of disease and other compromised health conditions, including bleaching. Juvenile colony surveys include morphology and size. Taxonomic identification of adult colonies is to the lowest taxonomic level possible and genus level for juveniles. Some segment observations were repeated for internal quality control starting in 2019, and indicated with a repeat segment flag and transectnum = -999; use column OG_OR_RPT_SEG to filter for only original segments. Refer to data dictionaries for details on column conditions.
The raw data also include individual observations of crustose coralline algae diseases and Alcyonarian disease type and lesion size as well as the presence of other Anthozoans, such as other cnidarians including Alcyonareans, Zoantharians, Corallimorpharians, and Antipatharians (does not apply to PMNM surveys conducted in 2014 and 2015 or surveys conducted after 2017).
Notes: Only a maximum of 4000 characters will be included.
Notes: Data collection is considered ongoing if a time frame of type "Continuous" exists.
Notes: All time frames from all extent groups are included.
Notes: All geographic areas from all extent groups are included.
(e.g., digital numeric data, imagery, photographs, video, audio, database, tabular data, etc.)
(e.g., satellite, airplane, unmanned aerial system, radar, weather station, moored buoy, research vessel, autonomous underwater vehicle, animal tagging, manual surveys, enforcement activities, numerical model, etc.)
2. Point of Contact for this Data Management Plan (author or maintainer)
Notes: The name of the Person of the most recent Support Role of type "Metadata Contact" is used. The support role must be in effect.
Notes: The name of the Organization of the most recent Support Role of type "Metadata Contact" is used. This field is required if applicable.
3. Responsible Party for Data Management
Program Managers, or their designee, shall be responsible for assuring the proper management of the data produced by their Program. Please indicate the responsible party below.
Notes: The name of the Person of the most recent Support Role of type "Data Steward" is used. The support role must be in effect.
4. Resources
Programs must identify resources within their own budget for managing the data they produce.
5. Data Lineage and Quality
NOAA has issued Information Quality Guidelines for ensuring and maximizing the quality, objectivity, utility, and integrity of information which it disseminates.
(describe or provide URL of description):
Lineage Statement:
Coral demographic data has been collected at stratified-random sites (StRS) across US Pacific Island Regions for the National Coral Reef Monitoring Program (NCRMP) since 2013. The methodology for selecting StRS and analyzing this data can be found in the sources listed below. The methodology has continued to evolve. Starting in 2023, some data was collected with GIS analysis alongside in-water diver surveys.
The benthic SfM survey methodology, employed by the NOAA Ecosystem Sciences Division (ESD), began in 2019. Benthic imagery is collected at stratified random sites and generated into orthomosaic images using Agisoft Metashape software. Orthomosaics are imported into ArcGIS for annotation. Annotation of SfM belt orthomosaics is modelled after in situ diver adult and juvenile coral belt surveys employed by the PIFSC Ecosystem Sciences Division and includes coral species identification, maximum diameter measurement, partial mortality estimates, and disease and condition identification.
Process Steps:
- A stratified random sampling (StRS) design was employed to survey the coral reef ecosystems through the U.S. Pacific regions. The survey domain encompassed the majority of the mapped area of reef and hard bottom habitats. The stratification scheme included island, reef zone, and depth in all regions, as well as habitat structure type in the Main Hawaiian Islands. The habitat structure types included simple, complex, and coral-rich. Depth categories of shallow (0-6 m), mid (> 6-18m) and deep (>18-30 m) were also incorporated into the stratification scheme. Allocation of sampling effort was proportional to strata area and variance in coral density. Sites were randomly selected within each stratum. A geographic information system (GIS) and digital spatial databases of benthic habitats (NOAA National Centers for Coastal Ocean Science NCCOS), reef zones (IKONOS satellite imagery, NDGC 1998) bathymetry (NDGC 1998, CREP benthic mapping data), and marine reserve boundaries (NOAA) were used to facilitate spatial delineation of the sampling survey domain, strata, and sample units. Map resolution was such that the survey domain could be overlain by a grid using a GIS with individual cells of size 50 m by 50 m in area. A one-stage sampling scheme following Cochran (1977) was employed. Grid cells containing at least 10% hard-bottom reef habitats were designated as primary sample units (referred to as sites). (Citation: Winston M, Couch C, Ferguson M, Huntington B, Swanson D, Vargas-Angel B. 2019. Ecosystem Sciences Division Standard Operating Procedures: Data Collection for Rapid Ecological Assessment Benthic Surveys, 2018 Update. NOAA Tech. Memo. NOAA-TM-NMFS-PIFSC-92, 66 p. doi:10.25923/w1k2-0y84)
- At each site, benthic Rapid Ecological Assessment (REA) surveys were conducted within one 10 sq. meter belt transect. Transects were 18 meters in length. Adult coral colonies (>=5 cm) were surveyed within four (1.0 x 2.5 m) segments at 5 meter increments along the 18 meter transect in the following manner: 0-2.5 m (segment 1); 5.0-7.5 m (segment 3); 10-12.5 m (segment 5); and 15-17.5 m (segment 7). All colonies whose center fell within 0.5 m on either side of each transect line were identified to lowest taxonomic level possible (species or genus), measured for size (maximum diameter to nearest cm), and morphology was noted. In addition, partial mortality and condition of each colony was assessed. Partial mortality was estimated as percent of the colony in terms of old dead and recent dead. The cause of recent mortality was identified if possible. The condition of each colony including disease (not attributed to recent tissue loss) and bleaching was noted along with the extent (percent of colony affected) and level of severity (range from moderate to acute). The lowest taxonomic level of coral identification was genus except for a select number of species consistently identified to species by all divers conducting the surveys. The number of species may change for each survey year depending on the experience and training of the benthic divers conducting the surveys. The list of coral species is included for each survey year. Juvenile coral colonies (<5 cm) were surveyed within three (1.0 x 1.0 m) segments along the same transect: 0-1.0 m (segment 1); 5.0-6.0 m (segment 3); and 10.0-11.0 m (segment 5). Juvenile colonies were distinguished in the field by a distinct tissue and skeletal boundary (not a fragment of a larger colony). Each juvenile colony was identified to lowest taxonomic level (genus or species) and measured for size by recording both the maximum and perpendicular diameter to the nearest 2 mm. (Citation: Winston M, Couch C, Ferguson M, Huntington B, Swanson D, Vargas-Angel B. 2019. Ecosystem Sciences Division Standard Operating Procedures: Data Collection for Rapid Ecological Assessment Benthic Surveys, 2018 Update. NOAA Tech. Memo. NOAA-TM-NMFS-PIFSC-92, 66 p. doi:10.25923/w1k2-0y84)
- Structure from Motion (SfM) benthic surveys were conducted by both the fish and benthic teams to record the benthic community composition. A 3 x 20 m plot was visually established centered on a transect line (at sites >18 m, a 3 x 13 m plot was visually established). 3-4 ground control points (GCPs) were placed within the plot for scale. White balance settings were adjusted in situ using a 15% gray card. JPEG images were collected using a Canon SL2 or SL3 DSLR camera in an underwater housing with a dome port. Images were collected at each site by swimming in a back-and-forth motion 1 m above the substrate capturing images continuously to achieve a 60-80% overlap. (Citation: Suka R, Asbury M, Couch C, Gray A, Winston M, Oliver T. 2019. Processing Photomosaic Imagery of Coral Reefs Using Structure-from-Motion Standard Operating Procedures. U.S. Dept. of Commerce, NOAA Technical Memorandum NOAA-TM-NMFS-PIFSC-93, 54 p. doi:10.25923/h2q8-jv47)
- SfM images for each site were evaluated for image quality and images deemed unsatisfactory (e.g. overexposed, images of blue water or images of divers, or images not taken perpendicular to the reef) were removed from the image set. Imagery was color and exposure corrected using Adobe Lightroom if white balance was not achieved in situ or a substantial number of images were dark, respectively. Image adjustments were batch run per site for consistent correction. The SfM approach produces an accurately scaled, two-dimensional (2D) orthomosaic model created from the overlapping imagery. Raw imagery was imported into Agisoft Metashape software (AgiSoft Metashape Professional Version 1.6.1 or later) where images were aligned and used to build 3D dense point clouds (DPCs) following parameters described by Suka et al. (2019). DPCs were then imported into Viscore, a visualization software (Petrovic et al., 2014) where they were scaled and oriented using the GCP information. The ground sample distance (GSD) of the scaled DPC, which estimates the resolution per pixel by measuring the size of each pixel on the ground, ranged from 2-4 mm/pix. A geometrically accurate 2D projection of the DPC (orthoorthomosaic) is exported from Metashape and uploaded into ArcGIS for annotation. (Citation: Suka R, Asbury M, Couch C, Gray A, Winston M, Oliver T. 2019. Processing Photomosaic Imagery of Coral Reefs Using Structure-from-Motion Standard Operating Procedures. U.S. Dept. of Commerce, NOAA Technical Memorandum NOAA-TM-NMFS-PIFSC-93, 54 p. doi:10.25923/h2q8-jv47)
- In ArcGIS, each SfM site was set up for annotation by manually digitizing the transect and segments as a shapefile using the same sampling design as the PIFSC ESD-led in situ coral demography surveys, and setting up the attribute table in a geodatabase to mirror the in situ visual survey database (Swanson et al. 2018; Winston et al. 2019). To record and extract data from the orthomosaic, each coral colony was annotated following the in situ visual survey methods. Each colony was measured by digitizing a line across the maximum diameter of the colony. Coral ID (to lowest taxonomic level), estimated percent old mortality, and bleaching extent and severity were recorded. During annotation, the original JPEG imagery was viewed alongside the orthomosaic with the Viscore Image View feature to see fine scale colony details, observe colonies from multiple angles and locate colonies not visible in the orthomosaic (e.g. under ledges). (Citation: Suka R, Asbury M, Couch C, Gray A, Winston M, Oliver T. 2019. Processing Photomosaic Imagery of Coral Reefs Using Structure-from-Motion Standard Operating Procedures. U.S. Dept. of Commerce, NOAA Technical Memorandum NOAA-TM-NMFS-PIFSC-93, 54 p. doi:10.25923/h2q8-jv47)
- SfM derived annotations created in ArcGIS are quality controlled using a multi-stage process. Data are first exported from ArcGIS and quality controlled in R with specific queries to identify and correct data entry errors (e.g. misspelled species names, missing segments, data in incorrect columns, partial mortality >100%). Then data are summarized to the segment-level by annotator to identify potential issues (e.g. anomalously high or low mean values for specific metrics relative to other annotators). If issues are identified for a given annotator, that annotator reviews and corrects each site if they did find errors. (Citation: Couch CS, Oliver, TA, Suka R, Lamirand M, Asbury M, Amir C, Vargas-Angel B, Winston M, Huntington B, Lichowski F, Halperin A, Gray A, Garriques J, & Samson J. 2021. Comparing coral colony surveys from in-water observations and structure-from-motion imagery shows low methodological bias. Frontiers in Marine Science, 8. https://doi.org/10.3389/fmars.2021.647943 )
- Raw data from in situ and SfM surveys include individual colony records with the corresponding physical data which reflect the description of the site. An individual colony record for adult corals includes colony species/genus identification, morphology, maximum diameter (cm), percent old dead, percent recent dead, cause of recent dead general category, cause of recent dead specific category, condition, extent and severity. Juvenile coral colony records include colony species/genus identification, morphology, maximum diameter (cm) and perpendicular diameter (cm). For species level identification of adult and juvenile corals, check the species list to ensure species identification across all divers conducting the surveys. The physical data for all records includes the following: region, island, site, date (day, month, year), latitude (dd), longitude (dd), transect, segment, segment length, segment width, minimum depth (ft), maximum depth (ft), and survey type. (Citation: Winston M, Couch C, Ferguson M, Huntington B, Swanson D, Vargas-Angel B. 2019. Ecosystem Sciences Division Standard Operating Procedures: Data Collection for Rapid Ecological Assessment Benthic Surveys, 2018 Update. NOAA Tech. Memo. NOAA-TM-NMFS-PIFSC-92, 66 p. doi:10.25923/w1k2-0y84)
(describe or provide URL of description):
Quality control of the data occurred at a few stages from data entry to data ingestion into the Oracle database. Observations, including species identification, are periodically checked during expeditions for consistency between and among divers. Data entry is usually conducted on the same day as the surveys using a data entry interface with several data controls employed, and are quality controlled by individual divers checking entry errors at a separate time. Following a mission, the data is then run through rigorous quality control checks by the data management team before the data are migrated to the Oracle database. The data is quality controlled against the physical data sheets following data entry. There are also several queries in the MS Access / Oracle database to flag errors based on pre-defined criteria. Given the size of the data set, there remains some possibility of typographical or other errors.
Prior to generating the 3D dense point clouds and 2D orthomosaics , the JPEG imagery was evaluated for image quality and images deemed unsatisfactory (e.g. overexposed, images of blue water or images of divers, or images not taken perpendicular to the reef) were removed from the image set. During annotation in ArcGIS, the original JPEG imagery was viewed alongside the orthomosaic using the Viscore Image View feature to see fine scale colony details, observe colonies from multiple angles and locate colonies not visible in the orthomosaic. Annotations created in ArcGIS were quality controlled using a multi-stage process. Data were first quality controlled in R with specific queries to identify and correct data entry errors (e.g. misspelled species names, missing segments, data in incorrect columns, partial mortality >100%). Then data were summarized to the segment-level by annotator to identify potential issues (e.g. anomalously high or low mean values for specific metrics relative to other annotators). If issues were identified for a given annotator, that annotator reviewed and corrected each site if they did find errors.
6. Data Documentation
The EDMC Data Documentation Procedural Directive requires that NOAA data be well documented, specifies the use of ISO 19115 and related standards for documentation of new data, and provides links to resources and tools for metadata creation and validation.
Missing/invalid information:
- 1.7. Data collection method(s)
(describe or provide URL of description):
7. Data Access
NAO 212-15 states that access to environmental data may only be restricted when distribution is explicitly limited by law, regulation, policy (such as those applicable to personally identifiable information or protected critical infrastructure information or proprietary trade information) or by security requirements. The EDMC Data Access Procedural Directive contains specific guidance, recommends the use of open-standard, interoperable, non-proprietary web services, provides information about resources and tools to enable data access, and includes a Waiver to be submitted to justify any approach other than full, unrestricted public access.
None
Notes: The name of the Organization of the most recent Support Role of type "Distributor" is used. The support role must be in effect. This information is not required if an approved access waiver exists for this data.
Notes: This field is required if a Distributor has not been specified.
http://accession.nodc.noaa.gov/0298217
http://accession.nodc.noaa.gov/0298217
Notes: All URLs listed in the Distribution Info section will be included. This field is required if applicable.
Data can be accessed online via the NOAA National Centers for Environmental Information (NCEI) Ocean Archive.
Notes: This field is required if applicable.
8. Data Preservation and Protection
The NOAA Procedure for Scientific Records Appraisal and Archive Approval describes how to identify, appraise and decide what scientific records are to be preserved in a NOAA archive.
(Specify NCEI-MD, NCEI-CO, NCEI-NC, NCEI-MS, World Data Center (WDC) facility, Other, To Be Determined, Unable to Archive, or No Archiving Intended)
Notes: This field is required if archive location is World Data Center or Other.
Notes: This field is required if archive location is To Be Determined, Unable to Archive, or No Archiving Intended.
Notes: Physical Location Organization, City and State are required, or a Location Description is required.
Discuss data back-up, disaster recovery/contingency planning, and off-site data storage relevant to the data collection
The data is captured in several locations: physical data sheets, MS Access cruise database, and PIFSC Oracle database. The physical data sheets are housed at PIFSC. The MS Access cruise database is regularly backed up by the cruise data manager while at sea. The PIFSC Oracle database is regularly backed up by PIFSC ITS.
9. Additional Line Office or Staff Office Questions
Line and Staff Offices may extend this template by inserting additional questions in this section.