Data Management Plan
GUID: gov.noaa.nmfs.inport:60108 | Published / External
Data Management Plan
DMP Template v2.0.1 (2015-01-01)
Please provide the following information, and submit to the NOAA DM Plan Repository.Reference to Master DM Plan (if applicable)
As stated in Section IV, Requirement 1.3, DM Plans may be hierarchical. If this DM Plan inherits provisions from a higher-level DM Plan already submitted to the Repository, then this more-specific Plan only needs to provide information that differs from what was provided in the Master DM Plan.
1. General Description of Data to be Managed
The Federal Emergency Management Agency (FEMA) required high accuracy classified LiDAR data in combination with raster digital elevation models and hydrographic breaklines. For this effort, Continental Mapping Consultants (Continental) collected and processed high accuracy classified LiDAR data in .LAS format as well as a combination of raster digital elevation models.
Branch County:
Acquired and processed 535 square miles of USGS defined Quality Level 2 LiDAR, collection of hydro-breaklines, and creation of Hydro-Flattened Digital Elevation Models.
Calhoun County:
Acquired and processed 745 square miles of USGS defined Quality Level 2 LiDAR, collection of hydro-breaklines, and creation of Hydro-Flattened Digital Elevation Models. The Calhoun County project area contains Palustrine wetlands and poorly drained hydric soils that retain water after snow melt and/or spring rains. Most soil types, according to the Soil Survey Geographic Database (SSURGO) for Calhoun County, have available water storage between 15-37 cm. Low confidence areas have been designated where it was apparent that the ground conditions were saturated and produced minimal returns.
The NOAA Office for Coastal Management (OCM) downloaded this data set from these USGS sites:
ftp://rockyftp.cr.usgs.gov/vdelivery/Datasets/Staged/Elevation/LPC/Projects/USGS_LPC_MI_BranchCo_2017_LAS_2019
ftp://rockyftp.cr.usgs.gov/vdelivery/Datasets/Staged/Elevation/LPC/Projects/USGS_LPC_MI_CalhounCo_2017_LAS_2019
These files were processed to the Data Access Viewer (DAV) and https. The total number of files downloaded and processed was 1533.
The breaklines were also downloaded and are available for download at the link provided in the URL section of this metadata record. Please note that this product has not been reviewed by the NOAA Office for Coastal Management (OCM) and any conclusions drawn from the analysis of this information are not the responsibility of NOAA or OCM.
Notes: Only a maximum of 4000 characters will be included.
Notes: Data collection is considered ongoing if a time frame of type "Continuous" exists.
Notes: All time frames from all extent groups are included.
Notes: All geographic areas from all extent groups are included.
(e.g., digital numeric data, imagery, photographs, video, audio, database, tabular data, etc.)
(e.g., satellite, airplane, unmanned aerial system, radar, weather station, moored buoy, research vessel, autonomous underwater vehicle, animal tagging, manual surveys, enforcement activities, numerical model, etc.)
2. Point of Contact for this Data Management Plan (author or maintainer)
Notes: The name of the Person of the most recent Support Role of type "Metadata Contact" is used. The support role must be in effect.
Notes: The name of the Organization of the most recent Support Role of type "Metadata Contact" is used. This field is required if applicable.
3. Responsible Party for Data Management
Program Managers, or their designee, shall be responsible for assuring the proper management of the data produced by their Program. Please indicate the responsible party below.
Notes: The name of the Person of the most recent Support Role of type "Data Steward" is used. The support role must be in effect.
4. Resources
Programs must identify resources within their own budget for managing the data they produce.
5. Data Lineage and Quality
NOAA has issued Information Quality Guidelines for ensuring and maximizing the quality, objectivity, utility, and integrity of information which it disseminates.
(describe or provide URL of description):
Process Steps:
- 2016-01-01 00:00:00 - The base stations determine where LiDAR can be collected with the highest confidence of accuracy as determined by reading the same satellites in the aircraft and on the ground. Base stations typilcally are set at airports and provide coverage within 20 to 25 miles of the base if possible in order to cover in its entirety. Due to the timing constraints of weather and leaf off conditions requirement, and the high quality of the CORS network, the MIBC station located in Battle Creek and the MICW station located in Coldwater were used for this project. Both stations collect a 1 second sampling rate and are maintained by the Michigan Department of Transportation. The purpose of boresighting is to determine the offset values for the IMU used in the LiDAR sensor. To determine the boresight offset values, the LiDAR sensor has to be flown in a certain configuration over a well-controlled site. The boresighting is done both prior to the flight of the project area and after. This insures that the quality of the LiDAR was maintained throughout the process.
- 2017-01-01 00:00:00 - Branch County: The aerial survey teams were deployed at the first opportunity based on availability of acceptable weather conditions. Due to snow cover and lake effect precipitation, the flight was delayed until March. ALTM-Nav Planner software was utilized to conduct the final flight planning. The sensor used was an Optech Gemini, which is owned and operated by GRW Aerial Surveys, Inc. Due to weather conditions at the collection site or acquisition logistics, 14 total lifts were completed for collection. There were 90 project flight lines, and 6 cross flights collected of which 2 (lines 68 and 77) were used for calibrating. There was also 1 line (line 76) not used in production due to excessive cloud coverage that was successfully collected for coverage in an adjacent lift. The LiDAR acquisition started on March 22, 2017 and ended on April 24, 2017. Altitude: 5500 feet Aircraft Ground Speed: 125 knots Pulse Rate: 70.0 kHz Scan Rate: 43.6 Hz Full Field of View: 16 degrees Multi-Pulse: Yes Full Swath Width: 2630 feet Swath Overlap: 50% Average Point Density: 2.0 pts/m^2 Calhoun County: The aerial survey teams were deployed at the first opportunity based on availability of acceptable weather conditions. Due to snow cover and lake effect precipitation, the flight was delayed until April. Leica’s AeroPlan software was utilized to conduct the final flight planning. The sensor used was a Leica ALS70, which is owned and operated by Kucera International. There were 37 project flight lines, and 1 cross flight collected. The LiDAR acquisition started on April 18, 2017 in which one lift was achieved. Due to weather concerns, two aircraft were initially deployed. Due to sensor complications, a re-flight of the northern portion of the county was performed 5 days later on April 23, 2017 in which two lifts were achieved. Altitude: 7500 meters Aircraft Ground Speed: 150 knots Pulse Rate: 263.2 kHz Scan Rate: 48.1 Hz Full Field of View: 38 degrees Multi-Pulse: Yes Full Swath Width: 4650 meters Swath Overlap: 20% Average Point Density: 2.3 pts/m^2 Solar activity reached low levels on 17-18 Apr due to isolated C-class flare activity from Region 2651 (N12, L= 070, class/area=Cso/150 on 23 Apr), but solar activity was at very low levels through the remainder of the period (19-23 Apr).
- 2017-01-01 00:00:00 - Branch County: Continental team utilized PosPac v7.1 software to process sbet and precision files. Optech Lidar Mapping Suite v2.4.1.14540 was used for LAS creation. Calhoun County: Continental team utilized Leica’s CloudPro to initially process data and convert to LAS format. TerraMatch was used to refine calibration of LiDAR dataset. Trajectory files and point cloud swaths are imported into GeoCue to perform project setup. Project set up phase sets project parameters, tiling scheme, and is platform for initial macro runs. After import, checkpoints are run against point cloud to verify accuracy of data prior to classification. After verifying the accuracy, processing continues. Multiple macros are run through TerraScan to flag overlap and classify ground. Due to differing terrain, this step may take multiple iterations. Once analyst has verified results with ground macro, ground classification QC begins. During QC phase, analysts reclassify point cloud in areas where macro was not able to or were mis-classified. Multiple macros are run on dataset after ground classification complete including water macros. Water macros utilize the hydro breaklines that were manually digitized. Digitized breaklines were classified as ponds and rivers. After the hydro features were digitized, the ponds were flattened. This process calculated lowest elevation of the feature, and used that elevation to populate remaining vertices. This process verifies that all ponds are flat. River polygons that were digitized were ran against a monotonicity tool. This tool utilized elevation of a centerline that had correct elevation and pushed that elevation to the river polygon. This process not only maintains the monotonicity of the river, but also ensures that the river is flat from bank to bank. Then rigorous quality steps are performed each classification level. Bare earth lidar points that were within 3 feet of the water were classified to class 10. After analysts completed QC process in TerraScan, raster files were produced into 32-bit floating GeoTiffs and Erdas Imagine IMG files using LP360 (Branch County) or Global Mapper (Calhoun County). These files were created using only ground class. DEMS are ran against proprietary tools to identify any remaining potential blunders.
- 2017-01-01 00:00:00 - The calibrated and controlled lidar files were processed using automatic point classification routines in proprietary software. These routines operate against the entire collection (all swaths, all lifts), eliminating character differences between files. The trajectory files and point cloud swaths were imported into GeoCue to perform project setup. This project set up phase set the project parameters, tiling scheme, and was the platform for initial macro runs. After import, checkpoints were run against the point cloud to verify the accuracy of the data prior to classification. After verifying the NVA accuracy, the processing continued. Multiple macros were run through TerraScan to classify low points, high points, ground, below surface etc. Geometrically unusable points below ground and above ground were classified to Withheld Low Noise (W7) and Withheld High Noise (W18). Points below ground surface that were identified as low points were classified to class 7. A bare earth ground surface was derived from the unclassed points and put to class 2 using a suitable macro function for the project's terrain type. Points above the treeline that were not identified as a feature were classified to class 18. Ground points inside of water features were classifed to class 9 to represent water and ground points outside of hydro features but within 3 feet of hydro breaklines were classified to class 10. All remaining points were classified to class 1. Final lidar LAS delivery classes for the fully classed LAS tiles consistent with ASPRS LAS classes to be compliant with USGS LiDAR Guidelines and Base Specifications v13 consist of: Class 1 - Unclassified; Class 2 - Ground; Class 7 – Low Noise; Class 9 - Water; Class 10 - Ignored Ground (including 3-foot buffer around water breaklines); Class 17 - Bridge; and Class 18 - High Noise. Once the analyst was comfortable with the ground macro results, the ground classification QC began using TerraScan. During the QC phase, analysts reclassified the point cloud in areas where the macro was not able to, or created misclassifications. The same rigorous quality steps were performed on each classification. Data were then distributed as virtual tiles to experienced lidar analysts for localized automatic classification, manual editing, and peer-based QC checks. Supervisory QC monitoring of work in progress and completed editing ensured consistency of classification character and adherence to project requirements across the entire project. Upon completion of point classification, an automated process was executed to turn any points to class 1 in the Fully Classified LAS files and to delete all points that fell outside of the provided project buffered boundary. Breaklines were digitized at water elevation for any bodies of water over the entire project area including streams greater than 100ft in nominal width, water bodies greater than 2.0 acres in area, and islands greater than 1.0 acre. A macro was then run to classify points that lay at nominal water elevation to class 9 from 2 that fell within bodies of water. Concurrently a 3-foot buffer zone around water polygons was derived from the ground class and put to class 10 as Ignored Ground. Water bodies and streams with flow were hydro-enforced using the Hydro shapefile to identify Ponds, Islands, and Double Line Drains to demonstrate the removal of unnatural surface artifacts in both ponds and streams and to show downward flow for streams. A DEM data set was generated for the Bare Earth LAS set as 32-bit Erdas Imagine .IMG files at a resolution of 2.0 ft (feet) supported by the Hydro Breaklines.
- 2017-01-01 00:00:00 - Continental utilized various software packages and techniques to verify the accuracy of the data. Utilizing QCoherent’s LP360, Continental ran a survey to las check, followed by seamline analysis (swath to swath analysis) to verify the absolute and relative accuracy of the dataset. The survey to las check calculates the deviation between the survey point elevation and the point cloud elevation and exports an RMSE report. This check was ran by Continental, utilizing the provided control. This check was also ran by Compass Data Inc. utilizing the NVA points. The second check, calculates the deviation between the seamlines of the point cloud swaths. This check is performed in QCoherent’s GeoCue after classifying the initial ground. The output of the seamline analysis is represented visually on an intensity image. These images were delivered with the project deliverables. The third and final check, the Vegetated Vertical Accuracy (VVA) testing occurred after the ground classification has been completed. The VVA testing was performed by Compass Data, Inc. The field survey and aerial survey teams were deployed at the first opportunity based on availability of acceptable weather conditions and base personnel (for coordination). The area of interest contains subareas of dense vegetation which present fewer bare ground returns, higher variability, and potentially less accuracy than typical vegetated areas. Per the table below the survey accuracy results meet industry standards for both NVA and VVA based on the ground survey control points collected in December. Once all of the deliverables have been produced and verified, the data was moved to the Quality office for final review. The Quality Office verifies that the correct procedures were followed, tests the data, and verifies that all of the deliverables in the SOW are finished.
- 2020-06-22 00:00:00 - The NOAA Office for Coastal Management (OCM) downloaded the laz files from these USGS sites: ftp://rockyftp.cr.usgs.gov/vdelivery/Datasets/Staged/Elevation/LPC/Projects/USGS_LPC_MI_BranchCo_2017_LAS_2019 Number of laz files: 648 ftp://rockyftp.cr.usgs.gov/vdelivery/Datasets/Staged/Elevation/LPC/Projects/USGS_LPC_MI_CalhounCo_2017_LAS_2019 Number of laz files: 885 The total number of files downloaded and processed was 1533. The data were in Michigan State Plane South (NAD83 2011), international feet coordinates and NAVD88 (Geoid12B) elevations in feet. From the provided report, the data were classified as: 1 - Unclassified, 2 - Ground, 7 - Low Noise, 9 - Water, 10 - Ignored Ground, 17 - Bridge Decks, 18 - High Noise. OCM processed all classifications of points to the Digital Coast Data Access Viewer (DAV). Classes available in the DAV are: 1, 2, 7, 9, 10, 17, 18. OCM performed the following processing on the data for Digital Coast storage and provisioning purposes: 1. An internal OCM script was run to check the number of points by classification and by flight ID and the gps and intensity ranges. 2. Internal OCM scripts were run on the laz files to convert from orthometric (NAVD88) elevations to ellipsoid elevations using the Geoid12B model, to convert from Michigan State Plane South (NAD83 2011), international feet coordinates to geographic coordinates, to convert from elevations in feet to meters, to assign the geokeys, to sort the data by gps time and zip the data to database and to http.
(describe or provide URL of description):
6. Data Documentation
The EDMC Data Documentation Procedural Directive requires that NOAA data be well documented, specifies the use of ISO 19115 and related standards for documentation of new data, and provides links to resources and tools for metadata creation and validation.
Missing/invalid information:
- 1.7. Data collection method(s)
- 3.1. Responsible Party for Data Management
- 5.2. Quality control procedures employed
- 7.1.1. If data are not available or has limitations, has a Waiver been filed?
- 7.4. Approximate delay between data collection and dissemination
- 8.3. Approximate delay between data collection and submission to an archive facility
(describe or provide URL of description):
7. Data Access
NAO 212-15 states that access to environmental data may only be restricted when distribution is explicitly limited by law, regulation, policy (such as those applicable to personally identifiable information or protected critical infrastructure information or proprietary trade information) or by security requirements. The EDMC Data Access Procedural Directive contains specific guidance, recommends the use of open-standard, interoperable, non-proprietary web services, provides information about resources and tools to enable data access, and includes a Waiver to be submitted to justify any approach other than full, unrestricted public access.
None
Notes: The name of the Organization of the most recent Support Role of type "Distributor" is used. The support role must be in effect. This information is not required if an approved access waiver exists for this data.
Notes: This field is required if a Distributor has not been specified.
https://noaa-nos-coastal-lidar-pds.s3.amazonaws.com/laz/geoid18/9120/index.html
Notes: All URLs listed in the Distribution Info section will be included. This field is required if applicable.
Data is available online for bulk and custom downloads.
Notes: This field is required if applicable.
8. Data Preservation and Protection
The NOAA Procedure for Scientific Records Appraisal and Archive Approval describes how to identify, appraise and decide what scientific records are to be preserved in a NOAA archive.
(Specify NCEI-MD, NCEI-CO, NCEI-NC, NCEI-MS, World Data Center (WDC) facility, Other, To Be Determined, Unable to Archive, or No Archiving Intended)
Notes: This field is required if archive location is World Data Center or Other.
Notes: This field is required if archive location is To Be Determined, Unable to Archive, or No Archiving Intended.
Notes: Physical Location Organization, City and State are required, or a Location Description is required.
Discuss data back-up, disaster recovery/contingency planning, and off-site data storage relevant to the data collection
Data is backed up to tape and to cloud storage.
9. Additional Line Office or Staff Office Questions
Line and Staff Offices may extend this template by inserting additional questions in this section.