Environmental Protection Agency
Enterprise Data Inventory - Volume and composition over time
M-13-13 Milestone 11 - May 31st 2016
OMB Review Complete: OMB has completed the agency review for this milestone. Agencies should contact their OMB desk officer if anything looks incorrect.
Leading Indicators
These indicators are reviewed by the Office of Management and Budget
| Review Status | complete |
|---|---|
| Reviewer | Bryant Renaud |
| Last Updated | August 17, 2016, 3:40 pm EDT by Bryant Renaud |
Assessment Summary
Other: Link quality reporting is unavailable is for EPA due to OMB-side technical issue.
Inventory Composition
Public Dataset Status
| Status | Indicator | Automated Metrics | ||
|---|---|---|---|---|
| Overall Progress this Milestone | ||||
| Inventory Updated this Quarter | ||||
| 2271 | Number of Datasets | |||
| 1241 | Number of APIs | |||
| 1 | Bureaus represented | |||
| 100.0% | Percentage of bureaus represented | |||
| 8 | Programs represented | |||
| 6.6% | Percentage of programs represented | |||
| 1963 | Number of public datasets | |||
| 10 | Number of restricted public datasets | |||
| 37 | Number of non-public datasets | |||
| -28.7% | Percentage growth in records since last quarter | |||
| To a very great extent (>75%) | To what extent is your agency’s Enterprise Data Inventory (EDI) complete? | |||
| See below | What steps have you taken to ensure your Enterprise Data Inventory is complete | |||
| In March 2015, the EPA CIO signed the EPA Enterprise Information Management Policy (EIMP), the EIMP Cataloguing Data Resources Procedure, and the EIMP Minimum Metadata Standards. The EIMP requires all EPA Organization officials, employees, and individuals or non EPA organizations, if applicable to ensure information is: The EIMP Cataloguing Data Resources Procedure states: The Agency's internal metadata catalog, the Environmental Dataset Gateway (EDG) was established in 2006. The EDG Team has worked with data owners since its inception to catalog Agency datasets. In response to Project Open Data requirements the EPA CIO issued an Agencywide data call asking all EPA organizations to register their datasets in EDG. The EDG Team worked with the Agency's Information Management Officers (IMOs) and the EDG's Stewardship Network and other key data owners to ensure that as many Agency data sets as possible were identified for registration. In addition, EPA's registry for IT systems, READ, was reviewed to ensure that all possible data owners were contacted. Since that time, the EDG team has established an ongoing relationship with the IMOs and has increased its network of stakeholders to ensure that any datasets not identified during the 2013 data call are registered in EDG. Quarterly meetings and training sessions held with these groups to educate them on Open Data requirements and metadata best practices as well as to encourage them to continue cataloguing their datasets. Targeted outreach, based on new entries in READ are conducted to ensure that all datasets are listed in the EDI. This includes working with Offices that have Confidential Business Information to ensure that we have a full registration of all data not shared with the public. In addition, EPA's Security Office is planning an audit of all Agency data systems and is coordinating with the EDG team to ensure that any uncatalogued datasets discovered in this process are registered in EDG and become part of EPA's EDI. And finally, EPA is also developing an | ||||
| Agency provides a public Enterprise Data Inventory on Data.gov | ||||
| Agency provided updated Enterprise Data Inventory to OMB | ||||
| 100% | License specified | Crawl details | ||
| Number of datasets with redactions | ||||
| Percent of datasets with redactions | ||||
| Status | Indicator | Automated Metrics |
|---|---|---|
| Overall Progress this Milestone | ||
| 2010 | Number of Datasets | Crawl details |
| 9 | Number of Collections | Crawl details |
| 1796 | Number of datasets not contained in a collection | Crawl details |
| Number of Public Datasets with File Downloads | Crawl details | |
| 2482 | Number of APIs | Crawl details |
| Number of public APIs | Crawl details | |
| Number of restricted public APIs | Crawl details | |
| Number of non-public APIs | Crawl details | |
| Total number of access and download links | Crawl details | |
| Quality Check: Links are sufficiently working | Crawl details | |
| Quality Check: Accessible links | Crawl details | |
| Quality Check: Redirected links | Crawl details | |
| Quality Check: Error links | Crawl details | |
| Quality Check: Broken links | Crawl details | |
| Quality Check: Percentage of download links in correct format as specified in metadata | Crawl details | |
| Quality Check: Percentage of download links in HTML | Crawl details | |
| Quality Check: Percentage of download links in PDF | Crawl details | |
| Percentage growth in records since last quarter | ||
| 100% | Valid Metadata | Crawl details |
| /data exists | Crawl details | |
| Provides datasets in human-readable form on /data | ||
| /data.json | Crawl details | |
| Harvested by data.gov | ||
| 1963 | Number of public datasets | Crawl details |
| 10 | Number of restricted public datasets | Crawl details |
| 37 | Number of non-public datasets | Crawl details |
| Percent growth of public datasets | ||
| Percent growth of restricted public datasets | ||
| Percent growth of non-public datasets | ||
| Percent datasets licensed as U.S. Public Domain | ||
| Percent datasets licensed as Creative Commons Zero | ||
| Percent datasets with other licenses | ||
| Percent datasets with no license |
| Status | Indicator | Automated Metrics | ||
|---|---|---|---|---|
| Overall Progress this Milestone | ||||
| Description of feedback mechanism delivered | Crawl details | |||
| Data release is prioritized through public engagement | ||||
| Provided narrative evidence of data improvements based on public feedback this quarter | ||||
| Feedback loop is closed, 2 way communication | ||||
| See below | Link to or description of Feedback Mechanism | |||
| https://developer.epa.gov/forums/forum/dataset-qa/ | ||||
| Provides valid contact point information for all datasets | ||||
| Status | Indicator | Automated Metrics | ||
|---|---|---|---|---|
| Overall Progress this Milestone | ||||
| Data Publication Process Delivered | Crawl details | |||
| Information that should not to be made public is documented with agency's OGC | ||||
| See below | Describe the agency's data publication process | |||
| EPA has a number of policies and procedures concerning the publication of Agency data. The Enterprise Information Management Policy requires all EPA Organization officials, employees, and individuals or non-EPA organizations, if applicable, to ensure information is cataloged and or labeled with metadata. This includes geographic references, as appropriate, in EPA and Federal-wide registries, repositories or other information systems. The EPA GeoPlatform Publishing Workflow Standard Operating Procedure and the EPA Environmental Dataset Gateway (EDG) Governance Structure and Standard Operating Procedure outline the details of EPA data publishing. EPA provides a range of tools and registry content (e.g. Reusable Component Services, Environmental Dataset Gateway, and Data Element Registry) through its System of Registries located at: www.epa.gov/sor. EPA is continuing efforts to document APIs through the development of an Agency-wide API Strategy. The proposed strategy is based on 18F’s API standards. The proposal encourages the use of api.data.gov’s API management platform. In addition, APIs produced by the EPA should be described using one of the common API definition formats (such as Swagger, API Blueprint and RAML). The strategy is being finalized and an Agency-wide communication plan is being developed. This communication plan will include Standard Operating Procedures (SOPs) that require API developers to register dataset APIs in the EPA’s Environmental Dataset Gateway, which will allow these APIs to become part of the EPA's EDI/PDL. All other APIs will be registered in EPA's Reusable Components Services (RCS). | ||||
Best Practice: Environmental Protection Agency has been highlighted for demonstrating a best practice on the Human Capital indicator
Automated Metrics
These metrics are generated by an automated analysis that runs every 24 hours until the end of the quarter at which point they become a historical snapshot