Leading Indicators Strategy Rubric

Updated for clarity ahead of Milestone 9: November 30, 2015

The Leading Indicators Strategy Rubric is intended to provide clarity and transparency on how OMB Staff review Leading Indicators in six categories: Enterprise Data Inventory, Public Data Listing, Public Engagement, Privacy & Security, Human Capital, and Use & Impact.

This Rubric may be updated as new Leading Indicators are informed by Project Open Data, the Open Data Cross-Agency Priority (CAP) Goal, and the Integrated Data Collection Open Data Progress, Use, and Impact questions.

Enterprise Data Inventory

Green

Agency provided a public EDI for this quarter that was of a high quality and indicated substantial progress towards completeness meeting all the requirements listed below:

  • Provides an updated public EDI JSON file for this quarter as a dataset in agency's data.json file
  • Shows an increase in progress to document non-public, restricted public, and public APIs
  • Shows an increase in progress to document all Public Domain/Licensing information
  • Shows an increase in the progress to expand and organize data assets through the use of Collections
  • Documents all non-public and restricted datasets, redactions, restrictive licenses and provides an explanation for non-disclosure in the `rights` field

Yellow

Agency provided a public EDI for this quarter but the agency EDI had one or more minor deficiencies listed below:

  • Provides a public EDI file, but fails to: update it for this quarter, provide it in the correct format, provide it as a dataset, redact it in accordance with Project Open Data reaction guidance, or include explanations for redacted material
  • Fails to document any outstanding APIs
  • Fails to document any outstanding Licensing information
  • Fails to organize data assets in to Collections
  • Fails to document all non-public and restricted datasets, redactions, restrictive licenses and provides an explanation for non-disclosure (in the rights field)

Red

Agency did not provide an updated public EDI for this quarter or the EDI had any of the serious deficiencies listed below:

  • EDI is less than PDL
  • Fails to document non-public, and restricted public datasets
  • Fails to provide a public EDI as a dataset in your data.json file.
  • Fails to make progress adding outstanding non-public, restricted public datasets for more than 1 quarter

Best Practice

Shows substantial growth (>10%) in cataloging non-public, restricted public, and public datasets.

Public Data Listing

Green

Agency provided a PDL for this quarter that was of a high quality and indicated substantial progress towards completeness meeting all the requirements listed below:

  • Has 100% valid data.json
  • Successfully provides a human-readable list of all datasets at [agency].gov/data

Yellow

Agency provided a PDL for this quarter but PDL had one or more minor deficiencies listed below:

  • Fails to have 100% valid data.json
  • Provides some datasets in human-readable form on /data, but not all
  • Insufficient link quality (20% or more broken and error links this quarter)

Red

Agency didn’t provide a PDL for this quarter or provided a PDL that was not updated or any serious deficiencies:

  • PDL does not validate against current schema/ Does not have a valid data.json file successfully harvesting to Data.gov
  • Fails to create a human-readable list of all datasets
  • Insufficient link quality (20% or more broken or error links for more than 1 quarter)

Best Practice

Shows substantial growth (>10%) in cataloging public datasets.

Public Engagement

Green

Agency provides a transparent two-way feedback mechanism and shows evidence of this mechanism on their digital strategy page.

Yellow

Agency provides a transparent two-way feedback mechanism but shows no evidence of active engagement.

Red

Fails to provide a transparent two-way feedback mechanism; e.g. only provides an email or contact form to engage public.

Best Practice

Agency has demonstrated evidence that two-way feedback mechanism has transformed existing process within agency.

Privacy and Security

Green

Provides an up-to-date Data Publication Process on the /digitalstrategy page.

Yellow

Fails to provide an up-to-date Data Publication Process on the /digitalstrategy page.

Red

Fails to provide an up-to-date Data Publication Process on the /digitalstrategy page for more than 1 quarter.

Best Practice

Agency demonstrates a robust privacy and security plan to support M-13-13 implementation.

Human Capital

Green

Agency provided a person and current contact information for all six required data points of contact.

Yellow

Agency provided a person and contact information for some but not all of the required data points of contact.

Red

Agency did not provided a person and contact information for any of the required data points of contact.

Best Practice

Agency demonstrates strong human capital in the area of data management (e.g. has a Chief Data Officer, data working group).

Use & Impact

Green

Agency provided answers for all IDC questions reflecting progress this quarter including a list of 5 dataset improvements.

Yellow

Agency provides a poor quality responses to required IDC questions.

Red

Agency did not provide a response to any of the required IDC questions or failed to provide responses that reflect progress this quarter.

Best Practice

Agency has demonstrated that one or more dataset improvements has had a significant impact to agency or to the public.

You can help edit this page on GitHub.