IATI’s Datastore and Validator tools launched
IATI is delighted to announce the launch today of its new Datastore and Validator, fulfilling a commitment to members to provide robust tools to improve the quality and use of development and humanitarian data published to IATI.
- Why use the new IATI Datastore?
- Why use the new IATI Validator?
- Get support: online guidance and webinars
- What’s changed since the testing phase?
- Deprecation of old Datastore and Public Validator - 22 March 2021
- Upcoming work
Developing and building these new tools has been the result of a close collaboration between the IATI Secretariat and suppliers Zimmerman and Data4Development, and thanks are also due to our dedicated community for testing the new tools and helping us to improve them.
Thanks are due to our dedicated community for testing the new tools and helping us to improve them.
The following information is provided to enable our community of publishers and users to easily transition to using the new tools and get help through webinars and detailed guidance. It also outlines planning for subsequent phases of development to continually improve our suite of IATI tools.
Why use the new IATI Datastore?
The new IATI Datastore allows for much more detailed queries and searches of IATI data than previously possible. Use the Query Builder to search data across 39 different fields (elements and attributes) of the IATI Standard and choose to download data in three different formats (CSV, XML and JSON). For the more technical user, the Datastore API enables searches of IATI data across all elements of the IATI Standard, enabling more complex data analysis.
Use the Query Builder to search for development and humanitarian activities responding to COVID-19
The Query Builder and API also provide users with data that complies with the most up-to-date version of the IATI Standard (version 2 and beyond). The IATI Datastore complements existing online tools for accessing IATI data such as d-portal.
Why use the new IATI Validator?
Organisations are strongly encouraged to use the new IATI Validator to check if their data is accessible and of good quality, increasing its utility for partner countries, civil society and other data users. Organisations have the option of checking for errors in their data before publishing their files, or after data has been published (using the Public Data Viewer).
Use the new IATI Validator to check the quality of IATI data.
Most organisations will find at least some errors in their data and we know this may take some time for them to correct. IATI’s Secretariat is providing support in the following ways to help organisations make these changes to improve the quality of their data.
Get support: online guidance and webinars
The IATI Secretariat has provided instructions and useful information on both tools through online guidance: IATI Datastore Query Builder User Guide, IATI Datastore API documentation and the IATI Validator Q&A. The Secretariat will also be proactively contacting IATI publishers in the coming months to offer direct support to improve their datafiles using the IATI Validator.
You can learn how to make the most of both tools by joining upcoming webinars held by IATI’s Technical Team:
- Webinar: Using IATI’s new Validator, Wednesday 7 October Register
- Webinar: Using IATI’s new Datastore, Wednesday 28 October Register
As always, the IATI Technical Team welcomes your questions and feedback at: [email protected].
What’s changed since the testing phase?
The new Datastore and Validator were pre-released for extensive testing. Both have benefitted from a huge number of improvements in response to feedback from IATI’s community of data users and publishers. Here are just a few examples:
Datastore
- Additional filters have been added to the Datastore Query Builder
- Users can now output CSV columns in the order of the IATI XML Schema
- The Query Builder has a new multi-sector and multi-country expansion option
- The Datastore is now integrated with the Validator. This means that the Validator will check that data is fed into the Datastore and ensure only data that follows the IATI schema is available to users. The IATI schema provides the exact order and structure for publishing the XML. Data that does not follow the IATI XML schema will not be outputted through the Datastore. For further information see IATI Validator Q&A (page 5-6).
Validator
- All bugs raised by users following the testing phase have been fixed
- There is a new feature to allow the bulk-uploading of files
- Users can now receive validation reports via email.
The IATI Technical Team has worked with suppliers Zimmerman (who continue to host the new IATI Datastore) and Data4Development (developers of the new IATI Validator, now hosted by the IATI Secretariat) to implement changes to both products.
Important: Deprecation of old Datastore and Public Validator - 22 March 2021
As previously announced, the new IATI Datastore and Validator replace the old Datastore CSV Query Builder, API and Public Validator. These older versions will be deprecated on Monday 22 March 2021 and after this date, users will no longer be able to access these services.
- Old Datastore API: Between now and 22 March 2021 users can choose to either use existing API calls to return data from the old Datastore, or use a Redirect version from the Old Datastore to the new Datastore. Redirects can be tested by replacing "datastore.iatistandard.org" in previous queries with "datastoreredirect.iatistandard.org." By using this version, existing API calls will be redirected and return data from the new Datastore. After 22 March 2021, the old Datastore will be deprecated and all queries to the Old Datastore will automatically be redirected to corresponding queries on the new Datastore.
For any questions or support to transition to using the new Datastore and Validator, please email the IATI Secretariat: [email protected].
Upcoming work
IATI’s Secretariat is committed to maintaining and updating all IATI digital products to meet users’ needs. Some examples of planned work include:
IATI’s Secretariat is committed to maintaining and updating all IATI digital products to meet users’ needs.
Datastore:
- Enable transaction and activity level data to be output from a single query
- Enable output of normalised USD values for transactions, regardless of published currency
- Enable output of validation results.
Validator:
- Key priority post-launch is working proactively with publishers in fixing data errors and improving their data quality.
To provide feedback on either of these two projects, please raise an issue on Github: IATI Datastore Validator or email [email protected].