About Open Auburn
Open Auburn is an open database of community and university public data, designed to support Auburn-specific development and insights.
Motivation
The City of Auburn and Auburn University both host massive amounts of public data online. Much of this data has obvious value and could be used to derive important information or applied to products and services that directly aim to help the community.
However, this data is almost exclusively scattered acrossed multiple pages, unnormalized, and intricately wrapped in HTML. In order to make anything of Auburn data, you would have to spend hours developing some web-scraper, normalizing the data, and reliably storing it. This poses a serious barrier to community-centred software development, data mining, etc.
Open Auburn distinguishes itself by providing not just public data, but open data - data that is not only accessible to humans but also readily available and machine-readable. The people behind this project figured it would be best if this data was scraped and processed once, and then served in its usable form for all.
Architecture
Open Auburn is currently comprised of the following 4 systems.
1. Central Database
A PostgreSQL database acts as central storage for the datasets that offer programmatic access. This database also stores relevant metadata on these collections.
2. Data Access
Data Access is a RESTful web API that allows programmatic access to Open Auburn's data. It supports user-defined pagination, filtering, sorting, and selective field hiding. The root endpoint for Data Access is https://data.openauburn.org/and documenation for it can be found here.
3. Scrape Engine
The scrape engine allows command-line access to a collection of scripts that scrape data and publish it to Data Access. The engine supports error reporting and tolerance, partial and complete dataset updates, and browser driving.
3. Data Portal
The data portal (this site) offers a central place to explore and contribue data.
Team
Open Auburn is maintained by Matthew Rogers. The project looks to expand both its data and operational capabilities. If you have made something cool with Auburn data or want to contribute to the project directly, please review the contribution page.