Process Automation for an Oregon Charitable Foundation

2026-05-22T00:00:00+00:00

The Oregon Scottish Rite Education Foundation awards scholarships annually to students across Oregon who are entering into college or trade schools. Over the past few years, their foundation has grown, requiring them to distribute more scholarships than ever. However, since the foundation board is small and composed entirely of volunteers, they were looking for some way to automate their application process to reduce the number of manual steps required.

Initial Approach

Members of the board approached me with a wish list of features and requirements. They wanted a system to integrate seamlessly with their website, while incoming applications should be automatically checked, sorted, and categorized by county. They already had a spreadsheet system for tracking applicants as well as a cloud storage system for storing supporting documents; they wanted the automation system to integrate with these existing components.

Since the board was already utilizing Google Business products for cloud documents and storage, I decided that the best approach would be to lean into Google Cloud services. Following their list of requirements, I implemented a Google Cloud Run service in Go that processed incoming applications from their website, then updated the spreadsheet while preparing their cloud storage system for each application’s supporting documents. The spreadsheet now contained all the application data, including a link to an automatically generated PDF version of the application and a link to the cloud storage location.

This initial approach cut out an enormous amount of manual work. Previously, applications were submitted via the website and emailed to a board member, who then manually sorted them by county and generated PDFs for storage and later distribution.

Implementation Challenges

The application submission window for the Oregon Scottish Rite Education Foundation runs from December through March of each year, which gave us a limited amount of time to observe incoming live data and how the automation was performing.

My initial approach worked fairly well, and handled 94% of incoming applications properly. However, the last 6% ran into various problems that prevented correct processing. One error arose from a misspelling in a county name, which was caught early and corrected. Another series of errors arose from processing errors with the Google Sheets API, which has both rate limiting and more mysterious, inexplicable errors. I added additional error correction and error checking, but there was still some additional manual work to enter data from a handful of applications.

By the end of the application period, we had managed to reduce the amount of manual data entry and processing by a large percentage, maybe as much as 60%. After discussing the current process automation with the board, we began to see opportunities for further automation and improvement.

Additional Development

After the first year’s application submission window, I began working with one of the board members to plan out how the process could be improved even more. My list of improvements were primarily on the backend, and involved robuster error detection and reporting, as well as a backup data store that would allow the automation to recover from the Google Sheets API errors more elegantly.

The board proposed an additional feature set that would further reduce manual data processing, by allowing applicants to upload supporting documents such as letters of recommendation and high school transcripts. These uploaded documents would then be processed and stored in the proper cloud storage location, thus reducing the amount of manual file management required by volunteers.

Implementing these two new features went very smoothly. It required the development of a new frontend, using Typescript and Svelte. This frontend allowed applicants to upload their supporting documents via a unique link given to each of them by automated email. It used both Google Cloud Storage and the Google Drive API to store these files.

The improvements on the backend used Google Cloud Datastore and a more robust error recovery system. It now backed up all incoming scholarship applications until they could reliably and more slowly be added to the spreadsheet with the Google Sheets API, thus nearly eliminating all rate limiting errors. In addition, progress reports and error output could now be viewed using a new frontend administrative interface.

Results

This round of improvements produced significant results. Over the entire span of the second year of application submissions, error rates dropped from 6% to just one unrecoverable error. While I had been hoping for improvement, even I was surprised by how much of an improvement we achieved!

The board reported that the amount of manual processing was reduced by another 80-90%, with only a few applicants resorting to email for sending in their supporting documents. While we have discussed further improvements to the process automation for the next application submission window, the board is satisfied with the improvements provided by the level of automation implemented so far.

Lessons

This project benefited a lot by reviewing requirements and expectations frequently with the client. They were great at providing me with very clear goals from the beginning, and as we progressed and it became clear where the most volunteer time was being spent, we were able to identify more tasks for automation.

On the development side of things, three things were clear. First, cloud services really make the development and deployment of process automation tools like this a breeze. Costs can be kept low while still providing robust infrastructure for hosting.

Second, I learned once more the importance of unit tests. I wrote unit tests for most elements of the automation software, which allowed me to catch breaking changes and test features very quickly. Test-driven development can be a great way to run a project, and though I did not follow all of its parameters strictly, that particular philosophy did guide my decision.

Third, once more I was reminded of how the best-planned software runs into difficulties when it collides with the real world. I failed to anticipate the Google Sheets API issues and did not implement enough comprehensive error correction early on. That is not an error I shall repeat again!

Finally, I learned that even with a project that seems as straightforward as this one, it is important to control expectations early. This turned into a multiyear project because we saw room for improvement. I think it is important for a business or organization looking for process automation to understand that it can be an ongoing series of improvements; not everything will be accomplished or perfected after one pass.