DevOps: Sprint 3 (July 15-28, 2020)

Note: We did not record a formal sprint review because the nature of the work accomplished is not something that translates well to a demo.

What was the goal of Sprint 3?
The goals of sprint 3 were to finish consolidating the duplicate shell scripts and the Starlight deploy pipelines, to set up Google authentication for each of our production and staging instances, to figure out the best way to migrate data between Starlight instances, and to ensure appropriate monitoring is set up for all applications in production.

What is the milestone that this sprint is supporting?
The overarching goal of this milestone was to tackle outstanding DevOps issues in the backlog to ensure that our in production applications (Shoreline and Starlight) are stable and can be deployed via CI/CD.

Accomplishments of Sprint 2:
Consolidated duplicate shell scripts and rask tasks used to configure Solr for each application

• Learned how to write internal dependencies that can be shared across multiple applications

Hooked up existing Google authentication service accounts to the new containerized Starlight instances

Tested Starlight data migration options

• In our test of the documentation site, we discovered that json export/import had major issues and determined that it was not viable option for large-scale migration at this time: https://gitlab.com/surfliner/surfliner/-/issues/590
• Currently testing out migration strategy leveraging Kubernetes Jobs and Helm post-install hooks to import data. These jobs assume access to an AWS/Minio bucket containing an exported postgreSQL database and all images stored on disk (in public/images). Job summary:
  - Data Import
    + Use psql client to import database export file
    + Use aws-cli sync command to copy files into mounted volume
  - Solr Reindex
    + Reindex all exhibits, thereby populating a new Solr index.

Began coordinating with the UCSB campus on setting up email for the new Google Connect system

Made the decision to drop support for Rancher questions.yml file for Helm charts

Did we do everything we set out to accomplish?
No. We did not begin the monitoring work that we had hoped to accomplish during this sprint.

What’s next?
The next sprint is the last of the current workcycle. The top priority is to continue work on the Starlight migration work; it is important to both campuses that this work be wrapped up in the near future. We also want to start work on monitoring and observability before this workcycle ends, even if we are not able to complete all tasks related to this effort.

GitLab Link: https://gitlab.com/surfliner/surfliner/-/milestones/50