Optimize GitHub Actions For Cloud Run & Desktop Builds
Hey guys! Let's dive into how we can optimize our GitHub Actions workflows for Cloud Run deployments and desktop builds. This is all about making our processes smoother, faster, and more reliable. We'll be consolidating workflows, tweaking configurations, and ensuring everything plays nicely together. So, buckle up and let's get started!
Objective
Our main goal here is to consolidate and optimize our GitHub Actions to support automated and reliable Cloud Run deployments, as well as desktop builds. We want this to be a seamless process while still respecting our repository preferences. This means streamlining what we have, ensuring things run when they should, and preventing any conflicts.
Acceptance Criteria
To make sure we're on the right track, we've set some clear acceptance criteria. These are the milestones we need to hit to consider this project a success:
- [x] Consolidate workflows and remove unused ones: We’re going to clean house and keep only the essential workflows. This means keeping
.github/workflows/cloudrun-deploy.yml
for Cloud Run (triggered bypush
and manualworkflow_dispatch
) and.github/workflows/build-release.yml
for desktop releases (triggered byv*
tags and manual). - [x] Remove CodeQL/security scans: We're streamlining our builds by removing CodeQL and other security scans from our continuous integration (CI) process. This will help speed things up.
- [x] Ensure
cloudrun-deploy.yml
runs automatically on push to main: The Cloud Run deployment workflow should automatically kick off whenever there’s a push to the main branch, but only for relevant paths. This ensures our deployments are always up-to-date. - [x] Add concurrency groups: To prevent any overlapping deployments to the same environment, we’ll implement concurrency groups. This is crucial for maintaining stability and avoiding conflicts.
- [x] Optimize caching: Caching is key to faster builds. We'll be optimizing our caching strategy by:
- Using
actions/setup-node@v4
with npm cache for API and streaming. - Caching Pub (
~/.pub-cache
) and pinning the Flutter SDK version for Flutter projects.
- Using
- [x] Configure WIF inputs in
cloudrun-deploy.yml
: We'll set up WIF (Workload Identity Federation) inputs in our Cloud Run deployment workflow. This includes configuringvars.WIF_PROVIDER
andvars.WIF_SERVICE_ACCOUNT
for secure deployments. - [x] Use
vars.GCP_PROJECT_ID
andvars.GCP_REGION
consistently: To maintain consistency, we’ll use environment variables for our Google Cloud project ID and region, preferringus-central1
as our region. - [x] Keep deployment configs preserved: We're making sure our existing deployment configurations are preserved. We'll add Cloud Run as an additional path without replacing any existing setups.
- [x] Document workflow triggers and usage: Documentation is vital! We’ll clearly document our workflow triggers and usage in both issue notes and our project documentation.
Workflow Consolidation and Optimization
Okay, so let's break this down. Consolidating workflows is a big win for keeping things manageable. Imagine having a bunch of different scripts doing similar things – it’s a recipe for confusion and duplicated effort. By streamlining our workflows, we make it easier to understand what’s going on and where to make changes. We're focusing on two main workflows: cloudrun-deploy.yml
and build-release.yml
. The first one handles our Cloud Run deployments, which are critical for our live applications. The second takes care of building and releasing our desktop applications. By keeping these workflows clean and focused, we ensure that our deployments and releases are reliable and efficient.
Removing unused workflows is like decluttering your workspace; it makes everything feel cleaner and more organized. No one wants to wade through a bunch of old files to find what they need. By getting rid of the unnecessary stuff, we reduce the chances of accidentally using an outdated workflow or getting confused about which one to use. This step alone can save us a lot of headaches down the road. Streamlining builds by removing the CodeQL and security scans from our CI process might sound a bit risky at first, but it's a strategic move. Security is crucial, but these scans can add significant time to our builds. We're not eliminating these checks altogether; we're just moving them to a different part of our process, perhaps a nightly build or a separate security-focused workflow. This ensures that our regular builds are fast and efficient, while security remains a top priority. Remember, it’s about balancing speed and security to achieve the best overall outcome.
Automated Cloud Run Deployments
Ensuring that our cloudrun-deploy.yml
workflow runs automatically on push to the main branch for relevant paths is a game-changer for our deployment process. Think about it: every time we push code to the main branch, we want our changes to be reflected in our Cloud Run environment as quickly as possible. This automation reduces the manual effort required and minimizes the chances of human error. By setting up triggers that watch for changes in specific paths, we can ensure that only relevant code changes kick off a deployment. This targeted approach saves resources and time, making our deployment pipeline more efficient. Imagine the peace of mind knowing that your code changes are automatically deployed without you having to lift a finger – that's the power of this automation!
Concurrency Groups
Adding concurrency groups is like setting up a traffic control system for our deployments. Without it, we could have multiple deployments trying to update the same environment simultaneously, which can lead to chaos and conflicts. Concurrency groups ensure that only one deployment runs at a time for a specific environment. This prevents overlapping deployments and guarantees that our environments are updated in a controlled and orderly manner. This is especially important for production environments where stability and consistency are paramount. Think of it as a safety net for your deployments, preventing any accidental collisions and ensuring a smooth deployment process. This feature alone can save us from potential downtime and headaches, making it a critical part of our optimization efforts.
Caching Optimization
Caching is where the magic happens when it comes to speeding up our builds. It's like having a shortcut that avoids repetitive tasks. By caching dependencies and other build artifacts, we can significantly reduce the time it takes to run our workflows. Using actions/setup-node@v4
with npm cache for API and streaming projects is a smart move. Node.js projects often have a large number of dependencies, and downloading them every time we run a build can be time-consuming. By caching the npm packages, we can reuse them across builds, saving valuable minutes. Similarly, for Flutter projects, caching the Pub cache (~/.pub-cache
) and pinning the Flutter SDK version can make a huge difference. Flutter SDKs can be quite large, and downloading them for every build is inefficient. By caching the SDK and Pub packages, we can drastically reduce build times. Optimized caching is a cornerstone of efficient CI/CD pipelines, and it’s an area where small tweaks can lead to big improvements.
WIF Configuration and GCP Variables
Configuring WIF (Workload Identity Federation) inputs in our cloudrun-deploy.yml
workflow is all about enhancing security. WIF allows our GitHub Actions workflows to securely access Google Cloud resources without needing to store long-lived credentials. By configuring vars.WIF_PROVIDER
and vars.WIF_SERVICE_ACCOUNT
, we ensure that our deployments are authenticated using a more secure mechanism. This reduces the risk of credential leaks and enhances our overall security posture. Using vars.GCP_PROJECT_ID
and vars.GCP_REGION
consistently is crucial for maintaining consistency and avoiding configuration errors. By defining these variables at a central place and reusing them across our workflows, we ensure that all our deployments target the correct project and region. This not only simplifies our configuration but also reduces the chances of deploying to the wrong environment. Consistent variable usage is a best practice that can save us from many potential issues.
Deployment Configuration Preservation
Keeping our deployment configurations preserved while adding Cloud Run as an additional path is about respecting what we already have in place. We don't want to break existing deployments while introducing new ones. By adding Cloud Run deployments without replacing our existing setups, we ensure a smooth transition and avoid any disruptions. This approach allows us to gradually adopt new technologies without affecting our current infrastructure. It’s a thoughtful way to evolve our deployment strategy, ensuring that we move forward without leaving anything behind. Think of it as adding a new lane to a highway without closing the existing ones – it improves traffic flow without causing a jam. This careful approach to deployment configuration is key to a successful migration.
Documentation of Workflow Triggers and Usage
Finally, documenting our workflow triggers and usage is about making our processes transparent and understandable. Good documentation is essential for collaboration and maintainability. By clearly documenting how our workflows are triggered and how they should be used, we make it easier for others to understand and contribute to our CI/CD pipeline. This includes documenting the triggers for the cloudrun-deploy.yml
and build-release.yml
workflows, as well as any specific configuration requirements. We’ll document this information in both our issue notes and our project documentation, ensuring that it’s easily accessible to anyone who needs it. Clear documentation is the glue that holds everything together, making our workflows more robust and easier to manage.
Links
Here are some useful links for reference:
- Workflows:
.github/workflows/cloudrun-deploy.yml
,.github/workflows/build-release.yml
- Docs:
docs/DEPLOYMENT/CI_CD_PIPELINE_GUIDE.md
,docs/DEPLOYMENT/CLOUDRUN_IMPLEMENTATION_SUMMARY.md
,docs/DEPLOYMENT/CLOUDRUN_DEPLOYMENT.md
Estimated Effort
We estimate this will take around 5 points, which translates to about 2–3 days of work. Let's get this done!