HashiCorp streamlines cross-repository code search and fixes with Sourcegraph

Every day, millions of developers and DevOps engineers rely on HashiCorp to efficiently provision, manage, and secure their cloud infrastructure. HashiCorp is a company that has brought best-in-class developer experience to infrastructure teams through its portfolio of tools which includes HashiCorp Terraform, Vault, Nomad, Consul, Packer, Boundary, Waypoint, and Vagrant, which together form a platform for automating cloud infrastructure. Just as HashiCorp prioritizes developer experience for its users and customers, the internal developer experience at HashiCorp enables it to produce industry-shifting tools.

Sourcegraph enables HashiCorp's engineering team to sustain its pace of innovation as the team and codebase grow. Bryce Kalow, Senior Web Engineer and tech lead of the Web Platform team, recalls four specific areas where Sourcegraph has helped his team:

1. Mapping and managing dependencies across multiple teams and codebases
2. Speeding up big refactors by an estimated 4x and combating code ossification by making more large-scale changes possible
3. Eliminating unused code and technical debt
4. Driving adoption of a new component library

HashiCorp saves time managing dependencies across multiple web properties

Since the initial release of its first open-source dev tool, Vagrant, HashiCorp has grown its product suite into a rich, integrated toolkit that covers nearly every area of infrastructure management, from resource provisioning to container orchestration to secrets management. As the product suite has expanded, so have its codebases. Prior to Sourcegraph, it was difficult and time-consuming to manage and track dependencies across the company’s websites.

According to Bryce, “Previously, searching repositories was a time-consuming, manual, and repetitive process. If the team was trying to find, say, the version of a specific dependency, like an npm package, it could take an hour or more to repeat that process across multiple repositories. Now, with Sourcegraph, we just do a search query, find the dependency within a minute, and can see that a particular version is used in these places across these different repositories."

He adds, “Using Sourcegraph has resulted in some significant changes to the way we work at HashiCorp.” Previously, changes would first require a lot of manual effort to build a mental model of the adjacent parts of code that the author was unfamiliar with. Due to friction points, developers often wouldn't acquire as thorough an understanding of the existing code as they might like, which often meant late-cycle realizations and wasteful rework.

“With Sourcegraph, it's easier and less aggravating to grok the existing code, so you end up getting a better understanding of it early, which saves a lot of time,” Bryce says. “All this means we have more time to focus on useful work, like building a new component library.”

Web team accelerates developer velocity by an estimated 4X

Bryce’s team maintains a dozen different websites spread across different repositories. These sites include hashicorp.com and each product’s reference documentation site. A major pain point was making changes to core components and rolling them out. “Oftentimes some sites would be missed, leading to UI inconsistencies and dependency drift across our apps.”

The task of keeping dependencies uniformly up to date was manual and mundane, a form of toil that sucked up valuable engineering time that could be better spent on truly creative endeavors.

Today, the team uses Sourcegraph Batch Changes to automate such large-scale code changes. “Batch Changes provides a very repeatable process to go into each app and, for example, run ‘npm install’ or ‘npm upgrade’ and whatever additional updates need to be made to upgrade the dependency. You wire up Batch Changes. You roll it out. It’s that simple, and it’s been really great.”

In addition to dependency updates, another Batch Changes use case involves updating the alert banners that HashiCorp uses on its websites to announce new features and releases. The alert banner is configured by a static JSON file. Bryce notes, “Before Sourcegraph, when it was time to change that banner, we had to tediously repeat the same process across all the properties. Batch Changes allows us to automate these and other menial tasks, which saves us a lot of time. Plus, when we have to do it again, we just update the content in Batch Changes and rerun it.”

Instead of one to two days to get a change rolled out, reviewed, and merged, Batch Changes cuts that timeline to a few hours. Bryce estimates that Sourcegraph accelerates developer velocity for changes impacting all apps by a factor of four.

Visibility into code usage reduces technical debt

Like virtually all engineering organizations, HashiCorp’s Web Platform team fights a continual battle with tech debt. According to Bryce, Sourcegraph was “transformative,” allowing the team to determine instantly where deprecated code was still being used or referenced. Without Sourcegraph, the only way to do this was to manually search through multiple repositories, which took so long that the effort was rarely made. As a result, tech debt accrued unchecked over time until it became a drag on the overall development process, akin to the "frog in boiling water" effect.

With Sourcegraph, the team can complete large refactors and make sure every instance of a pattern is updated. They can also check if a function or API is used infrequently or not at all, helping to reduce the total size of the codebase while maintaining the same functionality.

“Folks always want to address tech debt but rarely can make time for it,” Bryce says. “The capabilities and insights that Sourcegraph gives us make it much easier to track things like deprecated APIs. With an approachable process, people are more inclined to fix outdated or unused code. Equally important, Sourcegraph gives us the ability to not just find tech debt, but really pinpoint it and then make it clear where it exists and how to address it at a high level.”

Creating a new component library

The Web Platform team is building an open-source component library for Next.js called Swingset. Bryce explains, “Our applications are already using Swingset. It enables us to document our components and, through the Sourcegraph API, track usage of these components. We're using the Sourcegraph API to pull usage data and present it in the component library UI. This way, we can see when an application is running on an older version of a component and address that.”

He concludes, “Just as HashiCorp enables repeatable workflows and reduces toil when working with infrastructure as code, Sourcegraph enables repeatable refactors and eliminates manual effort in understanding source code. It blends in nicely with the way we ideally like to operate, as well as with our philosophy and principles around enabling developer productivity.”