Orchestration Platform for Apps at Viasat: Project VonBraun

I am a Software Engineering Intern working in Viasat’s Seattle office. My team’s internship project, VonBraun, is a next-generation orchestration platform for 12-factor apps at Viasat, to meet the need for a simple platform to run general-purpose (e.g. web) apps with little operational overload. It is basically a Heroku-style platform for internal use at Viasat. We aim to dramatically increase the speed at which certain kinds of apps can be developed and deployed, such as:

  • Web management dashboards for various systems
  • Databus loopbacks/aggregators
  • Web apps that fall into the 12-factor app idealism

Workflow

The project workflow is basically divided into 4 parts –

  • App Creation
  • New Repository/Commit
  • New Repository in the AWS S3 Bucket
  • App Deployment

VonBraun has 4 major components – the server, the PostgreSQL database, and the Builder and the Runner containers – all of which live inside a Kubernetes cluster. Apart from that, we have the VonBraun CLI, which is what the user will use to interact with the VonBraun system. We have an AWS S3 bucket to upload and download the repositories and builds from.

We decided to have two separate containers because we wanted to avoid doing docker builds in the Kubernetes cluster itself, which requires root access and can be dangerous in production. So instead, we made a build environment that uploaded the build result to S3 for the runtime container to pick up the completed build and run it. This circumvented the security issue.

VonBraun was developed using Golang for the most part, with some shell scripting to help with different processes. Our primary reason behind choosing Go as the main programming language was that a large number of tools for cloud (Docker, Kubernetes) are also written in Go, which would facilitate us in adding patches and using ready components and libraries in our code base.

Workflow Architecture: App Creation

The users will first interact with the VonBraun CLI to register their applications which are residing in a GitHub repository. We authenticate every call to our server with a Viasat.io token, which expires after a certain amount of time and the users have to create a new one by logging into VonBraun for additional security. Once the app registration is done, VonBraun server fetches the users’ app from GitHub and pushes it onto the AWS S3 bucket after zipping it.

The builder container is triggered as soon as a new repository is pushed onto the bucket. The builder then downloads the app, takes care of all its dependencies and builds it. If the build is successful, the builder pushes the executables onto the S3 and updates the database with the build status. The runner then downloads this executable and runs it, and the users are provided a URL to view their live app!

Workflow Architecture: New Commit

So, the users have their application deployed the first time they register with VonBraun. What next? Continuous deployment! VonBraun users can keep working on their application and push their commits into their GitHub repositories, and since our server constantly polls their repositories for new commits, it downloads their repositories once it detects new commits. These repositories are then uploaded onto the S3 bucket.

Workflow Architecture: New Repository in S3

Once the newer versions of the repositories are available in the S3 bucket, the builder container downloads them, and builds and uploads the executables back to the S3.

Workflow Architecture: Application Deployment

If the build was successful in the previous step, the runner will fetch the executables and deploy the app and provide URLs to the users.

Getting Started

As a new user, you would have to register your GitHub repository with VonBraun. The way to do this is to authenticate yourself by using the `gitauth` CLI command.

$ vb gitauth
Username: KKhara
Password:
Authentication Successful!

This adds the VonBraun service account to your repository as a collaborator which gives it the permissions to poll for new commits and download the repository when needed.

If the build was successful, you can deploy your app with a simple command-

$ vb deploy --app <app-name>

Once VonBraun is registered with your repository, you don’t need to worry about any managing or deploying your changes. Just commit and push your changes to your repository and the newer deployments will reflect the changes.

You can use various CLI commands to display/change your app configuration.

$ vb --help
CLI to interact with VonBraun


Usage:
vb [command]


Available Commands:
deploy      Deploy your application by entering your app name
deployments Get information about deployments
builds      Get information about builds
apps        Get information about apps
env         Get information about environments
gitauth     Add the VB client to your repository on git.viasat
help        Help about any command
secrets     Updating your secrets
users       View and update user profiles
version     Version of VB CLI


Flags:
--config string   config file (default is $HOME/.vb.yaml)
-h, --help            help for vb
-t, --toggle          Help message for toggle


Use "vb [command] --help" for more information about a command.

For example, you can use the `env` command to display information about environment variables inside an app:

$ vb env -g KKhara_helloapp,envname2.0
{"EnvId":1,"appName":"KKhara_helloapp","envVar":"Key: dbName and Value: vbDatabase      Key: dbPass and Value: samplePass      ","name":"envname2.0"

Final Thoughts

We take care of all the aspects of application development lifecycle (thus, VonBraun as a PaaS), and so a user can simply deploy their application by pushing commits into their repository. Our goal is to provide a resilient and optimized environment to create new software quickly and efficiently, with the benefit of simplicity and convenience for our users.

One of the main advantages of using VonBraun is that it’s extremely easy to start – you don’t need to deal with the infrastructure – it really just works. For people developing 12-factor apps that don’t want to spend time on setting up their configurations and managing infrastructure, it’s a true time saver, while also helping them follow best practices for software development in general.

We tested VonBraun by successfully deploying a sample Django web app. Our future work for the project includes having support for more languages. Right now, we support Python and Golang apps. Additionally, we would like to develop a web UI to help our users visualize their application development lifecycle.

Internship Takeaways

Interning at Viasat has been a great learning experience. We greatly developed our knowledge of Kubernetes, Docker and AWS. Learning to build and develop a large-scale system from the ground up was a comprehensive experience as it challenged us daily to think in terms of our system design and architecture as a whole. Collaborating in a team with varying experiences, skills and opinions made our collaboration skills much better.

Moreover, our supervisor, Gabriel Albacarys, did a great job in guiding us throughout the summer, while making sure that we have the complete ownership of our project. Daily stand-ups to discuss our progress, along with weekly sprints, made the development of this project quite rhythmic.

Overall, it has been a great summer working at Viasat. From working on a meaningful project to all the intern events including a major intern Hackathon event, we got a taste of Viasat’s culture, mission, values and much more, and the most important part is that we had fun all along!

Team

  • Keval Khara (Master of Science in Computer Engineering, Boston University, Expected Graduation: January 2019)
  • Gurpreet Chabada (Master of Science in Computer Science, University of Illinois at Chicago, Expected Graduation: May 2019)
  • Uttara Kulkarni (Master of Science in Software Engineering, San Jose State University, Expected Graduation: May 2019)

Leave a Reply