Create your own git server easily with Chef and the Vagrant AWS plugin: Part 3.

https://netdna.bootstrapcdn.com/font-awesome/3.2.1/css/font-awesome

Table of Contents

• Part 1: Get the latest Chef on your EC2 target.
• Part 2: Install your private Git server.
• Part 3: Automatically backup your repositories to S3.

Part 3 Goal. Backup your repositories to S3

Continuing from Part 2 in where we finished creating a personal Git server using Chef and the Vagrant AWS plugin.

1. Create a S3 bucket called git-server-repos from the AWS console.
2. Add the following s3cmd cookbook to the Berksfile.

site :opscode

cookbook 'gitolite', git: 'git@github.com:julionc/chef-gitolite.git'
cookbook 's3cmd', git: 'git@github.com:freerobby/s3cmd.git'
metadata

3. Also add depends 's3cmd' to metadata.rb.

name             'git-server'
maintainer       'YOUR_NAME'
maintainer_email 'YOUR_EMAIL'
license          'All rights reserved'
description      'Installs/Configures git-server'
long_description IO.read(File.join(File.dirname(__FILE__), 'README.md'))
version          '0.1.0'

depends 'apt'
depends 'git'
depends 'gitolite'
depends 's3cmd'

4. Edit the recipes/default.rb as so:

include_recipe 'apt'
include_recipe 'git'
include_recipe 'gitolite'

gitolite_user node.gitolite.username do
  home node.gitolite.home
  version node.gitolite.version
  ssh_key node.misc.ssh_key
end

include_recipe 's3cmd'

5. Update the Vagrantfile from Part 2 with the s3cmd attributes highlighted.

Vagrant.configure("2") do |config|
  config.vm.hostname = "git-server"
  config.vm.box = "aws-basic"
  config.vm.boot_timeout   = 120
  config.omnibus.chef_version = :latest
  config.berkshelf.enabled = true

  config.vm.provision :chef_solo do |chef|
    chef.json = {
      misc: {
        ssh_key: File.read(ENV['MY_PUBLIC_SSH_KEY_PATH'])
      },
      s3cmd: {
        aws_access_key_id: ENV['AWS_ACCESS_KEY_ID'],
        aws_secret_access_key: ENV['AWS_SECRET_ACCESS_KEY']
      }
    }
    chef.run_list = [
      'recipe[git-server::default]'
    ]
  end
  config.vm.provider :aws do |aws, override|
    aws.access_key_id = ENV['AWS_ACCESS_KEY_ID']
    aws.secret_access_key = ENV['AWS_SECRET_ACCESS_KEY']
    aws.keypair_name = ENV['AWS_KEYPAIR_NAME']
    aws.security_groups = ['your-security-group-here']
    aws.instance_type = "t1.micro"
    aws.ami = "ami-d0f89fb9"
    override.ssh.username = "ubuntu"
    override.ssh.private_key_path = ENV['MY_PRIVATE_AWS_SSH_KEY_PATH']
  end
end

6. Now run vagrant provision. Below is the partial output for a successfully installed s3cmd.

....
[2013-10-10T11:23:31+00:00] INFO: directory[/usr/local/share/s3cmd] created directory /usr/local/share/s3cmd
[2013-10-10T11:23:31+00:00] INFO: git[/usr/local/share/s3cmd] cloning repo git://github.com/s3tools/s3cmd.git to /usr/local/share/s3cmd
[2013-10-10T11:23:32+00:00] INFO: git[/usr/local/share/s3cmd] checked out branch: v1.1.0-beta3 reference: 7d047bd9d4df6cbfa1ea91c9171091cd9347cd9f
[2013-10-10T11:23:32+00:00] INFO: execute[build_s3cmd] ran successfully
[2013-10-10T11:23:32+00:00] INFO: link[/usr/local/bin/s3cmd] created
[2013-10-10T11:23:32+00:00] INFO: template[s3cfg] backed up to /var/chef/backup/root/.s3cfg.chef-20131010112332
[2013-10-10T11:23:32+00:00] INFO: template[s3cfg] updated file contents /root/.s3cfg
[2013-10-10T11:23:32+00:00] INFO: template[s3cfg] mode changed to 600
[2013-10-10T11:23:32+00:00] INFO: Chef Run complete in 8.671023313 seconds
....

7. Test that s3cmd installed properly by using it to list the new bucket created in Step 1.

$ vagrant ssh -c "sudo su - root -c 's3cmd ls'"
# output
# stdin: is not a tty
# 2013-10-11 11:16  s3://git-server-repos

IMPORTANT: If you don't get a bucket listing with the above command make sure that your AWS credentials are correct and then re-provision.
8. Add a Cron job to backup your repositories daily. Start by adding depends 'cron' to metadata.rb.

name             'git-server'
maintainer       'YOUR_NAME'
maintainer_email 'YOUR_EMAIL'
license          'All rights reserved'
description      'Installs/Configures git-server'
long_description IO.read(File.join(File.dirname(__FILE__), 'README.md'))
version          '0.1.0'

depends 'apt'
depends 'git'
depends 'gitolite'
depends 's3cmd'
depends 'cron'

9. Include the cron recipe as well as the backup command to be run.

include_recipe 'cron'

cron_d 'backup_git_repos_to_s3' do
  minute  0
  hour    3
  command "s3cmd sync #{node.gitolite.home}/repositories s3://git-server-repos/"
end

DISCUSSION: A file called backup_git_repos_to_s3 will be created in the /etc/cron.d directory and be run by the root user.
10. Run vagrant provision and then test that all went as planned.

$ vagrant ssh -c 'sudo cat /etc/cron.d/backup_git_repos_to_s3'
# Crontab for backup_git_repos_to_s3 managed by Chef. Changes will be overwritten.

0 3 * * * root s3cmd sync /home/git/repositories s3://git-server-repos/

IMPORTANT: By default Ubuntu Server is set for UTC time so for example, 3 AM UST would be 10 PM EDT.
NOTE: I will leave the final testing of the repository backup to you. It is best to test the Cron job by vagrant ssh'ing in and manually editing the time within /etc/cron.d/backup_git_repos_to_s3. Then when you are done testing run vagrant provision again and /etc/cron.d/backup_git_repos_to_s3 will be overwritten with your preferred time and command.

Finished

This concludes a 3-part tutorial where we installed an automatically-backed-up Git server on EC2 with a minimal amount of code thanks to the combined power of Chef and the Vagrant AWS plugin. I think that you will find Vagrant is a fine tool for production environments. You can re-visit Part 1 to read the benefits of using Vagrant for production, staging, or QA environs. Lastly, you will find full source code of this tutorial on Github.