Zshawn Syed

Running rails db:migrate in kubernetes

Thu, 25 Nov 2021 00:00:00 +0000

Overview

I am working on a dev project that is written in rails but deployed within kubernetes. While doing local development, I am running the cluster using skaffold dev. This allows me to make changes locally and have it auto-sync to the running container without having to rebuild the image and push it anywhere. One issue with this approach is that in order to run database migrations locally, creating a deployment or job is a little bit heavy handed. Instead, we can simply connect to the running pod and run our db:migrate command there.

Connecting to a kubernetes pod

In order to run rails db:migrate we need to connect to a running pod. In order to determine which pod we need to connect to, we can use kubectl to get the name of the pod that is running the rails server. In my project, I have named that pod simply server-deployment.

> kubectl get pods
NAME                                   READY   STATUS    RESTARTS   AGE
server-deployment-6886cb479c-tt4mf   1/1     Running   36         73d

With the above, we can grab the full pod name and run the following command which will connect to the running pod, run the command, and exit.

kubectl exec server-deployment-6886cb479c-tt4mf -- /bin/bash -c "rails db:migrate"

However, we can do this in one single step instead of two. The single line command looks like this:

kubectl exec $(kubectl get pods | grep server | sed 's/ .*//') -- /bin/bash -c "rails db:migrate"

The command above is doing the following:

The sub-command $(kubectl ...) is getting all the pods, grepping for our server pod name, and then stripping everything but the first word that contains our full pod name
This is then substituted into the kubectl exec <our_pod_name>
Then finally, the /bin/bash -c "rails db:migrate" runs a one off command and auto-exits (we are not opening an interative connection
Profit!

TLDR

To run rails db:migrate during our development flow, we can connect to the running rails pod and run that command via:

kubectl exec $(kubectl get pods | grep server | sed 's/ .*//') -- /bin/bash -c "rails db:migrate"

Ignoring .git directory when using skaffold dev

Mon, 22 Nov 2021 00:00:00 +0000

I was using skaffold dev to do my development work locally. Skaffold allows you to run your kubernetes cluster locally without requiring building and pushing your container image anywhere. Instead it builds it locally and deploys patched changes as you edit the files. I was running into an issue where I set up my sync directive to listen to my ruby and javascript files (rails stack) and map them over. However, for all other file extensions, skaffold by default will attempt to rebuild the image and restart it instead of hot patching the files only.

I noticed, whenever interacting with git, it would cycle through that entire process and generally slow things down for a minute or so. This was both annoying and frequent enough for me to investigate if its possible to ignore certain files and directories. The good thing is that not only is it supported but its not even required in skaffold. Skaffold honors anything within the .dockerignore file. So to fix the issue, I simply created one and added the .git folder there.

My .dockerignore file looks like this:

# Ignore the .git folder entirely
.git
log/
*.tmp
*.temp

Installing the eventmachine gem in WSL

Sat, 31 Oct 2020 00:00:00 +0000

I ran into this issue a few times and kept forgetting the fix for it which is why I am writing a quick post on it. Generally when adding ruby gems, we add them to our Gemfile and just throw bundle install at it to get everything configured. However I kept running into a strange issue when attempting to get event machine installed for a gem I was writing myself.

The installation steps are actually quite simple if you run into this error.

The error

When trying to install eventmachine (1.2.7) on WSL, I kept getting this stack error:

user@DESKTOP:/mnt/f/code/project$ bundle install
Fetching gem metadata from https://rubygems.org/......
Using rake 10.5.0
Fetching eventmachine 1.2.7
Installing eventmachine 1.2.7 with native extensions
Gem::Ext::BuildError: ERROR: Failed to build gem native extension.

    current directory: /home/user/.rvm/gems/ruby-2.7.1/gems/eventmachine-1.2.7/ext
/home/user/.rvm/rubies/ruby-2.7.1/bin/ruby -I /home/user/.rvm/rubies/ruby-2.7.1/lib/ruby/2.7.0 -r
./siteconf20201031-507-no2sha.rb extconf.rb --with-ssl-dir\=c:/OpenSS
-----
Cannot find OpenSSL include path c/include:/OpenSS/include
-----
*** extconf.rb failed ***
Could not create Makefile due to some reason, probably lack of necessary
libraries and/or headers.  Check the mkmf.log file for more details.  You may
need configuration options.

Provided configuration options:
        --with-opt-dir
        --without-opt-dir
        --with-opt-include
        --without-opt-include=${opt-dir}/include
        --with-opt-lib
        --without-opt-lib=${opt-dir}/lib
        --with-make-prog
        --without-make-prog
        --srcdir=.
        --curdir
        --ruby=/home/user/.rvm/rubies/ruby-2.7.1/bin/$(RUBY_BASE_NAME)
        --with-ssl-dir
        --with-ssl-include
        --without-ssl-include=${ssl-dir}/include
        --with-ssl-lib
        --without-ssl-lib=${ssl-dir}/lib

To see why this extension failed to compile, please check the mkmf.log which can be found here:

  /home/user/.rvm/gems/ruby-2.7.1/extensions/x86_64-linux/2.7.0/eventmachine-1.2.7/mkmf.log

extconf failed, exit code 1

Gem files will remain installed in /home/user/.rvm/gems/ruby-2.7.1/gems/eventmachine-1.2.7 for inspection.
Results logged to /home/user/.rvm/gems/ruby-2.7.1/extensions/x86_64-linux/2.7.0/eventmachine-1.2.7/gem_make.out

An error occurred while installing eventmachine (1.2.7), and Bundler cannot continue.
Make sure that `gem install eventmachine -v '1.2.7' --source 'https://rubygems.org/'` succeeds before bundling.

In Gemfile:
  project was resolved to 0.1.0, which depends on
    action_cable_client was resolved to 3.1.0, which depends on
      websocket-eventmachine-client was resolved to 1.3.0, which depends on
        websocket-eventmachine-base was resolved to 1.2.0, which depends on
          eventmachine

The fix!

To bypass this stack, the solution is to first install eventmachine locally before running bundle install.

The command is simply:

gem install eventmachine --pre

After this, you can successfully run bundle install to install the rest of your dependencies.

Add RuboCop to Existing Projects

Sun, 21 Jun 2020 00:00:00 +0000

Adding RuboCop to a project might seem daunting but is completely do-able. When working with an existing code base that might have grown in size, it can seem impossible to consider fixing all the RuboCop issues up front. To accomodate this, RuboCop has a built-in functionality to create a TODO file of existing violations that can then slowly be tackled over time.

Installing RuboCop

The following command will install rubocop globally:

gem install rubocop

If you desire an installation directly into the project, you can add it to the Gemfile

gem 'rubocop', require: false

Generating .rubocop_todo.yml

By default rubocop will turn off the entire cop if there are too many violations. This is not what we want, because if we let the entire cop become disabled then any new code is allowed to continue that violation. Instead when we generate the whitelisting file, we’ll tell rubocop to keep whitelisting files instead of disabling the entire cop. To do that, we will run this command:

rubocop --auto-gen-config --auto-gen-only-exlude --exclude-limit 10000

auto-gen-config - runs RuboCop against the codebase and captures all violations into a .rubocop_todo.yml file.

auto-gen-only-exclude - ensures that RuboCop adds Exclude and not Max when generating the results. This flag is used in conjuction to the --exclude-limit. We need to ensure that the second flag is high enough otherwise auto-gen-only-exclude will revert back to using Max.

exclude-limit - limits how many files will be added to the Exclude block for each cop. The default is very low (15) and if we do not adjust this higher, we will end up turning off the entire cop if there are too many violations. This flag is the key element to our whitelisting approach. As long as this value is high enough, none of our cops will be outright disabled and we can continue onboarding new code at higher standards while working to fix all the violations on the existing code.

As a sanity check, once generating the file is complete make sure that no cops are disabled. If they are, re-run the command above with a higher --exclude-limit.

Addressing Existing Violations

The key takeaway with generating the .rubocop_todo.yml is not to forget the old violations but to slowly tackle them while enforcing all new code to higher standards. This is a continuious improvement effort. When fixing the issues, make sure that the commits are solely for those and not regular feature work as well. It is much easier to push along changes that are code style improvements and provides a nice commit history of how the improvements were handled.

RuboCop provides a --auto-correct option that will attempt to clean up the violations for the supported cop. The feature is still experimental as of this post but worth starting off with. It’s easy to run the command and run a quick git diff to make sure that the new changes are valid and of course running your test suite to make sure there aren’t bugs. This is another reason why its much better to commit refactors and code cleanups into their own isolated commits.

Conclusion

RuboCop makes it a lot easier to have newer engineers onboard to a Ruby project by leveraging consistent coding styles and enforcing new commits to adhere to them. For new or old projects, RuboCop can easily be part of the workflow to keep the code quality high and remove the burden of reviewing coding styles within the code review process. I hope that at this point it should clear that onboarding RuboCop is very straight forward.

To learn more about the RuboCop command line flags, check out the official docs. RuboCop uses the rules defined by the official ruby style guide.

Interesting Tech Podcasts - Part 1

Sat, 20 Jun 2020 00:00:00 +0000

These days I find myself listening to a variety of podcasts while working from home and doing random chores. I wanted to share the podcasts in the software field that I currently enjoy and have enjoyed over the past few years.

Podcast Apps

You can use any app, your browser, or even just download the audio files to listen to podcasts. You don’t need anything fancy. I do have to call out my favorite podcast app however. I have been a long time user and fan of Podcast Addict. This is a great option with a lot of functionality on top of just listening to podcast. It supports discovery, automatically downloading episodes in your feed, and custom playlists. It keeps track of the episodes you have listened to and provides offline backups too. The only drawback is that this is an Android only app.

1. Ruby on Rails Podcast

Link: Ruby on Rails Podcast

The Ruby on Rails Podcast is hosted by Brittany Martin who invites a guest each week to discuss topics related to Ruby on Rails. The topics range from discussions about the Ruby ecosystem, Ruby on Rails, open source work, and stories about individual developers in the field. This is a great podcast to listen to that is not super technical and instead focuses more on the engineers and industry. This podcast has an easy entry bar for listening in.

Favorite Episodes

287: Recruitment on Rails with Brian Mariani - Interesting episode from the perspective of a company the places Rails engineers. Brian from Mirror Placement, goes over how the company started, what the experience is like placing engineers, and what engineers can expect from the process. This was the most listened to episode for this podcast in 2019.

306: Ask For The Job with Brian Mariani (Part II) - This is the second episode that follows up and dives more into the actual recruiting process and provides helpful tips for engineers going through the process or interested in started the placement process.

283: Kill All Mutants with Dave Aronson - Very interesting podcast to listen to about mutation based testing. This is a radically different testing approach that performs bytecode manipulation during runtime to validate that your tests do fail on the right behaviour and that the tests themselves are testing the right things. Mutation testing can be considered a way of testing on tests. It validates whether we have gaps in the actual testing scenarios and makes sure that what are in actually testing is meaningful.

2. The Ruby Blend

Link: The Ruby Blend Podcast

This podcast is a relatively new one but already has a decent range of episodes. The podcast has a collection of interviews and discussions amongst ruby engineers in the field. The discussions are more technical but still very approachable regardless of experience. The podcast is ruby focused but still takes time to cover what its like from a personal perspective of being a Rails engineer.

Favorite Episodes

Episode 6: Working From Home - With COVID-19 forcing most tech companies to quickly adopt a work from home policy, this episode provides some great insights into how engineers are handling the sudden transition. This is great episode to listen to if you have been working from home for the first time or are interesting in hearing what other’s experiences have been so far. There are several suggestions that the hosts discuss that will make sure that your experience is a positive one.

Episode 8: Tests and Webpacker - Testing is a common theme in the ruby community. This episode covers some interesting points in testing along with webpacker adoption into Rails. The hosts go into detail about what their experiences so far have been and how they approach testing legacy Rails applications. The hosts also talk a bit more about how working from home has been going, which builds a little bit on top of episode 6, which was an episode dedicated to that topic.

Episode 9: ViewComponent at GitHub with Joel Hawksley - ViewComponent extends functionality when building views. This was initially designed to be integrated into Rails but ended up being an optional drop in. The library allows for using ruby objects to generate views. This allows for creating better tests again the generated views. The library was created by Github and is used internally by them. If you are interested in seeing some interesting changes to how views could be generated in Rails and better testing against them, check this episode out. Joel Hawksley is the maintainer and goes into details on what inspired its creation, why it did not get merged into the Rails 6 code, and what he believes to be the future of this gem.

3. Ruby Rogues

Link: Ruby Rogues

This podcast is part of the devchat.tv network which has a large collection of podcasts. Ruby rogues focuses on Ruby and Ruby on Rails but occasionally brings in guests from other stacks. This is a great podcast because it has so many episodes (approaching 500 at the time of this post). The podcast has a core panel with a few rotating hosts. Recently the podcast has been diving into Rails 6 and exploring javascript.

Favorite Episodes

RR 446: Development Environments - Everyone has their own preference for local development regardless of tech stacks. I enjoyed this episode because its always great to see if there are other tools and setups that others are using that might make you more effecient. The hosts go over what they use locally, why they use it, and what works or doesn’t work with their setup. Its a great episode to see what other rubyists are using for local development. Another good episode on this topic is RR 389: Developer Environment with the Panelists.

RR 434: Surviving Webpack with Ross Kaffenberger - This is the episode that convinced me to take a look at using webpack, vue, and Rails 6. Ross goes into great detail on what his experience was adopting modern javascript and webpack into Rails along with the entire process of upgrading Rails 6. This is an excellent episode to listen in on if you are curious on what that process looks like, and some great things to watch out for as well. I ended up listening to this one several times.

RR 388: RubyoCop and Code Linting with Bozhidar Batsov - The ruby community heavily invests in a testing culture and keeping readability in mind. The most popular code linting tool for ruby seems to be RuboCop. The author of the library, Bozhidar Batsov, is invited to discuss what inspired him to create it and what his experience is being the lead developer for it. Highly recommend listening into this one to learn more about the history of RuboCop and get an idea of its primary goals.

4. Developer Tea

Link: Developer Tea

Developer Tea is a podcast that covers what its like being a software engineer from both a social and emotional point of view. The host of the podcast, Jonathan Cutrell hosts this podcast and provides small inspirations in each episode. The purpose of the podcast is to help software engineers that are mindful about growing their career by providing tips and suggestions that will help that goal. The episodes themselves are lightweight and usually only last about 10 minutes, which is easy to fit into your tea break :)

Favorite Episodes

3 Ways to be a Better PR Author - This episode goes over some amazing tips and a change in thinking to create pull requests. This one is a great listen and the biggest takeaway is to step away from just creating pull requests to becoming a reviewer of your own code. Many people, myself included, initially either are defensive about the code we write or create the pull requests only to see that it takes a while to get reviewed. These can all be addressed by changing your mindset into becoming a reviewer of your own code so you become part of the process as well as changing exactly how to create the pull requests to get them processed faster.

4 Ways to be a Better PR Reviewer - This episode goes hand in hand with the one above. Everyone creates pull requests but everyone also reviews pull requests. In your career you will review many more pull requests than you will create them. This episode explains how changing the way you approach code reviews will also in turn change the way you create your own pull requests and have them processed.

Meetings can Trick Your Emotions - Most software engineers have at least some recurring meetings along with meetings that are scheduled on an as needed basis. Meetings are very different from coding or just working within your team. Meetings become a social activity where there are many different perspectives, backgrounds, and levels of power. This episode dives into how to make sure that meetings are useful, productive, and don’t leave in a false sense of positivity.

[Rails 6 Series] Configuring Vue and Webpack

Mon, 15 Jun 2020 00:00:00 +0000

Posts in Series

Rails 6 Series - Starting Rails 6 with Vue on AWS
Rails 6 Series - Setting up AWS resources
Rails 6 Series - Deploying to AWS
Rails 6 Series - Configuring Vue and Webpack

Overview

We have successfully deployed the basic Rails 6 sample application to AWS. All that remains now is to configure both Vue and Webpack for production use. In local development we can serve our packs through bin/webpack-dev-server but in production we need everything pre-compiled so that Nginx can serve the content in the packs directory.

In this post we will go over configuring webpack so that we can import Vue globally in each pack as required. We will configure Vue and some basic Vue plugins and also make it available in all packs as needed. Lastly we will go over how webpack differs from the traditional Rails Asset Pipeline.

Webpack Configuration

In the root directory of our application we have the webpack configuration files by stage. You can find these under config/webpack/*.js. The main file we need to update is environment.js. This file imports webpack, vue, and any other top level library we need.

When we created our initial Rails structure, we told rails that we will be using webpack and vue via the --webpack=vue flag in the cli. By default this will create the following content in environment.js

To properly configure Vue for production, lets make sure that Webpack is configured to load the right payload for it. Create a new file named vue_config.js within the config/webpack directory and copy the contents below into it.

After that, update the environment.js file from earlier to load our custom vue config and override the default development payload for vue. This will configure it to load correctly in production.

Package.json

With webpack configured, we can sanity check that our packages are up to date and included. If any are missing or if you need additional packages, add them to your package.json file in the root directory of our application.

Vue Configuration

The final bit is to load in Vue into our application pack found in app/javascript/packs/application.js. We will import both Vue and BootstrapVue. If you do not want to use BootstrapVue, feel free to import just Vue. Since Rails uses Turbolinks, there is also a vue adapter for it. We will be importing all 3. Only Vue is absolutely required.

Migrating away from Asset Pipeline

Using webpack lets us migrate out of the Asset Pipeline for javascript files. This is a bit of a shift in thinking if you have been using Rails for a while. The key difference is that you can write your javascript files within app/javascript/ instead of app/assets. The way to model this approach is that you build your directory structure within app/javascript/* and use require statements to cross load all your local dependencies. Everything here will be compiled and minified by webpack during application build.

Intead of using javascript_include_tag, use the javascript_pack_tag to load files inside of the app/javascript/packs folder. Each pack can be considered a mini application bundle. You can generate as many as you need or bundle it all into 1 pack.

Conclusion

At this point everything has been setup for both local development and pushing to production. Webpack will load the correct version of Vue and Vue has been configured to pull in both Turbolinks and BootstrapVue. With this, we can start building out our application and creating as many packs as we need. Webpack will handle the compilation, minification, and tree-shaking for us!

To learn more about Webpack take a look at the concepts page at the official docs. This was sorely missing in the Rails Guides and helped me understand how webpack works when learning how to use webpack with rails.

Rails 6 Series

Rails 6 Series - Starting Rails 6 with Vue on AWS
Rails 6 Series - Setting up AWS resources
Rails 6 Series - Deploying to AWS
Rails 6 Series - Configuring Vue and Webpack

[Rails 6 Series] Deploying to AWS

Wed, 10 Jun 2020 00:00:00 +0000

Posts in Series

Rails 6 Series - Starting Rails 6 with Vue on AWS
Rails 6 Series - Setting up AWS resources
Rails 6 Series - Deploying to AWS
Rails 6 Series - Configuring Vue and Webpack

Overview

Now that we have our empty Rails project and created our Elastic Beanstalk environment, we are ready to add our configuration scripts so that we may deploy our application.

Beanstalk Platform Hooks

The latest beanstalk environment uses Amazon Linux 2 which no longer supports beanstalk extensions (ebextensions). Instead it introduces Platform Hooks. This allows us to create configuration files and setup scripts that will run on the EC2 hosts during the various build and deployment stages. AWS has a nice graph that shows the new platform hook process.

Nginx Setup

By default webpack generated content will not be served by Nginx. We need to add an Nginx config file to our beanstalk environment that will add the packs directory to Nginx. Elastic Beanstalk has a default nginx.conf file that loads all files within the conf.d/elasticbeanstalk folder. With Amazon Linux 2, we will leverage platform hooks.

Create a new folder .platform in the root directory of our Rails application. Within there, create a file in the following path .platform/nginx/conf.d/elasticbeanstalk/webpack.conf

# .platform/nginx/conf.d/elasticbeanstalk/webpack.conf

location /packs {
    alias /var/app/current/public/packs;
    gzip_static on;
    gzip on;
    expires max;
    add_header Cache-Control public;
}

With the above, whenever we deploy our application or an instance gets replaced, nginx will always automatically be configured to serve everything within the packs directory.

Yarn and Node Setup

As we did with the nginx configuration, we can install Yarn and Node JS automatically onto our EC2 hosts when beanstalk deploys the application. Both are requirements for using webpack and we will use platform hooks for this. Create a new file within the prebuild hooks folder .platform/hooks/prebuild/01_install_yarn.sh. Make sure that this file has executable permissions chmod +x 01_install_yarn.sh.

Database Setup

The next step is to configure our application to connect to our database in production. For this we will use both environment variables and Rails encrypted credentials.

Rails 6 Master Key

We need to let our beanstalk application have access to the master key to decrypt the credentials. In order to do this we need to go into our application and fetch the encryption key.

$ cd config
$ cat master.key
<copy this key>

As an aside, do not commit the config/master.key file!! It has already been added to the .gitignore file when we ran rails new.

RDS Connection Details

The next piece of information we need is our database name and database endpoint. Log into the AWS console and go to the RDS page. Select your application database where you will find the connection details under the Connectivity & security tab.

Beanstalk Environment Variables

From the AWS console, head over to the beanstalk configuration page for your environment
Go to the Software section configuration and hit Edit
Create 3 new environment variables with the values we fetched earlier
1. Database Name
2. Database Endpoint
3. Master Key

Encrypted Credentials

To store the username and password for connecting to our RDS instance, we will be using Rails encrypted credentials. The file is located in config/credentials.yml.enc. We will not be able to open the file without decrypting it first, for which there is a rails command available.

EDITOR="vim" rails credentials:edit

This will open up the file for editing and saving. The format of this file is just simple YAML. While we have this open we will add a few things to it.

Saving this file will encrypt it before saving it back to disk. WIth that in place we need to make a few adjustments. Open config/environments/production.rb and set config.require_master_key = true to make sure that the application checks for the master key during start up.

After that is done, accessing the values is super simple. From anywhere in our application we can access the yaml structure.

Rails.application.credentials.production.database_username # => will return username

Database.yml

With our connection details entered in both the environment variables and encrypted credentials, the last step is to update our config/database.yml file.

Deply to AWS

In the previous post we created our necessary AWS resources and in this post we created our initialization scripts. The last step we need to do is hook it to beanstalk and deploy it! Make sure you have committed all the files we created and edited locally.

Ensure that you have eb installed so that we can initialize our application
From your project root run eb init
The command will ask you which region you want to use, select the one we created our beanstalk environment
Next select which platform to use. Select the one that has Ruby 2.7 (Puma)
Select no when it prompts for ssh and CodeCommit
Run eb status to make sure everything is hooked up, you should see Status: Ready
Run eb deploy and wait for it to deploy to your environment before visiting the page!

Conclusion

With the past few posts we installed our prerequisites, setup our AWS resources and beanstalk environment, and lastly configured our application to install our dependencies within EC2 and serve webpack content through Nginx. At this point your application should be up and running on AWS managed completely through beanstalk.

Any further changes to your application can be done locally, saved via git, and deployed via eb deploy. This will allow fast development cadence and remove the operational burden of maintaining a stack on AWS!

The next post will go over actually using Vue with webpack now that our stack is deployable. We will import Vue, configure it, and lastly configure webpack so that we can create multiple packs

The next post will go over actually using Vue with webpack now that our application can be deployed. We will import Vue, configure both Vue and Webpack, and learn how to migrate from using the Asset Pipeline to creating packs instead.

Rails 6 Series

Rails 6 Series - Starting Rails 6 with Vue on AWS
Rails 6 Series - Setting up AWS resources
Rails 6 Series - Deploying to AWS
Rails 6 Series - Configuring Vue and Webpack

[Rails 6 Series] Setting up AWS resources

Tue, 02 Jun 2020 00:00:00 +0000

Posts in Series

Rails 6 Series - Starting Rails 6 with Vue on AWS
Rails 6 Series - Setting up AWS resources
Rails 6 Series - Deploying to AWS
Rails 6 Series - Configuring Vue and Webpack

Overview

To begin hosting our application in AWS we will first go into the Console and set up our environment configuration on AWS Elastic Beanstalk manually. Beanstalk can create your RDS instance as well however we will be creating our separately. This is because resetting or tearing down your beanstalk stack also deletes the RDS instance which may not always be desired. This post will go through the Console but everything can be done via the CLI as well. Since this is a one time setup, I have opted to just use the Console directly.

Let’s go into the Console and log in. After that select the desired region to use, I will be using us-west-2.

VPC

For our beanstalk application, we will create our own VPC instead of using the default VPC. This is not strictly required but does make things cleaner down the road having the application separated into its own resources. Doing so will not incur any additional cost either!

From the main console page, head over to the VPC page.

Create a VPC, you can name it whatever you want. I chose my project name for my VPC name. For the CIDR block we do not need to restrict ourselves so we will create a large VPC with a /16 block. If you feel that is excessive feel free to reduce it to /18 or /20. We are also not using Dedicated tenancy so we will stick to Default.

The next thing to ensure is that DNS resolution is enabled within your VPC. If this is not toggled then you will be unable to use an ELB with your EC2 targets. Simply go to Edit DNS resolution and ensure that the DNS resolution checkbox is enabled.

Subnet

Best practice is to create at least 2 subnets so that you can have hosts in 2 different availability zones. This increases the availability story of your application. Let’s go ahead and create 2 subnet’s in our new VPC (or default VPC if you decided not to create a new VPC). Head over to the VPC overview page and select the Subnets link from the left navbar. Hit the Create Subnet button at the top of the page.

First select the VPC you want to create the subnet
Pick an Availability Zone (make sure each subnet has a different one)
Create your CIDR range so that its a subset of the main VPC. For example use 172.30.1.0/24 and 172.30.2.0/24.

To make life a bit easier let’s make sure to enable public ip’s on our instances. We will be applying a security group so that only the ELB can talk to the instances but this will allow us to expand that later if need be.

Select each subnet on the Subnets page for our VPC
Select Actions at the top and navigate to Modify auto-assign IP settings
Ensure that the checkbox is enabled

Internet Gateway

In order to route traffic to your ELB and ultimately your application, your VPC must be able to have ingress/egress from the internet. On the same VPC page, select the Internet Gateways link and create a new Internet Gateway. This will allow us to attach it to the VPC. Once the Internet Gateway is created, from the Actions dropdown attach it to the desired VPC.

Route Table

The next step is to make sure that your route table can route traffic correctly between resources inside your VPC and from the internet through your internet gateway.

From the VPC page, select the Route Tables tab
Select the route table for your VPC
Under Actions select Edit Routes
In the routes page, make sure you have your VPC ip CIDR map to local and 0.0.0.0/0 route to your internet gateway

EC2

We will now setup our EC2 resources which will just consist of a new security group for our application. Go back to the console and head over to the EC2 page.

Security Group

We need 2 security groups for our application. The first one will be for our ELB/targets and the second will be for our RDS instance.

Application Security Group

From the left bar select Security Groups which will be under the Network & Security heading. After that hit the Create security group button.

You can name the security group anything, I chose app_name-web
Select your VPC
Outbound rules can be simple: All Traffic : 0.0.0.0/0
Inbound rules
1. Make sure to lock it to HTTP and HTTPS traffic only

RDS Security Group

Create another security group following the same steps as above with a few adjustments.

You can name the security group anything, I chose app_name-db to distinguish that this is for RDS
Select your VPC
Outbound rules can be simple: All Traffic : 0.0.0.0/0
Inbound rules
1. Use Protocol TCP
2. Enter the port that you will use for the RDS instance, default Postgres port is 5432
3. Source should be the security group id from our application security group, app_name-web

This will lock down the RDS instance to just the instances within our security group for the application.

RDS

The last thing we need to setup before we create our Elastic Beanstalk environment is the database. RDS provides a free tier for most of the databases offered. From the main console page, go to the RDS page.

From the RDS page, hit the Create database button
Select Standard Create
Select desired DB type, I went with PostgreSQL
Make sure to select Free Tier under Templates unless you are creating an actual Production stack
Under the Settings section, pick your database name, master username and password
Leave DB instance size as default
Leave Storage as default
In the Connectivity section make sure to select your VPC (not the default unless you want to)
1. Expand the Additional connectivity configration area
2. Make the DB publicly accessible (note: this is a quality of life thing if we want to connect from our local machine to the DB, it will be locked down via the security group)
3. Remove the Default VPC security group and instead select the RDS security group we created earlier (ie, app_name-db)
4. Pick any availability zone, ideally one of the ones that we selected for the subnets earlier
5. Pick a database port, or use the default one for Postgre (5432)
Scroll to the bottom and hit the Create database button

Elastic Beanstalk

Now we get to the final part of our AWS setup, which is to create our Elastic Beanstalk environment and application. All the pre-work we did up to the point will make the configuration very straight forward.

Environment & Application

Go to the Elastic Beanstalk page
From the left navbar, select Environments
Create a new Environment
1. This will also create a new application at the same time
2. Create a Web server environment and hit Select
3. Enter an application name and description
4. Enter an environment name and let EB generate your domain
5. Select Managed platform and use Platform: Ruby, Platform branch: Ruby 2.7 running on 64bit Amazon Linux 2, and Platform version: 3.0.2
6. Use the Sample application for now, we will push our code eventually
Hit the Create environment button

For the next sections, simply go to your application in beanstalk and click on the environment name. This will take you to a page that looks like this:

Beanstalk Configuration - Software

Go to the Software section of the EB configuration section
Under the Environment Properties section make sure the following environment variables are set
1. BUNDLER_DEPLOYMENT_MODE = true
2. BUNDLE_WITHOUT= (blank value)
3. NODE_ENV = production
4. RACK_ENV = production
5. RAILS_ENV = production
6. RAILS_SKIP_ASSET_COMPILATION = false
7. RAILS_SKIP_MIGRATIONS = false

Beanstalk Configuration - Instances

Go to the Instances section of the EB configuration section
Under the EC2 security groups section select the web security group we created earlier along with the default EB security group that is already checked
1. The security group that should be added is app_name-web
Leave everything else as default values

Beanstalk Configuration - Capacity

This step is completely optional and you can skip this if you do not want to use an ELB.

Go to the Capacity section of the EB configuration section
Under the Auto Scaling Group section
1. Swap environment type to Load balanced
2. [Optional] Set max instances to desired max count if scale out desired
3. [Optional] Set Instance type to a larger host depending on your application needs, this will incur charges!
Under the Scaling triggers section
1. Update the scaling triggers to what your application needs or just leave it to the default values for now, we can change this at any time

Beanstalk Configuration - Load Balancer (ELB)

This step is completely optional and you can skip this if you do not want to use an ELB.

Go to the Load Balancer section of the EB configuration section
Under the Listeners section
1. Make sure you have at least an HTTP listener
2. Add in your HTTPS listener with your cert (you can use AWS ACM which will provision a free certificate)
Not much is required, you can also leave everything as default to have an ALB with an HTTP listener by default

Beanstalk Configuration - Network

This step is completely optional and you can skip this if you do not want to use an ELB.

Go to the Network section of the EB configuration section
Under the Load balancer settings section
1. Make your ELB Public
2. Select the subnets created earlier
Under the Instance settings section
1. Make sure to enable Public IP addreess on your instances so that the ELB will be able to send traffic to them
2. Enable all subnets here as well

Conclusion

Now our AWS stack is ready for us to deploy and iterate on top of! At this point we have created security groups, a VPC and subnets for it, configured the routing table and opened the VPC to the internet via an internet gateway, and lastly spun up an RDS database. With all of this and our elastic beanstalk environment, we can make changes and deploy our application with super ease, a simple call to eb deploy will get us up and running. We will launch our application in the next post in this series.

To check out the default application that elastic beanstalk has created for us, navigate to the endpoint that beanstalk provisioned to see the sample stack up and running.

Rails 6 Series

Rails 6 Series - Starting Rails 6 with Vue on AWS
Rails 6 Series - Setting up AWS resources
Rails 6 Series - Deploying to AWS
Rails 6 Series - Configuring Vue and Webpack

[Rails 6 Series] Starting Rails 6 with Vue on AWS

Mon, 01 Jun 2020 00:00:00 +0000

Posts in Series

Rails 6 Series - Starting Rails 6 with Vue on AWS
Rails 6 Series - Setting up AWS resources
Rails 6 Series - Deploying to AWS
Rails 6 Series - Configuring Vue and Webpack

I recently started a personal project and wanted to try out Rails 6 with Vue (and Webpack). I was able to get this stack working locally with relative ease. However, when I attempted to push this idea to production I was met with many roadblocks. My goal was to use AWS to host the project. AWS has migrated to using “Amazon Linux 2” which has resulted in a lot of outdated tutorials and code snippets. Furthermore, I also found a lack of conscise and up to date documentation for pushing Rails 6 with Webpack to production.

I will be documenting my journey on pushing my personal project to AWS through a series of posts to hopefully help with the lack of documentation. To kick the series off, I will note down which resources I will be using:

Ruby 2.7
Rails 6
Vue.js
Webpack
AWS
- EB (Elastic Beanstalk)
- RDS (Postgresql)
- EC2 (VPC/Security Group setup)
- [Optional] ELB (Elastic Load Balancing)

For the most part we can get by using the free tier on AWS. However, because I will be using this to document my personal project some resources may not actually fall into the free tier. I will call these out as optional items or provide an alternative that is within the free tier.

Getting Started

For the purpose of this series, I will not go into the details on how to create an AWS account. If you do not have an account already I would highly advise just following the official guide.

Install Ruby

To install ruby locally there are several tools available that let you install multiple versions of ruby. We will use RVM.

First install RVM: https://rvm.io/rvm/install. Once that is complete proceed with installing the latest ruby (2.7.1 at the time of this post).

rvm install ruby-2.7.1

In case you have multiple versions of ruby installed, you can switch to using the one we just installed with the following command:

rvm use ruby-2.7.1

You can see if you have other ruby versions installed as well with rvm list and confirm which one is currently being used:

> rvm list
   ruby-2.4.0 [ x86_64 ]
   ruby-2.4.3 [ x86_64 ]
 * ruby-2.6.3 [ x86_64 ]
=> ruby-2.7.1 [ x86_64 ]

# => - current
# =* - current && default
#  * - default

Install Rails

Installing rails should be fairly straight forward. Simply run gem install rails and then you can call rails from your terminal.

Confirm you have rails installed

rails --version
Rails 6.0.3

Install Elastic Beanstalk

Elastic Beanstalk is a deployment orchestration engine provided by AWS. You can configure beanstalk to launch databases in RDS, spin up ELB’s, and launch the EC2 instances that will host your application. There are other options to perform this but for ease we will rely on AWS elastic beanstalk for the heavy lifting. The beanstalk binary is not part of the AWS SDK so we will have to install it separately. In order to get the CLI in your environment, AWS provides a nice automated installer.

Bootstrap Project

Now that we have installed the prerequisites, let’s go ahead and create the project in our local environment. We wil be calling the rails command and telling it to use Vue with webpack and Postgres as our database. If you wish to either avoid RDS or use another database type, feel free. One option is to leverage sqlite but the catch is that this will run on the EC2 host itself. Having said that, we will be using the free tier with RDS.

rails new app_name --webpack=vue --database=postgresql
cd app_name
bundle install

Conclusion

At this point we created our local environment and installed the necessary binaries. The next post will dive into setting up AWS resources including the beanstalk environment, RDS instance, and VPC/Security Groups.

Rails 6 Series

Rails 6 Series - Starting Rails 6 with Vue on AWS
Rails 6 Series - Setting up AWS resources
Rails 6 Series - Deploying to AWS
Rails 6 Series - Configuring Vue and Webpack

Github search api

Sun, 31 Jan 2016 00:00:00 +0000

I recently created the pr_status gem to be able to easily get the number of pull requests I opened in the current month. The inspiration came from being able to track my progress to my commit of opening 2 pulls. I should hopefully sometime soon be able to use this gem and officially keep track of my goal and progress each month.

I found the documentation for the api a bit lacking as I was initially following the url structure provided. Something like the following:

# did NOT work!
@octokit.search_issues('author:zsyed91+type:pr')

It turns out that the values are URI escaped and so using + is not needed. A simple space is sufficient. The full api documentation is available here.

Find issues and pull requests by author

This will return all issues and pull requests opened by the author.

@octokit.search_issues('author:zsyed91')

Find pull requests only

This will return all pull requests opened by the author.

@octokit.search_issues('author:zsyed91 type:pr')

Find pull requests opened after a certain date

The date fields follow the yyyy-mm-dd format. This is how to get all pull requests opened by the author on or after January 1st, 2016.

@octokit.search_issues('author:zsyed91 type:pr created:>=2016-01-01')

In order to exclude the starting date simply change >= to >.

@octokit.search_issues('author:zsyed91 type:pr created:>2016-01-01')

In order to get all pull requests opened on a specific date remove the > or >= qualifier.

@octokit.search_issues('author:zsyed91 type:pr created:2016-01-01')

Find pull requests by language

In order to get pull requests opened for a specific language for a certain user do the following:

@octokit.search_issues('author:zsyed91 type:pr language:ruby')

Conclusion

The gist of this post is just to show some examples on how to actually use the Octokit::Client api for search_issues. As stated before, the full api options can be found here. Keep in mind the api rate limit as of January 2016 is 10 requests per minute for unauthenticated requests and 30 requests per minute for authenticated requests.