Optimizing database queries

by

rails_eager_loading_large

Most projects you will work on will have a database of some sort. For this reason, optimizing your queries allows you to use all that the database provides without having your application try to reinvent the wheel. In this blog, I’ll be using Postgres and Ruby on Rails to demonstrate some ways that you can optimize things. Much of the second section is Postgres specific, but the concepts should apply to whatever ORM you are using. Hopefully by the end you will have learned some new tricks to improve your project.

EMR Interoperability and the cost of Healthcare

by

maxresdefault

Healthcare in the United States costs far more than in any other country in the world, and yet the quality of the care Americans receive is rated 11th among first world countries. The high price of healthcare burdens employers, increases the national debt, and historically has left many without access to healthcare. We all want cheaper, better healthcare, but it is difficult to agree on specifics of how to get there. Why is healthcare so expensive? 

Release Practices at Grio

by

launch

In a previous blog post, I covered the “textbook” definition of continuous integration, along with a handful of tools and practices that fulfill or help to fulfill said definition. These tools and practices include breaking up your app into components (e.g. front-end and back-end, or, for much larger projects, using microservices), utilizing “watch” utilities locally to iteratively run tests, and choosing test-oriented frameworks (e.g. Rails, Django, Grails, etc.). However, I didn’t talk much about any specific continuous integration setup, nor some of the third-party services that go together to make an efficient release process. I also didn’t talk much about how continuous integration fits into the larger cycle of deployment and release management. I aim to cover some of those topics here, and fill in the larger picture of how CI helps to ensure code quality and stability in a software project.

Much like projects themselves, release processes can be more or less complex depending upon the requirements of the project and problem domain. Adapting a release workflow depends on a number of factors, including the complexity of the project, team size, and stage of the product. For example, for one of our larger clients, Rivals.com, we use a combination of more than a half-dozen third-party services and custom tools to manage the full release cycle. Rivals is a large, high-traffic, well-established product with a comparatively large development team (currently eight developers on the web team alone). Insight into the changes that are going into any given release is critical for site reliability purposes, and orchestrating the activity of this many team members is more involved than with a solo or dual development team. I’ll contrast Rivals’ setup with the one we are using for our internal billing system, Moment. On Moment, anywhere from one to two developers work at a time on a smaller scale, beta product. At this stage in the product cycle, fast iteration takes precedence over change management.

Rivals.com

As mentioned above, Rivals is a larger project, in terms of complexity, team size, and userbase. As a result, agility needs to be carefully balanced with site reliability. What this means in practice is that in addition to the usual CI/CD tools (i.e. auto-triggered test builds and deployments), we also use a handful of dedicated code quality and change management tools.

Pivotal Tracker

Pivotal Tracker is one of the first and last stops in the release cycle. Tasks are created here and organized by priority into sprints, and then closed out as work is completed. Though this can be managed via the Tracker web UI, we prefer to change task states by tagging commits with a Tracker ID, i.e. [1234567] or [finished 1234567]. Tagging them in this way allows us to auto-close issues, and also allows us to determine which tasks will eventually be included in the release, using a custom tool we’ll talk about a little later.

Github

At Grio, like many agencies, we rely on Github for version-control. However, Github additionally supports a ton of third-party integrations for triggering various builds and analyses in response to new commits. On Rivals, for example, Github notifies Snap CI of new commits in order to run various test suites, Code Climate in order to lint and perform analyses on the code, and Pivotal Tracker in order to start/finish tasks.

Snap CI and Code Climate

As mentioned above, Snap CI and Code Climate are both configured to perform automated test runs and code analyses in response to Github webhooks. Github allows for strict enforcement of these integrations (i.e. by disabling the merge button on a pull request until all of the checks pass). We do not enforce these checks to this degree on our project, but the vast majority of our PRs do pass these checks, or are modified until they do. One of the critical tasks performed by Snap CI is build tagging; after each successful build, the corresponding commit is tagged along the lines of acceptance-[date-string]. We use these tags to find a good build prior to a release, as described below.

Release Notes Generator

After some number of cycles of committing, testing, analyzing, and merging (typically every half to full sprint, i.e. about twice weekly), we are ready to perform a release. In preparation, we use a custom Rake task to manually generate release notes in the form of a simple HTML page. By reading from the git history and Pivotal Tracker, this Rake task lists all of the commits and Tracker stories between a given acceptance tag and the most recent production tag. (Production tags are generated automatically after each successful deployment.) Once a tag is identified that contains only accepted stories, we run an automated deployment script to push the latest version of the app to Pivotal Web Services.

New Relic and Airbrake

Though only tenuously related to the deployment itself, New Relic and AirBrake are nonetheless indispensable parts of the full release cycle. During and after each release, we monitor New Relic and AirBrake for performance and stability data. For example, using New Relic, we may determine that a new controller method has a bottleneck in a particular query, and prioritize a performance optimization story for the next sprint. Likewise, AirBrake can auto-generate bug tickets in Tracker in response to errors, which are then prioritized accordingly.

Moment – Time-tracking Software

In contrast to Rivals.com, on Moment we use only a handful of tools as part of our release process. Jira is used instead of Pivotal Tracker, albeit without webhooks for automatic update of task state (i.e. started/finished). There is less velocity and a smaller team size, so this manual overhead is acceptable. Github once again acts as the communication hub for the project (pun intended), triggering automated test runs. In contrast to Rivals, we use Codeship for testing purposes, though this is equivalent to Snap CI for purposes of this blog post (a comparison of these CI tools is worthy of a blog post unto itself). Prior to release, we determine the contents of a release by manual inspection, i.e. reading over the most recently completed stories in Jira. On a larger project (in terms of either team size or traffic), this degree of manual intervention would be suboptimal, but, again, is acceptable for this stage in Moment’s lifespan. Deployment is handled on Heroku via their simple git push mechanism. New Relic is not yet integrated, though we’ll be adding this before too long. I mention New Relic here only to point out that it is not a strict requirement of a release setup. This is arguably a very abbreviated description of the release process on Moment, but hopefully highlights the degree of flexibility that is possible across projects of different sizes.

Takeaways

Adapt Process to Project

Process can and should adapt to a project’s lifecycle. Enforcing too much process early on in a project’s lifespan can have an adverse affect on velocity, and may also enforce practices that are ill-suited or wasted from the perspective of the product’s final state. Certainly, an issue tracker, version control, automated testing, and an automation-friendly hosting solution are essential. Beyond that, process should grow, shrink, and otherwise adapt as needed.

Leverage Third-Party Services

There is a multitude of off-the-shelf solutions for common automation, release, and deployment problems. Some of these are mentioned above (Github, Snap CI, Codeship, etc.), and numerous others that may be better suited to the financial and technical requirements of a given project. These should at least be considered before choosing a potentially more costly self-hosted or custom solution. Taking this approach can provide considerable cost and time savings for the purposes of getting an MVP to market. For a small startup, these delays and costs can mean the difference between succeeding versus never getting past the starting line.

Into the Core…OS

by

coreos-wordmark-horiz-color

At some point in March I received an email stating that I had roughly $40 in DigitalOcean (referral link – get $10 credit!) credit which was going to expire on the first of May. I wanted to do something cool, learn new things, and leverage a large portion of my remaining credits.

My initial plan was to spin up a CoreOS cluster, as it is something that I have had my eye on. Once running I would get Kubernetes running for management/orchestration of deployed applications. During this journey I learned a lot of new things, had a lot of fun, and even got a cool cluster running. However, I never made it to the end goal of Kubernetes in time for my presentation (and this post). Looking back, I don’t consider this to be defeat: I learned a lot of new concepts along the way. I can be certain that I will leverage and use that knowledge in the future too!

The Importance of Good Posture for Software Developers

by

A photo of a cartoon man with symptoms of iHunch.

Posture, defined as the position of one’s body while sitting or standing, is something most developers take for granted. The cliche of the programmer hunched over their desk typing away on their laptop is alive and well in many offices. What, exactly, is the problem with poor posture? Is the job necessarily tied to poor back and neck health? What are the long term effects of poor posture? Is there a way to avoid pain down the road?

Tech and Education Today

by

Tech in Education

Looking into a classroom today, things might not look much different than they did 10+ years ago. But if you take a closer look, amongst the books and desks, there are computers, smart boards, and iPads. Technology has always had an influence on education, and in the digital age of today, modern tech companies and education institutions are working together to build effective learning tools.