2022 In The Bag

To say: 'it's been a bit of year', would be more than an understatement. However...

It's been a bit of a year

A Year of Rails

For all the folks who don't know, I spent the year transitioning from Glossier to Orbit, having Orbit implode financially, racing to find and land a gig before the end of 2022, and finally finding a space at PayNearMe right before 2022 slammed shut. With Orbit and PayNearMe both being large Rails apps. Honestly, if in 2021 I was asked what technology I would be using in 2023, Rails would not have been my answer. Not even in the top 5...

Throughout most of 2020 and 2021 Glossier aimed to transition their own monolith to microservices. This work was done by opting into the growing serverless world offered by AWS. Ultimately, many of those products never saw the light of day. And eventually the organization opted to replatform to Shopify instead. Having worked on and even lead some of those initatives, the world of serverless was top of mind.

The Rails market though has exploded in the last couple of years. Making a comeback that reminds me of the late 2012's throughout 2015. Most of what a successful job search looks like (more on this soon) is managing relationships with recruiters. Those recruiters right now have large holes in their Ruby or Rails pipelines. The flip side of this is the dearth of Serverless roles at the moment.

Serverless maybe something that's talked about widely by internal team members but, there's little of it talked about (openly) outside oranizations. Or rather, outside the echo chambers that are HackerNews and IndieHackers. Doubly so when chatting about specific roles with hiring managers or internal recruiters. Even bringing up concepts that apply to well architected applications lands flat with all but, the most advanced engineers in interview loops.

Walking into a new Rails gig isn't as easy as it was even 5 years ago. Although most folks think Rails applications follow some kind of standard template, many are their own little unique snowflake. Sometimes with sharp edges all over the place. After working on three different and 'hefty' Rails apps in 2022, the following are 'a few tidbits' of what others can expect if they find themselves working on Rails in the future... Or today.

Seriously Barry, how much is a liter?

APM Metrics (logs don't count) are hit or miss when it comes to most apps. Some may have pooly implemented systems, others have several systems that compete with one another, and others may have no way at all of collecting runtime metrics. None of this depends on the age of the application, the age of the engineers, or even the number of senior engineers on the team.

No, sadly metrics as well as corresponding alarms and alerting, are indeed: a crap-shoot. If, for some reason the application is generating relevant metrics, the resulting charts and graphs around these metrics will be largly made up of whatever defaults the APM tool of choice selected. Because this work tends to be fairly time consuming, any proposed changes to a 'working' implementation comes across as impossible to sell. All of it leaves on call members fighting to figure out what alarms are relevant, or simply snoozing them until thresholds fall back to normallity.

The scarriest scenario of all is you find yourself at a company that has no tooling on this front. None. Zilch, Nada. It can only get moderatly worse if the company makes money. A rather odd man named: "Jeremy Clarkson" put it best when he was meandering about French and Italian cars (1:19 through 1:34)

It's honestly surprising how bad I have seen it in this domain with Rails over the past 4 to 5 years. The reason behind this is likely due to the complexity driven by competing cloud services. This space though is inherently hard and SRE's, good ones, are hard to come by. Not to mention hard to hire full time for most startups. Coupling this with a space that most developers actively avoid, leads to a recipie for oncall disasters.

Power Blackout!

Documentation in any project takes a thoughtful amount of time. Not just today but, tomorrow, the next day, and certainly every day afterward. This type of documentation could be in a Wiki, a Readme, inline with code, or an external tool. However, guarenteed, if you sit down at most Rails jobs today, that documentation will be spectacularly out of date. If you are lucky it refrerences the correct database version. What's much more likely though is a host of missing infrastructure requirements, out of date setup 'best practices', and a Readme that's a jumbled mess of 18 sections of which no developer has read in its' entirety in the past 6 months.

I am not entirely sure if it's the productivity that Ruby affords or if many Rubyists still think that all of their code is self documenting. However, the appauling state of most Ruby and Rails project's documentation would shock and awe almost all developers. Perhaps all of this stems from the whole Yard Vs Rdoc debacle but, it's highly unlikely. A much more likely scenario is: Ruby and likely more Rails developers haven't yet comprehended that documentation is a force lever.

It's been interesting being around > 3 larger Rails apps in 2022 as well as quit a few smaller Ruby apps. Documentation in this realm sits at two dipolar ends of the spectrum. It's either the most well thought out and hierarchal brilliant piece of prose ever... Or the documentation is so borderline useless, it's better off deleted. Sadly I have never seen an 'in-between' or grey area. Due to this, thoughtful approaches to this problem should be top of mind for all Rubyists entering a new role or kickstarting a new startup.

Maybe A Mutiny Of Clowns

Most... Perhaps even all Rails apps are shaped different than what most toy Rails apps look like. This is always a bit of a surprise since one of the core concepts of Rails is to keep to a 'strict' set of conventions. However, engineers will be engineers and customizations will creep in. The three forms of intense customizations that have been most prevalant are:

  1. The Hidden Gem
  2. The Layered Onion
  3. The Method Missing Addict

The Hidden Gem

Everything, and I do mean EVERYTHING is a gem. Perhaps there's a need to sell goods, pull in Solidus. Need a CMS? There's a gem for that! Building a feature to take on the complexity of another gem? MAKE it a Gem!

It's almost as if the whole NPM debacle never permeated back to the Ruby community. That or (and this is highly likely) this is some form of modularization or 'services' to some Rubyists. Whatever it is, it almost never is created due to a concrete boundary in the code. In addition, the idea is usually created by some newer engineer who's impetus is to avoid learning the existing system.

Regardless of the intent, the result is a system that has a blurry set of business rules locked behind doors that no one has figured out how to open. Yes, Solargraph isn't bad, nor is Syntax Suggest or Syntax Tree. The amount of developers though who know how to wrangle these powertools, is far and fleeting.

The Glass Onion

Rack middleware is at the heart of the Rails framework. Due to this, developers are free to build and add in their own middleware layers. However, like most things that come with 'great power', many apps fall prey to embaracing an overload of middleware layers. Many of these layers reach into data stores, either locally or in the 'cloud'.

This is where these approaches begin to break down. If (for some reason) some value can't yet write to S3, why should your entire application fail to render? Sure, there are error handling techniques to mitigate this but, if a team is moving fast, they are unlikely to build them in.

The Method Missing Addict

Lastly, the most common type of Rails application that one may run into, is the one where no one heeded the warning about metaprogramming.

Every class and concern is peppered with method_missing definitions. Leading to an impossibly-large and blurry boundary between objects that represent the system. On top of classes that respond to almost any message, the objects themselves are so God like that it's impossible to discern their actual shape.

Domain modeling and design maybe a core part of Object Oriented Programming but, it's fallen out fashion as of late. Doubly so in the Rails community. Without this, many larger apps tend to fall down with the weight of a few bloated classes. The anthesis of this would be a constellation of well defined classes that are small and focused.

Well designed (and documented) systems lead to productive engineers that push features successfully. Systems that are hard to understand not only jeapordize engineer's productivity, it compounds within the organization with each new hire.

Grover Cleveland called, He Wants His Watch Back

Every organization that has a rather large Rails app or perhaps multiple 'mini-monoliths', has a wizard on payroll. This human has likely been around the organization for awhile but, they definitely have the power to wield any and all of Ruby's DSLs. If RSpec is in play, it's almost guarentted the human is a wizard at that.

The preverbial RSpec wizard is as common as the household fly. They know every esoteric edge case when it comes to the absolute shit show that is alternative spec frameworks. Confused and need to know how to deeply test a complext JSON object with compound matchers? Grab some time to pair with your local wizard because, everyone claims that they (and only they) have an answer.

These folks are great at generating green dots but, pin them down to speed up a test suite, run one effectively on CI, or even kickstart one for a new application and they usually come up exceptionally short.

If your organization has one of these individuals, take the time to learn what their kryptonite is. Often times they have an alergic reaction to more normal items in the development workflow. Either parts of Git, newer norms in development like 'UUIDs', or even simple ideas like decoupling complexity using POROs and small classes. Just like with metrics and performance sampling, if an organization that you work at employs and upholds the nonsense of said wizard...

Danger Zone!

2023 is going to be huge...

For Rails. Beyond the aforementioned insanity, the Rails framework still is a great way to kickstart almost anything. Sure, apart from deployments (it's still a monster), getting literally anything to an MVP and to any type of investment, is still easier than figuring out what a Dockerfile is for.

With that, here's to another spectacular year zooming by wearing Ruby tinted glasses