matduggan.com

Modern infrastructure work is, by many measures, better than it has ever been. We live in a time when a lot of the routine daily problems have been automated away by cloud providers, tooling or just improved workflows. However in the place of watching OS upgrades has come the endless tedium of writing configuration files. Between terraform, Kubernetes, AWS and various frameworks I feel like the amount of time I spend programming gets dangerously low.

I'm definitely not alone in feeling the annoyance around writing configurations, especially for Kubernetes. One tool every business seems to make at some point in their Kubernetes journey is a YAML generator. I've seen these vary from internal web apps to CLI tools, but the idea is that developers who don't have time to learn how to write the k8s YAML for their app can quickly generate the basic template and swap out a few values. You know people hate writing YAML for k8s when different companies on different continents all somehow end up writing variations on the same tool.

These internal tools vary from good to bad, but I'm hear to tell you there is a better way. I'm going to tell you about what worked well for me based on size of the company and the complexity of how they were using k8s. The internal tools are typically trying to solve the following common use-cases:

• I need to quickly generate new YAML for launching a new service
• I have the same service across different clusters or namespaces for a non-production testing environment and a production environment. I need to insert different environmental variables or various strings into those files.

We're going to try and avoid having to write too much internal tools and keep our process simple.

Why would I do any of this vs use my internal tool?

My experience with the internal tools goes something like this. Someone writes it, ideally on the dev tools team but often it is just a random person who hates doing the research on what to include in the YAML. The tool starts out by generating the YAML for just deployments, then slow expands the options to include things like DNS, volumes, etc. Soon the tool has a surprising amount of complexity in it.

Because we've inserted this level of abstraction between the app developer and the platform, it can be difficult to determine what exactly went wrong and what to do about it. The tool is mission critical because you are making configs that go to production, but also not really treated that way as leadership is convinced someone checks the files before they go out.

Spoiler: nobody ever checks the file before they go out. Then the narrative becomes how complicated and hard to manage k8s is.

Scenario 1: You are just starting out with k8s or don't plan on launching new apps a lot

In this case I really recommend against writing internal tools you are going to have to maintain. My approach for stacks like this is pretty much this:

1. Go to https://k8syaml.com/ to generate the YAML I need per environment
2. Define separate CD steps to run kubectl against the different testing and production namespaces or clusters with distinct YAML files
3. Profit

One thing I see a lot of with k8s is over-complexity starting out. Just because you can create all this internal tooling on top of the platform doesn't mean you should. If you aren't anticipating making a lot of new apps or are just starting out and need somewhere to make YAML files that don't involve a lot of copy/pasting from the Kubernetes docs, check out k8syaml above.

Assuming you are pulling latest from some container registry and that strings like "testdatabase.url.com" won't change a lot, you'll be able to grow for a long time on this low effort approach.

Here's a basic example of a config generated using k8syaml that we'll use for the rest of the examples.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: Testing
  labels:
    app: web
spec:
  selector:
    matchLabels:
      octopusexport: OctopusExport
  replicas: 1
  strategy:
    type: RollingUpdate
  template:
    metadata:
      labels:
        app: web
        octopusexport: OctopusExport
    spec:
      volumes:
        - name: test-volume
          persistentVolumeClaim:
            claimName: test-claim
      containers:
        - name: nginx
          image: nginx
          ports:
            - containerPort: 80
          env:
            - name: database
              value: testdata.example.com
      affinity:
        podAntiAffinity:
          preferredDuringSchedulingIgnoredDuringExecution:
            - weight: 100
              podAffinityTerm:
                labelSelector:
                  matchExpressions:
                    - key: app
                      operator: In
                      values:
                        - web
                topologyKey: kubernetes.io/hostname

Scenario 2: You have a number of apps and maintaining YAML has started to turn into a hassle

This is typically where people start to over-complicate things. I advise folks to try and solve this problem as much as you can with the CI/CD process as opposed to attempting to build tooling that builds configs on the fly based on parameters passed to your tools CLI. It's often error-prone and even if it works now it's going to be a pain to maintain down the line.

Option 1: yq

This is typically the route I go. You create some baseline configuration, include the required environmental variables and then insert the correct values at the time of deploy from the CD stack by defining a new job for each target and inserting the values into the config file with yq. yq like the far more famous jq is a YAML processor that is easy to work with and write scripts around.

There are multiple versions floating around but the one I recommend can be found here. You will also probably want to set up shell completion which is documented in their help docs. Here are some basic examples based on the template posted above.

yq eval '.spec.template.spec.containers[0].name' example.yaml returns nginx. So in order to access our environmental variable we just need to run: yq eval '.spec.template.spec.containers[0].env[0].value' example.yaml. Basically if yq returns a - in the output, it's an array and you need to specify the location in the array. It's a common first-time user error.

So again using our example above we can define the parameters we want to pass to our YAML in our CD job as environmental variables and then insert them into the file at the time of deployment using something like this:

NAME=awesomedatabase yq -i '.spec.template.spec.containers[0].env[0].value = strenv(NAME)' example.yaml

Ta-dah our file is updated.

  containers:
    - name: nginx
      image: nginx
      ports:
        - containerPort: 80
      env:
        - name: database
          value: awesomedatabase

If you don't want to manage this via environmental variables its also equally easy to create a "config" YAML file, load the values from there and then insert them into your Kubernetes configuration. However I STRONGLY recommend adding a check to your CD job which confirms the values are set to something. Typically I do this by putting in a garbage value into the YAML and then checking inside the stage to confirm I changed the value. You don't want your app to fail because somehow an environmental variable or value got removed and you didn't catch it.

yq has excellent documentation available here. It walks you through all the common uses and many less common ones.

Why not just use sed? You can but doing a find replace on a file that might be going to production makes me nervous. I'd prefer to specify the structure of the specific item I am trying to replace, then confirm that some garbage value has been removed from the file before deploying it to production. Don't let me stop you though.

Option 2: Kustomize

Especially now that it is built-into kubectl, Kustomize is a robust option for doing what we're doing above with yq. It's a different design though, one favoring safety over direct file manipulation. The structure looks something like this.

Inside of where your YAML lives, you make a kustomization.yaml file. So you have the basics of your app structure defined in YAML and then you also have this custom file. The file will look something like this:

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

resources:
  - service.yaml
  - deployment.yaml
  - test.yaml

Nested as a sub-directory under your primary app folder would a directory called sandbox. Inside of sandbox you'd have another YAML, sandbox/kustomization.yaml and that file would look like this:

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
bases:
- ../../base
patchesStrategicMerge:
- test.yaml

So we have the base file test.yaml and the new test.yaml. Here we can define whatever changes we want merged into the top-level test.yaml at the time of running. So for example we could have a new set of options:

apiVersion: autoscaling/v2beta2
kind: HorizontalPodAutoscaler
metadata:
  name: backend-test
spec:
  minReplicas: 2
  maxReplicas: 4
  metrics:
  - type: Resource
  resource:
    name: cpu
    target:
      type: Utilization
      averageUtilization: 90

Then when we deploy this specific option we could keep it simple and just run kubectl apply -k  overlays/sandbox. No need for manually merging anything and it is all carefully sorted and managed.

This is really only scratching the surface of what Kustomize can do and it's well worth checking out. In my experience though dev teams don't want to manage more YAML, they want less. However spending the time breaking configuration into these templates is both extremely safe (since we're distinctly calling the configuration we want merged in) and has minimal tooling overhead, it's worth exploring.

I thought you said you use yq yeah I do but I should probably use Kustomize. Typically though when apps start to reach the level of complexity where Kustomize really shines I start to push for option 3, which I think is the best of all of them.

Option 3: Use the client library

As time goes on and the complexity of what folks end up wanting to do in k8s increases, one issue is that the only way anyone knows how to interact with the cluster is through INSERT_CUSTOM_TOOL_NAME. Therefore the tool must be extended to add the support for whatever new thing they are trying to do, but this is obviously dangerous and requires testing by the internal dev tools person or team to ensure the behavior is correct.

If you are an organization who: lives and dies by Kubernetes, has an internal YAML generator or config generator but am starting to reach the limit of what it can do and don't want to spend the time breaking all of your environments into Kustomize templates, just use the official client libraries you can find here. Especially for TDD shops, I don't know why it doesn't come up more often.

I've used the Python Kubernetes client library and it is great, with tons of good examples to jumpstart your work. Not only does it open up allowing you to manage Kubernetes pretty much any way you would like, but frankly it's just a good overview of the Kubernetes APIs in general. I feel like every hour I've spent writing scripts with the library and referencing the docs, I learn something new (which is more than I can say for a lot of the Kubernetes books I've read in my life....). You can check out all those docs here.

Conclusion

The management of YAML configuration files for Kubernetes often comes up as a barrier to teams correctly managing their own application and its deployment. At first the process of writing these configs is an error-prone process and it is only natural to attempt to leverage the opportunity k8s provides to make an automated approach to the problem.

I would encourage folks starting out in k8s who have encountered their first headache writing YAML to resist the urge to generate their own tooling. You don't need one, either to make the base config file or to modify the file to meet the requirements of different environments. If you do need that level of certainty and confidence, write it like you would any other mission critical code.

Finding new CIDR blocks in a VPC can get increasingly time consuming as you grow. I found a nice little tool which will quickly output to STDOUT the available ranges. You can find that here.

You can install it with: pip install aws-cidr-finder

Run with: aws-cidr-finder --profile aws-profile-name

Output looks like this:

CIDR               IP Count
---------------  ----------
172.31.96.0/19         8192
172.31.128.0/17       32768
Total                 40960

New AWS blueprint project gives you a much better new EKS cluster experience. After reading the terraform they provide, this should be the way we all do it from now on. You can find the new project here.

I know I'm probably the last one to find this out, but this has been an issue for a long time for me. I've been constantly building and storing SQL files in the actual container layers, thinking that was the correct way to do it. Thankfully, I was completely wrong.

Here is the correct way to do it.

Assuming you are in a directory with the SQL files you want.

FROM postgres:12.8

COPY *.sql /docker-entrypoint-initdb.d/

On startup Postgres will load the SQL files into the database. I've wasted a lot of time doing it the other way and somehow missed this the entire time. Hopefully I save you a bunch of time!

For Mongo

FROM mongo:4.2

COPY dump /docker-entrypoint-initdb.d/
COPY restore.sh /docker-entrypoint-initdb.d/

The restore.sh script is below.

#!/bin/bash

cd /docker-entrypoint-initdb.d || exit
ls -ahl
mongorestore /docker-entrypoint-initdb.d/

Just do a mongodump into a directory called dump and then when you want it, load the data with the restore.sh script inside the container.

I was talking to a friend recently, a non-technical employee at a large company. He was complaining about his mouse making strange noises. Curious, I asked what he had been doing. "Oh well I need to fill in these websites every quarter with information I get off an Excel sheet. It takes about 30 hours to do and I need to click around a lot to do it." I was a bit baffled and asked him to elaborate, which he did, explaining that there was an internal business process where he got an Excel spreadsheet full of tags. He had to go into an internal website and make sure the tags on their internal app matched the spreadsheet.

"Surely you all have a development team or someone who can help automate that?" I asked, a little shocked at what he was saying. "No well we asked but they just don't have the spare capacity right now, so we're gonna keep doing it this year." He went on to explain this wasn't even the only one of these he had to do, in fact there were many web pages where he needed to manually update information, seemingly only possible through his browser. I thought of the thousands of people at just this company who are spending tens of thousands of hours copy and pasting from a spreadsheet.

When I was a kid and first introduced to computers this was exactly the sort of tedium they were supposed to help with. If you did all the work of getting the data into the computer and had everything set up, part of the sales pitch of a computer in every workplace was the incredible amount of time saved. But this didn't seem any better as compared to him pulling paper files and sorting them. What happened? Weren't we supposed to have fixed this already?

My First Automation

My first introduction to the concept of programming was a technology called HyperCard. HyperCard was a very basic GUI scripting program, which allowed you to create stacks of black and white cards. Apple gave you a series of GUI controls that you could drag and drop into the cards and you linked everything together with HyperTalk, a programming language that still structurally looks "correct" to me.

put the value of card field "typehere" into theValue is an example of a HyperTalk command, but you actually had a lot of interesting options available to you. This was my first introduction to the idea of basically prompting the user for input with commands like:

onmouseUp
answer "This is an example of the answer command" with "Reply 1" or -. "Reply 2" or "Reply 3"
endmouseUp

One of the things I enjoyed doing a lot was trying to script art, making it seem like I had drawn something on the computer that in fact had been done through HyperTalk. Those commands looked something like this:

choose spray can tool
drag from 300,100 to 400, 200 wait 1 second
choose eraser tool
drag from 300,100 to 400, 200 choose browse tool
endmouseUp

The syntax was very readable and easy to debug along with being very forgiving. HyperCard stacks weren't presented as complicated or difficult, they were supposed to be fun. What I remember about HyperCard vs the programming tools I would be exposed to later was it was impossible to get into an unrecoverable state. What I mean is backing out of a card while keeping the stack intact didn't require any special knowledge, a kid could figure it out with very little instruction.  

For those who never got the chance to try it out: check out this great emulator here.

Automator and AppleScript

Years went by and my interest in IT continued, but for the most part this didn't involve a lot of scripting or programming. In the late 90s and early 2000s the focus for IT was all about local GUI applications, trying to transition power tools out of the terminal and into a place where they looked more like normal applications. Even things like server management and LDAP configuration mostly happened through GUI tools, with a few PowerShell or Bash scripts used on a regular basis.

The problem with this approach is the same problem with modern web applications, which is its great for the user who is doing this for the first time to have all this visual information and guides on what to do. For someone doing it for the 5000th time, there's a lot of slowness inherit in the design. The traditional path folks took in IT was to basically abandon all GUI tooling at that point if possible, leaving people lower on the totem pole to suffer through clicking through screens a thousand times.

I was primarily a Mac user supporting other Mac users, meaning I was very much considered a niche. Apple was considered all but dead at the time, with IT teams preferring to silo off the few departments they couldn't force to adopt windows onto a junior helpdesk person. Mostly my job was setting up the odd Mac server, debugging why Active Directory logins weren't working, nothing too shocking. However it was here when I started to encounter this category of non-programmers writing programs to solve surprisingly important problems.

You would discover these Macs running in higher education or in design studios with a post-it note on it saying "Don't turn off". Inevitably there would be some sort of AppleScript and/or Automator process running, resizing files or sending emails, sometimes even connecting to databases (or most often using Excel as a database). I saw grading tools that would take CSVs, convert them into SQL and then put them into a database as a common example.

For those who haven't used Apple Automator, part of the reason I think people liked it for these tasks in Mac-heavy environments is because they're difficult to mess up. You drag and drop a series of commands into a Workflow and then run the Workflow.

After discussing it with users over the years, a similar story emerged about how these workflows came to exist.

1. Someone technical introduced the user to the existence of Automator because they started using it to automate a task.
2. The less-technical user "recorded" themselves doing something they needed to do a thousand times. The Automator record basically just captured what you did with a mouse and ran it again when you hit play.
3. Eventually they'd get more comfortable with the interface and move away from recording and move towards the workflows since they were more reliable.
4. Or if they did batch file processing, they'd go with a Folder Action, which triggered something when an item was placed in a folder.
5. At some point they'd run into the limit of what that could to and call out to either a Bash script or an AppleScript depending on whatever the more technical person they had access to knew.

Why do this?

For a lot of them, this was the only route to automation they were ever going to have. They weren't programmers, they didn't want to become programmers but they needed to automate some task. Automator isn't an intimidating tool, the concept of "record what my mouse does and do it again" makes sense to users. As time went on and more functionality was added to these homegrown programs, they became mission-critical, often serving as the only way that these tasks were done.

From an IT perspective that's a nightmare. I found myself in an uncomfortable place being asked to support what were basically "hacks" around missing functionality or automation in applications. I decided early on that it was at least going to be a more interesting part of my job and so I steered into it and found myself really enjoying the process. AppleScript wasn't an amazing language to write in but the documentation was pretty good and I loved how happy it made people.

For many of you reading I'm sure you rolled your eyes a bit at that part, how happy it made people. I cannot stress enough some of the most impactful programming I maybe have ever done was during this period. You are, quite literally, freeing people to do other things. I had teachers telling me they were staying late 5+ hours a week to do the things we could automate and run. This form of basic programming with the end customer right there providing you direct feedback on whether it works is incredibly empowering for both parties.

It was also very easy to turn into a feature request. Think of all the bad internal tooling feature requests you get, with complicated explanations or workflows that make no sense when they're written down. Instead you had the full thing, from beginning to end, with all the steps laid out. I noticed a much shorter pipeline from "internal request" to "shipped feature" with these amateur automations I think in part to the clarity of the request that comes hand-in-hand with already seeing how the request works end to end.

As time went on, less and less work happened in desktop applications. Platform specific tooling like Automator and AppleScript became less relevant as the OS hooks they need to operate aren't present or can't be guaranteed. You can't really depend on mouse placement on a web page where the entire layout might change any second.

Now we're back at a place where I see people often wasting a ton of time doing exactly the sort of thing computers were supposed to prevent. Endless clicking and dragging, mind-numbing repetition of selecting check-boxes and copy/pasting values in form fields. The ideas present in technology like HyperCard and AppleScript have never been more needed in normal non-technical peoples lives, but there just doesn't seem to be any tooling for it in a browser world. Why?

What about RPA?

RPA, or "robotic process automation", is the attempted reintroduction of these old ideas into the web space. There are a variety of tools and the ones I've tried are UIPath and OpenRPA. Both have similar capabilities although UIPath is certainly more packed with features. The basic workflow is similar, but there are some pretty critical problems with it that make it hard to put into a non-technical users hands.

For those unaware the basic workflow of an RPA tool is mostly browser-based, often with some "orchestrator" process and a "client" process. A user installs a plugin to their browser that records their actions and allows them to replay it. You can then layer in more automation, either using built-in plugins to call APIs or by writing actions that look for things like CSS tags on the page.

Here are some of the big issues with the RPA ecosystem as it exists now:

1. There isn't a path for non-technical users to take from "Record" to building resilient workflows that functions unattended for long periods of time. The problem is again that websites are simply not stable enough targets to really program against using what you can see. You need to, at the very least, understand a bit about CSS in order to "target" the thing you want to hit reliably. Desktop applications you control the updates, but websites you don't.
2. The way to do this stuff reliably is often through that services API but then we've increased the difficulty exponentially. You now need to understand what an API is, how they work and be authorized by your organization to have a key to access that API. It's not a trivial thing in a large workplace to get a Google Workspaces key, so it unfortunately would be hard to get off the ground to begin with.
3. The tools don't provide good feedback to non-technical users about what happened. You just need too much background information about how browsers work, how websites function, what APIs are, etc in order to get even minimal value out of this tooling.
4. They're also really hard to setup on your computer. Getting started, running your first automation, is just not something you can do without help if you aren't technically inclined.

I understand why there has been an explosion of interest in the RPA space recently and why I see so many "RPA Developer" job openings. There's a legitimate need here to fill in the gaps between professional programming and task automation that is consuming millions of hours of peoples lives, but I don't think this is the way forward. If your technology relies on the use of existing APIs and professional tech workers to function, then its bringing minimal value to the table over just learning Python and writing a script.

The key secret for a technology like this has to be that the person who understands the nuance of the task has to be able to debug it. If it doesn't work, if the thing doesn't write to the database or whatever, maybe you don't know exactly why that happened, but because you have some context on how the task is done manually there exists the possibility of you fixing it.

You see this all the time in non-technical workplaces, a "black box" understanding of fixes. "Oh if you close that window, wait five minutes and open it again, it'll work" is one I remember hearing in a courthouse I was doing contract work in. Sure enough, that did fix the issue, although how they figured it out remains a mystery to me. People are not helpless in the face of technical problems but it also isn't their full-time job to fix them.

Why not just teach everyone to program?

I hate this ideology and I always have. Human civilization progresses in part due to specialization, that we as a society don't have to learn to do everything in order to keep things functional. There's a good argument to be made that we possibly have gone too far in the other direction, that now we understand too little about how the world around us functions, but that's outside the scope of this conversation.

There is a big difference between "having a task that would benefit from automation" and "being interested in investing the hundreds of hours to learn how to program". For most people it has nothing to do with their skills or interest and would be something they would need to learn in their off-time. But I love to program, won't they? My parents both love being lawyers but I can't think of anything I would want to do less right now.

Giving people a path from automation to "real programming", whatever that means, is great and I fully support it. But creating the expectation that the only way for people to automate tasks is to become an expert in the space is selfish thinking on the part of the people who make those services. It is easier for us if they take on the cognitive load vs if we design stuff that is easier to use and automate.

Is there a way forward to help people automate their lives?

I think there is, but I'm not sure if the modern paradigm for how we ship website designs works with it. Similar to website scraping, automation based on website elements and tags is not a forever solution and it can be difficult, if not impossible, for a user to discover that it isn't working anymore without adding manual checks.

There are tools users can use, things like Zapier that work really well for public stuff. However as more and more internal work moves to the browser, this model breaks down for obvious reasons. What would be great is if there was a way to easily communicate to the automation tool "these are the hooks on the page we promise aren't going to go away", communicated through the CSS or even some sort of "promise" page.

If some stability could be added to the process of basically scraping a webpage, I think the tooling could catch up and at least surface to the user "here are the stable targets on this page". As it exists now though, the industry treats APIs as the stable interface for their services and their sites as dynamic content that changes whenever they want. While understandable to some extent, I think it misses this opportunity to really expand the possibility of what a normal user can do with your web service.

It's now become very much the norm for employees to get onboarded and handed a computer where the most used application is the web browser. This has enabled incredible velocity when it comes to shipping new features for companies and has really changed the way we think about the idea of software. But with this has come cost incurred on the side of the actual user of this software, removing the ability to reliably automate their work.

If there was a way to give people some agency in this process, I think it would pay off hugely. Not only in the sheer hours saved of human life, but also with the quality of life for your end users. For the people who found and embraced desktop automation technology, it really allowed normal people to do pretty incredible things with their computers. I would love to see a future in which normal people felt empowered to do that with their web applications.

An opinionated take on the tool I use the most

Like many of you, my terminal emulator is probably my most used piece of software. My day begins with getting a cup of coffee, opening up Slack and iTerm 2, my terminal emulator for years. iTerm 2 has an incredible number of features, almost too many to list. Here are some of my most-used features just off the top of my head:

• Hotkey global terminal dropdown, meaning I can get into the terminal from any application I'm in
• Really good search, including support for regex
• Paste history, which like come on who doesn't use that 100 times a day
• Password manager. With the death of sudolikeaboss I've come to rely on this functionality just to deal with the mess of passwords that fill my life. Also if you know a replacement for sudolikeaboss that isn't the 1Password CLI let me know.
• Triggers, meaning you can write basic actions that fire when text matching a regex pattern is encountered. Nice for when you want the icon to bounce in the dock when a job is done in a dock or when you want the password manager to automatically open when a certain login prompt is encountered.

With all this flexibility comes complexity, which smacks you in the face the second you open the Preference pane inside of iTerm 2. I don't blame the developers for this at all, they've done a masterful job of handling this level of customization. But I've seen new users jaw drop when they click around this preference pane:

I have very few complaints with iTerm 2, but I'm always open to try something new. Someone on Twitter told me about Warp, a new terminal emulator written in Rust with some very interesting design patterns. I don't know if its the right terminal for me but it definitely solves problems in a new way.

What makes a good terminal emulator?

This is a topic that can stir a lot of feelings for people. Terminal emulators are a tool that people invest a lot of time into, moving them from job to job. However in general I would say these are the baseline features I would expect from a modern terminal emulator:

• Control over color, people don't all have the same setups
• Tabs, they're great in browsers and even better with terminals
• Profiles. Different setups for different terminals when you are doing totally isolated kinds of work. I like a visual indicator I'm working in production vs testing, for instance.
• Access to command history through the tool itself
• Bookmarks, while not a must-have are nice so you don't need to define a ton of bookmarks in your bash profile.
• Notifications of some sort.
• Control over fonts. I love fonts, it's just one of those things.

So why am I reviewing a terminal emulator missing most of these features or having them present in only limited configurations? Because by breaking away from this list of commonly agreed-upon "good features" they've managed to make something that requires almost no customization to get started. Along the way, they've added some really interesting features I've never seen before.

I requested an invite on their site and a few weeks later got the email inviting me to download it. First, huge credit to the Warp team. I respect the hell out of software with an opinion and Warp has a strong point of view. The default for development tools is to offer options for everything under the sun and to see someone come to the conversation with a tool that declares "there is a right way to do this" is intriguing. Here is what you see when you open warp:

From launch it wants you to know this is not your normal terminal emulator. It is trying to get you to do things the warp way from minute 1, which is great. The Command Palette is a lightning fast dropdown of all the Warp commands you might need. Search commands is just bringing up the previous commands from your history.

Executing commands in Warp is unlike anything I've ever seen before. Every command is broken into a Block which is a total rethink of the terminal. Instead of focusing primarily on the manipulation of text, you are focused on each command run as an independent unit you can manipulate through the UI. This is me trying to show what it looks like:

You'll notice the space and blocking between each command. Clicking those 3 dots gives you this dropdown:

All this functionality is available on your local machine but they are also available on machines you SSH (if the remote host is using bash). This opens up a massive collection of power text editing functionality on remote machines that might not be configured to be used as a "development machine". Check out that list here and think of how much time this might have saved you in your life.

Sharing

I think the primary sales point of Warp is the Sharing functionality, which allows for some very interesting workflows. It appears at some point you'll be able to add things like approval before you run commands (which I think is kind of a weird anti-pattern but still I applaud the idea). Right now though you can generate links to your specific block and share them with folks.

I made an example link you can see here. However I don't know how long the links last so here is a quick screenshot.

I love this for a million uses:

• it beats sharing snippets of text in Slack all the time
• it's a game changer for folks trying to do coding meetups or teaching a class
• adding in concepts like approval or review to commands would be mind-blowing for emergency middle of the night fixes where you want a group of people to review it. It doesn't seem to have this functionality yet but appears to be coming.

Daily Usage

Alright so I love a lot of the concepts but how much do I like using it as a daily driver? Let's focus on the positive stuff on Mac. I'm testing this on a 16 inch MacBook Pro with 32 GB of RAM, so about as powerful as it gets.

Pros

• Warp is just as fast as iTerm 2, which is to say so fast I can't make it choke on anything I tried.
• Steps into an area of the market that desperately needs more options, which is the multi-platform terminal emulator space. Warp is going to eventually be on Linux, Windows and Mac which right now is something only a handful of emulators can say, the biggest being alacritty.
• All this functionality comes out of the box. No giant configuration screens to go through, this all works from launch.

• I can't stress how "baked" this software feels. Everything works pretty much like they promise every time. Even with weird setups like inside of a tmux or processing tons of text, it kept working.
• The team is open to feedback and seems to be responsive. You can see their repo here.

Cons

• I don't love how much space the blocks end up taking up, even with "compact mode" turned on. I often take my laptop to my balcony to work and I miss the screen real estate with Warp that I get with iTerm 2.
• Missing profiles is a bummer. I like to be able to tweak stuff per workflow.
• I'd love some concept of bookmark if I'm going to lose so much space to the "Block" concept.
• My workflow is heavily invested in tmux and Vim, meaning I already have a series of shortcuts for how to organize and search my data into distinct blocks. I can change it for Warp, but right now that would be more of a lateral move than something that gives me a lot of benefits today.
• You really don't get a lot of customization. You can change the theme to one of their 7 preset themes. In terms of fonts, you have one of 11 options. My favorite themes and fonts weren't on this list.
• I would love some documentation on how I might write a plugin for Warp. There's stuff I would love to add but I couldn't really see how I might do that.
• A lot of the game-changing stuff is still in the pipeline, things like real-time collaboration and shared environmental variables.
• I wish they would share a bit more about how the app works in general. Even opening up the app bundle didn't tell me a lot. I have no reason to not trust this program, but anything they would be willing to share would be appreciated.

A Rust Mac App?

I'm very curious how they managed to make a Rust GUI application on the Mac. I'd love to see if there is some Swift UI or AppKit code in there or if they managed to get it done with the referenced Rust library. If they managed to make an application that feels this snappy without having to write Swift or Objective-C, all the more credit to this team. I'd love more information on how the app is constructed and specifically how they wrote the client front-end.

This does not feel like a "Mac app" though. Immediately you'll notice the lack of Preference pane underneath the "Warp" header on the menu bar. Help is not a drop-down but a search and in general there aren't a lot of MacOS specific options in the menu bar. This likely fits with their model of a common work platform across every OS, but if feeling "Mac-like" is important to you, know this doesn't. It's just as fast as a native application, but it doesn't have the UI feel of one.

Summary

If you are just starting out on the Mac as a development machine and want to use a terminal emulator, this is maybe the fastest to start with. It comes with a lot of the quality of life improvements you normally need to install a bunch of different pieces of software for. Also if you teach or end up needing to share a lot of code as you go, this "Sharing" functionality could be a real game-changer.

However if you, like me, spend your time mostly editing large blocks of text with Vim in the terminal, you aren't going to get a ton out of Warp right now. I just don't have a workflow that is going to really benefit from most of this stuff and while I appreciate their great tab completion, most of the commands I use are muscle memory at this point and have been for years.

I wish this team all the success though and cannot stress enough how great it is to see someone actually experimenting in this space. There are great ideas here, well executed with an eye for stability and quality. I'm going to keep a close eye on this product and will definitely revisit as things like "command reviews" are added down the line.

Like this? Think I'm out of my mind? Ping me on Twitter.