Being a DM is one of the more rewarding things you can do in the gaming space. It is the only opportunity I've ever had to truly create anything I want. After some time DMing you will struggle to enjoy videogames the same way, longing for the same levels of creative freedom. On the other hand, the pressure has never been higher.
Running a game can be very daunting for folks not used to it, so I thought I would put together some resources I've found extremely useful over the last 12 months of DMing. I'll try to update this post with new information as I find it.
Once you have all that, sit down and read through all of the basic rules. Feel annoyed and overwhelmed? Welcome to being a DM at first. Power through and then take a break and listen to a podcast.
https://www.jointhepartypod.com/ is a well-made DnD 5e podcast that explains all the rules as they go. This means you will get an explanation for what the characters did and why the GM did what they did.
After listening to the rules and the first 10 episodes of Join the Party, you are almost ready to start DMing. Here are some mistakes I made and what I should have done, so you don't make the same errors.
Don't let players roll their own characters at home. Make them do it in front of you because even otherwise good people tend to fudge some numbers and make some statistically unlikely characters.
Ask people to stick to classes and races in the PHB, or the players handbook, at first. This reduces the number of things you'll need to account for as a new DM and also prevents players who enjoy creating super-powered characters from taking too much advantage.
Do a few practice sessions of combat where you alternate between being the DM and being the player. It'll get you more experienced with combat.
Spells work under more complicated rules, so try a few of those out as well.
Have a session 0. This is where you all meet, chat about what you expect from the game and what you like. Here are some things you are going to want to discuss before you start playing together:
* Do you like exploration, combat or role-playing? See what parts of DnD your particular group likes
* How important is following the rules vs keeping the flow going? You are going to have to make judgement calls in the middle of games that might not be right or consistent with the rules. See if players are ok with this.
* Do you care if we do a custom campaign vs pre-written adventures? At first you'll want the security of the pre-written stuff
Post Session-0
Alright you professional DM, you are out there rocking and rolling, going through some pre-written adventures. Hopefully you are getting the hang of combat encounters and figuring out what parts of DMing you like vs which are a chore. For me I hate combat but love improv storytelling, where I try to say yes to the player whenever I can and build on top of their responses. But everyone is different.
Here are some things you will want to start to think about as you master the basics and begin to feel more comfortable in the chair:
Do you want to roll in front of players or behind the screen? This is a style decision and one I encourage people to think about. Rolling in front of players can be exciting since you also don't control what happens, but you need to be ok with what might happen as a result.
Start to add more flavor to your scenes. Maybe instead of "the sword misses" it "embeds itself into the stone". The more layers you start to add to the world, the more your players will get invested.
Don't be afraid to let things go off the rails. Games are often the most fun when players do something really unexpected, so don't be afraid to let that ride as long as it doesn't break the whole session. It is often as exciting for you as for them.
Maps! Visual aids help a lot when running a game. For in-person games I use a dry erase board and online I tend to steal maps from the following:
Q: Can I buy the PDFs of official DnD 5e adventures?
A: No, but you can buy them in the DnDBeyond app and search them that way
Q: How hard it is to write a custom campaign?
A: Totally depends! I recommend starting with a pre-made campaign and then if you feel more comfortable, writing small branching adventures off that main story. It takes some of the pressure off the daily prep work and lets you start to make a world.
I'm coming up on the halfway mark for this year of sponsoring a student from Norway. For those who stumble across this wondering "was the experience worth it" let me reassure you right away. It's an amazing experience that I wouldn't trade for the world. I was nervous with us being a childless couple in our 30s but it turned out to be a delightful learning experience for both parties. It pushes you out of your comfort zone and make you see your community and routine through new eyes.
One of my frustrations before my student came was how little information was out there. I hope people considering sponsoring a student will find some use in these tips. I'll continue to edit these as we discover more.
Registering for School Will Take Way Longer Than You Think
I was as surprised by this one as anyone. When we first accepted our I went to local high school and met with a counselor months before I needed to. I filled out a lot of his forms and was told "he's ready to go, just have him come back before school starts and we'll finish it up". Sounds good right?
What followed was a week of us being on the phone or at the school. We learned about the catch-22 of not being able to play in sports until you had a minimum number of course hours. There are questions about what grade he should be in. Every school official you meet is both eager to help and also telling you that you are missing some critical document or item. I provided many of the same documents several times.
All this means expect to spend some time in the school when your student arrives. Make a binder with copies of their important documents.
Have them cook once a week
One of the things I struggled with when our student first came was "what do I have him do". Some of the chores were pretty obvious. We asked him to keep his room clean and when he got home from school to walk the dog. I wanted something that would be more of a learning experience for both of us though. My wife told me about something she did when she was a foreign exchange student that I loved.
Put $100 in a drawer. Tell the student that once a week they need to cook dinner. They can pick the day but they need to go to the store themselves, pick out the items they need, get them home and cook them. It sounds simple but has actually proven to be one of the more useful exercises we've engaged in.
For the student it's a good immersion exercise. They need to plan ahead with a recipe, then go to the store with the limited funds and get the items they need. Finally they need to come home and have dinner ready by roughly the time the family eats.
Homesickness can sneak up on both of you
Homesickness is a constant challenge for foreign exchange students. We were ready for it around the 2-3 month mark. It didn't come though and it seemed like maybe our student just wasn't going to experience it. I chalked it up to social media ensuring he was still in regular contact with friends and family along with a deep passion for watching soccer and talking about it with his dad, a tradition he kept up.
Then the holidays hit. The photos of his family all together at the vacation cabin definitely caused him to experience some of the homesickness we had been ready for months earlier. It caught me off guard. It took me longer then maybe it should have to recognize what was going on and to help him deal with those issues. The point of this is a lot of the literature pounds into you that around 2-3 months they'll experience homesickness. That is by no means a hard and fast rule. Everybody is different.
You need more food
When you live with a partner in your 30s over time you start to mentally adjust sizes. You know how much laundry detergent to buy because you know in the back of your mind roughly how quickly you burn through it. Same with staples like bread and milk. These numbers start to become patterns on buying that you replicate every time you go to the store for grocery shopping. But with a kid you'll likely need a lot more food and it'll need to be easier to prep. I forgot how teenagers do not have things like basic knife skills and so prepping more complex foods is an time consuming process.
So for your first couple weeks keep an eye on what you are buying and what you are running out of. You'll start to develop the new list of things you buy along with the amounts you'll need.
IKEA Smart Home: Everything you love and hate about IKEA at once
Living in Denmark usually means I get everything late compared to the US. Movies come out a bit later, technology often isn’t available, etc.
So I delighted to learn (likely due to the extreme proximity between Denmark and Sweden) that I could get some IKEA Kadrilj smart blinds. After setting them up and having them work I have some feelings about the product.
The stuff I like
First IKEA couldn’t have made this process more simple. Out of the box, the Kadrilj blinds come with everything you need. There’s the signal repeater, the up/down button, the battery pack, the micro-USB cord to charge the battery pack and a USB wall plug. The instructions were pretty simple to install the blinds and I was pleased by how there didn’t seem to be any tricks to the installation itself. Working with the up/down button that comes out of the box was great. It was already paired to the signal repeater, so testing that basic functionality was charging up the smart blinds, plug in the repeater and then hit up/down to make them work. They also look quite nice and aren’t very loud which is a nice touch. The price is also excellent for what you get.
Outside of the blinds the IKEA gateway you need to connect the devices to Google Home, Alexa or Homekit is very simple to use. It is an ethernet-only device that only goes out to the internet to check for updates and update its time from NTP. Some people may see this as a downside but TRÅDFRI is pretty secure as far as IoT goes. The device itself is going to get an unencrypted JSON to download a signed file (so unless there is some local DNS hijacking this seems like a great approach). The local communication over your network is UDP but uses DTLS which means the UDP packets are encrypted. The key used for a handshake between devices is on the bottom of the device and seems long enough to be secure. I have a lot of experience with UDP but not that much with DTLS.
Above the networking layer for the gateway, the TRÅDFRI is running CoAP (Constrained Application Protocol) which gives a REST API. In testing with simple scripts written in Python, you have a lot of functionality open to you that isn’t available in the app yet. It’s great to know that in the future if IKEA were to drop support for the device I could still write scripts to manage the device without having to worry about IKEA relying on server-side communication to keep the device working. I also appreciate how IKEA limited themselves out of the gate and gave us something that is designed for years of operation. The device itself is nice looking although it did irritate me that the lights telling you of the status don’t change colors when they are working. Instead “blinking white” means not set up, “steady white” means it is working. I prefer colors to tell me status but this is not critical once you grasp how it works.
If you are interested in exploring the product on your own I found the pytradfri library to be super easy to use out of the box. It was easier to get information about the status of the gateway and the overall state of the world through this library than through the actual app. I don’t know if I would rely on it to do anything critical but it does seem like the kind of thing if you ever need to tinker with the stack it would let you do it.
Because the wireless protocol is Zigbee you don’t need the IKEA TRÅDFRI gateway, but I don’t have anything else that can serve in that control function and I wanted to connect the devices to Google Home. Be aware you can save a bit of money if you already own a Philips Hue stack. Also props to IKEA for including the USB cable for power, the wall outlet, and the Ethernet cable with the gateway. It’s a small thing but it is nice to not have to go into a box and dig up another Ethernet cable.
The stuff I didn't like
On the blinds themselves, it was hard to get the two to line up length. You have to hit the down button, then hold it and wait for it to get close to the next one and let go. If you miss the blind needs to roll back up to the top, you need to reset it by hitting one of the buttons twice to reset the length, then try again. It’s a small thing but it seems natural to me that you might have many blinds next to each other and would have loved a way to sync length between them. I also didn’t love how the battery pack that powers the blinds doesn’t seem to give me any feedback on its charging status. Once again IKEA has decided to go with one color LED here, another white LED. So I don’t know how charged the battery pack is. You let it charge for an hour, shrug and then plug it in.
In the smart home section, while I like the technical design of the gateway the app has some problems. First, the way you connect devices is super confusing at first. You take the up/down remote, reset it by unscrewing the back metal panel, then hitting a small button 4 times to reset it. You then take it over to the gateway, hold down the same button, wait for the app to say it has paired with the remote, then repeat with the signal repeater. Finally, you do the same thing with the blinds. This takes a LONG time. I couldn’t get it to pair with the signal repeater on the first 10 tries. Then I took a break, ate some dinner, came back and got it on the first try. It’s not clear to me why I need to use the remote to connect these devices and wish there was a less manual way to add stuff to the network. But, I suspect this might be a security thing since it requires me to get within 3 cm of the devices to pair.
Integrating it with Google Home, once everything was set up in the IKEA app, was super basic and worked great. Voice commands with Google Home worked on the first try and I didn’t encounter any surprises. Timers and other functionality in the IKEA app also worked. Since there is no internet functionality would appear to live on my local gateway which is great. But, I am a little worried about this stack.
Since everything is set up manually with me having to get quite close to each thing, I’m not sure how comfortable I am installing a lot of lights. There are reports online of people losing all their stored devices with software updates (which get applied to the gateway without any user intervention). I’m confident I could block it from hitting the update server (or doing something with DNS to trick the gateway to going to a local copy of the JSON file with the current firmware listed) but I’m not sure if that is a good idea since there isn’t a change log and I have no idea if there are critical security updates going out to the devices with these updates. Setting up three blinds again would be fine but setting up like 20+ devices with light bulbs and everything else is a multiple-hour job and since there is no backup to the internet I assume if the gateway dies or gets a bad software update I need to start from scratch.
Conclusion
I’m pretty pleased in general with these blinds. They work well with Google, they weren’t too bad to set up and I appreciate that IKEA has thought this product through to be defensively designed, which is refreshing in a world full of IoT devices that suck at security. The downside is that you are reliant on the physical gateway box that you purchased to continue working to keep your configurations and you are reliant on IKEA, who is not a software company, to keep all this running. I suspect a lot of the way they designed this product was with the assumption that if it proved to not be profitable it wouldn’t be dangerous for users to continue to run for years. I also appreciate that they used open standards and stacks without any ridiculous restrictions on access. If you have the gateway, you have the secret you need to make API calls.
The only thing I’m worried about is the lifespan of the blinds. Who knows if I’ll be able to buy replacement batteries in the future or where I will be able to find them. The lack of ability to wire these devices into power means I can’t imagine they have a lifespan beyond 5 years which is a bummer both for sustainability and also as a practical thing (I doubt the motor in there is only rated to work that long). But you can buy more of the battery packs (I didn’t see any inside of IKEA but they’re quite good about replacement parts so I’m going to give them the benefit of the doubt here). I’m excited though to start expanding my smart home collection with a few light bulbs and a few outlets. Yet, I wouldn’t put anything critical on this technology yet. It seems like IKEA is on its way to figuring it out, but if you were relying on this to keep AC going for people with medical conditions or other more niche use cases I would keep looking. The software isn’t quite there yet. For normal home use though it should be fine.
If you have a beard, you need to trim it. Like so many products in modern life, this is something that seems simple. You'll need a device that lets you select how much to trim off and it will need to be relatively easy to operate. You probably won't use it on a daily basis but a few times a week you will pull it out. This should be a product that is painfully simple to buy. If everything we think we know about mass manufacturing holds true, this should be something where I walk in, get what I need and leave.
So why have I owned so many of the things? I've owned this one and I've tried that one. These are good brands (or at least I thought) with reasonable features that seemed to meet every need I could imagine. They have lots of high reviews and the price seems right in line with what I would expect to pay for a small motor with guides. But I kept running into problems.
These devices just never seemed to work that well. Mostly the battery simply didn't last long enough to get the job done. So I did what anyone does. First I started by taking my Remington beard trimmer apart. How they work is pretty simple. The cam of a DC motor is inserted into the middle of the cutter assembly and makes a motion which then allows the blades to move back and forth. There are two blades in the cutter, one which doesn't move and another that does. All of this is powered by a pretty normal looking DC motor that (according to google) is rated to move around 6,000 RPM. All of these are pretty common pieces.
With the razor apart I focused on the PCB components and the battery. As far as I could tell, all the PCB was doing is regulating the power supply to the motor and charges the battery (along with controlling the LED that shows whether the thing is charging or not). The power supply in this case was a rechargeable AA battery. According to Google this battery takes 14-16 hours to fully charge and should run a motor like mine for 20-30 minutes.
This, to me, is the problem with these beard trimmers. Nothing about this list of features is wrong except it is asking me, as the consumer, to make peace with all sorts of crazy compromises. 14-16 hour charge time? That's not portable! I was thinking of this device like I would a laptop or a phone, but of course it isn't. I need to plan ahead in order to trim my beard. So either I leave the trimmer sitting on my counter all the time (that looks super nice, right?) or I plan ahead. I guess I should be putting reminders in my calendar for the night before.
The other issue is the run time. I don't think 20-30 minutes of running on a single charge works. It's clear this battery isn't powerful enough to keep this device running for years. In fact, everything about this trimmer is designed to have a very finite lifespan for absolutely no reason. This battery at its peak can trim my beard twice with a charge after a night of charging. The performance will, of course, decrease as cycles increases on the battery. But the motor is still working fine. The basic mechanics of how the blades work will continue to function with lubrication for thousands more hours than the battery will. I have two highly understood pieces of machinery held back on purpose by the limitation of the power supply.
Ok but what if we just ran off the battery when we needed to? This would meet all of our needs. As far as I can tell neither of these products do that. It's always running off of the battery even when plugged into the wall. This has left me multiple times with half a beard trimmed. It's not a simple problem but there are designs for power delivery that allow you to work off the battery when the power cord isn't connected and off the wall power when it is. This seems like such a common sense design that you would, of course want. Because I would argue the worst case scenario for a beard trimming is to be left in a half-complete state. I'd rather you not even try if I can't complete the operation.
So many modern products make me feel this way. A series of compromises that don't help me and seem to be exclusively designed in order to ensure that I gain as few benefits from the product as possible. They know the DC motor is the same DC motor that a more expensive trimmer uses. They know that if power was not an issue you would need to buy one of these trimmers in your life (short of accidental damage).
This isn't just a beard trimmer problem. My new phone doesn't have a headphone jack, ensuring I go out and buy bluetooth headphones with lifespans limited by the internal battery of those devices. My laptop doesn't have any ports I might need or a replaceable battery, ensuring I need to both buy new versions of the things I already have. In a world where we are increasingly seeing more and more signs that we must slow down the rate of consumption of natural resources, we are surrounded by products designed to have a finite end date.
So what did I end up doing for my trimmer? Well once I took it apart it turns out the problem was pretty simple. It has a 600mah battery. I swapped it for a 1900 mah battery (after taking the entire thing apart even though there was no reason to make it so complicated) and suddenly my device was working so much better. Now that the one restriction on the correct functionality of the device is removed suddenly this trimmer has at least another year of life in it.
You might ask then "What is the problem Mat?" You solved the problem. First, I solved the problem because I'm not afraid to take things apart. Nothing about this device is designed to be taken apart. The designers of this product might as well be screaming "Don't do it" with the way this device is put together. Second this feels like a "tech tax". We don't teach people how PCBs work or how to take things apart. I only know how to do this because of my personal hobbies. But people who, for economic reasons, can't afford to keep replacing these items with new ones every six months might not have that same information.
So I ended up replacing my trimmer with this one. It's a cheap unit that operates on most of the same principals we laid out before with the exception being that this one just pulls power from the wall. With a small amount of light machine oil this thing will last me for years and years with normal operation. I never need to think about whether its charged or what kind of battery it has in it. It isn't designed with the idea of "what is the bare minimum we need to ship as a product" but instead is a very simple to operate and maintain machine that does exactly what you want with a minimum amount of fuss.
I would argue we need fewer tools designed to fail and more items designed like this. It is certainly an older fashioned way to design products. It means fewer units sold year over year and will likely limit the potential future growth of your company. However we are coming up to a lot of hard decisions around the future of the human race in relationship to the consumption of natural resources and one of the things we need to start to do is design the items in our lives to operate for longer. There is no recycling program as effective as making something we never need to recycle (or need to recycle much less).
One of the biggest stories right now in the tech world is the bombshell dropped by Bloomberg that the biggest names in tech have been hacked. Not through a software exploit or through some gap in a firewall, but by the Chinese government infiltrating their server supply chain. You can read the story here.
This resonates with tech professionals for a lot of good reasons. First, the people who know how to make these servers do something useful for an end user are rarely the same people who understand the specifics of how they work. I might know how to bootstrap and set up a bunch of Dell servers, but if asked to explain in intricate detail how iDRAC functions or what specifically can be done to harden the server itself, I would have to plead some ignorance. There's just so much to learn in this field that one person can't do it all. At some point I have to trust the thing I just unboxed from Dell or Supermicro or whomever is safe to use.
So when the assertion came out that the supply chain itself for the hardware we all rely on may have been breached, it shook us all to our cores. First we rely on these companies to provide us with services not easily duplicated elsewhere. I don't know a lot of stacks anymore that don't rely on some AWS functionality, so saying all of that information may have been released to the Chinese government is akin to your bank saying it might have lost the keys to the safety deposit boxes. There's also the knowledge that we cannot individually all vet these vendors. I lack the training to disable every PCB inside of every model of device in my datacenter and determine whether it's safe and up to spec. I don't even have the diagrams to do this assuming I could! Basically Bloomberg asserted everyone's worse fears.
As time goes on we start to hear more problems with the actual reporting. One of the few named sources in the story, who provided technical background and context, admits it feels strange that almost everything he told the reporter as to how these attacks might happen is later apparently confirmed by different sources. I would encourage folks to listen to Joe Fitzpatrick here. Then the people who WERE supporting the narrative started to come out of the woodwork and they raised more questions than they answered. Yossi Appleboum, CEO of Sepio Systems (a firm with deep ties to the intelligence community based on their companies "About" page) comes out swinging that this is a much bigger problem! The scope is indeed even higher than Bloomberg asserts. You can read his take here.
Someone is lying, clearly. Either this hack didn't happen or it did happen and companies are willing to lie on the record to everyone involved. The later scenario feels unlikely for a number of reasons. One, because we know from recent events like the Facebook hack and the Google+ hack, the penalty for being caught leaking user data isn't that high. It would be a stock-price hit for Apple and Amazon if indeed this was true, but certainly one they could recover from. However the PR hit would appear to be relatively minor and you could bury the entire thing in enough technical details to throw off the average reader. Like I doubt if I told a random person on the street who uses and enjoys their iPhone that some of the servers making up the iCloud infrastructure might have been compromised by Chinese, they would swear off the company forever.
If it isn't "this hack happened and everyone involved is attempting to cover it up", then what is it? A false flag attack by the intelligence community on China? Sure maybe, but why? I've never seen anything like this before where a major news outlet that seemingly specializes in business news gets shut down by major industry players and continues writing the story. Whatever this story is, it feels like these companies coming out strongly against it is sending a message. They're not going to play along with whatever this narrative is. It also doesn't appear that foreign nationals would even need to go through all this trouble. Turns out the Supermicro firmware wasn't the best to start with on its own.
At the end of this saga I think either some folks at Bloomberg need to be let go or we need to open investigations into being deceived by officers of publicly traded companies. I don't see any other way for this to resolve.
I've worked in IT for my entire life in one capacity or another. Starting out as an "intern" for a local computer repair shop in my hometown, I developed strong opinions about many technologies. None of them have been as fundamental or as frustrating as DNS. I decided to take some time to do a series of posts on what DNS is, how it works and workarounds for common problems.
At a basis level DNS is the process of allowing humans to use easy to remember names like "google.com" while still allowing computers the ability to resolve specific IP addresses to send their requests. It also allows Google to change the IP address for google.com without having to run a global advertising campaign.
Why is DNS frustrating though?
DNS is frustrating because it works such a high percentage of the time. Like any system with a high track record of success, failures don't often come to mind when you are working in the system. If I'm troubleshooting why one system can't find a resource or speak to another (especially a third party system), my first inclination is to look inside the application stack, then the hosts networking and firewall, etc etc etc. In many ways this speaks to the reliability of DNS that it doesn't pop up in my mind.
However many experienced folks resist the urge to rely on DNS for mission-critical applications if at all possible. Some of this is the result of tribal thinking, where one experienced person dislikes relying on it and everyone follows. Some of it is a desire to simplify the troubleshooting scope if there is a problem. If a host can't reach an IP address I know where I need to start working. A lot of it is though that many people today don't have a super firm grasp on what DNS is doing or how to work with the tools it provides. Especially for younger DevOps folks its just not something they think about. Route 53 or other similar DNS providers are their DNS and other than that its best not to think about it too hard.
Ah /etc/hosts
Like any good story this one begins with a bit of a hack. It's the 70s and ARPAnet, or Internet Alpha, is a small community that likes to connect with each other. Much like today when someone doesn't want to manage DNS, they rely on a single text file of host names that is then turned into /etc/hosts. Much like modern times the burder of maintaining the single text file, in this case HOSTS.TXT became too much. People were constantly wanted to add or remove from it and it became a chore to ensure you had the latest copy. A more complicated system was required. Since these are engineers it'll likely be some sort of database.
It is a database!
DNS, or Domain Name System, is at its heart a distributed database. It has a structure which allows you to control the local pieces of that database while still allowing the worldwide distribution of all data. When we say DNS servers what we're talking about are nameservers. These servers hold the information for some part of the database and make that information available to the clients when asked. These clients, called resolvers, are relatively simple programs that create queries and send them to nameservers.
So what does DNS look like? Well if you've spend some time in the Linux filesystem it should look pretty familar to you. Here is the Linux directory structure.
Ok then here is the DNS database structure:
At the top is the root node or null node, shown as " " or in text as a single dot. We then break into the categories of names much like directories in Linux. Each domain name tells the DNS system where that domain name is in the database. So for instance, if I tell you to go to /var/log you read that and think "ok I'm going to start at / which is root, then go to var and inside of that I'll find log. With DNS its similar but in the opposite order. When you make a request to matduggan.com, the system is first reading that as "go to .com and then traverse from there". Much like larger file systems I have control over some parts but not others. For instance the root node is managed by ICANN (those folks who seem to keep coming up with bizarre TLDs or Top Level Domains like .genting). The .com directory is managed by Verisign (I know right? I had no idea). Finally I control the matduggan "zone" and it's managed by me. That means I can add subdomains at will. This system both prevents duplicates at the higher level while still giving me the freedom to do what I want at the lower level.
Apparently there is a level of depth to this directory structure of 127 which is pretty bonkers when you think about it. I'm not even sure how you'd get to 50.
Fun fact this is why when you are considering some DNS services you have to include a period at the end. So some software will require www.matduggan.com. since the . in this case is the root node. Domain names written in this manner are called fully qualified domain names.
So we've established that this is a database and you are accessing something. The something you are accessing is the resource records. There are different classes of records which tell you different things. We'll talk about all the records in their own post but for right now just mentally know the path ends at resource records.
Alright so we understand how a domain name works. But how does it work out practically that you control matduggan and not com. That has to do with zones. Nameservers generally have complete information about some part of the database structure, which we call a zone. That name server is then the authority for that zone. So how does that work in practice?
Starting again at www.matduggan.com. Verisign nameservers are the authority for .com and are responsible for keeping track of all the zones. In this scenario matduggan is its own zone and inside of that zone can be different zones for things like awesome.matduggan and terrible.matduggan. However if you don't have any subdomains you can just mentally think to yourself that your zone and your domain name are the same thing for now.
Nameservers
Now a bit more nuts and bolts time. The servers running this whole system are nameservers and there are two kinds. We have primary master nameservers and secondary master nameservers. A primary master nameserver for a zone has the information for that zone on a local file. The secondary master nameserver gets its zone data from the other host. You can also nest secondaries as needed.
Despite the naming both servers are authoritative for that zone. It's designed to make administration easier so you can worry about managing the one primary box and let the rest automatically update off of that nameserver. The local file the primary master nameserver is reading is the zone datafiles or just datafiles. (Which is a ridiculous name since all files have data but I guess we're going to let that go). Datafiles for secondary nameservers are generated and then used if the master can't be reached.
Resolvers
So we've got our nameservers. In this scenario I have my zone, matduggan. It's being managed by 10.20.30.40 as a primary master and 10.20.30.41 which is the secondary master. Requests are successfully flowing to each of these. But the thing actually making these requests are the resolvers. These are the clients that send the requests to our nameservers.
Resolvers have three jobs:
Query the nameserver
Interpreting responses (either errors or records)
Giving the information back to the program that asked for it
With BIND (we'll talk about BIND later but its the most common DNS service) the resolver isn't even a process running all the time. It's just a set of shared libraries that programs use to put together a query, send it, wait and then resend if it doesn't work. All the complex stuff is usually happening on the server. We call these simple resolvers stub resolvers.
Getting you the answer you want
Let's say I try to ssh into test.aws.com. The request is send to 10.20.30.40 as my local DNS server. Now my nameserver isn't the authoritative nameserver for aws.com. That'll likely be an Amazon server. My name server is going to go through a process called resolution to get me the information I need as a client. It starts with a request to the root nameservers we talked about before. So in this case 10.20.30.40 goes out and hits the root nameserver (assuming it has never done any requests before since these are mostly cached) and from there is told to go to the com nameserver. The com nameserver responds and tells my server to go to the aws server. This whole time my resolver is just sitting quietly waiting because all it understands, as a stub resolver, is either "here is the record you wanted or that record doesn't exist sorry". This type of resolution query is called a recursive query where the nameserver is basically going through the whole database structure and the client is doing nothing.
There's another design though. Some clients send an iterative query. This means 10.20.30.40 would respond not with the definite answer (unless it had that answer in a cache) but with the best possible information it has. The client then hits that new DNS server that is received what we call a referral to. Basically my DNS server was like "I don't know the answer but I know this server is closer to the answer, go ask them". The client does and is told "well yes I am closer but really you want to talk to THIS server". In this way the client walks the directory structure of DNS as opposed to my DNS server.
Alright so you understand now that the way the nameserver finds the answer depends a lot on how the question is asked by the client. But how does the nameserver decide where to start asking the question? There is not one root server but 13 distinct addresses. BIND, the software running on our nameserver picks the one it is going to use with a metric called roundtrip time or RTT. This works for nameservers of all kinds. It gets a list of nameservers at first, assigns a random low RTT time, starts sending queries and measures the time. By keeping track of which took the least time to respond it now knows which servers to send requests to in the future.
Wait though, this process has to work both ways right? Like if I hit an IP address looking for a domain name I should be able to look it up. How does that work?
Databases all the way down
Let's say we're talking about our test.aws.com box we're trying to SSH into. The address for this box is 12.13.120.180. There is a domain in DNS called the in-addr.arpa domain that keeps track of all IP addresses for us. Now with the IP address since we want to follow the same "least specific to most specific pattern" reading like a normal IP address isn't going to work. Instead you reverse the order. So when the IP 12.13.120.180 is read in a domain name its actually seen as 180.120.13.12 since that lets us efficiently go through the in-addr.arpa structure. This allows us to do the same authority delegation we do with domain names. So in our example AT&T is the authoritive nameserver for the 12 block, but let's say it doesn't control the last octet and that is controlled by a different name server. I can still effectively look up that information with the same process that let me look up the domain name.
This seems really slow
Doesn't it? Well thankfully DNS servers don't need to do this whole process very often. Every time our nameserver goes out into the internet we learn a lot about the world around us and we cache it locally. This saves us a ton of time in the future. Nameservers cache both successful results and unsuccessful results so the amount of traffic flying around is kept as low as possible.
These caches don't live forever though because otherwise a lot of the dynamic updating of the internet would break. Every zone gets to decide how long the data about their zone should last. This is called the TTL or time to live. Set a short one and you'll get more traffic to your nameservers, but less delay when you implement changes. Longer one, less traffic but more delay. Like all things there are tradeoffs.
That's the basics!
There you have it folks. The basics of DNS lookups. In the next post I'll discuss BIND, how to set it up, what the different options means, etc.
Git is an amazing tool that does a lot of incredible things. It's also a tool that is easy to screw up. Here is a list of tips I've found useful for when working with git.
Git Up
So when you run git pull chances are what you actually want to do is rebase over them. You also are probably working in a "dirty workspace" which means you want to pull the stuff ahead of you and rebase over it but until you are ready to commit, you don't want your work to be reflected. (I'm saying all this like its a certainty, I'm sure for a lot of people it's not. However I work in an extremely busy repo all day).
We're gonna set a new git alias for us that will do what we actually want to do with git. git config --global alias.up 'pull --rebase --autostash'
Autostash basically makes a temporary stash before you run the rest of the rebase and then applies it back after the operation ends. This lets you work out of a dirty directory. Make sure you sanity check your changes though since the final stash application has the potential to do some weird things.
Approving Stuff As You Git Add
Been working for awhile and ready to add stuff to a commit but maybe don't remember all the changes you've made? Not a problem.
git add -p lets you step through your changes one by one and only add the ones to this commit you need. I use this as a final sanity check to ensure some fix I didn't mean to go out with this deploy doesn't sneak out.
Go Back In Commit History
git reflog will show you a list of everything done across all branches along with the index HEAD@{index}
git reset HEAD@{index} will let you fix it
Commited to the wrong branch
Go to the branch you want: git checkout right-branch
Grab your commit: git cherry-pick SHA of commit (get this from git log) or if its the last commit on a branch just the name of the branch
If needed, delete the commit from the wrong branch: git checkout wrong-branch then git reset HEAD~ --hard to knock out the last commit
You can pass an argument to HEAD to undo multiple commits if needed, knucklehead: git reset HEAD~3 to undo three
Squash Commits Down
First just do a soft reset passing HEAD the argument for how many commits you want to go back: git reset --soft HEAD~3
Then take your previous commit messages and make them your new unified commit: git commit --edit -m"$(git log --format=%B --reverse HEAD..HEAD@{1})"
Fix Messed Up Commit Message
git commit --amend and then just follow the prompts
Everyone who has worked with specific vendors servers for awhile has likely encountered megacli. This is the comman line tool to manage your RAID controller and disks. Since often I'll only end up doing this every six months or so, I usually forget the syntax and decided to write it down here.
Install megacli
sudo apt-get install megacli
Common Parameters
Controller Syntax: -aN
This is telling MegaCLI the adapter ID. Don't use the ALL flag, always use this because you'll get in a bad habit and forget which boxes have multiple controllers
Physical drive: -PhysDrv [E:S]
I hate this format. E is the enclosure drive ID where the drive is and S is the slot number starting at 0. You get this info with the megacli -EncInfo -aALL
Virtual drive: -Lx
Used for specifying the virtual drive (where x is a number starting with zero or the string all).
Set the drive offline with megacli -PDOffline -PhysDrv[E:S] -aN
Mark the drive as missing: megacli -PDMarkMissing -PhysDrv [E:S] -aN
Get the drive ready to yank out: megacli -PDPrpRmv -PhysDrv [E:S] -aN
Change/replace the drive: megacli -PdReplaceMissing -PhysDrv[E:S] -ArrayN -rowN -aN
Start the drive: megacli -PDRbld -Start -PhysDrv [E:S] -aN
Fix Drive marked as "foreign" when inserted:
This happens when you steal a drive from a existing RAID to put in your more critical array. I know, you've never done anything so evil. Anyway when you need to here's how you do it.
Flip to good: megacli -PDMakeGood -PhysDrv [E:S] -aALL
Clear the foreign (that feels offensive but we're gonna move on): megacli -CfgForeign -Clear -aALL
